Excerpt | ||
---|---|---|
| ||
Aspire in 40 minutes using the distribution archetype |
The previous quick-start used a pre-built Aspire distribution which was fixed for a certain set of components.
In this quick-start, you'll learn how to build an Aspire distribution from scratch, using the Aspire Maven Distribution archetype. Also, all of the components will be automatically downloaded from Maven repositories - making system configuration and deployment a breeze.
On this page:
Table of Contents |
---|
Before you begin, you need to be registered to use Aspire (go to http://aspire.searchtechnologies.com/) if you haven't already done that.
You will need your user registration name and password in order to complete this tutorial.
The version of Java you should use depends on the Aspire version you are targeting to:
We recommend installing the Java JDK (Java Development Kit), just in case you want to create your own Aspire components/applications in the future. But only the JRE (Java Runtime Environment) is absolutely required.
Aspire is tightly integrated with Apache Maven, more so than any other software tool. New distributions are built using a Maven archetype and Aspire will automatically download components and application bundles stored in Maven repositories.
1. Please refer to Step 1 above for the instructions on accessing your Environment Variables via the Control Panel.
2. Add the M2_HOME system variable:
Now that you have Maven and Java (yay!), you will need to configure Maven so that it understands where the Search Technologies Maven Repository is located. This is the location on the internet where all of the Aspire components are located. The URL is http://repository.searchtechnologies.com/artifactory/simple/community-public/. It is password protected.
Note: You will need to use your registered user name and password to access the Maven repository. Go to Registration for more information.
Once Maven understands where the ST repository is located, Maven can automatically download Aspire and all of its pieces to create new Aspire distributions with a single command.
More detailed documentation for creating new distributions is here: Use The Maven Distribution Archetype
Now that Maven is installed, creating a new Aspire distribution is relatively easy. Our recommendations are to:
mvn archetype:generate -DarchetypeGroupId=com.searchtechnologies.aspire -DarchetypeArtifactId=aspire-distribution-archetype -DarchetypeVersion=3.3 -DrepositoryId=stPublic
The command should be executed exactly as is. Do not modify the parameters (they specify where to locate the instructions for how to build a new Aspire Distribution).
Note that, sometimes, the "mvn" command will fail to locate the Search Technologies repository (it should be specified in your new settings.xml file). If this is a problem, add the following argument to the "mvn" command from above:
-DarchetypeRepository=http://repository.searchtechnologies.com/artifactory/simple/community-public/
After executing the command, Maven will download a bunch of different JAR files. Most of these are various Maven plug-ins which Maven itself requires to execute. These plug-ins are downloaded automatically from the "master maven repository in the sky", namely http://repo.maven.apache.org.
Finally, once Maven itself is ready to go, you will be prompted for the following information:
After you enter the final information and press the Enter key, you are prompted to confirm the entered information. Enter Y and press the Enter key again.
Here is an example output:
>mvn archetype:generate -DarchetypeGroupId=com.searchtechnologies.aspire -DarchetypeArtifactId=aspire-distribution-archetype -DarchetypeVersion=3.3 -DrepositoryId=stPublic [INFO] Scanning for projects... Downloading: http://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.4.1/maven-clean-plugin-2.4.1.pom . . . Define value for property 'groupId': : com.searchtechnologies Define value for property 'artifactId': : aspire-demo Define value for property 'version': 1.0-SNAPSHOT: : 0.1-SNAPSHOT Define value for property 'mavenPassword': : : xxx Define value for property 'mavenUsername': : : [email protected] [INFO] Using property: package = pom Confirm properties configuration: groupId: com.searchtechnologies artifactId: aspire-demo version: 0.1-SNAPSHOT package: pom Y: : y [INFO] ---------------------------------------------------------------------------- [INFO] Using following parameters for creating project from Archetype: aspire-distribution-archetype:2.2.2 [INFO] ---------------------------------------------------------------------------- [INFO] Parameter: groupId, Value: com.searchtechnologies [INFO] Parameter: artifactId, Value: aspire-demo [INFO] Parameter: version, Value: 0.1-SNAPSHOT [INFO] Parameter: package, Value: pom [INFO] Parameter: packageInPathFormat, Value: pom [INFO] Parameter: version, Value: 0.1-SNAPSHOT [INFO] Parameter: package, Value: pom [INFO] Parameter: groupId, Value: com.searchtechnologies [INFO] Parameter: artifactId, Value: aspire-demo [INFO] Parameter: mavenPassword, Value: xxx [INFO] Parameter: mavenUsername, Value: [email protected] [INFO] project created from Archetype in dir: C:\Users\pnelson\Desktop\AspireDemo\aspire-demo [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 45.292s [INFO] Finished at: Wed Jan 26 13:28:55 EST 2014 [INFO] Final Memory: 9M/109M [INFO] ------------------------------------------------------------------------
So now you can run Aspire, right? Well, actually, there's one more step.
The purpose of the Maven archetype (the previous step) was to build a Maven distribution project. Essentially this is a template distribution. We will now need to use Maven to build the distribution itself.
The sequence goes like this:
Aspire Distribution Archetype --mvn--> Aspire Distribution Project --mvn--> Aspire Distribution
Why is it so complicated? Well, Aspire is meant for industrial-strength production installations. It is not a toy system. This means:
The Maven Distribution Project amply fulfills these needs. It provides a template for building copies of your distribution. Maven "profiles" can be used to customize the template for different environments, specifying different server addresses and database names for the Development environment vs the Staging environment, for example.
Okay, so let's actually build our Aspire distribution. Fortunately, this is very easy.
Again, since this is your first time executing "mvn package", Maven will download a bunch of plug-ins, which it needs to do its job. But once that's done, it will build the Aspire distribution and you're ready to go.
> cd aspire-demo > mvn clean package [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Aspire Distribution Project 0.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ Downloading: http://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.5/maven-resources plugin-2.5.pom Downloaded: http://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.5/maven-resources plugin-2.5.pom (7 KB at 15.1 KB/se c) Downloading: http://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/19/maven-plugins-19.pom . . . [INFO] Reading assembly descriptor: distribution.xml [INFO] Copying files to C:\Users\pnelson\Desktop\AspireDemo\aspire-demo\target\aspire-demo-0.1-SNAPSHOT-distribution [WARNING] Assembly file: C:\Users\pnelson\Desktop\AspireDemo\aspire-demo\target\aspire-demo-0.1-SNAPSHOT-distribution is not a regular file (it may be a directory). It cannot be attached to the project build for installation or deployment. [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3:12.857s [INFO] Finished at: Wed Jan 25 13:46:04 EST 2012 [INFO] Final Memory: 9M/51M [INFO] ------------------------------------------------------------------------
There are a number of warning message you may see when building the distribution. Don't let this worry you. Basically, if you see the following message, everything worked OK :
[INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------
However, here's an explanation of warnings you may see.
This warning refers to the fact we don’t hard code the encoding for the ascii files we use. We tried to do this in the very early stages and it gave us more problems than it fixed. This shouldn’t cause any issues as long as you’re not working in any languages that use extended character sets.
[WARNING] Using platform encoding (MacRoman actually) to copy filtered resources, i.e. build is platform dependent!
We recently changed the build process so that you could include dependencies in the pom file to cause “application bundles” to be copied from Maven in to the target bundles/aspire directory at build time. This allows Aspire to use those application bundles on hosts which have no internet access. You've always been able to do that with component bundles. The message below tells you that you’ve not got any dependencies with the group id com.searchtechnologies.appbundles.
[WARNING] The following patterns were never triggered in this artifact inclusion filter: o 'com.searchtechnologies.appbundles:*'
Some versions of Maven's mvn command (mainly 2.x) produce an unwanted file and copy it to the target bundles/aspire directory . This warning indicates your version didn’t (as this line warns you the file is not being excluded when things are being copied to the target directory)
[WARNING] The following patterns were never triggered in this artifact exclusion filter: o 'com.searchtools:aspire-demo:jar:0.3-SNAPSHOT'
This warning just refers to the fact that the build is copying to a directory and therefore can not produce something that can be place back in a Maven repository.
[WARNING] Assembly file: /Users/st/aspire-test-3/aspire-demo/target/aspire-demo-0.3-SNAPSHOT-distribution is not a regular file (it may be a directory). It cannot be attached to the project build for installation or deployment.
In Maven 2.x, you can get away with running the following command to build the distribution:
mvn install
In Maven 3.x, you can't run this command to build a distribution, so make sure you use mvn package as detailed above. If you do use install, the build will fail and you'll see the following:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.3.1:install (default-install) on project test-project: The packaging for this project did not assign a file to the build artifact -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
The following diagram shows the directory structure you have created within your Maven distribution project so far:
The default Aspire distribution is a fully functional version of Aspire. Let's launch Aspire.
Here is example output:
Removing Felix-Cache and AppBundle-Cache directories ************************************************************************ * * ASPIRE BOOTLOADER * * Bundle id : 10 * * Location : file:bundles/boot/aspire-bootloader-3.3.jar * . . . . 2014-02-04T20:38:43Z INFO [/Workflow]: Installed component: /Workflow/WfManager 2014-02-04T20:38:43Z INFO [aspire]: Successfully started appBundle: /Workflow (location: com.searchtechnologies.aspire:app-workflow-manager) AUTOSTART: No applications to start
NOTE: If you are downloading Aspire Community, ignore the error message about being unable to download the com.searchtechnologies.aspire:aspire-dcm-enterprise component. The aspire-dcm-enterprise component is available only with Enterprise systems (and is used for Distributed Processing and Failover).
At this point, you can use the standard administration interface (http://localhost:50505) to install pre-packaged applications such as connectors and search engine publishers. Go to UI Introduction for more details.
If you see an error like this:
2014-11-19T10:20:35Z INFO [BOOTLOADER]: Fetching: com.searchtechnologies.aspire:aspire-application:2.0.3 2014-11-19T10:20:43Z ERROR [BOOTLOADER]: Cannot get file from repository (http://repository.searchtechnologies.com/artifactory/simple/community-public/) - check the component is properly deployed. File: http://repository.searchtechnologies.com/artifactory/simple/community-public/com/searchtechnologies/aspire/aspire-application/3.3/aspire-application-3.3.jar org.apache.maven.wagon.authorization.AuthorizationException: Access denied to: http://repository.searchtechnologies.com/artifactory/simple/community-public/com/searchtechnologies/aspire/aspire-application/3.3/aspire-application-3.3.jar at org.apache.maven.wagon.providers.http.LightweightHttpWagon.fillInputData(LightweightHttpWagon.java:119) at org.apache.maven.wagon.StreamWagon.getInputStream(StreamWagon.java:116) at org.apache.maven.wagon.StreamWagon.getIfNewer(StreamWagon.java:88) at org.apache.maven.wagon.StreamWagon.get(StreamWagon.java:61) at org.sonatype.aether.connector.wagon.WagonRepositoryConnector$GetTask.run(WagonRepositoryConnector.java:511) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:662)
and Aspire exits:
ERROR: Failed to load Aspire application Shutting down OSGI
the first thing to check is that you have the correct username and password in the config/settings.xml file. See here for details.
A simple application.xml file is provided with your application. Let's go to the debug interface and start it up to see what it does.
This will start up the sample application. Aspire will take a minute or two to launch the first time because (just like Maven) Aspire will automatically download components from the Search Technologies Maven Repository. These are stored in the Maven local repository (the .m2 directory) just like Maven does.
You should see something that looks like this:
> cd target\aspire-demo-0.1-SNAPSHOT-distribution > bin\startup.bat Removing Felix-Cache and AppBundle-Cache directories ->Removing Felix-Cache and AppBundle-Cache directories ************************************************************************ * * ASPIRE BOOTLOADER * * Bundle id : 10 * * Location : file:bundles/boot/aspire-bootloader-3.3.jar * . . . Starting bundle: 13 - file:/C:/Users/pnelson/.m2/repository/com/searchtechnologies/aspire/aspire-extract-domain/2.2.2/aspire-extract-domain-2.2.2.jar Started component factory: aspire-extract-domain (bundle 13) Registering component: /FeedOneExample/StandardPipeManager/ExtractDomain Registering component: /FeedOneExample/StandardPipeManager/PrintToFile AUTOSTART: Complete
Once the sample application is started, you can return to the debug console: http://localhost:50505/aspire
The debug-console provides status information and some low-level admin controls for Aspire. Through the debug console you can:
So you can see, the debug console contains quite a lot of useful functionality. Spend some time exploring it before going on to the next (and final) step.
Let's process a URL with the default pipeline.
After clicking on "feed", you should see the "Job status" table that indicates your job has been submitted to the Aspire pipeline.
The steps above submit a URL down the default pipeline, which will:
So, the pipeline doesn't actually send the document to a search engine to index it, but it certainly could. That would only take adding one additional stage to the pipeline.
The job should take only a small fraction of a second to process. You can see that it's done (and that everything worked correctly) by refreshing the page. If it was successful, you should see "Successful" in neon green text.
You can click on the "detail" link in the job status to see the results of the pipeline. What you should see (you may need to scroll down a bit) is an XML representation of Aspire's internal document structure, which contains the result of all document processing. It will contain a <content> tag which contains the text content (from the text extraction component), http headers and web server information (from communicating with the web server at searchtechnologies.com), the <content> (added by the text extractor), and the <domainName> (extracted from the host name of the URL).
Congratulations! You have built a brand-new distribution of Aspire from scratch using the distribution archetype, and you launched the default Aspire installation and processed a URL. Cool!
Here are some suggestions on what to do next:
For example, the following is a minimal Groovy Scripting component:
<component name="SampleGroovyScript" subType="default" factoryName="aspire-groovy"> <config> <script> <![CDATA[ println "Hello World!"; println "The Aspire Document is this: "; println doc.toString(/*pretty:*/true); ]]> </script> </config> </component>
You can add this component to your default Aspire configuration using the following steps: