Aspire in 40 minutes using the distribution archetype.
Learn how to build an Aspire distribution from scratch using the Aspire Maven Distribution archetype. All of the components will be downloaded automatically from Maven repositories, making system configuration and deployment a breeze.
Have your user registration name and password ready before you begin this tutorial.
Make sure you have completed: Installing Java and Installing the Maven Command Line
Anchor | ||||
---|---|---|---|---|
|
Now that you have Maven and Java (yay!), let's configure Maven so that it understands where the Search Technologies Maven Repository is located. This is the location on the internet where all of the Aspire components are located. The URL is http://repository.searchtechnologies.com/artifactory/simple/community-public/. It is password protected.
Note |
---|
Use your registered user name and password to access the Maven repository. Go to Registration for more information. |
After Maven understands where the ST repository is located, Maven can download Aspire and all of its pieces automatically to create new Aspire distributions with a single command.
Anchor | ||||
---|---|---|---|---|
|
Anchor | ||||
---|---|---|---|---|
|
For more detailed documentation for creating new distributions, see Using The Use the Maven Distribution Archetype
Now that Maven is installed, you can create a new Aspire distribution easilyy. Our recommendations are to:
Anchor | ||||
---|---|---|---|---|
|
Code Block | ||||
---|---|---|---|---|
| ||||
mvn archetype:generate -DarchetypeGroupId=com.searchtechnologies.aspire -DarchetypeArtifactId=aspire-distribution-archetype -DarchetypeVersion=4.0 -DrepositoryId=stPublic |
The command should be executed exactly as is. Do not modify the parameters; they specify where to locate the instructions for how to build a new Aspire Distribution.
Note: Sometimes the mvn command will fail to locate the Search Technologies repository. (It should be specified in your new settings.xml file.) If this is a problem, add the following argument to the mvn command from above:
Code Block | ||||
---|---|---|---|---|
| ||||
-DarchetypeRepository=http://repository.searchtechnologies.com/artifactory/public |
After executing the command, Maven will download a bunch of different JAR files. Most of these are various Maven plug-ins that Maven requires. These plug-ins are downloaded automatically from the "master maven repository in the sky", namely http://repo.maven.apache.org.
Finally, after Maven is ready to go, you will be prompted for the following information:
After you enter the final information and press Enter, you are prompted to confirm the entered information. Enter Y and Enter again.
Example output:
Code Block | ||||
---|---|---|---|---|
| ||||
>mvn archetype:generate -DarchetypeGroupId=com.searchtechnologies.aspire -DarchetypeArtifactId=aspire-distribution-archetype -DarchetypeVersion=4.0 -DrepositoryId=stPublic [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Maven Stub Project (No POM) 1 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] >>> maven-archetype-plugin:2.4:generate (default-cli) > generate-sources @ standalone-pom >>> . . . Define value for property 'groupId': : com.searchtechnologies Define value for property 'artifactId': : aspire-demo Define value for property 'version': 1.0-SNAPSHOT: : 0.1-SNAPSHOT [INFO] Using property: package = pom [INFO] Using property: aspireAdminPort = 50505 Define value for property 'mavenPassword': : xxx Define value for property 'mavenUser': : [email protected] Confirm properties configuration: groupId: com.searchtechnologies artifactId: aspire-demo version: 0.1-SNAPSHOT package: pom aspireAdminPort: 50505 mavenPassword: xxx mavenUser: [email protected] Y: : y [INFO] ---------------------------------------------------------------------------- [INFO] Using following parameters for creating project from Archetype: aspire-distribution-archetype:4.0 [INFO] ---------------------------------------------------------------------------- [INFO] Parameter: groupId, Value: com.searchtechnologies [INFO] Parameter: artifactId, Value: aspire-demo [INFO] Parameter: version, Value: 0.1-SNAPSHOT [INFO] Parameter: package, Value: pom [INFO] Parameter: packageInPathFormat, Value: pom [INFO] Parameter: version, Value: 0.1-SNAPSHOT [INFO] Parameter: package, Value: pom [INFO] Parameter: mavenUser, Value: [email protected] [INFO] Parameter: aspireAdminPort, Value: 50505 [INFO] Parameter: groupId, Value: com.searchtechnologies [INFO] Parameter: mavenPassword, Value: xxx [INFO] Parameter: artifactId, Value: aspire-demo [INFO] project created from Archetype in dir: C:\Users\AspireDemo\aspire-demo [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:19 min [INFO] Finished at: 2016-02-22T00:29:09-06:00 [INFO] Final Memory: 14M/172M [INFO] ------------------------------------------------------------------------ |
Anchor | ||||
---|---|---|---|---|
|
So now you can run Aspire, right? Well, actually, there's one more step.
The purpose of the Maven archetype (the previous step) was to build a Maven distribution project. Essentially, this is a template distribution. Now let's use Maven to build the distribution itself.
The sequence goes like this:
Code Block | ||
---|---|---|
| ||
Aspire Distribution Archetype --mvn--> Aspire Distribution Project --mvn--> Aspire Distribution |
Well, Aspire is meant for industrial-strength production installations. It is not a toy system. This means:
The Maven Distribution Project amply fulfills these needs. It provides a template for building copies of your distribution. Maven "profiles" can be used to customize the template for different environments, specifying different server addresses and database names for the Development environment vs the Staging environment, for example.
Okay, so let's actually build our Aspire distribution. Fortunately, this is very easy.
Again, since this is your first time running "mvn package", Maven will download a bunch of plug-ins, which it needs. But after that's done, it will build the Aspire distribution and you're ready to go.
Code Block | ||||
---|---|---|---|---|
| ||||
> cd aspire-demo >mvn clean package [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Aspire Distribution Project 0.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ Downloading: http://repository.searchtechnologies.com/artifactory/public/com/searchtechnologies/aspire/aspire-bootloader/4.0/maven-metadata.xml . . . [INFO] Reading assembly descriptor: distribution.xml [WARNING] Cannot include project artifact: com.searchtechnologies:aspire-demo:jar:0.1-SNAPSHOT; it doesn't have an associated file or directory. [WARNING] Cannot include project artifact: com.searchtechnologies:aspire-demo:jar:0.1-SNAPSHOT; it doesn't have an associated file or directory. [WARNING] The following patterns were never triggered in this artifact exclusion filter: o 'com.searchtechnologies:aspire-demo:jar:0.1-SNAPSHOT' [WARNING] Cannot include project artifact: com.searchtechnologies:aspire-demo:jar:0.1-SNAPSHOT; it doesn't have an associated file or directory. [WARNING] Cannot include project artifact: com.searchtechnologies:aspire-demo:jar:0.1-SNAPSHOT; it doesn't have an associated file or directory. [INFO] Copying files to C:\Users\AspireDemo\aspire-demo\target\aspire-demo-0.1-SNAPSHOT-distribution [WARNING] Assembly file: C:\Users\AspireDemo\aspire-demo\target\aspire-demo-0.1-SNAPSHOT-distribution is not a regular file (it may be a directory). It cannot be attached to the project build for installation or deployment. [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 7.855 s [INFO] Finished at: 2016-02-22T00:37:05-06:00 [INFO] Final Memory: 14M/168M [INFO] ------------------------------------------------------------------------ |
There are a number of warning message you may see when building the distribution. Don't let this worry you. Basically, if you see the following message, everything worked.
Code Block | ||||
---|---|---|---|---|
| ||||
[INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ |
However, here's an explanation for warnings you may see.
This warning refers to the fact we don’t hard code the encoding for the ascii files we use. We tried to do this in the very early stages and it gave us more problems than it fixed. This shouldn’t cause any issues as long as you’re not working in any languages that use extended character sets.
Code Block | ||||
---|---|---|---|---|
| ||||
[WARNING] Using platform encoding (MacRoman actually) to copy filtered resources, i.e. build is platform dependent! |
We recently changed the build process so that you could include dependencies in the pom file to cause “application bundles” to be copied from Maven in to the target bundles/aspire directory at build time. This allows Aspire to use those application bundles on hosts which have no internet access. You've always been able to do that with component bundles. The message below tells you that you’ve not got any dependencies with the group id com.searchtechnologies.appbundles.
Code Block | ||||
---|---|---|---|---|
| ||||
[WARNING] The following patterns were never triggered in this artifact inclusion filter: o 'com.searchtechnologies.appbundles:*' |
Some versions of Maven's mvn command (mainly 2.x) produce an unwanted file and copy it to the target bundles/aspire directory . This warning indicates your version didn’t (as this line warns you the file is not being excluded when things are being copied to the target directory)
Code Block | ||||
---|---|---|---|---|
| ||||
[WARNING] The following patterns were never triggered in this artifact exclusion filter: o 'com.searchtools:aspire-demo:jar:0.3-SNAPSHOT' |
This warning just refers to the fact that the build is copying to a directory and therefore can not produce something that can be place back in a Maven repository.
Code Block | ||||
---|---|---|---|---|
| ||||
[WARNING] Assembly file: /Users/st/aspire-test-3/aspire-demo/target/aspire-demo-0.3-SNAPSHOT-distribution is not a regular file (it may be a directory). It cannot be attached to the project build for installation or deployment. |
In Maven 2.x, you can get away with running the following command to build the distribution:
Code Block | ||||
---|---|---|---|---|
| ||||
mvn install |
In Maven 3.x, you can't run this command to build a distribution, so make sure you use mvn package as detailed above. If you do use install, the build will fail and you'll see the following:
Code Block | ||||
---|---|---|---|---|
| ||||
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.3.1:install (default-install) on project test-project: The packaging for this project did not assign a file to the build artifact -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException |
The following diagram shows the directory structure you have created within your Maven distribution project so far:
Anchor | ||||
---|---|---|---|---|
|
The default Aspire distribution is a fully functional version of Aspire. Let's launch Aspire.
Excerpt | |||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Here is example output:
At this point, you can use the standard administration interface (http://localhost:50505) to install pre-packaged applications such as connectors and search engine publishers.
ErrorsIf you see an error like this:
and Aspire exits:
the first thing to check is that you have the correct username and password in the config/settings.xml file. See General Settings for details.
For more information on how to start, stop, install as service, etc. Visit: Launch Control |
A simple application.xml file is provided with your application. Let's go to the debug interface and start it up to see what it does.
This will start up the sample application. Aspire will take a minute or two to launch the first time because (just like Maven) Aspire will automatically download components from the Search Technologies Maven Repository. These are stored in the Maven local repository (the .m2 directory) just like Maven does.
You should see something that looks like this:
Code Block | ||||
---|---|---|---|---|
| ||||
> cd target\aspire-demo-0.1-SNAPSHOT-distribution > bin\startup.bat Removing Felix-Cache and AppBundle-Cache directories Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0 2016-02-22 00:42:43.916:INFO::FelixStartLevel: Logging initialized @1980ms 2016-02-22 00:42:44.010:INFO:oejs.Server:FelixStartLevel: jetty-9.2.12.v20150709 2016-02-22 00:42:44.162:INFO:oejsh.ContextHandler:FelixStartLevel: Started o.e.j.s.ServletContextHandler@5f98816f{/,null,AVAILABLE} 2016-02-22 00:42:44.173:INFO:oejs.Server:FelixStartLevel: Started @2237ms 2016-02-22 00:42:44.441:INFO:oejs.ServerConnector:FelixStartLevel: Started ServerConnector@3de207d8{HTTP/1.1}{0.0.0.0:50505} ************************************************************************ * * ASPIRE BOOTLOADER * * Bundle id : 13 * * Location : file:bundles/boot/aspire-bootloader-4.0.jar * 2016-02-22T06:42:46Z INFO [BOOTLOADER]: Maven repository: stPublic - http://repository.searchtechnologies.com/artifactory/public/ 2016-02-22T06:42:46Z INFO [BOOTLOADER]: Maven local repository: C:\Users\Potter/.m2/repository 2016-02-22T06:42:46Z INFO [BOOTLOADER]: Maven update policy: always 2016-02-22T06:42:46Z INFO [BOOTLOADER]: Default version: 4.0 2016-02-22T06:42:46Z INFO [BOOTLOADER]: Fetching: com.searchtechnologies.aspire:aspire-application:4.0 . . . Starting bundle: 13 - file:/C:/Users/.m2/repository/com/searchtechnologies/aspire/aspire-extract-domain/4.0/aspire-extract-domain-4.0.jar Started component factory: aspire-extract-domain (bundle 13) Registering component: /FeedOneExample/StandardPipeManager/ExtractDomain Registering component: /FeedOneExample/StandardPipeManager/PrintToFile AUTOSTART: Complete |
After the sample application is started, you can return to the debug console: http://localhost:50505/aspire
The debug-console provides status information and some low-level admin controls for Aspire. Through the debug console you can:
So you can see, the debug console contains quite a lot of useful functionality. Spend some time exploring it before going on to the next (and final) step.
Let's process a URL with the default pipeline.
After clicking on "feed", you should see the "Job status" table that indicates your job has been submitted to the Aspire pipeline.
So, the pipeline doesn't actually send the document to a search engine to index it, but it certainly could. That would only take adding one additional stage to the pipeline.
The job should take only a small fraction of a second to process. You can see that it's done (and that everything worked correctly) by refreshing the page. If it was successful, you should see "Successful" in neon green text.
You can click on the "detail" link in the job status to see the results of the pipeline. What you should see (you may need to scroll down a bit) is an XML representation of Aspire's internal document structure, which contains the result of all document processing. It will contain a <content> tag which contains the text content (from the text extraction component), http headers and web server information (from communicating with the web server at searchtechnologies.com), the <content> (added by the text extractor), and the <domainName> (extracted from the host name of the URL).
Congratulations! You have built a brand-new distribution of Aspire from scratch using the distribution archetype, and you launched the default Aspire installation and processed a URL. Cool!
Here are some suggestions on what to do next:
For example, the following is a minimal Groovy Scripting component:
Code Block | ||
---|---|---|
| ||
<component name="SampleGroovyScript" subType="default" factoryName="aspire-groovy"> <config> <script> <![CDATA[ println "Hello World!"; println "The Aspire Document is this: "; println doc.toString(/*pretty:*/true); ]]> </script> </config> </component> |
You can add this component to your default Aspire configuration using the following steps:
Info |
---|
For more information on the Groovy scripting language, go here http://groovy.codehaus.org/ |
Info |
---|
For more information on methods available on the "doc" variable in your Groovy script go to:
|