Page History
Aspire 3.3 (the first Aspire release as part of Accenture!) includes the successful integration of Aspire into the Hadoop ecosystem as a Cloudera parcel with Accenture AIP. However, Aspire can still be used normally in standalone mode, also as a parcel inside Cloudera.
Find more on the Cloudera parcel configuration at Aspire Parcel and Service for Cloudera.
A logical consequence of the integration into the Hadoop ecosystem is the support for HBase for crawl metadata and statistics (previously only MongoDB was supported), which will facilitate the use of Aspire as part of Big Data solutions. You can read about a successful and relevant use case here. This extended support made necessary some refactoring of the connector framework, implementing along the way several improvements over the last version (3.1.1).
All configuration steps needed for configuring HBase for crawl metadata can be found at HBase Settings.
Other new features of interest are Licensing and User Roles which improves the security control by separating users into "developers" and "administrators" with different roles and permissions over the Aspire configuration.
New connectors:
- HBase Connector
- Kafka Connector
- SMB2 Protocol Connector
- Kinesis (to be released)
- Exchange (to be released)
New publishers:
Please visit Release Notes for more technical information about this release.
Migrating from Aspire 3.x
When importing a content source from 3.x into 3.3, the following error will occur and the content source will show up in red "Failed" status:
| Code Block |
|---|
Error message: Unable to start appBundle: com.searchtechnologies.aspire:app-rap-connector
Caused by: com.searchtechnologies.aspire.services.AspireException: Failed to register components from appBundle: CONTENT_SOURCE_NAME (Parent: <null>)
at com.searchtechnologies.aspire.application.AspireApplicationImpl.registerAppBundleComponents(AspireApplicationImpl.java:945)
at com.searchtechnologies.aspire.application.AspireApplicationImpl.registerAppBundle(AspireApplicationImpl.java:980)
at com.searchtechnologies.aspire.application.AspireApplicationComponent.loadApplication(AspireApplicationComponent.java:696)
at com.searchtechnologies.aspire.application.AspireApplicationComponent.loadApplication(AspireApplicationComponent.java:692)
at com.searchtechnologies.aspire.configuration.ConfigurationManager.reloadApplication(ConfigurationManager.java:697)
at com.searchtechnologies.aspire.configuration.ContentSourcesModule.processSyncUnitUpdate(ContentSourcesModule.java:309)
at com.searchtechnologies.aspire.configuration.SynchronizedModule.run(SynchronizedModule.java:289)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.searchtechnologies.aspire.services.AspireException: The value ("${waitForWfApps}") of element <waitForWfApps> is improperly formatted for a boolean - must be either "true" or "false"
at com.searchtechnologies.aspire.framework.ComponentImpl.getBooleanFromConfig(ComponentImpl.java:2634)
at com.searchtechnologies.aspire.connector.framework.controller.CrawlControllerImpl.initialize(CrawlControllerImpl.java:260)
at com.searchtechnologies.aspire.framework.ComponentFactoryImpl.registerComponent(ComponentFactoryImpl.java:446)
at com.searchtechnologies.aspire.application.ComponentManagerImpl.registerComponents(ComponentManagerImpl.java:328)
at com.searchtechnologies.aspire.application.ComponentManagerImpl.initialize(ComponentManagerImpl.java:93)
at com.searchtechnologies.aspire.application.PipelineManagerImpl.initialize(PipelineManagerImpl.java:75)
at com.searchtechnologies.aspire.framework.ComponentFactoryImpl.registerComponent(ComponentFactoryImpl.java:446)
at com.searchtechnologies.aspire.application.ComponentManagerImpl.registerComponents(ComponentManagerImpl.java:328)
at com.searchtechnologies.aspire.application.ComponentManagerImpl.initialize(ComponentManagerImpl.java:93)
at com.searchtechnologies.aspire.framework.ComponentFactoryImpl.registerComponent(ComponentFactoryImpl.java:446)
at com.searchtechnologies.aspire.application.AspireApplicationImpl.registerAppBundleComponents(AspireApplicationImpl.java:941) |
This happens because the 3.3 connectors contains several new configuration options that the content source to import does not have.
To fix this:
- Click on the content source to enter the configuration page
- Click on "Save" and then "Done"
By doing this Aspire will generate the new options and save them into the configuration files.
MongoDB Changes
Any migration from Aspire 3.x requires a Full Crawl of all content sources since there was a major refactor on the MongoDB provider component. In specific the following tables changed:
| Collection | Fields in 3.x | Fields in 3.3 | Compatible |
|---|---|---|---|
| audit |
|
| Yes |
| errors |
|
| No |
| hierarchy |
|
| No |
processQueue and scanQueue |
|
| Yes |
| snapshot |
|
| No |
| statistics |
|
| No |
| status |
|
| No |
*These fields are available since 3.1
Overview
Content Tools