Aspire 3.3
Find more on the Cloudera parcel configuration at Aspire Parcel and Service for Cloudera.
A logical consequence of the integration into the Hadoop ecosystem is the support for HBase for crawl metadata and statistics (previously only MongoDB was supported), which will facilitate the use of Aspire as part of Big Data solutions. You can read about a successful and relevant use case here. This extended support made necessary some refactoring of the connector framework, implementing along the way several improvements over the last version (3.1.1).
All configuration steps needed for configuring HBase for crawl metadata can be found at HBase Settings.
Other new features of interest are Licensing and User Roles which improves the security control by separating users into "developers" and "administrators" with different roles and permissions over the Aspire configuration.
New connectors:
New publishers:
Please visit Release Notes for more technical information about this release.
includes significant enhancements with this new version.
Note: This version requires MongoDB or HBASE to be installed along with Aspire depending on the client’s environment.
An important new feature is the release of a Google Cloud Search (GCS) publisher. (You can read a blog post about it here).
Other enhancements include:
You can refer to the Release Notes for information on bug fixes and enhancements addressed in this version.
When importing a content source from 3.1.1 x into 3.3, the following error will may occur and the . The content source will may show up in with a red "Failed" status:.
Code Block |
---|
Error message: Unable to start appBundle: com.searchtechnologies.aspire:app-rap-connector Caused by: com.searchtechnologies.aspire.services.AspireException: Failed to register components from appBundle: CONTENT_SOURCE_NAME (Parent: <null>) at com.searchtechnologies.aspire.application.AspireApplicationImpl.registerAppBundleComponents(AspireApplicationImpl.java:945) at com.searchtechnologies.aspire.application.AspireApplicationImpl.registerAppBundle(AspireApplicationImpl.java:980) at com.searchtechnologies.aspire.application.AspireApplicationComponent.loadApplication(AspireApplicationComponent.java:696) at com.searchtechnologies.aspire.application.AspireApplicationComponent.loadApplication(AspireApplicationComponent.java:692) at com.searchtechnologies.aspire.configuration.ConfigurationManager.reloadApplication(ConfigurationManager.java:697) at com.searchtechnologies.aspire.configuration.ContentSourcesModule.processSyncUnitUpdate(ContentSourcesModule.java:309) at com.searchtechnologies.aspire.configuration.SynchronizedModule.run(SynchronizedModule.java:289) at java.lang.Thread.run(Thread.java:748) Caused by: com.searchtechnologies.aspire.services.AspireException: The value ("${waitForWfApps}") of element <waitForWfApps> is improperly formatted for a boolean - must be either "true" or "false" at com.searchtechnologies.aspire.framework.ComponentImpl.getBooleanFromConfig(ComponentImpl.java:2634) at com.searchtechnologies.aspire.connector.framework.controller.CrawlControllerImpl.initialize(CrawlControllerImpl.java:260) at com.searchtechnologies.aspire.framework.ComponentFactoryImpl.registerComponent(ComponentFactoryImpl.java:446) at com.searchtechnologies.aspire.application.ComponentManagerImpl.registerComponents(ComponentManagerImpl.java:328) at com.searchtechnologies.aspire.application.ComponentManagerImpl.initialize(ComponentManagerImpl.java:93) at com.searchtechnologies.aspire.application.PipelineManagerImpl.initialize(PipelineManagerImpl.java:75) at com.searchtechnologies.aspire.framework.ComponentFactoryImpl.registerComponent(ComponentFactoryImpl.java:446) at com.searchtechnologies.aspire.application.ComponentManagerImpl.registerComponents(ComponentManagerImpl.java:328) at com.searchtechnologies.aspire.application.ComponentManagerImpl.initialize(ComponentManagerImpl.java:93) at com.searchtechnologies.aspire.framework.ComponentFactoryImpl.registerComponent(ComponentFactoryImpl.java:446) at com.searchtechnologies.aspire.application.AspireApplicationImpl.registerAppBundleComponents(AspireApplicationImpl.java:941) |
This happens could happen because the Aspire 3.3 connectors contains several new contain configuration options that the "content source to import does not have." lack. To fix this error:
By doing this Aspire will generate generates the new options and save saves them into the configuration files.
Any migration from Aspire 3.1.1 requires a Full Crawl of all content sources since there was a major refactor on the MongoDB provider component. In specific the following tables changed:
processQueue
and scanQueue
.
@total (Int32)