The following are the NoSQL DB providers supported by Aspire 3.2 release:
- MongoDB version 3.4.10
- HBase version 1.2.4
Below you can find the list of the updates for this version.
New Features
- NoSQL DB Provider
- AIP Integration related features (Aspire Cloudera Parcel mode)
- Audit Logs
- Log Aspire actions via Jetty.
- Cloudera Parcel updated.
- Licensing
- Licensing to build entitlements for AIP
- Licensing to Access Aspire Application: Annual, Perpetual and Trial.
- Log Tika errors as warnings.
- Logging improvements .
- Added processes and scripts to delete old logs.
- Security Access Control Configuration.
- User Role Authentication with LDAP.
- Security
- Access to file system/APIs via groovy.
This page maintains a list of all of the updates for version 3.2 of Aspire.
On this page:
New Features
- Web Crawler named Aspider Connector replaces the Legacy Heritrix Connector.
- Salesforce Connector has been refactored to include the following features:
- Runs in the new connector framework.
- Supports execution in a distributed environment.
- Allows concurrent crawling of multiple endpoints.
- Provides faster incremental crawls.
- Uses snapshots.
- New way to manage Failed Documents for all of the Source Connectors:
- Allows document reprocessing that previously failed in both processing and publishing stages.
- Avro Reader Extractor Application and Avro Publisher.
Status |
---|
subtle | true |
---|
colour | Green |
---|
title | Alpha version |
---|
|
- Parquet Extractor Application.
Status |
---|
subtle | true |
---|
colour | Green |
---|
title | Alpha version |
---|
|
- SMTP Extractor.
Status |
---|
subtle | true |
---|
colour | Green |
---|
title | Alpha version |
---|
|
- Support of Azure Authentication on the SharePoint Online Connector.
- New features for the SharePoint Connector (2007/2010):
- Supports default snapshots on incremental crawls.
- Supports crawling-specific views on lists.
- Implemented a single security key-store throughout all of Aspire.
- Updated SharePoint 2007/2010 Web Service Extensions.
To Be Released
- Connectors
- Publishers
- CDH Hadoop
- Elasticsearch in Azure
- Web HDFS
- Services
- Azure Group Expansion
- HTTP Listener
- HTTP Service
External Technical Limitations
- HBase: When running Aspire with long-term, large ingestion (with HBASE as the underlying HBase libraries may eventually stop returning results without throwing any error back, degrading the crawl performance down until it stalls completely. When this happens the only solution is to restart the affected aspire servers so the underlying HBase library threads get to connect from scratch.
- SalesForce connector: Due to SalesForce API limitations, the connector has the following limitations:
- For incremental crawls, the getUpdated and getDeleted methods are used, but when an attachment is updated from any item, that action will not be processed by the methods mentioned.
- Security and incremental related limitations:
- In security, we are only supporting 'Supported elements'.
- For sharing related incremental crawling, unsharing of Salesforce item is not working.
- For incremental crawling of Salesforce task items, we are only supporting tasks based on accounts.
- If removal of sharing occurs for a item (e.g., removing sharing of an account), it is not reflect in the incremental crawl.
- Pricebook sharing ACLs are not supported.
- We are only supporting Tasks that are based on accounts for incremental crawling.
- Chatter security
- Chatter ACLs only will be retrieved if the “Filter TrackedChange feeds” option is checked.
- Chatter ACLs are only supported for items that were created by a User or a Group, otherwise no ACL will be generated for the item.
- The public chatter groups will have two ACLs, one for the public group and a PUBLIC:ALL ACL.
- Private and Unlisted chatter groups will have one ACL for the group.
- The followers of a chatter user will be treated as a private group called “<username>’s followers”, all the feed items created by a user to their followers will have this ACL.
- The chatter item attachments will inherit the parent item ACLs.
- Reducing the users retrieval scope might lead to a loss of ACLs, since no ACLs won’t be generated for followers of users outside the scope of the user retrieval.
- Salesforce Compatibility limitation
- Every 3 months Salesforce releases a new version of their API and, sometimes makes changes to the data structures, after each update there is a possibility that the compatibility between the connector and Salesforce will break.
Anchor |
---|
| ItemDeprecate |
---|
| ItemDeprecate |
---|
|
Items Deprecated on Aspire 3.2
Applications
- Archive Extractor
- Deletes are not handled properly for incremental crawls.
- Delete by query option is not working for ElasticSearch versions greater than 5.0. Ask to the Core Engineering team for a workaround if you need it.
Aspire Core
- Loading Application message trying to add connector but it does not load.
- Intermittent black screen showing in the UI for less than a second (flickering screen).
- Failover:
- Triple instance full crawl (double-interrupted): Having missing jobs.
- Dual Test Full Crawl interrupted: after aspire shutdown in one instance the other instance continue the crawl but never ends, Not all Docs are published on Solr.
- Full test interrupted: after aspire shutdown and restarted docs are not published on Solr.
- NoSQL provider:
- Configurations - Encrypted fields is not working.
- Missing 'NoSQL provider unavailable' message when provider is down.
Connectors
The following items are deprecated on this Aspire version:
- RDB Snapshot
- Crawl was not finishing with the Use Slices option and set bad Extract SQL.
- No error was reported when setting a wrong ACL SQL.
- Wrong sql statement in Full crawl was not showing errors.
- RDB Tables
- Action column was ignored for the incremental crawl.
- Service Now
- Displayed incorrect URL field in Knowledge Articles (XML representation).
- Inclusion\Exclusion pattern was not working for attachments.
- Aspire error when two images files were attached and a full crawl was run.
- Social Cast
- Tag nonTextDocument was missed in the Aspire Object.
- SharePoint 2007
- Error on console and UI while crawling an item updated on root using both Index Containers and Scan Recursively disabled.
- NPE processed container after changing ACL on an Incremental crawl.
- ACLs showed the same item as group and user.
- SharePoint 2010
- Minor fixes to the tooltips.
- SharePoint 2013
- Incremental reported duplicate jobs when adding a subsite.
- Delete job had the incorrect displayUrl and fetchUrl after renaming a file.
- SharePoint Online
- Adding specific site collections made incremental crawl everything.
- Renaming an item returned an add, update and delete on the same crawl.
- Error when crawling site URL with encoded blank spaces.
Publishers
- Publish to Solr
- Deletes were not working correctly.
Services
- Add Service button was not working.
- Azure Group Expander
- Azure GE and SharePoint Online GE were not deleting users.
- CEWS Listener
- PropertyOflong and PropertyOfArrayOflong were not working.
- Fast Content API
- Group Expansion Manager
- Fixed 'Missing version number' error when service was loaded.
- Some validations were missed.
- LDAP Cache
- Some validations were missed.
- Problems with tooltips for LDAP Attribute in Cache user and Cache group options.
External Technical Limitations
- Zip files are not crawled with the Activity Incrementals when they are created inside Jive Documents.
To Be Released
- Amazon S3
- Box
- CEWS Listener
- FTP
- GSA Publisher
- IBM Connections
- PST Extractor
- Publish to HDFS
- Publish to SharePoint 2013
- Publish to SharePoint 2013 (Install & Setup)
- Salesforce
- Subversion
- Teamforge
Items to Deprecate on Aspire 3.2
The following items are marked to be deprecated on the next Aspire version:
- Elasticsearch bootloader
- aspire-elastic-bootloader
- DCM
- aspire-dcm-enterpriseaspire-amazonec2-dm
- aspire-zk-dm
The old Admin UI(s)- Parts of aspire-application
- Big Data
- app-semantic-co-occurrence-hadoop
- app-semantic-co-occurrence-hadoop-soln
- aspire-hadoop-job-launcher
- aspire-hadoop-hdfs
- aspire-hadoop-wiki-dict-generator
- aspire-load-hdfs
- Connectors
- Staging Repo Connector (File System)
- SVN
- Services:
- Fast Components
- Fast Content API
- Fast Query Completion Listener
- Fast Query Listener
- SolutionsSolutions
- Publishers
- Cloudsearch
- SharePoint 2013
- Staging Repo Publisher (File System)