The following are the NoSQL DB providers supported by the Aspire 4.0 release:

• MongoDB version 3.4.10
• HBase version 1.2.4

Below you can find the list of the updates for this version.

Anchor

ConnectorsReleased

ConnectorsReleased

• NoSQL DB Provider
  • HBase supported.
• AIP Integration related features (Aspire Cloudera Parcel mode)
  • Audit Logs
    
    • Log Aspire actions via Jetty.
  • Cloudera Parcel updated.
  • Licensing
    
    • Licensing to build entitlements for AIP
    • Licensing to Access Aspire Application: Annual, Perpetual and Trial.
  • Log Tika errors as warnings.
  • Logging improvements .
    
    • Added processes and scripts to delete old logs.
  • Security Access Control Configuration.
    
    • User Role Authentication with LDAP.
  • Security
    
    • Access to file system/APIs via groovy.

Items Released

• Applications
• Connectors
•  Publishers
  • Google Cloud Search
• Services

• Applications
  
  • HDFS Binary File Writer.
  • Connectors
    
    • HBase
    • Jive
      
      • Simplified way to bring Security ACLs.
      • Support version 9.
      • New option to crawl specific places.
    • Kafka
    • Salesforce
      
      • Chatter Feeds Endpoint updated to use the SOAP API and Accounts management.
    • Service Now
      • New option to choose what to use for the Include/Exclude patterns. Either URLs or Knowledge Article Id.
    • SharePoint 2016
    • SMB
    • Framework
      
      • Updated start scripts to fail with versions of Java less than 1.8.
      • Normalize time zone for Aspire including logs and statistics.
      • Remove Recovery Policy options from the UI, since they are no longer used.
      • Aspire Archetype - Add required Felix properties for Kerberos authentication in the Aspire Distribution.
      • Jetty web server modifications for Aspire: HTTP TRACE method and server header info.
      • Option enable to change Aspire's port when distribution is created with maven command.
      • LDAP Cache collection names changed for something more meaningful.
    • Services
      
      • HAR Compactor
    • Publishers
      
      • HBase
      • Kafka
      • Solr Cloud (SolrJ library)
    • UI
      
      • New grid/list view on Content Source page.
  • Publishers
    • Azure Blobs

Anchor
bugfixes bugfixes

Bug Fixes

Anchor
AspireCorebugs AspireCorebugs

Aspire Core

• NPE killing Aspire distribution
• Error when second server joins a distributed crawl.

Anchor
auditingbugs auditingbugs

Auditing

• Page Navigation not showing all the jobs.

Anchor
connectorsbugs connectorsbugs

Connectors

• Aspider

  • Crawl getting stuck (Error claiming the entry from the queue).

  • Decoded characters in URLs when no needed.

  • Case-sensitive option not considering robots.txt file.

  • NPE when running an incremental crawl using an invalid URL in the seed file.

  • NPE when running a full crawl using default options.

  • Full Crawl - NPE while running a Full Crawl using default options.

  • Minor UI issues.

• CIFS
  
  • Connector not allowing special characters as paths to crawl.
  • Missing placeholder on "Path to URLs field.
• Jive
  • ACLs not getting updated when the privacy of a group changed.
  • Hierarchy for documents with attachments not constructed correctly.
  • Extract Text stage is not being executed with default settings.
  • Activity Incremental + Creation Date Filter not catching updates after the first one.
  • Activity Incremental crawl generating duplicate key error collection.
  • Unicode characters not handled at all in several Jive contents.
  • Hierarchy for Social Group's blogs being generated incorrectly.
  • Error not being displayed retrieving ACLs from Entitlement API using on-premise Jive.
  • No UI error when Aspire Security Plugin is disabled.
  • The first level of the hierarchy populating the content source name in "name" field.
• RDB Snapshot
  
  • Minor UI validations.
• Salesforce
  
  • ACLs error with default connector configuration.
  • ACLs for Groups/Users not reflected correctly after updates.
  • Error crawling Feed deleted.
  • Error on fetching Attachments.
  • Error getting deletes on Incremental for Tasks and Profiles.
  • Error on invalid session.
  • Error ending crawl.
• Service Now
  •  Deletes  Deletes being reported over attachments not updated.
• StageR
  
  • Connector doing reprocessing for incremental crawl.

Anchor
publishersbugs publishersbugs

Publishers

• StageR
  
  • Validation for Document Key field when field does not exist.

Anchor
UIbugs UIbugs

UI

• Importing connector with special characters in the path fields not loading correctly.

Anchor
ToRelease ToRelease

To Be Released

• Connectors
  
  • Lotus
  • RightNow
  • TeamForge
• Publishers
• CDH Hadoop
• Elasticsearch in Azure
• Web HDFS
• Services
  • Azure Group Expansion
  • HTTP Listener
  • HTTP Service

Anchor
ExtTechLimit ExtTechLimit

External Technical Limitations

• HBase: When running Aspire with long-term, large ingestion (with HBASE as the underlying HBase libraries may eventually stop returning results without throwing any error back, degrading the crawl performance down until it stalls completely. When this happens the only solution is to restart the affected aspire servers so the underlying HBase library threads get to connect from scratch.
• SalesForce connector: Due to SalesForce API limitations, the connector has the following limitations:
  • For incremental crawls, the  getUpdated and getDeleted methods are used, but when an attachment is updated from any item, that action will not be processed by the methods mentioned.
  • Security and incremental related limitations:
    • In security, we are only supporting 'Supported elements'.
    • For sharing related incremental crawling, unsharing of Salesforce item is not working.
    • For incremental crawling of Salesforce task items, we are only supporting tasks based on accounts.
    • If removal of sharing occurs for a item (e.g., removing sharing of an account), it is not reflect in the incremental crawl.
    • Pricebook sharing ACLs are not supported.
    • We are only supporting Tasks that are based on accounts for incremental crawling.
  • Chatter security
    • Chatter ACLs only will be retrieved if the “Filter TrackedChange feeds” option is checked.
    • Chatter ACLs are only supported for items that were created by a User or a Group, otherwise no ACL will be generated for the item.
    • The public chatter groups will have two ACLs, one for the public group and a PUBLIC:ALL ACL.
    • Private and Unlisted chatter groups will have one ACL for the group.
    • The followers of a chatter user will be treated as a private group called “<username>’s followers”, all the feed items created by a user to their followers will have this ACL.
    • The chatter item attachments will inherit the parent item ACLs.
    • Reducing the users retrieval scope might lead to a loss of ACLs, since no ACLs won’t be generated for followers of users outside the scope of the user retrieval.
  • Salesforce Compatibility limitation
    • Every 3 months Salesforce releases a new version of their API and, sometimes makes changes to the data structures, after each update there is a possibility that the compatibility between the connector and Salesforce will break.

Anchor
knownissues knownissues

Known Issues

Anchor
ItemDeprecate ItemDeprecate

Items Deprecated on Aspire 4.0

Anchor
AspireCore1 AspireCore1

Applications

• Archive Extractor
  • Deletes are not handled properly for incremental crawls.
  • Delete by query option is not working for ElasticSearch versions greater than 5.0. Ask to the Core Engineering team for a workaround if you need it.
    

Anchor
AspireCore2 AspireCore2

Aspire Core 

• Loading Application message trying to add connector but it does not load.
• Failover:
• Triple instance full crawl (double-interrupted): Having missing jobs.
• Dual Test Full Crawl interrupted: after aspire shutdown in one instance the other instance continue the crawl but never ends, Not all Docs are published on Solr.
•  Full test interrupted: after aspire shutdown and restarted docs are not published on Solr.
• NoSQL provider:
• Configurations - Encrypted fields is not working.
• Missing 'NoSQL provider unavailable' message when provider is down.

Anchor
AspireCore3 AspireCore3

Connectors

• Aspider

  • Crawl statistics are shown more items than actually processed.

• Documentum

  • Connector fails during group expansion with large number of users and groups. 

• Jive
  • Resume button

The following items are deprecated on this Aspire version:

• Elasticsearch bootloader
  • aspire-elastic-bootloader
• DCM
  
  • aspire-amazonec2-dm
  • aspire-zk-dm
• Big Data
  • app-semantic-co-occurrence-hadoop
  • app-semantic-co-occurrence-hadoop-soln
  • aspire-hadoop-job-launcher
  • aspire-hadoop-hdfs
  • aspire-hadoop-wiki-dict-generator
  • aspire-load-hdfs
• Connectors
  • Staging Repo Connector (File System)
  • SVN
• Services:
  • Fast Components
  • Fast Content API
  • Fast Query Completion Listener
  • Fast Query Listener
•  Solutions
  • OCR
  • Semantic Co-ocurrence
• Publishers
  • Cloudsearch
  • SharePoint 2013
  • Staging Repo Publisher (File System)