New and Enhanced Features

Aspire Core and Framework Components

• Salted Challenge Response Authentication Mechanism (SCRAM) support has been added to the MongoDB used with Aspire.
• The ability to dynamically load jar files has been added to Aspire with Java 9.
• When starting Aspire either normally or in debug mode, the debug line in the settings.xml file is handled appropriately. 

• A section has been added to the settings.xml file for HBase information.

• Logging of remote IP addresses for successful or failed logins will now occur.

• The Mongo Provider now encrypts/hashes IDs.
• Record fields have been improved.

• Entitlements checking now longer checks missing components at every restart.

• Time zones have been normalized for Aspire, including logs and statistics.

• The documentation has been updated for Keytab/Kerberos.
• Improvements have been added to Job usage.
• Updates have been made to the ExtractText default configuration limit for text extracted from a stream.
• List page retrieval and metadata extraction have been improved in Sharepoint Commons.

Aspire UI

• To re-fetch entitled components (after deleting the Resources folder), an "Allow Refresh" button has been added.

• The ability to show Provider Information has been added.

Connectors

• Aspider
  • Added a Headless browser for rendering dynamically generated pages (client side JavaScript pages).
• IBM Connections
  • Connector and dependency versions have been updated for the 3.3-SNAPSHOT IBM Connections.

• Sharepoint 2010

  • On the Multiple Urls drop-down, when the 'Site Discovery' option is set, the 'Set List View' option is removed. 

• Sharepoint Online
  • NPE at crawl end if bad credentials were used.
  • Incremental crawls no longer detect containers as updated items.

  • Scan recursively is working as expected.

Publishers

• Elasticsearch
  • Case sensitive index names can be handled properly now.
    
• Google Cloud Search
  • A new Google Cloud Search (GCS) publisher receives content from Aspire connectors and uses the Java Client library to index the content into Cloud Search.
• HBase
  • Content can now be deleted.
  • During a full crawl, the publisher now defaults to clean.
    

  • When not in file configuration mode, the publisher can now be used without security. 

• Publish to StageR

• Field level help has been added for the special scope $record.

Services

Applications

• The Entitlements Admin application has been updated.

Bug Fixes

Aspire Core and Framework Components

• Admin UI
  • one
• Aspire Application
  • one
  • two
  • three
  • four
• Connector Framework
  • • one
• MongoDB Provider
  • one
  • two
• Sharepoint Commons
  • one
• The Aspire Archetype had "http" rather than "https" repository and entitlement URLs.
• Failed to connect to Artifactory with custom keystore. Artifactory certificates were added to the distribution. See: https://contentanalytics.digital.accenture.com/pages/viewpage.action?spaceKey=aspire33&title=Crawling+via+HTTPs
• AspireObject was casting an incorrect numeric type when created from JSON.

• The AspireObject isEmpty method returned true even if the object had children.

• The processDeletes (String) was missing a Status page.
• The Aspire Connector Framework was not using shouldScan during incremental crawls.
• When running a full crawl, a "Provider 'encrypted' not installed" message could occur.

• The Mongo Provider generated an invalid JSON object during document conversion.

• Audit logs were incomplete.

• For AIP integration, the logout action was not being logged.

• Publisher framework retryDelay, retryDelayMultiplier and maxRetryDelay properties were not supported by Dynamic XML Forms (DFX).

• The Aspire-Services jar file was missing noSQL package.

• The "Loading Application" message could display whether a connector was loading or not.

• Extract Text
  • Use the Apache Tika SAX Parser for Microsoft documents.
• Scheduler
  • The option to create a Cache Groups scheduler was not being displayed.

Aspire UI

• A Connector component might not show the actual state of a crawl.
• The link that points to the Confluence wiki has been updated.

Connectors

• Aspider
  • An authentication form error could occur indicating "Target host is not specified while crawling".
  • Neither NTLM nor ADFS authentication was occurring when a host was specified in the Credentials.

  • On any port, the Port field was not working correctly with any value except "-1".

  • A crawl could cause a warning about duplicate IDs in MongoDB.

  • To indicate that the Gateway was not working, the exception message in ADFS needed updating. 

• Confluence

  • ACLs info appeared inside the hierarchy section.

  • A batch error could display while publishing to Elasticsearch 6.3.0.
    
• Documentum
  • Exception was being thrown during Group Expansion.
• File System
  • Starting Directories in the File option was not working as expected.

• IBM Connections

  • one
  • two
  • three
  • four
  • five
• Kafka
  

  • A "NO-NAME" field could occur.

• Sharepoint 2010

  • one
  • two
  • three
  • four
  • five
  • six

• Sharepoint 2013

  • one two three four five six seven eight nine ten

• Sharepoint 2016

• Sharepoint Online

  • one two three four five six seven eight nine ten

• Standalone Mode

  • When a user added a custom connector, feedback needed to be provided by the Aspire UI. 
• Staging Repository

  • A global variable was not working when configuring the server in the Staging Repository connector.

  • When crawling over multiple documents and publishing at two different scopes, the items published could be duplicated.

  • The Stager connection could be broken when running a full crawl.

Publishers

• DBC Plugin 

  • one
• Elasticsearch
  • DeleteByQuery was not being used with Elasticsearch 6.1.1.

• GCS Publisher

  • one
  • two
  • three
  • four
• Kafka

  • An error could be masked when running a non-batched job.

• Publish to Avro

  • Validation needed to be added to the Time Rollover Threshold field.

• Sharepoint Security Pre Trimmer

  • one

Services

• Group Expansion failed if user data exceeded the Mongo Max Document Limit (16MB).

• For Aspire Distributed mode, Services in the master node were not starting automatically after saving changes.
• Errors to reflect failed Services were not being generated.

• Services that were set up in an Aspire cluster were not synced up correctly.

• Azure Active Directory Group Expander

  • Users were not being removed.

• Group Expansion Service
  • The userGroupCache map was accessed when the Group Expansion Service was running.
  • two

• LDAP Cache Service

  • The controls did not display and the Schedule was set to Advanced even if Minutes or Hourly were set.

  • two
  • three
  • four
  • five

Applications

• Archive Extractor

  • The "Send delete by query first" option could throw an exception.

  • Deleting files inside of an archive file was not handled properly for incremental crawls.

• AVRO Extractor

  • During an incremental crawl, a "duplicate key error" message could display.

Known Issues

Connectors 

• FTP 

  • FTP connector is only working with Unix systems and not in Windows.