Released on March 7th, 2023

NoSQL DB provider supported by the Aspire 5.1 release:

  • New Elasticsearch/Kibana versions supported (7.17.7).
  • OpenSearch v. 1.3.7.

Java version supported

  • OpenJDK 17. 

New Features 

  • Aspire Framework
    • Cluster-wide resource management (Files).
    • New workflow events (onCrawlStart, onCrawlEnd, and onSubJob).
    • Advanced Groovy component.
    • Bulk action improvements (Seeds and Schedules).
    • Single Sign-on authentication method now supported (UI and API).
    • Support for hierarchical ACLs (Intersection ACLs) supported for Confluence crawls, Confluence Identity crawls, and Group Expansion.
  • Connectors. Newly migrated.
    • Confluence. Now supporting On-Premise and Cloud Servers.
    • OneDrive.
  • New Python Migration Scripts for Aspire 4.0 configurations. Available for the following connectors:
    • File System.
    • SMB.
    • Sharepoint Online.
  •  Publishers. Newly migrated.
    • Solr v. 8.9. 
  • Services. Newly migrated.
    • HTTP Listener.
  • Workflow. Newly migrated.
    •  Components
      • Archive Extractor. 
      • Tabular Files Extractor. 

Improvements

  • Aspire Framework
    •  General
      • JRE v. 17 now embedded on the distribution. 
      • Improved queue management, allowing timeouts.
      • Improved requests to nodes in case of communication failure.
      • Clarified the escape characters for specific URLs on Scope fields (Connections).
      • Changed MavenRepository implementation to use Apache Maven Resolver instead of Maven Aether Provider.
    • NOSQL Providers
      • Implemented index rollover management.
    • Scan Reports
      • Security Assessments: Critical, high, medium, and low tickets fixed (Dynamic and Static Application Assessments).
      • Vulnerabilities: Critical vulnerabilities fixed.
    • Resource Manager
      • Configuration item manager instead of using file paths.
  •  Connectors
    • Azure Identity
      • Allows multiple threads to speed up crawling.
    • DB Server
      • Support for the SAP HANA and IRIS database ports. 
      • Separated fields for Table and View Metadata query.
    • REST
      • 'Stop on Scan Error' option added. 
      • Pagination by next page link.
      • Support for expiration date format for Bearer authentication.
      • Include children metadata info inside customizable fields.
    • S3
      • MinIO support.
    •  ServiceNow
      • Inherit Knowledge Base User Criteria for Knowledge Article items. Known issue on 5.0.4.
      • Custom Search Query option for Knowledge Articles.
  •  Workflow
    • Summarizers
      • Job Summarizer Executor
        • Row sampling options added.
        • 'Fetch row using slices' option added.
      • Saga Summarizer Executor
        • Allows to detect Saga tags in columns of tabular files.
        • 'Python server manager & cache' option added.
        • 'Limit Entity Samples' option added to matched tags.

Bugs

  • Aspire Framework
    • API
      • NullPointerException executing a schedule with no seeds.
    •  General
      • Workers nodes not closing RAP Connections after completion.
      • Pausing a crawl closing in use connections immediately.
      • AspireObject not allowing valid characters in JSON format. Known issue on 5.0.4.
      • Items marked as excluded being processed in the Scan phase.
      • Non released items for dead workers.
      • Aspire not recovering after errors on standalone mode.
  • Connectors
    • Aspider:
      • Crawling a URL including the meta tag "robots" with content="noindex,nofollow".
      • Credentials not being encrypted correctly.
    • Group Expansion Manager
      • Issue merging entities.
      • No option to map groups based on attributes.
      • Some duplicated entries.
      • Some other minor issues.
    •  ServiceNow
      • Missing metadata fields for Knowledge Articles and their attachments.
    • Sharepoint Online
      • Crawl failing due to folder with special characters.
      • Hotfixes migrated to this version.
      • Some minor issues.
  • Publishers
    • Elasticsearch
      • Publisher failing having ES server configured to use self-signed certificates.
  • Schedules
    • Schedules triggered when it was not time for execution.
    • Duplicated entries for schedules on the Debug Console.
    • Crawl executed for disabled Sequence Schedule.
  • UI
    • Improved redirect after creating configuration items.
    • Some columns case-sensitive on the Seeds page when filtering.
  • Workflow
    • Summarizers
      • Saga Summarizer Executor
        • 'Entity Metadata' option removed.

Known Issues

  • Aspire Framework
    • Throttling is considering control items during the crawl.
    • Resource Manager
      • Resources (drivers) loaded not working for the RDB connections.
  • Connectors
    • REST
      • Bearer authentication failing with unknown expiring time format.
    • ServiceNow
      • Include pattern option not getting items by using the Short Description pattern field.
  • UI
    • Edit page for Seeds loading empty rare times.
    • UI actions are limited for users with both Admin and Operator roles.
  • Workflow
    • Extract Text
      • The Custom mappings option is not working.

Migrating from 5.0.2 to 5.1

Update the aspire-mappings index to refresh mappings:

aspire-mappings
PUT aspire-audit/_mapping
{
  "properties": {
    "audit.timestamp": {
      "type": "date"
    },
    "audit.seedId": {
      "type": "keyword"
    },
    "audit.crawlId": {
      "type": "keyword"
    },
    "audit.type": {
      "type": "keyword"
    },
    "audit.worker": {
      "type": "keyword"
    },
    "audit.statistics.seedId": {
      "type": "keyword"
    },
    "audit.statistics.connectionId": {
      "type": "keyword"
    },
    "audit.statistics.connectorId": {
      "type": "keyword"
    },
    "audit.statistics.id": {
      "type": "keyword"
    }
  }
}



  • No labels