Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Elasticsearch provider can be configured to use Basic authentication if required by the  Elasticsearch server administrator. Username and password must be provided. Password must be encrypted by standard Aspire encryption utilities.

Claim

...

pre-fetch size

Code Block
languagexml
firstline78
linenumberstrue
<!-- noSql database provider for the 4.0 connector framework -->
<noSQLConnectionProvider>
    <implementation>com.searchtechnologies.aspire:aspire-elasticsearch-provider</implementation>
    <url>http://localhost:9200</url>
    <claimPrefetch>300</claimPrefetch>
    <claim>100</claim>
</noSQLConnectionProvider>

Elasticsearch provider can claim items from queues by larger units and after changing queue items status they are sent back to Elasticsearch as a bulk unit. By this technique we can achieve better performance.  We can configure claim unit size parameters based of the current Aspire installation (e.g. standalone/ distributed mode, etc.). This is how it works:

  • Pre-fetch "claimPrefetch" number of queue items of status "available". The default value is 10 000.
  • Take randomly "claim" number of items from the above pre-fetched set. The default value is 10 000.
  • Change the status of all "claimed" items from "available" to "inProgress" and sent this as a bulk update request to Elasticsearch.
  • Examine the result of the bulk request. All successfully updated items with no version conflicts can be sent one by one as a result of bulk claim to the connector framework.


Full configuration

Code Block
languagexml
firstline78
linenumberstrue
<!-- noSql database provider for the 4.0 connector framework -->
<noSQLConnectionProvider>
    <implementation>com.searchtechnologies.aspire:aspire-elasticsearch-provider</implementation>
    <url>http://localhost:9200</url>
    <claimPrefetch>300</claimPrefetch>
    <claim>100</claim>
    <keepSearchContextAlive>5m</keepSearchContextAlive>
    <authentication type="basic">
        <username>admin</username>
        <password>encrypted:password</password>
    </authentication>
    <debugOutFile>/tmp/aspire/profile.txt</debugOutFile>
    <maxRetries>3</maxRetries>
</noSQLConnectionProvider>