Page History

...

Elasticsearch provider can claim items from queues by in larger units and after changing queue items status from Available to InProgress they are sent back to Elasticsearch as a bulk unit. By With this technique we can achieve better performance.  We can configure claim unit size parameters based on the current Aspire installation (e.g. standalone/ distributed mode, etc.). This is how it works:

• Prefetch "claimPrefetch" number of queue items of status "available". The default value is 10 000.
• Take random Pick randomly a set of size "claim" from the above prefetched set. The default value is 10 000.
• Change the status of all "claimed" items from "available" to "inProgress" and sent this items back as a bulk update request to Elasticsearch.
• Examine the result of the bulk request. All successfully updated items with no version conflicts can be sent one by one as a result of bulk claim to the connector framework.

...

Elasticsearch provider iterators use Elasticsearch scroll technique. The scrolls are resources which should be deleted after their use. This is done explicitly whenever possible by calling iterators close method or when the iteration is over. There are cases though when the iteration cannot be completed and in that case "unused scrolls " stay persist and can might potentially reach the limit of available resources. The parameter "keepSearchContextAlive" controls how long the scrolls should stay before deletion. The default value is "5m".  The format of this parameter and other information is described here.

...

The Provider will bulk for better performance chosen operations if "useBulk" is true (default false). The above example values of remaining parameters are also their default values

Profiling Settings

...