Internal API
This is an internal API used for the communication between nodes and is not expected to be used by others.
The manager node API provides functionality that allows a manager and worker nodes to communicate.
Reset the status of any in memory batches that were sent to a failed worker but had not been acknowledged, so they may be sent to another worker. This endpoint needs at least the ADMINISTRATOR or MANAGER role to be executed.
PUT /aspire/_api/manager/release/worker/:id
PUT /aspire/_api/manager/release/worker
Path and Request Body Parameters
Name | Type | Required | Description |
---|---|---|---|
id | string | Required | The identifier(s) of the failed worker node(s) whose batches must be reset |
Examples
Release a single worker
PUT /aspire/_api/manager/release/worker/AAABcID5GBc= PUT /aspire/_api/manager/release/worker { "id": ["AAABcID5GBc="] }
Release more workers
PUT /aspire/_api/manager/release/worker { "id": ["AAABcID5GBc=", "AAABcIueWUc="] }
Response
{ "message": "optional response message" }
Status: 200, 404
Used by the main manager when pausing or stopping to release all the items for a seed from unsent batches. The manager will then forward the same to the workers. This endpoint needs at least the ADMINISTRATOR or MANAGER role to be executed.
PUT /aspire/_api/manager/release/seed/:id
PUT /aspire/_api/manager/release/seed
Path and Request Body Parameters
Name | Type | Required | Description |
---|---|---|---|
id | string | Required | The identifier(s) of the seed to be released |
Examples
Release a single seed
PUT /aspire/_api/manager/release/seed/AAABcID5GBc= PUT /aspire/_api/manager/release/seed { "id": ["AAABcID5GBc="] }
Release multiple seeds
PUT /aspire/_api/manager/release/seed { "id": ["AAABcID5GBc=", "AAABcIueWUc="] }
Response
{ "message": "optional response message" }
Status: 200, 404
Used by the worker to fetch a batch from the manager. This endpoint needs at least the ADMINISTRATOR or WORKER role to be executed.
GET /aspire/_api/manager/batch
Path and Request Body Parameters
Name | Type | Required | Description |
---|---|---|---|
worker | string | Required | The id of the worker asking for the batch |
type | string | Required | The type of batch required
|
tags | string | Optional | The tags this machine has |
items | number | Optional | The number of items to fetch |
Response
{ "batch": { "id": 0, "type": "process", "connector": "myConnectorId", "entry": [{ "_id": "item_id_1", "url": "item_url_1", "type": "com.accenture.aspire.connector.manager.ManagerComponent$FSItemType@filesystem", "status": "A", "action": "add", "timestamp": 1234567890, "shouldScan": true, "shouldProcess": true, "crawlRetries": 0, "isCrawlRootItem": false, "inCrawlRetries": 0 }, { "_id": "item_id_2", "url": "item_url_2", "type": "com.accenture.aspire.connector.manager.ManagerComponent$FSItemType@folder", "status": "A", "action": "update", "timestamp": 1234567890, "shouldScan": true, "shouldProcess": true, "crawlRetries": 0, "isCrawlRootItem": false, "inCrawlRetries": 0 }] }, "message": "optional response message" }
Status: 200, 406
Used by the worker node to acknowledge the worker has taken responsibility for the batch(es). This endpoint needs at least the ADMINISTRATOR or WORKER role to be executed.
PUT /aspire/_api/manager/batch/ack/:id
PUT /aspire/_api/manager/batch/ack
Query String Parameters
Name | Type | Required | Description |
---|---|---|---|
type | string | Required | The type of the batch
|
Path and Request Body Parameters
Name | Type | Required | Description |
---|---|---|---|
id | string | Required | The identifier(s) of the batch(es) to be acknowledged |
Examples
Acknowledge a single batch
PUT /aspire/_api/manager/batch/ack/AAABcID5GBc=?type=scan PUT /aspire/_api/manager/batch/ack?type=scan { "id": ["AAABcID5GBc="] }
Acknowledge multiple batches
PUT /aspire/_api/manager/batch/ack?type=scan { "id": ["AAABcID5GBc=", "AAABcIueWUc="] }
Response
{ "message": "optional response message" }
Status: 200, 404
Allows an administrator to manually re-balance the seed to manage allocation for active crawls. This endpoint needs the ADMINISTRATOR role to be executed
Allows handling the crawl failed state for the seed. This endpoint needs the ADMINISTRATOR or WORKER role to be executed
PUT /aspire/_api/manager/crawlFailed/:id
Response
{ "message": "optional response message" }
Status: 200, 404
Allows handling the throttle event happening in the connector for the seed. This endpoint needs the ADMINISTRATOR or WORKER role to be executed
PUT /aspire/_api/manager/throttle/:seedId/:pauseSeedUntil
Path and Request Body Parameters
Name | Type | Required | Description |
---|---|---|---|
seedId | string | Required | The identifier of the throttle seed |
pauseSeedUntil | number | Required | The time in milliseconds until which the seed must be paused |
Response
{ "message": "optional response message" }
Status: 200, 404