The Push Updates Listener is an Aspire Service that API allows external systems to push update requests into an Aspire Content Sourceseed. Each request will have one or more documents, each with one of the following actions: add, update and delete. The content source seed will then start a new incremental crawl and only process the documents that were sent on the request.

All requests will be stacked in a separate queue called

pushRequestQueue

fifoQueue, and each request will be processed sequentially using "first-in, first-out" policy.

Easy Heading Free

navigationTitle	On this Page
navigationExpandOption	expand-all-by-default

Filter body parameters

How to select audit items for aggregation. All fields are optional but we suggest at least one field is used (typically crawlId)

Section

Ingest Push

Update

Updates

Ingests push updates to the queueCreates basic crawl metrics report from the audit index. This endpoint needs the OPERATOR role to be executed.

Panel

borderColor	black
bgColor	#fafafb

GET POST /aspire/_api/metrics/:seedId/:crawlIdpushUpdates

Path and Request Body Parameters

idcrawlIdstring

Name	Type	Required	Description
seedId	string	Required	The seed
Id
documents
array	Required
The crawl id

Response

Code Block

language	js
theme	RDark

{
    "crawl-metrics": {
        "seedId": "0dda9332-ca52-44e1-ba63-95e51b46f951",
        "crawlId": "1658154843524",
        "metrics": {
            "add": 3,
            "enqueued": 4,
            "crawlBegin": 1,
            "scanned": 1,
            "crawlEnd": 1,
            "scannedNotIndexed": 1,
            "controlItem": 4
        },
        "workerDistribution": [
            {
                "workerId": "worker_192.168.112.1:41443//AAABghGLnak=",
                "metrics": {
                    "add": 3,
                    "enqueued": 4,
                    "crawlBegin": 1,
                    "scanned": 1,
                    "crawlEnd": 1,
                    "scannedNotIndexed": 1,
                    "controlItem": 4
                }
            }
        ]
    }
}

Status: 200

Section

Array with all documents that will be pushed.
documents.url	string	Required	Url or Id of the document.
documents.action	string	Required	add, update or delete.
documents.repItemType	string	Optional	Item type of the connector used for the seed. If not present Aspire will use its internal type for "pushed" documents
documents.metadata	object	Optional	Any extra metadata required by the content source to process the document.

Example

Code Block

theme	RDark

{
    "seedId": "b6c3c23a-2e4d-415a-8499-afd23815a5ad",
    "documents": [
        {
            "url": "file://C:tmp/test.txt"

Jobs performance statistics

Creates jobs performance statistics report from the audit index. This endpoint needs the ADMINISTRATOR role to be executed.

Panel

borderColor	black
bgColor	#fafafb

POST /aspire/_api/metrics/getJobsStatistics

Query String Parameters

Name Type Required Description

size

number

Optional

How many audit items should be included in the report in addition to the aggregation part. This does not affect the aggregation part of the report. (default = 0)

aggSource string Optional The audit item "metrics" part selected for aggregation. Can be a workflow stage, a workflow event or a workflow itself. Internal id is required here. Please check the audit index items for the idea which "id" should be used. (default = root)

aggSortField string Optional How to sort audit items requested by the "size" parameter. Descending order is used. (default = sumExecutionTime). Options: maxStageExecutionTime, minStageExecutionTime, avgStageExecutionTime, sumExecutionTime, countStages

Code Block

theme	RDark

{
  "filter" : {
    "seedId" : "seedId"
	"connectionId" : "connectionId",
	"connectorId" : "connectorId",
	"docId" : "docId",
    "crawlId": "crawlId",
	"timestampFrom" : millis from
	"timestampTo" : millis to
  }
}

Response

Code Block

language	js
theme	RDark

{
  "jobsStatisticsReport": {
    "configuration": {
      "JobStatisticsFilter": {
        "seedId": "0dda9332-ca52-44e1-ba63-95e51b46f951",
        "connectorId": null,
        "connectionId": null,
        "crawlId": null,
        "docId": null,
        "timestampFrom": null,
        "timestampTo": null
      },
      "size": 1,
      "aggSource": "root",
      "aggSortField": "sumExecutionTime"
    },
    "aggregations": {
      "documentsCount": {
        "value": 3
      },
      "metrics.root.countStages": {
        "count": 3,
        "min": 4.0,
        "max": 4.0,
        "avg": 4.0,
        "sum": 12.0
      },
      "metrics.root.minStageExecutionTimeaction": {
        "count": 3"add",
        "min": 0.0,
        "maxrepItemType": 3.0,
        "avg": 1.3333333333333333"FilesystemItemType@file",
        "sum": 4.0
      },
      "metrics.root.sumExecutionTime"metadata": {
        "count": 3,
        "mindisplayUrl": 1065.0,
        "max": 2044.0,
        "avg": 1579.3333333333333,
        "sum": 4738.0
      },
      "metrics.root.avgStageExecutionTime": {
        "count": 3,
        "min": 266.0,
        "max": 511.0,
        "avg": 394.6666666666667,
        "sum": 1184.0
      },
      "metrics.root.maxStageExecutionTime": {
        "count": 3,
        "min": 518.0,
        "max": 1507.0,
        "avg": 1014.0,
        "sum": 3042.0
      }
    },
    "audit": {
      "itemId": "c:"C:\\tmp\\ach1\\Directory4\\test2.txt",
      "seedId": "0dda9332-ca52-44e1-ba63-95e51b46f951",
      "connectionId": "0b069957-0f95-40b3-b036-f4f9c93614e1",
      "crawlId": "1658154843524",
test.txt",
         "crawlType": "full",
      "timestampfetchUrl": 1658154891061,
      "type": "add",
      "worker": "worker_192.168.112.1:41443//AAABghGLnak=",
      "status": "success",
      "retry": false,
      "statistics": {
        "jobId": "192.168.112.1:41443/2022-07-18T13:42:18Z/21"file:///C:/tmp/test.txt",
        "startTimestamp": 1658154882983,
        "stageconnectorSpecific": [
          {
            "source": "onAddUpdate",
            "sourceNamefield": "onAddUpdate",
  [
          "startTimestamp": 1658154883014,
            "stage": {
              "source": "627a709e-74ee-456c-a263-056a63a389f0",
              "sourceName@name": "fs2itbigBlb",
              "startTimestamp": 1658154883014,
              "stage": {
                "source$": "f0d17635-3fcc-475b-918b-86e157448df0",
                "sourceName": "onAdik",
                "startTimestamp": 1658154883014,
  PePo"
              "endTimestamp": 1658154883530
              },
              "endTimestamp": 1658154883530
            },]
            "endTimestamp": 1658154883530
          },
          {
            "source": "onPublish",
            "sourceName": "onPublish",
            "startTimestamp": 1658154883530,
            "stage": {
              "source": "627a709e-74ee-456c-a263-056a63a389f0",
              "sourceName": "fs2it",
              "startTimestamp": 1658154883531,
              "stage": [
                {
                  "source": "58be59a2-c88d-437e-abc5-6cf5ab804fb6",
                  "sourceName": "pepoScr1",
                  "startTimestamp": 1658154883531,
                  "endTimestamp": 1658154885038
                },
                {
                  "source": "9da23484-1523-4ab7-8e16-455390866c59",
                  "sourceName": "fsPubPep",
                  "startTimestamp": 1658154885038,
                  "endTimestamp": 1658154885039
                },
                {
                  "source": "8f07f20b-3934-4944-8046-583aa49e84ed",
                  "sourceName": "pubikE",
                  "startTimestamp": 1658154885039,
                  "endTimestamp": 1658154885059
                }
              ],
              "endTimestamp": 1658154885059
            },
            "endTimestamp": 1658154885059
          }}

Response

Code Block

language	js
theme	RDark

{
  "documents":{
        ],
        "endTimestamp": 1658154885059,
        "crawlId": "1658154843524",
        "seedId": "0dda9332-ca52-44e1-ba63-95e51b46f951",
        "id": "c:\\tmp\\ach1\\Directory4\\test2"url": "file://C:/tmp/test.txt",
        "connectorIdpushQueueId": "4048af9ef6c3c23a-55ae2e4d-4ee0415a-a730-66f1d87f14e1",
        "connectionId": "0b069957-0f95-40b3-b036-f4f9c93614e1",
        "metrics": {
          "root": {
            "maxStageExecutionTime": 1507,
            "minStageExecutionTime": 1,
            "avgStageExecutionTime": 511,
            "sumExecutionTime": 2044,
            "countStages": 4
          },
          "f0d17635-3fcc-475b-918b-86e157448df0": {
            "maxStageExecutionTime": 516,
            "minStageExecutionTime": 516,
            "avgStageExecutionTime": 516,
            "sumExecutionTime": 516,
            "countStages": 1
          },
          "58be59a2-c88d-437e-abc5-6cf5ab804fb6": {
            "maxStageExecutionTime": 1507,
            "minStageExecutionTime": 1507,
            "avgStageExecutionTime": 1507,
            "sumExecutionTime": 1507,
            "countStages": 1
          },
          "627a709e-74ee-456c-a263-056a63a389f0": {
            "maxStageExecutionTime": 1507,
            "minStageExecutionTime": 1,
            "avgStageExecutionTime": 511,
            "sumExecutionTime": 2044,
            "countStages": 4
          },
          "onPublish": {
            "maxStageExecutionTime": 1507,
            "minStageExecutionTime": 1,
            "avgStageExecutionTime": 509,
            "sumExecutionTime": 1528,
            "countStages": 3
          },
          "8f07f20b-3934-4944-8046-583aa49e84ed": {
            "maxStageExecutionTime": 20,
            "minStageExecutionTime": 20,
            "avgStageExecutionTime": 20,
            "sumExecutionTime": 20,
            "countStages": 1
          },
          "onAddUpdate": {
            "maxStageExecutionTime": 516,
            "minStageExecutionTime": 516,
            "avgStageExecutionTime": 516,
            "sumExecutionTime": 516,
            "countStages": 1
          },
          "9da23484-1523-4ab7-8e16-455390866c59": {
            "maxStageExecutionTime": 1,
            "minStageExecutionTime": 1,
            "avgStageExecutionTime": 1,
            "sumExecutionTime": 1,
            "countStages": 1
          }
        }
      }
    }
  }
}8499-afd23815a599"
  }
}

Status: 200

Section

Check queue item status

Check queue item status. This endpoint needs the OPERATOR role to be executed.

Panel

borderColor	black
bgColor	#fafafb

GET /aspire/_api/pushUpdates/:itemId

Path Parameters

Name	Type	Required	Description
itemId	string	Required	The queue item id (see "pushQueueId" in the response field above)

Examples

Get queue item status

Code Block

language	js
theme	RDark

GET /aspire/_api/pushUpdates/f6c3c23a-2e4d-415a-8499-afd23815a599

Response

Code Block

language	js
theme	RDark

{
  "pushQueueId": "f6c3c23a-2e4d-415a-8499-afd23815a599",
  "status": "A"
}

Status: 200, 404Status: 200

Page tree

Versions Compared

Old Version 1

New Version Current

Key

Ingest Push

Updates

Jobs performance statistics

Check queue item status

Page tree

Page History

Versions Compared

Old Version 1

New Version Current

Key

Ingest Push

Updates

Jobs performance statistics

Check queue item status