The Push Updates API allows external systems to push update requests into an Aspire seed. Each request will have one or more documents, each with one of the following actions: add, update and delete. The seed will then start a new incremental crawl and only process the documents that were sent on the request.

All requests will be stacked in a separate queue called pushRequestQueuefifoQueue, and each request will be processed sequentially using "first-in, first-out" policy.

Easy Heading Free

navigationTitle	On this Page
navigationExpandOption	expand-all-by-default

credential

Section

Ingest Push Update

Creates basic crawl metrics report from the audit indexIngests push updates to the queue. This endpoint needs the OPERATOR role to be executed.

Panel

borderColor	black
bgColor	#fafafb

POST /aspire/_api/pushUpdates

Body Parameters

Name	Type	Required	Description
seedId	string	Required	The seed Id

description

documents

string

array

Required

An optional description of the server

Array with all documents that will be pushed.
documents.url	string

OptionalCredential id can be specified for the serverdeleteIncrementalPolicy

Required	Url or Id of the document.
documents.action	string

OptionalDelete policy idthrottlePolicy

Required	add, update or delete.
documents.repItemType	string	Optional

Throttle policy idroutingPoliciesstring arrayOptionalRouting policies idspropertiesJSONRequiredDynamic JSON object with server properties

Item type of the connector used for the seed. If not present Aspire will use its internal type for "pushed" documents
documents.metadata	object	Optional	Any extra metadata required by the content source to process the document.

Example

Code Block

theme	RDark

{
    "seedId": "b6c3c23a-2e4d-415a-8499-afd23815a5ad",
    "documents": [
        {
            "url": "file://C:tmp/test.txt",
            "action": "add",
            "repItemType": "FilesystemItemType@file",
            "metadata": {
                "displayUrl": "C:\\tmp\\test.txt",
                "fetchUrl": "file:///C:/tmp/test.txt",
                "connectorSpecific": {
                    "field": [
                        {
                            "@name": "bigBlb",
                            "$": "PePo"
                        }
                    ]
                }
            }
        }
    ]
}

Response

Code Block

language	js
theme	RDark

{
  "documents":{
    "url": "file://C:/tmp/test.txt",
    "pushQueueId": "f6c3c23a-2e4d-415a-8499-afd23815a599"
  }
}

Status: 200

Section

Jobs performance statistics

Creates jobs performance statistics report from the audit index. This endpoint needs the ADMINISTRATOR role to be executed.

Panel

borderColor	black
bgColor	#fafafb

POST /aspire/_api/metrics/getJobsStatistics

Query String Parameters

Name	Type	Required	Description
size	number	Optional	How many audit items should be included in the report in addition to the aggregation part. This does not affect the aggregation part of the report. (default = 0)
aggSource	string	Optional	The audit item "metrics" part selected for aggregation. Can be a workflow stage, a workflow event or a workflow itself. Internal id is required here. Please check the audit index items for the idea which "id" should be used. (default = root)
aggSortField	string	Optional	How to sort audit items requested by the "size" parameter. Descending order is used. (default = sumExecutionTime). Options: maxStageExecutionTime, minStageExecutionTime, avgStageExecutionTime, sumExecutionTime, countStages

Filter body parameters

How to select audit items for aggregation. All fields are optional but we suggest at least one field is used (typically crawlId)

Code Block

theme	RDark

{
  "filter" : {
    "seedId" : "seedId"
	"connectionId" : "connectionId",
	"connectorId" : "connectorId",
	"docId" : "docId",
    "crawlId": "crawlId",
	"timestampFrom" : millis from
	"timestampTo" : millis to
  }
}

Response

Code Block

language	js
theme	RDark

{
  "jobsStatisticsReport": {
    "configuration": {
      "JobStatisticsFilter": {
        "seedId": "0dda9332-ca52-44e1-ba63-95e51b46f951",
        "connectorId": null,
        "connectionId": null,
        "crawlId": null,
        "docId": null,
        "timestampFrom": null,
        "timestampTo": null
      },
      "size": 1,
      "aggSource": "root",
      "aggSortField": "sumExecutionTime"
    },
    "aggregations": {
      "documentsCount": {
        "value": 3
      },
      "metrics.root.countStages": {
        "count": 3,
        "min": 4.0,
        "max": 4.0,
        "avg": 4.0,
        "sum": 12.0
      },
      "metrics.root.minStageExecutionTime": {
        "count": 3,
        "min": 0.0,
        "max": 3.0,
        "avg": 1.3333333333333333,
        "sum": 4.0
      },
      "metrics.root.sumExecutionTime": {
        "count": 3,
        "min": 1065.0,
        "max": 2044.0,
        "avg": 1579.3333333333333,
        "sum": 4738.0
      },
      "metrics.root.avgStageExecutionTime": {
        "count": 3,
        "min": 266.0,
        "max": 511.0,
        "avg": 394.6666666666667,
        "sum": 1184.0
      },
      "metrics.root.maxStageExecutionTime": {
        "count": 3,
        "min": 518.0,
        "max": 1507.0,
        "avg": 1014.0,
        "sum": 3042.0
      }
    },
    "audit": {
      "itemId": "c:\\tmp\\ach1\\Directory4\\test2.txt",
      "seedId": "0dda9332-ca52-44e1-ba63-95e51b46f951",
      "connectionId": "0b069957-0f95-40b3-b036-f4f9c93614e1",
      "crawlId": "1658154843524",
      "crawlType": "full",
      "timestamp": 1658154891061,
      "type": "add",
      "worker": "worker_192.168.112.1:41443//AAABghGLnak=",
      "status": "success",
      "retry": false,
      "statistics": {
        "jobId": "192.168.112.1:41443/2022-07-18T13:42:18Z/21",
        "startTimestamp": 1658154882983,
        "stage": [
          {
            "source": "onAddUpdate",
            "sourceName": "onAddUpdate",
            "startTimestamp": 1658154883014,
            "stage": {
              "source": "627a709e-74ee-456c-a263-056a63a389f0",
              "sourceName": "fs2it",
              "startTimestamp": 1658154883014,
              "stage": {
                "source": "f0d17635-3fcc-475b-918b-86e157448df0",
                "sourceName": "onAdik",
                "startTimestamp": 1658154883014,
                "endTimestamp": 1658154883530
              },
              "endTimestamp": 1658154883530
            },
            "endTimestamp": 1658154883530
          },
          {
            "source": "onPublish",
            "sourceName": "onPublish",
            "startTimestamp": 1658154883530,
            "stage": {
              "source": "627a709e-74ee-456c-a263-056a63a389f0",
              "sourceName": "fs2it",
              "startTimestamp": 1658154883531,
              "stage": [
                {
                  "source": "58be59a2-c88d-437e-abc5-6cf5ab804fb6",
                  "sourceName": "pepoScr1",
                  "startTimestamp": 1658154883531,
                  "endTimestamp": 1658154885038
                },
                {
                  "source": "9da23484-1523-4ab7-8e16-455390866c59",
                  "sourceName": "fsPubPep",
                  "startTimestamp": 1658154885038,
                  "endTimestamp": 1658154885039
                },
                {
                  "source": "8f07f20b-3934-4944-8046-583aa49e84ed",
                  "sourceName": "pubikE",
                  "startTimestamp": 1658154885039,
                  "endTimestamp": 1658154885059
                }
              ],
              "endTimestamp": 1658154885059
            },
            "endTimestamp": 1658154885059
          }
        ],
        "endTimestamp": 1658154885059,
        "crawlId": "1658154843524",
        "seedId": "0dda9332-ca52-44e1-ba63-95e51b46f951",
        "id": "c:\\tmp\\ach1\\Directory4\\test2.txt",
        "connectorId": "4048af9e-55ae-4ee0-a730-66f1d87f14e1",
        "connectionId": "0b069957-0f95-40b3-b036-f4f9c93614e1",
        "metrics": {
          "root": {
            "maxStageExecutionTime": 1507,
            "minStageExecutionTime": 1,
            "avgStageExecutionTime": 511,
            "sumExecutionTime": 2044,
            "countStages": 4
          },
          "f0d17635-3fcc-475b-918b-86e157448df0": {
            "maxStageExecutionTime": 516,
            "minStageExecutionTime": 516,
            "avgStageExecutionTime": 516,
            "sumExecutionTime": 516,
            "countStages": 1
          },
          "58be59a2-c88d-437e-abc5-6cf5ab804fb6": {
            "maxStageExecutionTime": 1507,
            "minStageExecutionTime": 1507,
            "avgStageExecutionTime": 1507,
            "sumExecutionTime": 1507,
            "countStages": 1
          },
          "627a709e-74ee-456c-a263-056a63a389f0": {
            "maxStageExecutionTime": 1507,
            "minStageExecutionTime": 1,
            "avgStageExecutionTime": 511,
            "sumExecutionTime": 2044,
            "countStages": 4
          },
          "onPublish": {
            "maxStageExecutionTime": 1507,
            "minStageExecutionTime": 1,
            "avgStageExecutionTime": 509,
            "sumExecutionTime": 1528,
            "countStages": 3
          },
          "8f07f20b-3934-4944-8046-583aa49e84ed": {
            "maxStageExecutionTime": 20,
            "minStageExecutionTime": 20,
            "avgStageExecutionTime": 20,
            "sumExecutionTime": 20,
            "countStages": 1
          },
          "onAddUpdate": {
            "maxStageExecutionTime": 516,
            "minStageExecutionTime": 516,
            "avgStageExecutionTime": 516,
            "sumExecutionTime": 516,
            "countStages": 1
          },
          "9da23484-1523-4ab7-8e16-455390866c59": {
            "maxStageExecutionTime": 1,
            "minStageExecutionTime": 1,
            "avgStageExecutionTime": 1,
            "sumExecutionTime": 1,
            "countStages": 1
          }
        }
      }
    }
  }
}

Status: 200

Page tree

Versions Compared

Old Version 2

New Version 3

Key

Ingest Push Update

Jobs performance statistics

Page tree

Page History

Versions Compared

Old Version 2

New Version 3

Key

Ingest Push Update

Jobs performance statistics