Versions Compared

Old Version 4

changes.mady.by.user Julian Ramirez

Saved on

compared with

New Version 5

changes.mady.by.user Julian Ramirez

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Welcome to the Push Updates Listener for Aspire. A central location for all information on the Aspire Push Updates Listener service.

How It Works

The Push Updates Listener is an Aspire Service that allows external systems to push update requests into an Aspire Content Source. Each request will have one or more documents, each with one of the following actions: add, update and delete. The content source will then start a new incremental crawl and only process the documents that were sent on the request.

All requests will be stacked in a separate queue called pushRequestQueue, and each request will be processed sequentially using "first-in, first-out" policy.

Properties

PropertyRequiredDescription
contentSourceYesThe name of the content source where documents will be pushed
pushTypeNo
  • Values: pushUpdates or cleanAndRestart
  • Default value when property is not sent is pushUpdates which will send the documents to the pushRequestQueue to be processed by the content source
  • cleanAndRestart will force release any in progress (status 'P') requests, clean up flags and restart the listener threads.
documentsYesArray with all documents that will be pushed

documents.url

YesUrl or Id of the document
documents.actionYesadd, update or delete
documents.metadataNo

Any extra metadata required by the content source to process the document

documents.shouldScanNoDefault is false when property is not present. This will force scan the document.

Examples:


Code Block
{
  "contentSource": <content-source-name>,
  "documents": [
    {
      "url": "file://C:/testData/file.xml",
      "action": "add",
      "metadata": {
          "displayUrl" : "C:\\testData\\file.xml",
          "fetchUrl" : "file:///C:/testData/file.xml"
      }
    },
     {
      "url": "file://C:/testData/subfolder/blog.docx",
      "action": "update",
      "metadata": {
          "displayUrl" : "C:\\testData\\subfolder\\blog.docx",
          "fetchUrl" : "file:///C:/testData/subfolder/blog.docx"
      }
    }
  ]
}
Code Block
{
  "contentSource": <content-source-name>,
  "pushType": "cleanAndRestart"
}

How to configure 

NamePropertyDefaultDescription
Servlet NameservletNameupdatesListener

The name of the http servlet that will accept push update requests. The value will appear as a servlet under the root of the server on which Aspire is running. For example, for "updatesListener" your updates listener requests will be handled at http://<server>:<port>/updatesListener.

Wait for RequestswaitRequestsTime5m

Time to wait for new requests to be pushed after the ones in progress are done. If no new requests are sent and time expires, the crawl will be stopped.

Update SnapshotpushUpdateSnapshotfalse

If enabled, the incoming updates will update the snapshot of the content source. This property only works with Aspire Snapshots. Any connector with custom incremental crawls/snapshots, will ignore this property.

Delete Complete RequestsdeleteCompleteRequestsfalse

If enabled, all requests marked as complete in the queue will be removed on the following crawl start.

Secure servletAPIKeyEnabledfalseUse an API Key to secure communication with the servlet.

FAQ & Troubleshooting 

  • Multinode

    • Supported Content Source Types

    FAQ & Troubleshooting

    Does it work on an Aspire cluster?

    Yes. In this case the node that gets the first request will be in charge of processing all requests sent during the crawl. If that node crashes during the crawl, remaining requests will be processed by any of the other nodes. Document processing is done in parallel by all nodes.

    What happens if the content source is disabled?

    Any requests sent to the content source while being disabled will get an error response.

    Disabled content source