You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

Each Connector installed as a Content Source in Aspire will create a new Database in MongoDB with its name, and multiple collections that it uses for crawling and serving security information. The following describes what every collection is used for as well as an explanation of each of the fields of the documents stored in them.

 

Document Queues and Metadata

  • processQueue

Manages the items that needs to be processed by the workflow, these items may or may not be sent to scanned.

Fields:

Field NameExample

Description

_idC:\test-folder\folderA\testDocument.txtThe unique id of the document
metadata[depends on each connector]The necessary metadata fields the connector needs to fetch or populate this document
type[depends on each connector]The serialized version of the ItemType of the document
statusC, P or A

The document processing status:

C: Completed, means it have been already processed

P: in Progress, means it is currently been processed

A: Available, means it is available for been processed

actionadd, update, deleteThe action to be performed to the search engine for the document
timestamp1465334398471The time-stamp when this document was added to the queue
signatureCBEC1210FE2D51A8166C3E70D38F8A07An MD5 signature, when a document changes this signature should also change
parentIdC:\test-folder\folderAThe id of the parent document, in other words the document that scanned the current document
processorFile_System-192.168.1.15:50505The identifier of the Aspire server that processed or is processing the current document
shouldScanfalseDetermines whether or not this document should be considered for scanning
shouldProcesstrueDetermines whether or not this document should be considered for being processed by the workflow
retries0The number of times this document has been retried
nametestDocument.txtThe name of this document
isCrawlRootItemfalseIndicates if this is one of the root crawl items (for internal control)
hierarchyIdC:\test-folder\folderA\testDocument.txtUnique Id for using to generate the hierarchy for this document, it may be different from the _id field

 

  • scanQueue

  • hierarchy

 

Statistics and Logging

  • audit

  • errors

  • statistics

 

Controlling and Incremental

  • status

  • snapshots

 

 

  • No labels