...
Each database holds several collections for its crawl usage:
audit
Holds all the actions of each of the items being processed
This is an example of a document in this collection:
Code Block | ||||
---|---|---|---|---|
| ||||
{
"_id" : ObjectId("571f94498cd956261c112156"),
"id" : "C:\\dev-temp\\testdata\\A\\0\\0\\0\\3.txt",
"crawlStart" : NumberLong(1461687363561),
"url" : "file://C:/dev-temp/testdata/A/0/0/0/3.txt",
"type" : "job",
"action" : "ADD",
"batch" : null,
"ts" : NumberLong(1461687366086)
} |
errors
Holds all errors that happened during the crawls.
This is an example of a document in this collection:
Code Block | ||||
---|---|---|---|---|
| ||||
{
"_id" : ObjectId("571f940e8cd956261c112151"),
"error" : {
"@time" : NumberLong(1461687310975),
"@crawlTime" : NumberLong(1461686819675),
"@cs" : "File_System_Source",
"@processor" : "File_System_Source-10.10.20.203:50505",
"@type" : "S",
"_$" : "Error starting crawl\ncom.searchtechnologies.aspire.services.AspireException: Bad 'exclude' regex pattern: C:\\dev-temp\\testdata\\A ..."
}
} |
hierarchy
Holds the parent hierarchy for all the container items. This is used to generate the item hierarchy in the Populate & Fetch stage.
This is an example of a document in this collection:
Code Block | ||||
---|---|---|---|---|
| ||||
{
"_id" : "C:\\dev-temp\\testdata\\B\\5\\9\\3",
"name" : "3",
"ancestors" : {
"_id" : "C:\\dev-temp\\testdata\\B\\5\\9",
"name" : "9",
"ancestors" : {
"_id" : "C:\\dev-temp\\testdata\\B\\5",
"name" : "5",
"ancestors" : {
"_id" : "C:\\dev-temp\\testdata\\B",
"name" : "B",
"ancestors" : {
"_id" : "C:\\dev-temp\\testdata",
"name" : "testdata",
"ancestors" : null
}
}
}
}
} |