The create storage unit GET/PUT/POST request requires the storage unit name.
PUT admin/create/<storage-unit-name> |
If the storage unit was created successfully, a 200 response code with an OK message is returned.
{"message": "OK"} |
If a storage unit with the given name already exists, a 400 response code with a STORAGE_UNIT_EXISTS message is returned.
{"message": "STORAGE_UNIT_EXISTS"} |
Deletes a storage unit from the staging repository server. All content and transactions for the storage unit are deleted.
The drop storage unit GET/DELETE/POST request requires the name of the storage unit to delete.
DELETE admin/drop/<storage-unit-name> |
If the storage unit was deleted successfully, a 200 response code with an OK message is returned.
{"message": "OK"} |
If the storage unit doesn't exist, a 400 response code with a STORAGE_UNIT_DOESNT_EXIST message is returned.
{"message": "STORAGE_UNIT_DOESNT_EXIST"} |
Lists all available storage units with the option of returning content and transaction statistics.
The list storage units GET/POST request can receive a withStats parameter to return content and transaction statistics for each storage unit.
GET admin/list?withStats |
A 200 response code with the list of storage unit names is returned.
[ {"name": "Management"}, {"name": "Management10"}, {"name": "Management4"}, {"name": "Management5"}, {"name": "testStorageUnit"} ] |
A 200 response code with the list of storage unit names and statistics is return when using the withStats parameter.
[ { "name": "Sales", "records": 0, "transactions": 0 }, { "name": "Products", "records": 32, "transactions": 32, "latestTransactionTime": "2015-11-12T20:13:13.737Z", "latestTransactionNum": "5644f2d997cc11144081d14f" } ] |
List all available scopes for a given Storage Unit.
The list scopes GET request receives the Storage Unit name
GET admin/listScopes/<storage-unit-name> |
A 200 response code with the list of scopes for the Storage Unit is returned.
[ "connector", "index" ] |
Gets the statistics for a storage unit.
The get storage unit statistics GET/POST request requires the name of the storage unit.
GET admin/getStatistics/<storage-unit-name> |
If the storage unit exists, a 200 response code and a JSON with the storage unit statistics is returned.
{ "name": "Management5", "records": 32, "transactions": 32, "latestTransactionTime": "2015-11-12T20:13:13.737Z", "latestTransactionNum": "5644f2d997cc11144081d14f" } |
If the storage unit doesn't exist, a 400 response code with a STORAGE_UNIT_DOESNT_EXIST message is returned.
{"message": "STORAGE_UNIT_DOESNT_EXIST"} |
Dumps all transactions and content records of a storage unit. Dump is a two step process: create the dump file and download the dump file.
The create dump GET request requires the name of the storage unit to dump.
GET admin/createDump/<storage-unit-name> |
If the storage unit exists, a 200 response code and a fileId to call the download dump request.
If the storage unit doesn't exist, a 400 response code with a STORAGE_UNIT_DOESNT_EXIST message is returned.
{"message": "STORAGE_UNIT_DOESNT_EXIST"} |
The download dump GET request requires the fileId returned by the create dump request.
GET admin/downloadDump/<file-id> |
If the file id exists, but the dump file is in process of being created, a 200 response code with an IN_PROGRESS message is returned.
{"message": "IN_PROGRESS"} |
If the file id exists and the dump file is ready, a 200 response code and the file are returned.
If the file id doesn't exist, a 400 response code with an INVALID_DUMP_FILE_ID message is returned.
{"message": "INVALID_DUMP_FILE_ID"} |
A storage unit has a set of events that are triggered during different actions performed on the storage unit. There are two different types of events: per document events and general events.
Per document events are triggered for each document when it is added, update, deleted or fetched.
General events are triggered when specific transactions are submitted to the storage unit:
Event actions are configured on external JavaScript modules placed inside the processing_modules folder of the Staging Repository server. Each module can implement one or more of the event functions. The name of the function needs to be the name of the implemented event. A module can have functions of both general and per document events.
Per document events receives five parameters:
/** * Creates a new content object with only url, acls and content fields, if they exist. */ exports.Process = function (key, content, context, settings, callback) { var newContent = {} if (content.url) { newContent.url = content.url } if (content.acls) { newContent.acls = content.acls } if (content.content && content.content.$) { newContent.content = content.content.$ } callback(null, newContent, context) }; |
General events receive three parameters:
exports.BatchStart = function (context, settings, callback) { context.size = 0 context.keysInRequest = [] callback(null, context) } |
The set content processing modules API call defines the content processing pipelines and settings that will be executed per scope when an event is trigger. A default list of modules can be configured for scopes that are not explicitly defined.
Each module can define it's own collection of settings. If a module has a setting with the same name as a global setting, the global setting gets overwritten for that module only.
When executing the events for each module, these will be executed in the order in which they appear in the configuration array.
{ "modules" : { "connector": [ { "module" : "AspireFieldMapping" }, { "settings" : { "elasticsearch-index" : "aspiredocs", "elasticsearch-type" : "aspiredoc" }, "module" : "ESPublisher" } ], "index" : [ { "module" : "FieldMapping" }, { "settings" : { "elasticsearch-index" : "researchdocs", "elasticsearch-type" : "researchdoc" }, "module" : "ESPublisher" } ], "research" : [ { "module" : "NormalizeCategory" } ] }, "settings" : { "elasticsearch-port" : 9200, "elasticsearch-server" : "localhost" } } |
This configuration defines a content processing pipeline for three different scopes of the storage unit: connector (AspireFieldMapping → ESPublisher), index (FieldMapping → ESPublisher) and research (NormalizeCategory).
The storage unit PUT/POST set content processing modules request requires the name of the storage unit in the URL and receives a JSON in the body with the content processing modules configuration.
Warning: This will replace any previous stored configuration for the storage unit.
POST admin/setContentProcessingModules/<storage-unit-name> { "modules" : { "connector": [ { "module" : "FieldMapping" }, { "settings" : { "elasticsearch-index" : "aspiredocs", "elasticsearch-type" : "aspiredoc" }, "module" : "ESPublisher" } ], ... }, "settings" : { "elasticsearch-port" : 9200, "elasticsearch-server" : "localhost" } } |
If the operation is successful, a 200 response code and an OK message are returned.
{"message":"OK"} |
Enables or disables content processing for a storage unit.
The storage unit PUT/GET/POST enable content processing request requires the name of the storage unit and a boolean value to indicate whether to enable (true) or disable (false) content processing. When a storage unit is created, content processing is enabled by default.
PUT admin/enableContentProcessing/<storage-unit-name>/true |
PUT admin/enableContentProcessing/<storage-unit-name>/false |
If the operation is successful, a 200 response code and an OK message are returned.
{"message":"OK"} |
Enables or disables content encryption for a storage unit. When enabled, the JSON documents are encrypted, with the configured key manager, before being stored in the database.
The storage unit PUT/GET/POST enable content encryption request requires the name of the storage unit and a boolean value to indicate whether to enable (true) or disable (false) content encryption. By default encryption is enabled.
Enable
PUT admin/enableContentEncryption/<storage-unit-name>/true |
PUT admin/enableContentEncryption/<storage-unit-name>/false |
If the operation is successful, a 200 response code and an OK message are returned.
{"message":"OK"} |
Enables or disables content compression for a storage unit. When enabled, the JSON documents are compressed before encryption happens for storage.
The storage unit PUT/GET/POST enable content compression request requires the name of the storage unit and a boolean value to indicate whether to enable (true) or disable (false) content compression. By default compression is disabled.
PUT admin/enableContentCompression/<storage-unit-name>/true |
PUT admin/enableContentCompression/<storage-unit-name>/false |
If the operation is successful, a 200 response code and an OK message are returned.
{"message":"OK"} |
Enables or disables the background reprocessing queue of a storage unit. When enabled, automatic and manual reprocess requests will execute the Process document events configured for documents being reprocessed.
The storage unit PUT/GET/POST enable reprocessing queue request requires the name of the storage unit and a boolean value to indicate whether to enable (true) or disable (false) the reprocessing queue execution. By default the reprocessing queue is enabled.
PUT admin/enableReprocessingQueue/<storage-unit-name>/true |
PUT admin/enableReprocessingQueue/<storage-unit-name>/false |
If the operation is successful, a 200 response code and an OK message are returned.
{"message":"OK"} |
Get the current storage unit configuration. This includes the content processing modules (settings field) and the configuration flags for encryption, compression, content processing and reprocessing queue.
The storage unit GET get configuration request requires the name of the storage unit in the URL.
GET admin/getConfiguration/<storage-unit-name> |
If the operation is successful, a 200 response code and the storage unit JSON configuration.
{ "settings":{ "modules" : { "connector": [ { "module" : "FieldMapping" }, { "settings" : { "elasticsearch-index" : "aspiredocs", "elasticsearch-type" : "aspiredoc" }, "module" : "ESPublisher" } ], ... }, "settings" : { "elasticsearch-port" : 9200, "elasticsearch-server" : "localhost" } }, "contentProcessing" : true, "compressContent" : false, "encryptContent": true "reprocessingQueue" : true } |