...

SSL Configuration

Secure access can be enabled in the Staging Repository to restrict access to the REST APIs through https connections using client certificates to authenticate and authorize the access.

...

CRDP-65: Option to allow all valid client certificates to access by adding '*' to the authList.

Encryption and Key Managers

All content stored in the Staging Repository is encrypted with the NodeJS crypto library. Content is encrypted with aes-256-cbc algorithm using an Initialization Vector (IV) and a Data Encryption Key (DEK) provided by a plugable key manager.

...

To create a custom key manager see.

Content Compression

The Staging Repository provides the option per Storage Unit to compress the content that will be stored in the database. Uses the zlib library from NodeJS.

...

GET admin/enableContentCompression/<storage-unit>/<true-false>

Content Processing

A Storage Unit has a set of events that are triggered during different actions performed while interacting with the Storage Unit. Content processing modules can be configured to be executed on these events. A content processing module is a javascript file that implements one or more events.

...

General events will usually be used to initialize common configuration to be used by document events, this configuration can be set in the context variable and it will be shared with all document events that belong to the same general event, for example all documents that belong to a batch will receive the same context variable that BatchStart returns, and BatchEnd will receive this same context variable with any modifications that could have been done by other events to the data/configuration of the context variable.

Foreign Key Joins

The Staging Repository provides a content processing module ForeignKeyJoin for automatic merging of records from different storage units based on record keys. 1-to-N relations can be specified between storage unit records. If configured,ForeignKeyJoin will run on Process and Fetch events of each document for the specified scope.

...

{
    "modules": {
        "connector": [
            {
                "module": "ForeignKeyJoin"
            },
            ...
        ]
    }
}

With this configuration, foreign key merges will happen for the connector scope on any Process or Fetch event.

Publishers

ElasticSearch Publicher

Content Processing example

Reprocessing Queue and Automatic Updates

Remote Replication