Run StageR

...

1. Download and install NodeJS.
2. Download and install MongoDB.
3. Download StageR from here.

4. Configure npm to connect to Search Technologies artifactory repository for local dependencies:

   1. Open cmd as Administrator 

      Code Block
      language text theme FadeToGrey
      npm install npmrc -g

      
      

      Code Block
      language text theme FadeToGrey
      npmrc -c local

      
      

      Code Block
      language text theme FadeToGrey
      npm config set registry https://repository.searchtechnologies.com/artifactory/api/npm/npm-virtual

      


   2. Get Authentication token (using curl or Postman)

      Code Block
      language text theme FadeToGrey
      curl -u<USERNAME>:<PASSWORD> https://repository.searchtechnologies.com/artifactory/api/npm/auth

      


   3. Append response to: C:\Users\<USERNAME>\.npmrcs\local

   4. Save file

5. Install dependencies:

   Code Block
   language text theme FadeToGrey
   npm install --production

   
   

6. Start MongoDB.

   Code Block
   language text theme FadeToGrey
   mongod

   
   

7. Launch StageR.

   Code Block
   language text theme FadeToGrey
   node server.js

   
   

Build StageR from source

1. Download and install git.

2. Clone the project from gitlab

   Code Block
   language bash theme FadeToGrey
   git clone https://stgit.searchtechnologies.com/core-research/stager.git

   
   

3. Checkout the development branch

   Code Block
   language bash theme FadeToGrey
   git fetch && git checkout develop

   
   

4. Install dependencies:

   Code Block
   language bash theme FadeToGrey

Storage

1. npm install

   
   

2. Run tests

   Code Block
   language bash theme FadeToGrey
   npm test

   
   

Storage

...

StageR data The Staging Repository data is stored in a No-SQL database. By default, the Staging Repository uses MongoDB as its MongoDB is the default storage option. Custom plugins can be developed for other No-SQL databases.

The Staging Repository will StageR will automatically create the database and all necessary collections (contents, transactions, settings, foreignkeys, and reprocesses) automatically that are used by each Storage Unit.

Environment Configurations

...

The Staging RepositoryStageR can hold the configuration for multiple environments. The config/env folder holds the different configuration files for all environments.  An environment configuration will inherit the configuration present in the config/env/all.js file and can will overwrite any configuration field by redefining it on the environments file. New configuration fields can be added to the environment file.

...

'use strict';'

var path = require('path')
module.exports = {
    app: {
        title: 'Staging RepositoryStageR',
        description: 'Staging Repository',
    keywords: 'Storage Unit, NoSQL, keywords: 'MongoDB, Express, Node.js'
    Replication, Content Processing, Staging'
  },
    port: process.env.PORT || 3000,
    templateEngine: 'swig',
  dbConn:  dbConn{
    type: {'mongo',
    hbase: {
   type   server: 'mongocentos-hadoop3',
      port: 9090
    },
    hbasemongo: {
      namespace: 'stager-',
      server: 'hadoop-thrift-serverlocalhost:27017',
      connectionOptions: {
      port: 9090  /* Uncomment to add secure configuration
        server: {
        }  ssl: true,
        mongo:{  sslValidate: true,
          sslCA:  server:'localhost'fs.readFileSync('./config/sslcerts/mongo/cacert.pem'),
          sslKey: fs.readFileSync('./config/sslcerts/mongo/testuser_key.pem'),
            port: 27017sslCert: fs.readFileSync('./config/sslcerts/mongo/testuser_cert.crt'),
            shardingsslPass: false'123456'
        },
      }  user: 'CN=testuser-aws,OU=CS,O=SearchTech,L=Herndon,ST=VA,C=US',
    keyManager    auth:{
          typeauthMechanism: 'filebased',
  MONGODB-X509'
        }*/
      },
      sharding: false
    }
  keysNumber},
  keyManager: 1000,{
    type: 'filebased',
    filebased: {
      keysNumber: 5,
      masterKeyLocation: 'config/MasterKey.txt',
        }dbName: 'DEK'
    },
    reqBodySizebasic: '1gb',{
     reprocessQueue:{ masterKey: 'MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMzI='
    }
  },
  batchSizereqBodySize: 20'1gb',
  reprocessQueue: {
     querySizebatchSize: 4020,
    querySize: 40,
    timeout: 30005000
  },
  replication: },{
    clusterModecompression: falsetrue
};

Environment specific configuration (secure.js)

In this example, the port and clusterMode parameter will be overridden. authList, logging, keyLocation, certLocation, caLocation, and passphrase will be added to the applications configuration.

'use strict';
module.exports =  },
  logging: {
    express: {
    port: 44300,
    authList :[  request: [
        // 'body', //only add if it's safe to display data
        'Aspireconnection.remoteAddress',
        'testuserauthorizedCN'
      ],
    logging:{
  response: [
        // 'body'  express: {//only add if it's safe to display data
      ]
    },
  request  transports: [{
      console: {
        level: 'connection.remoteAddress'error',
        enable: true
      },
      file: {
        level: 'authorizedCNinfo',
        enable: false

      ]}
    },
    response: {
       responseshowStack: [true
    }
  },
  workers: {
    restapi: ]
 1,
    reprocess: 1,
    replication: 1
  },
  fileHandler: {
    type: 'local',
    transportslocal: {
      destination: path.resolve(process.cwd(), 'uploads')
    }
  },
  tempFolder: path.resolve(process.cwd(), 'tmp'),
  zoneId: 'APP_ZONE'
}

Environment specific configuration (secure.js)

In this example, the port and clusterMode parameter will be overridden. authList, logging, keyLocation, certLocation, caLocation, and passphrase will be added to the applications configuration.

'use strict'

module.exports = {
  port: 44300,
  logging: {
    express: {
  console:{
                level: 'debugrequest: [
        // 'body', //only add if it's safe to display data
        'connection.remoteAddress',
        'authorizedCN'
      ],
  enable    response: false[
        // 'body' //only  },add if it's safe to display data
      ]
    },
    filetransports: {
      console: {
         level: 'info',
        enable: true
      },
      file: {
        level: 'info',
        enable: true

      }
    },
    response: {
      showStack: false
    }
  },
  clusterMode: false,
  secure: }{
    }adminUser: 'TestUser',
    keyLocation: 'config/sslcerts/sr_server/server_.key.pem',
    certLocation: 'config/sslcerts/server/sr_server_cert.crt',
    caLocation: 'config/sslcerts/cacertca/ca.crt',
    passphrase: '123456abC123456',
    clusterModerequestCert: true,
    rejectUnauthorized: false
};  }
}

To start the Staging Repository StageR with a specific environment configuration:

...