Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Run

...

STageR


Excerpt
hiddentrue

...

STageR in 20 min

...

STageR runs on top of NodeJS and uses a No-SQL database for storage.

...

It uses MongoDB by default.

On this page:

Table of Contents

Download and Configure


  1. Download and install NodeJS.
  2. Download and install MongoDB.
  3. Download 

...

  1. STageR from here.
  2. Configure npm to connect to Search Technologies artifactory repository for local dependencies:

    1. Open cmd as Administrator 

      Code Block
      languagetext
      themeFadeToGrey
      npm install npmrc -g


      Code Block
      languagetext
      themeFadeToGrey
      npmrc -c local


      Code Block
      languagetext
      themeFadeToGrey
      npm config set registry https://repository.searchtechnologies.com/artifactory/api/npm/npm-virtual


    2. Get Authentication token (

...

    1. using curl

...

    1.  or Postman)

      Code Block
      languagetext
      themeFadeToGrey
      curl -u<USERNAME>:<PASSWORD> https://repository.searchtechnologies.com/artifactory/api/npm/auth


    2. Append response to: C:\Users\<USERNAME>\.npmrcs\local

    3. Save file


  1. Install dependencies

...

  1. .

    • Unzip the file stager-1.0.0.zip (downloaded in previous step)
    • Change to this directory
    • npm install --production

    Code Block
    languagetext
    themeFadeToGrey
    npm install --production


  2. Start MongoDB.

    Code Block
    languagetext
    themeFadeToGrey
    mongod

...


  1. Launch STageR.

    Code Block
    languagetext
    themeFadeToGrey
    node server.js


Build

...

STageR from

...

Source


  1. Download and install git.
  2. Clone the project from gitlab.

    Code Block
    languagebash
    themeFadeToGrey
    git clone https://stgit.searchtechnologies.com/core-research/stager.git

...


  1. Check out the development branch.

    Code Block
    languagebash
    themeFadeToGrey
    git fetch && git checkout develop


  2. Install dependencies

...

  1. .

    Code Block
    languagebash
    themeFadeToGrey
    npm install


  2. Run tests.

    Code Block
    languagebash
    themeFadeToGrey
    npm test


Storage

...


  • STageR data is stored in a No-SQL database. 
  • MongoDB is the default storage option.

...

  • You can develop custom plugins for other No-SQL databases.

...

  • STageR will

...

  • create the database

...

  • automatically, including all necessary collections (contentstransactionssettingsforeignkeys, and reprocesses) that are used by each Storage Unit.


Environment Configurations

...


STageR can hold the configuration for multiple environments. The config/env folder holds the different configuration files for all environments.  An environment configuration will inherit the configuration present in the config/env/all.js file and will overwrite any configuration field by redefining it on the environments file. New configuration fields can be added to the environment file.


All configuration example:

Code Block
languagejs
themeRDark
'use strict'

var path = require('path')
module.exports = {
  app: {
    title: 'StageR',
    description: 'Staging Repository',
    keywords: 'Storage Unit, NoSQL, Replication, Content Processing, Staging'
  },
  port: process.env.PORT || 3000,
  templateEngine: 'swig',
  dbConn: {
    type: 'mongo',
    hbase: {
      server: 'centos-hadoop3',
      port: 9090
    },
    mongo: {
      namespace: 'stager-',
      server: 'localhost:27017',
      connectionOptions: {
        /* Uncomment to add secure configuration
        server: {
          ssl: true,
          sslValidate: true,
          sslCA: fs.readFileSync('./config/sslcerts/mongo/cacert.pem'),
          sslKey: fs.readFileSync('./config/sslcerts/mongo/testuser_key.pem'),
          sslCert: fs.readFileSync('./config/sslcerts/mongo/testuser_cert.crt'),
          sslPass: '123456'
        },
        user: 'CN=testuser-aws,OU=CS,O=SearchTech,L=Herndon,ST=VA,C=US',
        auth:{
          authMechanism: 'MONGODB-X509'
        }*/
      },
      sharding: false
    }
  },
  keyManager: {
    type: 'filebased',
    filebased: {
      keysNumber: 5,
      masterKeyLocation: 'config/MasterKey.txt',
      dbName: 'DEK'
    },
    basic: {
      masterKey: 'MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMzI='
    }
  },
  reqBodySize: '1gb',
  reprocessQueue: {
    batchSize: 20,
    querySize: 40,
    timeout: 5000
  },
  replication: {
    compression: true
  },
  logging: {
    express: {
      request: [
        // 'body', //only add if it's safe to display data
        'connection.remoteAddress',
        'authorizedCN'
      ],
      response: [
        // 'body' //only add if it's safe to display data
      ]
    },
    transports: {
      console: {
        level: 'error',
        enable: true
      },
      file: {
        level: 'info',
        enable: false

      }
    },
    response: {
      showStack: true
    }
  },
  workers: {
    restapi: 1,
    reprocess: 1,
    replication: 1
  },
  fileHandler: {
    type: 'local',
    local: {
      destination: path.resolve(process.cwd(), 'uploads')
    }
  },
  tempFolder: path.resolve(process.cwd(), 'tmp'),
  zoneId: 'APP_ZONE'
}

Environment specific configuration (secure.js)

In this example, the port and clusterMode parameter will be overridden. authList, logging, keyLocation, certLocation, caLocation, and passphrase will be added to the applications configuration.

Code Block
languagejs
themeRDark
'use strict'

module.exports = {
  port: 44300,
  logging: {
    express: {
      request: [
        // 'body', //only add if it's safe to display data
        'connection.remoteAddress',
        'authorizedCN'
      ],
      response: [
        // 'body' //only add if it's safe to display data
      ]
    },
    transports: {
      console: {
        level: 'info',
        enable: true
      },
      file: {
        level: 'info',
        enable: true

      }
    },
    response: {
      showStack: false
    }
  },
  clusterMode: false,
  secure: {
    adminUser: 'TestUser',
    keyLocation: 'config/sslcerts/server/server.key',
    certLocation: 'config/sslcerts/server/server.crt',
    caLocation: 'config/sslcerts/ca/ca.crt',
    passphrase: '123456',
    requestCert: true,
    rejectUnauthorized: false
  }
}

...

Start STageR with a specific environment configuration:

  1. Set the environment configuration name to the NODE_ENV environment variable. 
  2. If the variable is not set, the application will start with the development configuration by default.