Page History

Base Configuration (Without Authentication)

Parameters

• Parameter
  summary The name of the provider, used in the pipeline configuration when required by stages. default saga-provider name name required true

• Parameter
  summary The provider type. default OpenSearch name type required true

   See Resources for more information.

• Parameter
  summary Name for the prefix for each index used in the OpenSearch server. default saga name indexName

• Parameter
  summary List of OpenSearch hosts and ports (including schema) default http://localhost:9200 name nodeUrls

• Parameter
  summary Disables SSL certificate validation when connecting to ElasticSearchOpenSearch. (ONLY USE IT FOR DEVELOPMENT PURPOSES) default false name trustAllSSL

• Parameter
  summary Authentication to use, it could be basic, aws or azure. default none name authentication

   

  • Tip
    If none, it can be omitted

• Parameter
  summary Field in indices used as timestamp, automatically included when fetching data. name timestamp

• Parameter
  summary Time in seconds, of how much to wait between retries default 5 name delay

• Parameter
  summary In case of a connection issue, how many time it will try before throwing an error default 3 name retries

• Parameter
  summary Fields in indices to include when fetching data. name include

• Parameter
  summary Fields in indices to exclude when fetching data. name exclude

• Parameter
  summary Maximum amount of results to return per request. By default is -1, which means the engine's default will be not overwritten default -1 name maxResults type integer

• Parameter
  summary Indicates, if the engine must track always the real total of results available, otherwise it will return an estimated beyond certain point (e.g. gt 10000)) default true name track_total_hits type boolean

"providers": [
	{
		"name": "saga-provider",
  		"type": "OpenSearch",
		"nodeUrls": ["http://localhost:9200"],
		"timestamp": "updatedAt",
		"indexName": "saga",
		"trustAllSSL": false,
	    "timeout": 90,
		"delay": 5,
		"retries": 3,
		"include": [],
		"exclude": [],
		"track_total_hits": true,
		"maxResults": 10000
	}
]

Step-by-step guide (Basic Authentication)

To connect to an Elasticsearch with basic authentication you will need to encrypt your password and add the right values to the configuration.

Follow the next steps:

1. Encrypt the password.
   1. Locate on <saga>/bin the file saga-secure-<version>.jar

   2. In a terminal run, in Saga's root folder.

      Code Block
      language text title Console / Terminal / Command
      java -jar bin/saga-secure-<version>.jar -ep=<password> -config="config/config.json"

   3. Keep the generated pwd.txt file at hand, you'll need to reference it in the configuration.
2. Update the configuration in <saga>/config/config.json.

   1. Update "providers" using authentication factor as "basic".

      Code Block
      "providers": [ { "name": "saga-provider", "type": "OpenSearch", "nodeUrls": ["http://localhost:9200"], "timestamp": "updatedAt", "indexName": "saga", "encryptionKeyFile" : "./bin/saga.ek", "trustAllSSL": false,   "authentication": "basic", "user": "<username>", "password": "<path_to_pwd_file>", "timeout": 90, "delay": 5, "retries": 3, "exclude": [ "updatedAt", "createdAt" ] } ]

      Note
      Notice the values of "user" and "password" and "encryptionKeyFile"

Step-by-step guide (AWS)

To connect to an Elasticsearch with AWS authentication you will need to set your Amazon Web Services credentials locally as environmental variables or get them from the ECS or EC2 credentials.

Follow the next steps:

1. Set your AWS credentials.
   1. Set your credentials using the AWS CLI. You can see how to do that here.
   2. Or, you can load credentials from you ECS or EC2 instance. With IAM roles for Amazon ECS Tasks, you can specify an IAM role that can be used by the containers in a task to access AWS resources.

      
      

      Info
      title Note
      SagaElasticIndexer gets the credentials automatically by getting the credentials file. That is why you only need to specify region and service in the config file which is below this note.

2. Update the configuration in <saga>/config/config.json.

   1. Update "providers" using authentication factor as "aws".

   2. Then we have 2 options with Credential Chain Provider, or with Access & Secret Keys

      1. With Credential Chain Provider

         Code Block

         "providers": [ { "name": "saga-provider", "type": "OpenSearch", "nodeUrls": ["http://localhost:9200"], "timestamp": "updatedAt", "indexName": "saga", "encryptionKeyFile" : "./bin/saga.ek", "trustAllSSL": false,   "authentication": "aws", "awsRegion": "<region_where_the_aws_service_is_located>", "awsService": "<aws_service>", "useCredentialsProviderChain": true, "timeout": 90, "delay": 5, "retries": 3, "exclude": [ "updatedAt", "createdAt" ] } ]

         Note
         Notice the values of "awsService", "awsRegion" and "useCredentialsProviderChain"

      2. With Access & Secret Keys

         Code Block

         "providers": [ { "name": "saga-provider", "type": "OpenSearch", "nodeUrls": ["http://localhost:9200"], "timestamp": "updatedAt", "indexName": "saga", "encryptionKeyFile" : "./bin/saga.ek", "trustAllSSL": false,   "authentication": "aws", "awsRegion": "<region_where_the_aws_service_is_located>", "awsService": "<aws_service>", "awsAccessKey": "<service_access_key>", "awsSecretKey": "<service_secret_key>", "useCredentialsProviderChain": false, "timeout": 90, "delay": 5, "retries": 3, "exclude": [ "updatedAt", "createdAt" ] } ]

         Note
         Notice that now we also have the values of "awsAccessKey" and "awsSecretKey". You can ommit "useCredentialsProviderChain" if you want to

In case of Aspire Saga Parse (or any component using Saga)

If you are using Saga within Aspire, the configurations are the same for providers, but you will need to:

1. Copy Saga's encryption key file to <aspire>/bin/

2. Copy pwd.txt to <aspire>/bin/ as well.

3. Update the Saga's config file (the one within the Aspire configuration folder) to reflect the relative path of those files:

   Code Block

   { "config": { "security": { "encryptionKeyFile": "./bin/saga.ek" }, "libraryJars": [ "./lib" ], ... "providers": [ { "name": "filesystem-provider", "type": "FileSystem", "baseDir": "./config" }, { "name": "saga-provider", "type": "OpenSearch", "nodeUrls": ["http://localhost:9200"], "timestamp": "updatedAt", "indexName": "saga", "encryptionKeyFile" : "./bin/saga.ek", "trustAllSSL": false, "authentication": "basic", "user": "<username>", "password": "<path_to_pwd_file>", "timeout": 90, "delay": 5, "retries": 3, "maxResults": 2000000 "exclude": [ "updatedAt", "createdAt" ] } ] }

Connect to OpenSearch via HTTPS

To connect SAGA to a OpenSearch provider with HTTPS enabled you need to follow these steps:

1. Get/Extract the OpenSearch default certificate file (.crt or .pem file).
   1. You can set up custom certificate but that is on the OpenSearch side, here is a good start to do that.
   2. If you are using docker you can do 2 things:
      1. Extract the cert file via Docker Copy command.
      2. Use volumes to have a common place on the host to contain the certificate file and pass them easily to the contianers.
2. (OPTIONAL DOCKER/K8) Pass the certificate file to SAGA container/pod.
   1. You can move or copy the certificate file to any path on the container, but for consistency, you can use the SAGA config folder or the bin folder path to store the new certificate.
3. Import the certificate into the Java Keystore.

   1. Go to the folder where you copied the cert file and run this command (you may need to use sudo if not running as ROOT user):

      Code Block
      language bash
      sudo keytool -importcert -alias elastic_ca -cacerts -storepass changeit -file <CERT_FILE> -noprompt

4. Modify the SAGA config file to use HTTPS instead of HTTP for the nodeUrls property.

   1. Code Block
      language js
      { "name": "saga-provider", "type": "OpenSearch", "nodeUrls": ["https://localhost:9200"], <============= NOTICE the HTTPS protocol "timestamp": "updatedAt", "indexName": "saga", "encryptionKeyFile" : "./bin/saga.ek", "authentication": "none", "timeout": 90, "delay": 5, "retries": 3, "exclude": [ "updatedAt", "createdAt" ] }

5. Run SAGA normally.

Related articles