There are 3 different ways to connect with authentication: with Basic Authentication and Amazon Web Services (AWS).

Or, if you don't want any authentication, you can set that as well.

Base Configuration (Without Authentication)

Parameters

  • name ( type=string | default=saga-provider | required ) - The name of the provider, used in the pipeline configuration when required by stages.
  • type ( type=string | default=OpenSearch | required ) - The provider type. See Resources for more information.
  • indexName ( type=string | default=saga | optional ) - Name for the prefix for each index used in the OpenSearch server.
  • nodeUrls ( type=string | default=http://localhost:9200 | optional ) - List of OpenSearch hosts and ports (including schema)
  • trustAllSSL ( type=string | default=false | optional ) - Disables SSL certificate validation when connecting to OpenSearch. (ONLY USE IT FOR DEVELOPMENT PURPOSES)
  • authentication ( type=string | default=none | optional ) - Authentication to use, it could be basic, aws or azure.  

    • If none, it can be omitted 

  • timestamp ( type=string | optional ) - Field in indices used as timestamp, automatically included when fetching data.
  • delay ( type=string | default=5 | optional ) - Time in seconds, of how much to wait between retries
  • retries ( type=string | default=3 | optional ) - In case of a connection issue, how many time it will try before throwing an error
  • include ( type=string | optional ) - Fields in indices to include when fetching data.
  • exclude ( type=string | optional ) - Fields in indices to exclude when fetching data.
  • maxResults ( type=integer | default=-1 | optional ) - Maximum amount of results to return per request. By default is -1, which means the engine's default will be not overwritten
  • track_total_hits ( type=boolean | default=true | optional ) - Indicates, if the engine must track always the real total of results available, otherwise it will return an estimated beyond certain point (e.g. gt 10000))


"providers": [
	{
		"name": "saga-provider",
  		"type": "OpenSearch",
		"nodeUrls": ["http://localhost:9200"],
		"timestamp": "updatedAt",
		"indexName": "saga",
		"trustAllSSL": false,
	    "timeout": 90,
		"delay": 5,
		"retries": 3,
		"include": [],
		"exclude": [],
		"track_total_hits": true,
		"maxResults": 10000
	}
]



Step-by-step guide (Basic Authentication)

To connect to an Elasticsearch with basic authentication you will need to encrypt your password and add the right values to the configuration.

Follow the next steps:

  1. Encrypt the password.
    1. Locate on <saga>/bin the file saga-secure-<version>.jar

    2. In a terminal run, in Saga's root folder.

      Console / Terminal / Command
      java -jar bin/saga-secure-<version>.jar -ep=<password> -config="config/config.json"
    3. Keep the generated pwd.txt file at hand, you'll need to reference it in the configuration.
  2. Update the configuration in <saga>/config/config.json.

    1. Update "providers" using authentication factor as "basic".

      "providers": [
	{
		"name": "saga-provider",
  		"type": "OpenSearch",
		"nodeUrls": ["http://localhost:9200"],
		"timestamp": "updatedAt",
		"indexName": "saga",
		"encryptionKeyFile" : "./bin/saga.ek",
		"trustAllSSL": false,
	    "authentication": "basic",
		"user": "<username>",
		"password": "<path_to_pwd_file>",
		"timeout": 90,
		"delay": 5,
		"retries": 3,
		"exclude": [
			"updatedAt",
			"createdAt"
		]
	}
]

      Notice the values of "user" and "password" and "encryptionKeyFile"

Step-by-step guide (AWS)

To connect to an Elasticsearch with AWS authentication you will need to set your Amazon Web Services credentials locally as environmental variables or get them from the ECS or EC2 credentials.

Follow the next steps:

  1. Set your AWS credentials.
    1. Set your credentials using the AWS CLI. You can see how to do that here.
    2. Or, you can load credentials from you ECS or EC2 instance. With IAM roles for Amazon ECS Tasks, you can specify an IAM role that can be used by the containers in a task to access AWS resources.


      Note

      SagaElasticIndexer gets the credentials automatically by getting the credentials file. That is why you only need to specify region and service in the config file which is below this note.

  2. Update the configuration in <saga>/config/config.json.

    1. Update "providers" using authentication factor as "aws".

    2. Then we have 2 options with Credential Chain Provider, or with Access & Secret Keys

      1. With Credential Chain Provider

        "providers": [
	{
		"name": "saga-provider",
		"type": "OpenSearch",
		"nodeUrls": ["http://localhost:9200"],
		"timestamp": "updatedAt",
		"indexName": "saga",
		"encryptionKeyFile" : "./bin/saga.ek",
		"trustAllSSL": false,
	    "authentication": "aws",
		"awsRegion": "<region_where_the_aws_service_is_located>",
		"awsService": "<aws_service>",
		"useCredentialsProviderChain": true,
		"timeout": 90,
		"delay": 5,
		"retries": 3,
		"exclude": [
			"updatedAt",
			"createdAt"
		]
	}
]

        Notice the values of "awsService", "awsRegion" and "useCredentialsProviderChain"

      2. With Access & Secret Keys

        "providers": [
	{
		"name": "saga-provider",
		"type": "OpenSearch",
		"nodeUrls": ["http://localhost:9200"],
		"timestamp": "updatedAt",
		"indexName": "saga",
		"encryptionKeyFile" : "./bin/saga.ek",
		"trustAllSSL": false,
	    "authentication": "aws",
		"awsRegion": "<region_where_the_aws_service_is_located>",
		"awsService": "<aws_service>",
		"awsAccessKey": "<service_access_key>",
  		"awsSecretKey": "<service_secret_key>",
		"useCredentialsProviderChain": false,
		"timeout": 90,
		"delay": 5,
		"retries": 3,
		"exclude": [
			"updatedAt",
			"createdAt"
		]
	}
]

        Notice that now we also have the values of "awsAccessKey" and "awsSecretKey". You can ommit "useCredentialsProviderChain" if you want to

In case of Aspire Saga Parse (or any component using Saga)

If you are using Saga within Aspire, the configurations are the same for providers, but you will need to:

  1. Copy Saga's encryption key file to <aspire>/bin/

  2. Copy pwd.txt to <aspire>/bin/ as well.

  3. Update the Saga's config file (the one within the Aspire configuration folder) to reflect the relative path of those files:

    {
	"config": {
    	"security": {
			"encryptionKeyFile": "./bin/saga.ek"
        },
    	"libraryJars": [
            "./lib"
        ],
		...
	"providers": [
     	{
			"name": "filesystem-provider",
			"type": "FileSystem",
			"baseDir": "./config"
		},
 		{
			"name": "saga-provider",
  			"type": "OpenSearch",
			"nodeUrls": ["http://localhost:9200"],
			"timestamp": "updatedAt",
			"indexName": "saga",
			"encryptionKeyFile" : "./bin/saga.ek",
		 	"trustAllSSL": false,
			"authentication": "basic",
			"user": "<username>",
			"password": "<path_to_pwd_file>",
			"timeout": 90,
			"delay": 5,
			"retries": 3,
 			"maxResults": 2000000 
			"exclude": [
				"updatedAt",
				"createdAt"
			]
		} 

	]
}