You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Search API configuration can be found in config/config.py, as the extension implies it is a python file, making it configurable, meaning you can use libraries, constants, calculate keys, dates on the fly, and define environment variables.


This configuration is accessible anywhere in the code doing an import

from config import SERVER_CONFIG

Configuration Sections

Global

Starting with a reminder, the configuration file is a .py file, which means you can code in it, making a more dynamic configuration, and that’s what we did.

Starting top to bottom:

  1. Imports for the utility function used on the configuration creation
  2. Environment variables, just to show how the loading of Docker variables can be done.
  3. Declaration of the SERVER_PATH to be used in relative paths, later on


Server Address and Workers

Over here we start the actual configuration, and addressing the elephant in the room… Yes the configuration is in a variable CONFIG, which is a dictionary.

  • Host
    • Listening address of the server, by default 0.0.0.0, meaning listening to all request
  • Port
    • Listening port, by default 8085
  • Workers:
    • Number of webapp instances to start up in the server
    • Applicable only when using the provided uvicorn server
    • If using default server, the workers should be set according to the project needs


CORS

CORS (Cross-Origin Resource Sharing), controls the access to reasources for external request.

  • Allow Origins:
    • List of domains allow to make request to the server
  • Allow Credentials:
    • Allow credential header to be on the request
  • Allow Methods
  • Alllow Headers
  • Expose Headers
    • indicates which headers should be made available to scripts running in the browser


* means all


Logging

Logging options for the entire server, from the message format, date format to handlers

  • Message Formart (msgFormat)
  • Date Format
  • Level
    • Default message level, to report to the handlers
  • Handlers: Logs destinations
    • File
    • Console
    • NonSQL
      • Non-SQL starts recording after the connection to the engine was made
  • Loggers
    • Set the log level for specific loggers


Engines

List of search engine connections to handle, all the connections can be access via the connection manager, all across the code

From the properties, the ones to highlight are:

  • Name
    • Reference name, you gave to this engine connection, DEFAULT_ENGINE_NAME = Elastic
  • Type
    • Type of search engine, currently Opensearch or Elasticsearch (DEAFAULT_ENGINE_TYPE)
  • Default
    • The connection to use for internal transactions, like feedback, analytics, …, no default is define, the first engine in the list is the default
  • Engine URL
    • URL or list URLs to search engines



Security

In security section we found authentication, encryption, and roles (currently under development)

  • Authentication:
    • Enabled
    • Type
      • Indicates the type of authentication to use, same key field as the authentication type config
    • Secret
      • To be used in the JWT
    • Anonymous
      • Information regarding the anonymous user
    • Authentication Type Configuration
      • Local
      • Delegated
      • LDAP
      • SAML
    • Encryption
      • Secret Key

        • Secret for all the encryption and decryption done in the server
    • Roles
      • File
        • CSV with the roles and their respective value (currently under development)


Mailer

Search API also allows to configure a mail service, to sent emails based on templates extension .tlp

Currently supports sending emails via Gmail or a custom SMTP


Analytics

If enable, it will accept messages via web socket with the activity done in the UI (if one is used)

It will also record activity performed in the server, where it has been recorded

Each message is sent to the default engine connection, to an analytics index to be later process


Current Default Configuration Sample

The current default configuration can be found here in case the sample below is outdated config.py 

# ==============================================================================
#  Copyright ©2023 Accenture and/or its affiliates. All Rights Reserved.
#
#  Permission to any use, copy, modify, and distribute this software and its
#  documentation for any purpose is subject to a licensing agreement duly
#  entered into with the copyright owner or its affiliate.
#
#  All information contained herein is, and remains the property of Accenture
#  and/or its affiliates and its suppliers, if any.  The intellectual and
#  technical concepts contained herein are proprietary to Accenture and/or
#  its affiliates and its suppliers and may be covered by one or more patents
#  or pending patent applications in one or more jurisdictions worldwide, and
#  are protected by trade secret or copyright law. Dissemination of this
#  information or reproduction of this material is strictly forbidden unless
#  prior written permission is obtained from Accenture and/or its affiliates.
# ==============================================================================

#
#
import os
from os.path import basename, join, abspath

from models.engines import Authentications
from models.security import AuthenticationType
from utils.constants import DEFAULT_ENGINE_NAME, DEFAULT_ENGINE_TYPE
from utils.str import DEFAULT_ENCODING

os.environ['PORT'] = os.getenv('PORT', default='8085')
os.environ['HOST'] = os.getenv('HOST', default='0.0.0.0')
os.environ['DOMAIN_NAME'] = os.getenv('DOMAIN_NAME', default='0.0.0.0')
os.environ['OPENAI_API_KEY'] = os.getenv('OPENAI_API_KEY',default='placeholder')
os.environ['OPENAI_API_KEY_AZURE'] = os.getenv('OPENAI_API_KEY_AZURE',default='placeholder')
os.environ['OPENAI_API_BASE'] = os.getenv('OPENAI_API_BASE', default='https://api.openai.com/v1')
os.environ['OPENAI_API_BASE_AZURE'] = os.getenv('OPENAI_API_BASE_AZURE', default='https://sca-cio-acom-ai-search-south-central-us.openai.azure.com')
os.environ['JOBS_API_BASE'] = os.getenv('JOBS_API_BASE', default='https://acn-api.accenture.com/api/job/findjobs')
os.environ['JOBS_TENANT_ID'] = os.getenv('JOBS_TENANT_ID', default='e0793d39-0939-496d-b129-198edd916feb')
os.environ['JOBS_GRANT_TYPE'] = os.getenv('JOBS_GRANT_TYPE', default='client_credentials')
os.environ['JOBS_CLIENT_ID'] = os.getenv('JOBS_CLIENT_ID',default='placeholder')
os.environ['JOBS_CLIENT_SECRET'] = os.getenv('JOBS_CLIENT_SECRET',default='placeholder')
#https://search-a-com-ai-search-57ogxr2vij4gsypl3ealsjbhae.us-east-2.es.amazonaws.com:443
os.environ['ENGINE_URL'] = os.getenv('ENGINE_URL', default='https://search-a-com-ai-search-57ogxr2vij4gsypl3ealsjbhae.us-east-2.es.amazonaws.com:443')
os.environ['SAGA_URL'] = os.getenv('SAGA_URL', default='http://localhost:8080')
os.environ['EMBEDDINGS_MODEL'] = os.getenv('EMBEDDINGS_MODEL', default='text-embedding-ada-002')
os.environ['DEFAULT_ARTICLE_INDEX'] = os.getenv('DEFAULT_ARTICLE_INDEX', default='article_text_embedding_ada_002')
os.environ['IS_OPENSEARCH'] = os.getenv('IS_OPENSEARCH', default="True")

# *******************************************************************************
# AWS Elasticsearch Credentials
# *******************************************************************************

# Domain. If service is set then the AWS will be used
# os.environ['AWS_SERVICE'] = os.getenv('AWS_SERVICE', default='es')
# os.environ['AWS_REGION'] = os.getenv('AWS_REGION', default='us-east-1')

# ------------------------------------------------------
# Uncomment only if using Access Key and Session Token
# ------------------------------------------------------

# os.environ['AWS_ACCESS_KEY_ID'] = os.getenv('AWS_ACCESS_KEY_ID', default='default-key')
# os.environ['AWS_SECRET_ACCESS_KEY'] = os.getenv('AWS_SECRET_ACCESS_KEY', default='default-secret')
# os.environ['AWS_SESSION_TOKEN'] = os.getenv('AWS_SESSION_TOKEN', default='default-token')

SERVER_PATH = abspath(join(__file__[:-len(basename(__file__))], '', '..'))

CONFIG = {
    'host': os.getenv('HOST'),
    'port': os.getenv('PORT'),
    'workers': 1,
    'cors': {
        'allow_origins': [
            'http://localhost:8085',
            'http://localhost:3000',
            'http://127.0.0.1:3000',
            'http://44.197.233.154:3000',
            'http://127.0.0.1:8080',
            'http://localhost:8080',
            'http://10.0.0.8:8085',
            'http://10.0.0.8:3000',
            'https://login.microsoftonline.com'
            f'{os.getenv("HOST")}:3000',
            f'{os.getenv("HOST")}:{os.getenv("PORT")}'
        ],
        'allow_credentials': True,
        'allow_methods': ['*'],
        'allow_headers': ['*'],
        'expose_headers': ['*'],
        'max_age': 600
    },

    'web_app_config': {
        'title': '',
        'description': '',
        'default_lang': 'en',
        'available_langs': ['en'],
        'default_webview': 'config',
        'web_views': ['config'],
    },

    'logging': {
        'msgFormat': '%(asctime)s\t%(levelname)s\t%(name)s\t%(message)s',
        'dateFormat': '%Y-%m-%dT%H:%M:%S%z',
        'level': 'INFO',
        'handlers': {
            'file': {
                'enable': True,
                'encoding': DEFAULT_ENCODING,
                'backupCount': 5,
                'maxBytes': 5242880,  # 5Mb
            },
            'console': {
                'enable': True
            },
            'nonSQL': {
                'enable': True
            }
        },
        'loggers': {
            'werkzeug': 'info',
            'django.utils.autoreload': 'warning',
            'ldap3': 'info',
            'fastapi': 'notset',
            'passlib.utils.compat': 'info',
            'urllib3.connectionpool': 'info',
            'passlib.registry': 'info',
            'app.rest': 'info',
            'uvicorn.error': 'error'
        }
    },

    # *******************************************************************************
    #  Engines Configuration for ES features
    # *******************************************************************************
    'engines': [
        {
            'name': DEFAULT_ENGINE_NAME,
            'type': DEFAULT_ENGINE_TYPE,
            'default': True,
            'headers': {
                'Accept-Encoding': 'gzip'
            },
            'engine_url': os.getenv('ENGINE_URL').split(),
            'pool_connections': 10,
            'pool_maxsize': 100,
            'pool_block': True,
            'verify': True,
            'max_redirects': 30,
            'max_retries': 10,
            'retry_wait_time': 10,
            'timeout': 60,
            'allow_redirects': True,
            'trust_env': True,
            'use_throttling': True,
            'throttling_rate': 5000,
            'throttling_connection_rate': 50,
            'auth': {
                'type': Authentications.NONE,

                # #### For Basic Auth ####
                # 'username': '',
                # 'password': ''

                # #### For AWS Auth ####
                # 'aws_region': '',
                # 'aws_service': '',
                # 'aws_access_key': '',
                # 'aws_secret_key': ''

                # #### For AWS Auth With Credentials Provider ####
                # 'credentials_provider': True,
                # 'aws_region': '',
                # 'aws_service': ''

            },
            'log_requests': True
        }
    ],

    # *******************************************************************************
    # Security Configuration
    # *******************************************************************************
    'security': {

        'authentication': {
            'enabled': False,
            'type': AuthenticationType.DELEGATED,
            'secret': '52ecfd60e01b800355a8ce59780f9243b4662c3a236394ee',

            'anonymous': {
                'id': 'Anonymous',
                'account': '[email protected]',
                'name': 'Anonymous',
                'displayName': 'Anonymous'
            },
            # *******************************************************************
            # Local Authentication based on a CSV implements FORM and BASIC Auth
            #
            # NOTE: Only recommended for testing
            # *******************************************************************
            'local': {
                'file': join(SERVER_PATH, 'config', 'auth', 'users.csv')
            },

            # **************************************
            # DELEGATED Authentication
            # **************************************

            'delegated': {
                'jwks_url': 'https://login.microsoftonline.com/f3211d0e-125b-42c3-86db-322b19a65a22/discovery/v2.0/keys',
                'audience': 'https://sites-acn-api-dev1.ciodev.accenture.com',
                'attributesMapping': {
                    # key is the property name stored in the SEIA user profile,
                    # the value is the user attribute in LDAP

                    'id': 'email',  # _id is required
                    'account': 'email',  # account is for roles and group expansion
                }            },

            # **************************************
            # LDAP Authentication
            # **************************************
            'ldap': {
                'authentication': 'SIMPLE',
                'url': 'ldap://localhost:10389',
                'bindDN': 'uid=admin,ou=system',  # Bind DN or User
                'bindCredentials': 'secret',  # password
                'searchBase': 'ou=users,ou=system',
                'searchFilter': '(uid=%s)',
                'searchAttributes': ['uid', 'cn', 'sn', 'displayName'],
                'attributesMapping': {
                    # key is the property name stored in the SEIA user profile,
                    # the value is the user attribute in LDAP

                    'id': 'uid',  # _id is required
                    'account': 'uid',  # account is for roles and group expansion
                    'email': 'uid',
                    'firstName': 'cn',
                    'lastName': 'sn',
                    'name': 'cn',
                    'displayName': 'alias',
                    # if the alias is not given, one is created from the first and last name or roles
                    'groups': 'ou',
                    'photo': 'photo'
                }
            },

            # **************************************
            # SAML2 Authentication
            # **************************************
            'saml': {
                'debug': True,
                'clientID': 'f6d3696a-1780-4614-9792-7744b67ab462',
                'serviceUrl': 'https://login.microsoftonline.com/cc4e4bb7-5cce-4b65-80e1-f282b630ca4b/saml2',
                'logoutUrl': 'https://login.microsoftonline.com/cc4e4bb7-5cce-4b65-80e1-f282b630ca4b/saml2',
                # 'callbackUrl': 'http://localhost:8085/es/auth/saml/callback',

                'certPath': 'MIIDBTCCAe2gAwIBAgIQH4FlYNA+UJlF0G3vy9ZrhTANBgkqhkiG9w0BAQsFADAtMSswKQYDVQQDEyJhY2NvdW50cy5hY2Nlc3Njb250cm9sLndpbmRvd3MubmV0MB4XDTIyMDUyMjIwMDI0OVoXDTI3MDUyMjIwMDI0OVowLTErMCkGA1UEAxMiYWNjb3VudHMuYWNjZXNzY29udHJvbC53aW5kb3dzLm5ldDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMBDDCbY/cjEHfEEulZ5ud/CuRjdT6/yN9fy1JffjgmLvvfw6w7zxo1YkCvZDogowX8qqAC/qQXnJ/fl12kvguMWU59WUcPvhhC2m7qNLvlOq90yo+NsRQxD/v0eUaThrIaAveZayolObXroZ+HwTN130dhgdHVTHKczd4ePtDjLwSv/2a/bZEAlPys102zQo8gO8m7W6/NzRfZNyo6U8jsmNkvqrxW2PgKKjIS/UafK9hwY/767K+kV+hnokscY2xMwxQNlSHEim0h72zQRHltioy15M+kBti4ys+V7GC6epL//pPZT0Acv1ewouGZIQDfuo9UtSnKufGi26dMAzSkCAwEAAaMhMB8wHQYDVR0OBBYEFLFr+sjUQ+IdzGh3eaDkzue2qkTZMA0GCSqGSIb3DQEBCwUAA4IBAQCiVN2A6ErzBinGYafC7vFv5u1QD6nbvY32A8KycJwKWy1sa83CbLFbFi92SGkKyPZqMzVyQcF5aaRZpkPGqjhzM+iEfsR2RIf+/noZBlR/esINfBhk4oBruj7SY+kPjYzV03NeY0cfO4JEf6kXpCqRCgp9VDRM44GD8mUV/ooN+XZVFIWs5Gai8FGZX9H8ZSgkIKbxMbVOhisMqNhhp5U3fT7VPsl94rilJ8gKXP/KBbpldrfmOAdVDgUC+MHw3sSXSt+VnorB4DU4mUQLcMriQmbXdQc8d1HUZYZEkcKaSgbygHLtByOJF44XUsBotsTfZ4i/zVjnYcjgUQmwmAWD',
                'attributesMapping': {
                    'id': 'http://schemas.microsoft.com/identity/claims/objectidentifier',
                    'account': 'http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name',
                    'displayName': 'http://schemas.microsoft.com/identity/claims/displayname'
                },
                'cookie': {
                    'path': '/',
                    'samesite': 'lax',
                    'httponly': False,
                    'secure': False
                }
            }
        },

        # *******************************************************************************
        # Encryption
        # *******************************************************************************
        'encryption': {
            'secret_key': join(SERVER_PATH, 'config', 'auth', 'secret_key')
        },

        # *******************************************************************************
        # Roles
        # ******************************************************************************
        'roles': {
            'file': join(SERVER_PATH, 'config', 'auth', 'roles.csv')
        }
    },

    # *******************************************************************************
    # Mailer
    # *******************************************************************************
    'mailer': {
        'enable_endpoint': False,  # Enables an endpoint for direct access through http request
        'mailer_config': {
            # **********************
            # Using Gmail
            # **********************
            'service': 'gmail',
            'auth': {
                'user': '[email protected]',
                'pass': 'test'
            }

            # **********************
            # Using a Custom SMTP
            # **********************
            # 'host': os.getenv('SMTP_RELAY', default='localhost'),
            # 'port': os.getenv('SMTP_PORT', default=22),
            # 'secure': False,
            # 'logger': True,
            # 'debug': True,
            # 'tls': {
            #    'ca':[ Path(os.getenv('CERTIFICATES_PATH')).read_text() ],
            #    'rejectUnauthorized': False,
            # },
        },
        'from': '[email protected]',  # From to display in the email
        'test': True,  # send the emails to the to_test_email, instead to the actual destiny
        'to_test_email': '[email protected]',  # Test destination for all email send
        'default_subject': 'Email Suibject',  # default subject, if none is specified in code
        'data': {
            # This body will be injected as _data, un the actual, body used to map the email
            # templates (e.g {{{_data.url}}})
            'url': 'http://example.com/'
        },

        # **********************
        # Templates
        # **********************
        'plain_template_path': join(SERVER_PATH, 'config', 'templates', 'email_text.tlp'),
        'html_template_path': join(SERVER_PATH, 'config', 'templates', 'email_html.tlp')
    },

    # *****************************************************************************************
    # Analytics Logs all activity in the UI, and activity triggered by the user in the Server
    # *****************************************************************************************
    'analytics': {
        'enable': True
    },

    # *******************************************************************************
    # Chat
    # *******************************************************************************
    'chat_forum': {
        'enabled': False,
        'history_size': 100,  # Amount of messages to store in memory and display
    }
}

  • No labels