Starting with a reminder, the configuration file is a .py file, which means you can code in it, making a more dynamic configuration, and that’s what we did.
Imports for the utility function used on the configuration creation
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
from os.import os from os.path import join from models.engines import Authentications, EngineTypes from models.security import AuthenticationType from utils.constants import DEFAULT_ENGINE_NAME, SERVER_PATH from utils.str import DEFAULT_ENCODING import join from models.engines import Authentications, EngineTypes from models.security import AuthenticationType from utils.constants import DEFAULT_ENGINE_NAME, SERVER_PATH from utils.str import DEFAULT_ENCODING |
The configuration file sets several environment variables to configure the application's behavior. These variables are essential for defining network settings and integration parameters.
Name | Default | Description | ||
---|---|---|---|---|
UVICORN_WORKERS | 1 | Number of Uvicorn workers.
| ||
PORT | '8085' | Port number for the application. | ||
HOST | '0.0.0.0' | Host IP address for the application. | ||
DOMAIN_NAME | 'localhost' | Domain name used for the application. | ||
COOKIE_DOMAIN_NAME | '' (empty) | Domain name for setting cookies. | ||
ENGINE_URL | 'http://localhost:9200' | URL for the application engine. |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
os.environ['UVICORN_WORKERS'] = os.getenv('UVICORN_WORKERS', default=1) os.environ['PORT'] = os.getenv('PORT', default='8085') os.environ['HOST'] = os.getenv('HOST', default='0.0.0.0') os.environ['DOMAIN_NAME'] = os.getenv('DOMAIN_NAME', default='localhost') os.environ['COOKIE_DOMAIN_NAME'] = os.getenv('COOKIE_DOMAIN_NAME', default='') os.environ['ENGINE_URL'] = os.getenv('ENGINE_URL', default='http://localhost:9200') |
Commented-out AWS Elasticsearch credentials, to be used if the AWS service is chosen.
Name | Default | Description |
---|---|---|
AWS_SERVICE | 'es' | The AWS service name, defaulting to Elasticsearch. |
AWS_REGION | 'us-east-1' | The AWS region. |
AWS_ACCESS_KEY_ID | 'default-key' | AWS Access Key ID. |
AWS_SECRET_ACCESS_KEY | 'default-secret' | AWS Secret Access Key. |
AWS_SESSION_TOKEN | 'default-token' | AWS Session Token. |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
# os.environ['AWS_SERVICE'] = os.getenv('AWS_SERVICE', default='es') # os.environ['AWS_REGION'] = os.getenv('AWS_REGION', default='us-east-1') # os.environ['AWS_ACCESS_KEY_ID'] = os.getenv('AWS_ACCESS_KEY_ID', default='default-key') # os.environ['AWS_SECRET_ACCESS_KEY'] = os.getenv('AWS_SECRET_ACCESS_KEY', default='default-secret') # os.environ['AWS_SESSION_TOKEN'] = os.getenv('AWS_SESSION_TOKEN', default='default-token') |
CONFIG Variable
The CONFIG variable is a central configuration object that consolidates various settings and options for the application. This structured dictionary contains nested sections, each representing specific aspects of configuration, such as server parameters, security settings, and more. It enhances code readability and flexibility by providing a clear and organized way to customize the script's behavior without extensive code changes.
host
, port
, workers
)General settings include host IP, port number, and number of Uvicorn workers.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
CONFIG = {
'host': os.getenv('HOST'),
'port': os.getenv('PORT'),
'workers': os.getenv('UVICORN_WORKERS'),
'generate_settings_docs': True,
# ... (other configurations)
} |
Name | Type | Default | Description |
---|---|---|---|
host | String | localhost | Host to which the server will be listening |
port | Integer | 8085 | Port in which the server will be listening |
workers | Integer | 1 | Number to workers for the webapp |
generate_settings_docs | Boolean | True | If true, GAIA API will seach and generate html documentation for each data model, accessible via http://localhost:8085/es/docs/schemas/config |
Manages Cross-Origin Resource Sharing settings.
Name | Type | Default | Description |
---|---|---|---|
allow_origins | List | A list of allowed origins (URLs) that can access the server. | |
allow_credentials | Boolean | True | Indicates whether the server allows credentials (e.g., cookies, HTTP authentication) to be included in cross-origin requests. |
allow_methods | List | ['*'] | A list of HTTP methods (e.g., GET, POST) that are allowed for cross-origin requests. Use ['*'] to allow all methods. |
allow_headers | List | ['*'] | A list of HTTP headers that are allowed for cross-origin requests. Use ['*'] to allow all headers. |
expose_headers | List | ['*'] | A list of HTTP headers that are exposed to the response of cross-origin requests. Use ['*'] to expose all headers. |
max_age | Integer | 600 | The maximum time (in seconds) that the results of a preflight request (OPTIONS) can be cached. |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
'cors': { 'allow_origins': [ 'https://login.microsoftonline.com', f'http://{os.getenv("DOMAIN_NAME")}:{os.getenv("PORT")}', f'http://{os.getenv("DOMAIN_NAME")}:3000', f'https://{os.getenv("DOMAIN_NAME")}:{os.getenv("PORT")}', f'https://{os.getenv("DOMAIN_NAME")}:3000' ], 'allow_credentials': True, 'allow_methods': ['*'], 'allow_headers': ['*'], 'expose_headers': ['*'], 'max_age': 600 }, |
Settings related to the web application's interface.
Name | Type | Default | Description |
---|---|---|---|
title | String | '' | Title of the web app. |
description | String | '' | App description. |
default_lang | String | 'en' | Default language. |
available_langs | List | ['en'] | List of available languages. |
default_webview | String | 'config' | Default web view. |
web_views | List | ['config'] | List of web views. |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
'web_app_config': { 'title': '', 'description': '', 'default_lang': 'en', 'available_langs': ['en'], 'default_webview': 'config', 'web_views': ['config'], }, |
Specifies details for logging.
Name | Type | Default | Description | ||
---|---|---|---|---|---|
msgFormat | str | - | Log message format string | ||
dateFormat | str | - | Date format for log messages | ||
level | str | 'INFOSTAT' | Logging level (e.g., 'DEBUG', 'STAT', 'INFO', 'ERROR', 'CRITICAL') | ||
handlers | dict | - | Dictionary of log handlers | ||
handlers.file | dict | - | File log handler configuration | ||
handlers.file.enable | bool | True | Enable file log handler | ||
handlers.file.encoding | str | 'UTF-8' | File encoding for log files | ||
handlers.file.backupCount | int | 5 | Number of log file backups to keep | ||
handlers.file.maxBytes | int | 5242880 (5MB) | Maximum log file size (bytes) | ||
handlers.console | dict | - | Console log handler configuration | ||
handlers.console.enable | bool | True | Enable console log handler | ||
handlers.nonSQL | dict | - | Non-SQL log handler configuration
| ||
handlers.nonSQL.enable | bool | True | Enable non-SQL log handler | ||
loggers | dict | - | Dictionary of logger configurations | ||
loggers.werkzeug | str | 'info' | Werkzeug logger level | ||
loggers.django.utils.autoreload | str | 'warning' | Django autoreload logger level | ||
loggers.ldap3 | str | 'info' | LDAP3 logger level | ||
loggers.fastapi | str | 'notset' | FastAPI logger level | ||
loggers.passlib.utils.compat | str | 'info' | Passlib utils compat logger level | ||
loggers.urllib3.connectionpool | str | 'info' | Urllib3 connection pool logger level | ||
loggers.passlib.registry | str | 'info' | Passlib registry logger level | ||
loggers.app.rest | str | 'info' | App rest logger level | ||
loggers.uvicorn.error | str | 'error' | Uvicorn error logger level |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
'logging': { 'msgFormat': '%(asctime)s\t%(levelname)s\t%(name)s\t%(message)s', 'dateFormat': '%Y-%m-%dT%H:%M:%S%z', 'level': 'INFO', 'handlers': { 'file': { 'enable': True, 'encoding': DEFAULT_ENCODING, 'backupCount': 5, 'maxBytes': 5242880, # 5Mb }, 'console': { 'enable': True }, 'nonSQL': { 'enable': True } }, 'loggers': { 'werkzeug': 'info', 'django.utils.autoreload': 'warning', 'ldap3': 'info', 'fastapi': 'notset', 'passlib.utils.compat': 'info', 'urllib3.connectionpool': 'info', 'passlib.registry': 'info', 'app.rest': 'info', 'uvicorn.error': 'error' } }, |
List of search engine connections to handle, all the connections can be accessed via the connection manager, across the code
Name | Type | Default | Description | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
name | str | Name of the connection
| ||||||||||
type | Enum | Type of the engine (e.g., EngineTypes.ELASTIC)
| ||||||||||
default | bool | Indicates if it's the default engine | ||||||||||
headers | dict | Headers to include in requests to the engine | ||||||||||
engine_url | list | List of engine URLs (may be multiple) | ||||||||||
index_prefix | str | Prefix for Elasticsearch index names | ||||||||||
pool_connections | int | Number of connections in the connection pool | ||||||||||
pool_maxsize | int | Maximum size of the connection pool | ||||||||||
pool_block | bool | Whether to block when the pool is full | ||||||||||
verify | bool | Verify SSL certificates (False to disable) | ||||||||||
max_redirects | int | Maximum number of redirects to follow | ||||||||||
max_retries | int | Maximum number of retries on request failure | ||||||||||
retry_wait_time | int | Time to wait between retries (in seconds) | ||||||||||
timeout | int | Request timeout (in seconds) | ||||||||||
allow_redirects | bool | Allow redirects in requests | ||||||||||
trust_env | bool | Trust environment variables for configuration | ||||||||||
use_throttling | bool | Enable throttling for this engine | ||||||||||
throttling_rate | int | Request rate limit for throttling (requests/sec) | ||||||||||
throttling_connection_rate | int | Connection rate limit for throttling (connections/sec) | ||||||||||
auth | dict | Authentication configuration for this engine | ||||||||||
log_requests | bool | Log HTTP requests (True to enable) |
Info |
---|
You can get a more detail detailed explanation in Engine Framework |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
'engines': [ { 'name': DEFAULT_ENGINE_NAME, 'type': EngineTypes.ELASTIC, 'default': True, 'headers': { 'Accept-Encoding': 'gzip' }, 'engine_url': os.getenv('ENGINE_URL').split(), 'index_prefix': 'gaia', 'pool_connections': 10, 'pool_maxsize': 100, 'pool_block': True, 'verify': False, 'max_redirects': 30, 'max_retries': 10, 'retry_wait_time': 10, 'timeout': 60, 'allow_redirects': True, 'trust_env': True, 'use_throttling': True, 'throttling_rate': 5000, 'throttling_connection_rate': 50, 'auth': { 'type': Authentications.NONE, # With Authentications.BASIC # #### For Basic Auth #### # 'username': os.getenv('ENGINE_USER'), # 'password': os.getenv('ENGINE_PASSWORD') # With Authentications.AWS # #### For AWS Auth #### # 'aws_region': os.getenv('AWS_REGION'), # 'aws_service': os.getenv('AWS_SERVICE'), # 'aws_access_key': os.getenv('AWS_ACCESS_KEY_ID'), # 'aws_secret_key': os.getenv('AWS_SECRET_ACCESS_KEY') # With Authentications.AWS # #### For AWS Auth With Credentials Provider (AWS)#### # 'credentials_provider': True, # 'aws_region': os.getenv('AWS_REGION'), # 'aws_service': os.getenv('AWS_SERVICE') }, 'log_requests': False } ], |
Handles various security aspects of the application.
Name | Type | Default | Description |
---|---|---|---|
authentication.enabled | bool | False | Indicates whether authentication is enabled. |
authentication.type | str | OIDC | The type of authentication mechanism to use (e.g., OIDC, local, delegated, ldap). |
authentication.secret | str | 52ecfd | The secret key used for authentication (e.g., for OIDC). |
authentication. | bool | False | Specifies whether to force HTTPS original URL redirection when using OIDC. |
authentication.anonymous. id | str | Anonymous | The ID for anonymous users. |
authentication.anonymous. account | str | [email protected] | The account name for anonymous users. |
authentication.anonymous. name | str | Anonymous | The name of anonymous users. |
authentication.anonymous. displayName | str | Anonymous | The display name of anonymous users. |
authentication.local | object | 'config/auth/users.csv' | This section of the configuration deals with local authentication based on a CSV file. Local authentication allows users to log in using credentials stored in a CSV file |
authentication.delegated | object | object | This section of the configuration deals with delegated authentication, specifically using the OAuth 2.0 protocol. Delegated authentication allows users to log in to your application using their existing accounts with external identity providers |
authentication.ldap | object | This section of the configuration deals with LDAP (Lightweight Directory Access Protocol) authentication. LDAP is a protocol used for authenticating and authorizing users by accessing a directory service | |
authentication.oidc | object | The client ID for OIDCThis section of the configuration is related to OIDC (OpenID Connect) authentication. OIDC is an authentication protocol that builds on top of OAuth 2.0 and provides user authentication as well as information about the user. . | |
encryption.secret_key | str | 'config/auth/secret_key' | The path to the secret key used for encryption. |
roles.file | str | 'config/auth/roles.csv' | The file path for roles configuration. |
headers.include | list | [( 'strict-transport-security', 'maxage=31536000; includeSubDomains' )] | The list of headers to include in all API responses. |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
'security': { # ******************************************************************************* # Authentication # ******************************************************************************* 'authentication': { 'enabled': False, 'type': AuthenticationType.OIDC, 'secret': '52ecfd', 'force_https_original_url_redirection': False, 'anonymous': { 'id': 'Anonymous', 'account': '[email protected]', 'name': 'Anonymous', 'displayName': 'Anonymous' }, # ******************************************************************* # Local Authentication based on a CSV implements FORM and BASIC Auth # # NOTE: Only recommended for testing # ******************************************************************* 'local': { # ... }, # ************************************** # DELEGATED Authentication # ************************************** 'delegated': { # ... }, # ************************************** # LDAP Authentication # ************************************** 'ldap': { # ... }, # ************************************** # OIDC Authentication # ************************************** 'oidc': { # ... } }, # ******************************************************************************* # Encryption # ******************************************************************************* 'encryption': { 'secret_key': join(SERVER_PATH, 'config', 'auth', 'secret_key') }, # ******************************************************************************* # Roles # ****************************************************************************** 'roles': { 'file': join(SERVER_PATH, 'config', 'auth', 'roles.csv') }, # ******************************************************************************* # Include Headers. # Headers on this section will be added to all responses of the api # ****************************************************************************** 'headers': { 'include': [ ('strict-transport-security', 'max-age=31536000;includeSubDomains') ] } }, |
Info |
---|
You can get a more detail explanation in Security |
The 'mailer' configuration section is responsible for configuring email-related settings, including the SMTP service, email templates, and default email settings.
Nested Field | Type | Default | Description |
---|---|---|---|
enable | Boolean | False (Default) | Enables an endpoint for direct access through an HTTP request. |
mailer_config.service | String | "gmail" (Default) | Specifies the email service provider to use. You can choose between 'gmail' or configure a custom SMTP service. |
mailer_config.auth | Object | N/A | Contains authentication credentials required for the SMTP service. |
mailer_config.host | String | N/A (Uncomment to use) | Specifies the SMTP server hostname (uncomment if using a custom SMTP service). |
mailer_config.port | Integer | N/A (Uncomment to use) | Specifies the SMTP server port (uncomment if using a custom SMTP service). |
mailer_config.secure | Boolean | N/A (Uncomment to use) | Indicates whether the SMTP connection should be secure (uncomment if using a custom SMTP service). |
mailer_config.logger | Boolean | N/A (Uncomment to use) | Enables SMTP logging (uncomment if using a custom SMTP service). |
mailer_config.debug | Boolean | N/A (Uncomment to use) | Enables SMTP debugging (uncomment if using a custom SMTP service). |
mailer_config.tls | Object | N/A (Uncomment to use) | Defines TLS settings for SMTP (uncomment if using a custom SMTP service). |
from | String | N/A | Specifies the email address to display as the sender in outgoing emails. |
test | Boolean | True | When set to True , emails are sent to the 'to_test_email' address instead of the actual recipients for testing purposes. |
to_test_email | String | "[email protected]" (Default) | The email address to which test emails are sent when 'test' mode is enabled. |
default_subject | String | "Email Subject" (Default) | Specifies the default subject for outgoing emails. If not specified in the email content, this subject is used. |
data | Object | N/A | Contains data to be injected into the email content. |
plain_template_path | String | N/A | Specifies the file path to the plain text email template. |
html_template_path | String | N/A | Specifies the file path to the HTML email template. |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
'mailer': { 'enable': False, # Enables an endpoint for direct access through http request 'mailer_config': { # ********************** # Using Gmail # ********************** 'service': 'gmail', 'auth': { 'user': '[email protected]', 'password': 'test' } # ********************** # Using a Custom SMTP # ********************** # 'host': os.getenv('SMTP_RELAY', default='localhost'), # 'port': os.getenv('SMTP_PORT', default=22), # 'secure': False, # 'logger': True, # 'debug': True, # 'tls': { # 'ca':[ Path(os.getenv('CERTIFICATES_PATH')).read_text() ], # 'rejectUnauthorized': False, # }, }, 'from': '[email protected]', # From to display in the email 'test': True, # send the emails to the to_test_email, instead to the actual destiny 'to_test_email': '[email protected]', # Test destination for all email send 'default_subject': 'Email Suibject', # default subject, if none is specified in code 'data': { # This body will be injected as _data, un the actual, body used to map the email # templates (e.g {{{_data.url}}}) 'url': 'http://example.com/' }, # ********************** # Templates # ********************** 'plain_template_path': join(SERVER_PATH, 'config', 'templates', 'email_text.tlp'), 'html_template_path': join(SERVER_PATH, 'config', 'templates', 'email_html.tlp') }, |
The 'analytics' configuration section is responsible for configuring analytics and logging of all activity in the UI, as well as activity triggered by the user in the server.
Name | Type | Default | Description |
---|---|---|---|
enable | boolean | - | Specifies whether analytics tracking is enabled. |
Note |
---|
As of version 3.0.0, the Analytics endpoint is not being used by the UI and needs tuning on the development side. This will be improved in future versions. |
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
'analytics': { 'enable': True }, |
CONFIG_URL
Environment Variable)CONFIG_URL
environment variable is set and not empty.Environment variable replacement is performed within the configuration content, when ever a word is contained in ${}.
Info | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
If in the file you got
This will be replace with the environment variable HOST, lets say (HOST=192.168.1.1)
|
CONFIG_URL
as a URL:In this example, the CONFIG_URL
is set to a valid URL (https://example.com/config.json), which points to a remote configuration file hosted on a web server.
Code Block |
---|
export CONFIG_URL="https://example.com/config.json" |
CONFIG_URL
as a local file path:In this example, the CONFIG_URL
is set to a local file path (/path/to/local/config.json), which points to a configuration file located on the local file system.
Code Block |
---|
export CONFIG_URL="/path/to/local/config.json" |
From the Local File
CONFIG_URL
environment variable is not set or is empty, the configuration is loaded from a local configuration file (config/config.py
) and environment variables.config/env
if the GAIA_ENV
environment file was set to anything other than system_default
production
the local config/config.py will be merged with config/env/production.py
If you need to modify configuration for different environments, you need to create this new configuration in config/env. This configuration files can be exactly the same as the default configuration in config/config.py or just fragments of the overwrite configuration needed, like in the example below
Code Block | ||||
---|---|---|---|---|
| ||||
from os.path import join from models.security import AuthenticationType from utils.constants import SERVER_PATH CONFIG = { 'security': { 'authentication': { 'enabled': True, 'type': AuthenticationType.LOCAL, 'local': { 'file': join(SERVER_PATH, 'config', 'auth', 'users.csv') }, } } } |
This configuration will just overwrite the security authentication section, enabling the local type authentication and adding the configuration for it, everything else not specified will remain as it is in the default configuration.
Info | ||
---|---|---|
| ||
By default there is not environment set, so only config.py applies |
To specify which environment file you want to use, there are 2 options from which to choose, in both cases the environment name must match the name of the environment file
Note |
---|
Environment name = Environment File Name |
By adding the argument --env and the name of the environment
Code Block | ||
---|---|---|
| ||
python uvicorn_server.py --env=value |
Or if executing in docker
Code Block | ||
---|---|---|
| ||
CMD python -m uvicorn app.webapp:app --host 0.0.0.0 --port 8085 --workers 1 --no-server-header --env value |
By adding the environment variable SA_ENV and the name of the environment
To set an environment variable in Windows using CMD, you can use the set command:
Code Block | ||||
---|---|---|---|---|
| ||||
:: Set a temporary environment variable for the current session set SA_ENV=value :: Set a permanent environment variable for the current user (requires admin privileges) setx SA_ENV value :: Set a permanent environment variable for all users (requires admin privileges) setx SA_ENV value /M |
To set an environment variable in PowerShell, you can use either Set-Item or New-ItemProperty cmdlets:
Code Block | ||||
---|---|---|---|---|
| ||||
# Set a temporary environment variable for the current session $env:SA_ENV = "value" # Set a permanent environment variable for the current user (requires admin privileges) New-ItemProperty -Path "HKCU:\Environment" -Name "SA_ENV" -Value "value" -Force # Set a permanent environment variable for all users (requires admin privileges) New-ItemProperty -Path "HKLM:\System\CurrentControlSet\Control\Session Manager\Environment" -Name "SA_ENV" -Value "value" -PropertyType "String" -Force |
To set an environment variable in macOS or Linux using the terminal (bash), you can use the export
command:
Code Block | ||||
---|---|---|---|---|
| ||||
# Set a temporary environment variable for the current session export SA_ENV=value # Set a permanent environment variable for the current user echo 'export SA_ENV=value' >> ~/.bashrc # After updating the configuration file, apply the changes without restarting source ~/.bashrc |
The GAIA API provides access to the Server configuration through the use of SERVER_CONFIG from the config module. By importing SERVER_CONFIG, developers can access all the server configuration properties as if they were regular objects. This allows for seamless interaction with and customization of the server configuration within your application.
To access the Server configuration using the GAIA API, follow these steps:
Import the SERVER_CONFIG object from the config module in your Python code.
Utilize SERVER_CONFIG to access the desired server configuration properties.
Code Block | ||||
---|---|---|---|---|
| ||||
from config import SERVER_CONFIG # Accessing the server configuration properties host = SERVER_CONFIG.host port = SERVER_CONFIG.port auth_type = SERVER_CONFIG.security.authentication.type # Use the server configuration properties in your application print(f"Server Host: {host}") print(f"Server Port: {port}") print(f"Max Results: {auth_type}") |
In the above example, the SERVER_CONFIG object is imported from the config module. The server configuration properties such as host, port, and auth_type are accessed directly from SERVER_CONFIG as regular object attributes. These properties can then be used within your application for further processing or customization.
Note |
---|
Ensure that the config module is properly imported and available in your application environment to access the SERVER_CONFIG object. |