Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Prerequisites

Developers Recommended Tools

These are recommended tools when working with GAIA API, they are not required, but we have seen they are the ones that make the work the easiest

  • PyCharm (Community Version): Pycharm is a powerful Python integrated development environment (IDE) that offers comprehensive features and tools to facilitate efficient and productive Python development.
    • We prefer PyCharm for the:
      • Intelligent Code Editor: PyCharm offers syntax highlighting, code completion, and analysis for error-free coding.
      • Debugging and Testing: Powerful debugger and support for unit testing frameworks.
      • Code Refactoring: Tools for improving code structure and efficiency.
      • Intelligent Assistance: Code templates, quick fixes, and suggestions for more efficient coding.
    • It also provides a comprehensive Git support, which facilitates comparing and merging code from other projects
    • Additionally PyCharm has a plugin for Pydantic which makes working with Pydantic models easier

  •  GitKraken (Paid): GitKraken is a powerful Git client with a visually appealing interface that simplifies and enhances the Git version control workflow.
    • We prefer GitKraken for the:
      •   Visual Commit History: GitKraken provides an intuitive visual representation of commit history, making it easy to navigate and understand project changes.

      • Git Graphs and Branching: GitKraken's interactive graphs provide a clear view of branches and their relationships, simplifying branch management and enabling efficient collaboration.

    • But we only recommend this one if you are working in multiple projects, otherwise is not necessary

Hardware Requirements

Hardware requirements are tricky to establish because GAIA API is meant to be customized. It will be used for different purposes and using different components on every project it is implemented.

Please note numbers specified here are just an initial recommendation we calculated based on the vanilla configuration of the GAIA API which is a search application for the movies index.  There is no AI models loaded or vector calculation, just plain search queries to the search engine.

Assuming you configured GAIA API to work with 16 workers:

CPU: 4 cores

RAM: 16 Gb


Note

GAIA API comes configured by default with 1 worker.


Ask For Access And Branch Folder

Warning
titleContact Global Support for this!!

Make your request for the permissions group and project folder request with the CAGGlobalSupport Team

or create a ticket here Create Issue - Jira (accenture.com) (Keep in mind you need access to the IIS project in order to create the ticket)

Before start working, ask permission to access and clone the project GAIA API

https://source.digital.accenture.com/projects/ST/repos/gaia-api/browse

And the creation of a permissions group in active directory with the format as required, 

Code Block
digital.source.st.your-project.admins
digital.source.st.your-project.creators
digital.source.st.your-project.users

 along with branch folder specific for your project, where you and your team will have complete freedom to create more branches, add, update and delete code.

Code Block
your-project/develop
your-project/main
your-project/features
your-project/release

Here's an example or the ticket for the Support Team:


Info
iconfalse

For the following branch in bitbucket:

<branch name>

in the repository:

<repository name>

please provide the permission groups and include the following users in that group:

  • <user identifier 1>
  • <user identifier 2>
  • ...
Info
titleWhy have a branch folder, in the same repository and not have my own repository?

The branch folder is recommended to make synergy between the different project under development at the same time, features created on a project, may as well be used in another, all generic features are merge into the root develop branch, and from there can be distributed to other projects and user in future projects.

You still can have the code in another repository, but in that case we still recommend have the branch folder and synchronize changes, making the support of the project less invasive and the upgrade of the code, more fluid
Warning
titleDO NOT CLONE FROM ANOTHER PROJECT!!!


If you are going to generate your branches on your own, DO NOT CLONE FROM ANOTHER PROJECT!!!, always clone from the develop branch, develop branch is the one supported by the development team. If you clone from another project:

  • You increase the risk of bugs, which the develop team has no idea
  • You may fetch extra code you don't require for your project
  • The development team doesn't know the code implemented in each project, so the help given on custom code can be minimum
  • The other project may not be update with the latest base version, which means you may have bugs already fixed in the develop branch
Tip
titleCompare & Fetch from other project


If you need code implemented in other project, use your preferable tool for managing git, and do a Compare with Branch, this way you can fetch the code you need to the latest code in your project. 

IF YOU DON'T KNOW HOW TO DO THIS, you can read this article How to Move Changes Between Branches

Table of Contents

Download Python 

Download and install Python, currently version 3.11.6,

Warning

For version 1.0.0, Windows can manage 3.9.X due to a C compilation issue with a dependency. Issue fixed in version 1.1.0



Info
titleRead About FastAPI

The new backend of the Enterprise Search is built in Python, using FastAPI, which allow us to maintain a structure like ExpressJs, but with a better understanding and on-the-fly documentation with Swagger UI and Redoc

You can see more about the features of FastAPI here

  • For Windows 64bit version

    Note
    titleCompatibility Issue

    Windows currently has issues with xmlsec library, And it only supports Python 3.9.x

    Update: in GAIA version 2.0, we removed the library which required the xmlsec library, so higher versions of pyhton can be used, starting from GAIA version 2.x

  • For Linux

  • For macOs


Note
titleDisable the path length
Disable the path length limit to prevent issues in windows environment

Fahim-article-image



Setup Environment

Clone and Set The Branch

Tip
titleConnect to VPN

Connection to Accenture's VPN is required

  1. Clone the project from the repository

    Code Block
    git clone https://source.digital.accenture.com/scm/st/gaia-api.git
  2. Verify the current brach executing the command below, it should be main

    Code Block
    git branch
    1. If the branch wasn't main, checkout the main branch:

      Code Block
      git checkout main
  3. Create a branch to work on, using the branch folder assigned <branch_folder>/develop (we recommend using develop as your working branch)

    Code Block
    git checkout -b <branch_folder>/develop

Create & Use A Virtual Environment

We recommend the creation of a virtual environment to prevent python dependencies conflicts, is also helpful if you have multiple versions of python in the same machine

Column
width350px
Code Block
python -m venv {name_of_your_venv}
Column
width20px


 or

Column
width350px
Code Block
python3 -m venv {name_of_your_venv}


If you want to name you virtual environment test then the command would look like this

Column
width350px
Code Block
python -m venv test
Column
width20px


 or

Column
width350px
Code Block
python3 -m venv test

Activate Virtual Environment

To use your virtual environment you need to activated, the activation command changes depending on your OS

  • For Windows

    Code Block
    {name_of_your_venv}\Scripts\activate.bat
  • For Unix or macOS, run:

    Code Block
    source {name_of_your_venv}/bin/activate
    Note
    titleUsing Bash on Windows

    If using a bash console on Windows, then you have to execute

    Code Block
    source {name_of_your_venv}/Scripts/activate

Following the example of the venv named test, the commands would look like this:

  • Windows

    Code Block
    test\Scripts\activate.bat
  • Unix or macOS:

    Code Block
    source test/bin/activate

Install PyQPL local library

After the installation is successful, proceed to install PyQPL

Note

The pyqpl library comes in the lib folder of the GAIA API 

Code Block
pip install lib/pyqpl-VERSION_IN_PROJECT-py3-none-any.whl
Tip
titleDifferent version depending on the project and date of the code

 Is better is you write lib/pyqpl and then let the autocomplete of the terminal write the rest of the file

Why Use PyQPL?

PyQPL is our proprietary dependency, is a versatile scripting language that simplifies the construction of complex queries. It offers engine agnosticism, query parsing, manual query building, query translation, Saga coupling for advanced processing, synonym expansion, and support for custom operators. With PyQPL, you can write standardized queries irrespective of the underlying search engine, making it flexible and efficient for query processing tasks.

Is already seamlessly integrated into the Query Stage, and can be enable or disabled as need it, probably if you are working or had work with GAIA API previously, have used PyQPL without notice.

Info
titleAsk for help

If any issue raises or a feature is required please don't hesitate to contact the development team

Uninstall PyQPL local library

In case you need to upgrade the version of pyqpl, we recommend to uninstall the current library using the command

Code Block
pip uninstall pyqpl

Delete the whl file and replace it with the new one, and use the same command as above to install the new one


Install GAIA Core local library

Info

Since 3.0 version

What is the GAIA Core?

In the latest update to Version 3.0, significant architectural changes have been made to the GAIA API. The core functionalities of the GAIA API, including APIs, stages, and engines, have been migrated to a new, external library named GAIA Core. This strategic restructuring aims to segregate the foundational code from specialized implementations within the GAIA API.

Key Changes

  • Separation of Core and Specialized Code: The base code, comprising APIs, stages, and engines, has now been relocated to the GAIA Core library. Consequently, the GAIA API retains only the specialized code, tailored to specific applications and functionalities.

  • Benefits:

    • Enhanced Troubleshooting and Debugging: By isolating the core functionalities into a separate library, we streamline the process of identifying and resolving issues within the GAIA API ecosystem.
    • Simplified Migration: Upgrading to future versions of the GAIA API is now more straightforward. The upgrade process primarily involves reinstalling a single wheel file. In instances where modifications to specialized code are necessary, updates can be applied more efficiently and with minimal disruption.


To install GAIA Core execute:

Note

The gaia_core library comes in the lib folder of the GAIA API 

Code Block
pip install lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl
Tip
titleDifferent version depending on the project and date of the code

 It's better to write lib/gaia_core and then let the autocomplete of the terminal write the rest of the file

There are other optional dependencies, you can install depending on your necessities

  • If you want to use LDAP authentication

    Code Block
    pip install "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[ldap]"
  • If you want to use sentence splitter capabilities

    Code Block
    pip install "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[nltk]"
  • If you want to use genai and vectorization capabilities

    Code Block
    pip install "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[genai]"
  • If you want to use the unit testing

    Code Block
    pip install "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[test]"
  • If you want to install all required and optional dependencies

    Code Block
    pip install "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[all]"
Tip

you can even try combinations like pip install  "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[genai,test]"

Panel
borderColor#cecece
bgColor#fbfbfb
titleBGColor#eeeeee
borderStylesolid
titleWhat is the difference between required and optional dependencies?

Required Dependencies

  • Essential for your project to work.
  • If missing, your project won’t run.
  • Example: A library needed to call the GAIA API for your project's core functions.

Optional Dependencies

  • Enhance or add extra features to your project but aren’t crucial.
  • If missing, your project still works, just without some features.
  • Example: Without nltk you can't use the sentence splitter, but that's all




Uninstall GAIA Core local library

In case you need to upgrade the version of gaia_core, we recommend to uninstall the current library using the command

Code Block
pip uninstall gaia_core

Delete the whl file and replace it with the new one, and use the same command as above to install the new one



Install Python Dependencies

If you are working on Windows you may need to download Build Tools for Visual Studio and install the Universal Windows Platform build tools (Probably will require Admin rights). Unix bases OS should have any issues

Once the virtual environment is activated (if you did one, otherwise is the same process), install the dependencies stored in the pyproject.toml. To install them execute the following command:

Code Block
pip install -e .

This will install the bare minimum necessary dependencies in order to run GAIA API

Note

Since 1.1.0 version

There are other optional dependencies, you can install depending on your necessities

  • If you want to use LDAP authentication

    Code Block
    pip install -e ".[ldap]"
    Note

    Not applicable for versions 3.0 or higher

  • If you want to use genai and vectorization capabilities

    Code Block
    pip install -e ".[genai]"
    Note

    Not applicable for versions 3.0 or higher

  • If you want to use the unit testing

    Code Block
    pip install -e ".[test]"
  • If you want to install all required and optional dependencies

    Code Block
    pip install -e ".[all]"
Tip

you can even try combinations like pip install -e ".[genai,test]"


If that didn't work try

Column
width350px
Code Block
python -m pip install -e .
Column
width20px


 or

Column
width350px
Code Block
python3 -m pip install -e .

If none of the previous commands worked, try looking at this troubleshooting article.

Tip
titleUpdate the Pyproject.toml

If you have installed new python dependencies, you may to update the pyproject.toml, with the latest changes, you can see how in How to Add New Dependencies

Install with Development Dependencies

Same as above, but additionally it will install development dependencies necessary for testing a server mocking

Code Block
pip install -e ".[test]"

Startup The Server

Note
titleDon't Forget the Search Engine

In order to startup the server the Non-SQL engine needs to be up and running, and the connection to it must setup in the Configuration


In root folder execute

Column
width350px
Code Block
python uvicorn_server.py 
Column
width20px


 or

Column
width350px
Code Block
python3 uvicorn_server.py