You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 45 Next »

Prerequisites

Developers Recommended Tools

These are recommended tools when working with GAIA API, they are not required, but we have seen they are the ones that make the work the easiest

  • PyCharm (Community Version): Pycharm is a powerful Python integrated development environment (IDE) that offers comprehensive features and tools to facilitate efficient and productive Python development.
    • We prefer PyCharm for the:
      • Intelligent Code Editor: PyCharm offers syntax highlighting, code completion, and analysis for error-free coding.
      • Debugging and Testing: Powerful debugger and support for unit testing frameworks.
      • Code Refactoring: Tools for improving code structure and efficiency.
      • Intelligent Assistance: Code templates, quick fixes, and suggestions for more efficient coding.
    • It also provides a comprehensive Git support, which facilitates comparing and merging code from other projects
    • Additionally PyCharm has a plugin for Pydantic which makes working with Pydantic models easier

  •  GitKraken (Paid): GitKraken is a powerful Git client with a visually appealing interface that simplifies and enhances the Git version control workflow.
    • We prefer GitKraken for the:

      •   Visual Commit History: GitKraken provides an intuitive visual representation of commit history, making it easy to navigate and understand project changes.

      • Git Graphs and Branching: GitKraken's interactive graphs provide a clear view of branches and their relationships, simplifying branch management and enabling efficient collaboration.

    • But we only recommend this one if you are working in multiple projects, otherwise is not necessary

Hardware Requirements

Hardware requirements are tricky to establish because GAIA API is meant to be customized. It will be used for different purposes and using different components on every project it is implemented.

Please note numbers specified here are just an initial recommendation we calculated based on the vanilla configuration of the GAIA API which is a search application for the movies index.  There is no AI models loaded or vector calculation, just plain search queries to the search engine.

Assuming you configured GAIA API to work with 16 workers:

CPU: 4 cores

RAM: 16 Gb


GAIA API comes configured by default with 1 worker.


Ask For Access And Branch Folder

Contact Global Support for this!!

Make your request for the permissions group and project folder request with the CAGGlobalSupport Team

or create a ticket here Create Issue - Jira (accenture.com) (Keep in mind you need access to the IIS project in order to create the ticket)

Before start working, ask permission to access and clone the project GAIA API

https://source.digital.accenture.com/projects/ST/repos/gaia-api/browse

And the creation of a permissions group in active directory with the format as required, 

digital.source.st.your-project.admins
digital.source.st.your-project.creators
digital.source.st.your-project.users

 along with branch folder specific for your project, where you and your team will have complete freedom to create more branches, add, update and delete code.

your-project/develop
your-project/main
your-project/features
your-project/release

Why have a branch folder, in the same repository and not have my own repository?


The branch folder is recommended to make synergy between the different project under development at the same time, features created on a project, may as well be used in another, all generic features are merge into the root develop branch, and from there can be distributed to other projects and user in future projects.

You still can have the code in another repository, but in that case we still recommend have the branch folder and synchronize changes, making the support of the project less invasive and the upgrade of the code, more fluid

DO NOT CLONE FROM ANOTHER PROJECT!!!


If you are going to generate your branches on your own, DO NOT CLONE FROM ANOTHER PROJECT!!!, always clone from the develop branch, develop branch is the one supported by the development team. If you clone from another project:

  • You increase the risk of bugs, which the develop team has no idea
  • You may fetch extra code you don't require for your project
  • The development team doesn't know the code implemented in each project, so the help given on custom code can be minimum
  • The other project may not be update with the latest base version, which means you may have bugs already fixed in the develop branch

Compare & Fetch from other project


If you need code implemented in other project, use your preferable tool for managing git, and do a Compare with Branch, this way you can fetch the code you need to the latest code in your project. 

IF YOU DON'T KNOW HOW TO DO THIS, you can read this article How to Move Changes Between Branches

Download Python 

Download and install Python, currently version 3.11.6,

For version 1.0.0, Windows can manage 3.9.X due to a C compilation issue with a dependency. Issue fixed in version 1.1.0



Read About FastAPI

The new backend of the Enterprise Search is built in Python, using FastAPI, which allow us to maintain a structure like ExpressJs, but with a better understanding and on-the-fly documentation with Swagger UI and Redoc

You can see more about the features of FastAPI here

  • For Windows 64bit version

    Compatibility Issue

    Windows currently has issues with xmlsec library, And it only supports Python 3.9.x

    Update: in GAIA version 2.0, we removed the library which required the xmlsec library, so higher versions of pyhton can be used, starting from GAIA version 2.x

  • For Linux

  • For macOs


Disable the path length

Disable the path length limit to prevent issues in windows environment

Fahim-article-image


Setup Environment

Clone and Set The Branch

Connect to VPN

Connection to Accenture's VPN is required

  1. Clone the project from the repository

    git clone https://source.digital.accenture.com/scm/st/gaia-api.git

  2. Verify the current brach executing the command below, it should be main

    git branch

    1. If the branch wasn't main, checkout the main branch:

      git checkout main

  3. Create a branch to work on, using the branch folder assigned <branch_folder>/develop (we recommend using develop as your working branch)

    git checkout -b <branch_folder>/develop

Create & Use A Virtual Environment

We recommend the creation of a virtual environment to prevent python dependencies conflicts, is also helpful if you have multiple versions of python in the same machine

python -m venv {name_of_your_venv}


 or

python3 -m venv {name_of_your_venv}


If you want to name you virtual environment test then the command would look like this

python -m venv test


 or

python3 -m venv test

Activate Virtual Environment

To use your virtual environment you need to activated, the activation command changes depending on your OS

  • For Windows

    {name_of_your_venv}\Scripts\activate.bat

  • For Unix or macOS, run:

    source {name_of_your_venv}/bin/activate

    Using Bash on Windows

    If using a bash console on Windows, then you have to execute

    source {name_of_your_venv}/Scripts/activate

Following the example of the venv named test, the commands would look like this:

  • Windows

    test\Scripts\activate.bat

  • Unix or macOS:

    source test/bin/activate

Install PyQPL local library

After the installation is successful, proceed to install PyQPL

The pyqpl library comes in the lib folder of the GAIA API 

pip install lib/pyqpl-VERSION_IN_PROJECT-py3-none-any.whl

Different version depending on the project and date of the code

 Is better is you write lib/pyqpl and then let the autocomplete of the terminal write the rest of the file

Why Use PyQPL?

PyQPL is our proprietary dependency, is a versatile scripting language that simplifies the construction of complex queries. It offers engine agnosticism, query parsing, manual query building, query translation, Saga coupling for advanced processing, synonym expansion, and support for custom operators. With PyQPL, you can write standardized queries irrespective of the underlying search engine, making it flexible and efficient for query processing tasks.

Is already seamlessly integrated into the Query Stage, and can be enable or disabled as need it, probably if you are working or had work with GAIA API previously, have used PyQPL without notice.

Ask for help

If any issue raises or a feature is required please don't hesitate to contact the development team

Uninstall PyQPL local library

In case you need to upgrade the version of pyqpl, we recommend to uninstall the current library using the command

pip uninstall pyqpl

Delete the whl file and replace it with the new one, and use the same command as above to install the new one

Install GAIA Core local library

Since 3.0 version

What is the GAIA Core?

In the latest update to Version 3.0, significant architectural changes have been made to the GAIA API. The core functionalities of the GAIA API, including APIs, stages, and engines, have been migrated to a new, external library named GAIA Core. This strategic restructuring aims to segregate the foundational code from specialized implementations within the GAIA API.

Key Changes

  • Separation of Core and Specialized Code: The base code, comprising APIs, stages, and engines, has now been relocated to the GAIA Core library. Consequently, the GAIA API retains only the specialized code, tailored to specific applications and functionalities.

  • Benefits:

    • Enhanced Troubleshooting and Debugging: By isolating the core functionalities into a separate library, we streamline the process of identifying and resolving issues within the GAIA API ecosystem.
    • Simplified Migration: Upgrading to future versions of the GAIA API is now more straightforward. The upgrade process primarily involves reinstalling a single wheel file. In instances where modifications to specialized code are necessary, updates can be applied more efficiently and with minimal disruption.


To install GAIA Core execute:

The gaia_core library comes in the lib folder of the GAIA API 

pip install lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl

Different version depending on the project and date of the code

 It's better to write lib/gaia_core and then let the autocomplete of the terminal write the rest of the file

There are other optional dependencies, you can install depending on your necessities

  • If you want to use LDAP authentication

    pip install "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[ldap]"

  • If you want to use sentence splitter capabilities

    pip install "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[nltk]"

  • If you want to use genai and vectorization capabilities

    pip install "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[genai]"

  • If you want to use the unit testing

    pip install "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[test]"

  • If you want to install all required and optional dependencies

    pip install "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[all]"

you can even try combinations like pip install  "lib/gaia_core-VERSION_IN_PROJECT-py3-none-any.whl[genai,test]"

What is the difference between required and optional dependencies?

Required Dependencies

  • Essential for your project to work.
  • If missing, your project won’t run.
  • Example: A library needed to call the GAIA API for your project's core functions.

Optional Dependencies

  • Enhance or add extra features to your project but aren’t crucial.
  • If missing, your project still works, just without some features.
  • Example: Without nltk you can't use the sentence splitter, but that's all




Uninstall GAIA Core local library

In case you need to upgrade the version of gaia_core, we recommend to uninstall the current library using the command

pip uninstall gaia_core

Delete the whl file and replace it with the new one, and use the same command as above to install the new one


Install Python Dependencies

If you are working on Windows you may need to download Build Tools for Visual Studio and install the Universal Windows Platform build tools (Probably will require Admin rights). Unix bases OS should have any issues

Once the virtual environment is activated (if you did one, otherwise is the same process), install the dependencies stored in the pyproject.toml. To install them execute the following command:

pip install -e .

This will install the bare minimum necessary dependencies in order to run GAIA API

Since 1.1.0 version

There are other optional dependencies, you can install depending on your necessities

  • If you want to use LDAP authentication

    pip install -e ".[ldap]"

    Not applicable for versions 3.0 or higher

  • If you want to use genai and vectorization capabilities

    pip install -e ".[genai]"

    Not applicable for versions 3.0 or higher

  • If you want to use the unit testing

    pip install -e ".[test]"

  • If you want to install all required and optional dependencies

    pip install -e ".[all]"

you can even try combinations like pip install -e ".[genai,test]"


If that didn't work try

python -m pip install -e .


 or

python3 -m pip install -e .

If none of the previous commands worked, try looking at this troubleshooting article.

Update the Pyproject.toml

If you have installed new python dependencies, you may to update the pyproject.toml, with the latest changes, you can see how in How to Add New Dependencies

Install with Development Dependencies

Same as above, but additionally it will install development dependencies necessary for testing a server mocking

pip install -e ".[test]"

Startup The Server

Don't Forget the Search Engine

In order to startup the server the Non-SQL engine needs to be up and running, and the connection to it must setup in the Configuration


In root folder execute

python uvicorn_server.py


 or

python3 uvicorn_server.py
  • No labels