With gapi.py (project manager)

Info

Available since version 0.5

Rename in version 3.0 from sapi to gapi

On the root of the GAIA API project execute on the terminal the following command:

Code Block

language	bash

python gapi.py new endpoint <name>

This will generate a bare minimum endpoint with all the steps and changes view on the manual section, and some route examples

Table of Contents

Warning
Documentation below does not apply for version >= 3.0.0

Manual

Use this section if gapi.py is not avialable in your distribution

1) Create the Endpoint package

Create a folder in app/api/v1
Inside the folder create the __init__.py and the endpoint file
Tip
A descriptive name works the best

On the endpoint file

2) Add the imports

Special attention to APIRouter

Code Block

language	py
theme	DJango

from fastapi import APIRouter, Body
from fastapi.requests import Request

3) Declare the endpoint Router

This is essentially a small branch of the app, which will hold all our related routes
The prefix indicates the prefix of the endpoint
Info
All endpoint have the same base path, /es/api/v1, from here we attach the prefix, in this case being /Simple, so /es/api/v1/Simple

Code Block

language	py
theme	DJango

router = APIRouter(prefix='/Simple', tags=['Simple'])

4) Declare the routes

At this point, all the routes are pure FastAPI
The content of each route depends entirely on the one implementing the endpoint

Tip
If you want more information on how to build routes, and what features are available please go to the FastAPI tutorial, whenever you see @app, just replace that with @router

Code Block

language	py
theme	DJango

@router.post('/post')
async def post_request(req: Request, body: dict = Body()):
    return body

@router.post('/put/{name}')
async def post_request(req: Request, name: str):
    return {"name": name}

@router.post('/get/{id}')
async def post_request(req: Request, id: int, param_a: int = 0):
    return f'GET Request for id {id}, with number {param_a}'

Info

In this scenario the path for each route is:

/es/api/v1/Simple/post

/es/api/v1/Simple/put/lincon

/es/api/v1/Simple/get/42

on the __init__ file

5) Add the import for your router

Go to the __init__.py file
Import the router from the API file

Warning
If you don’t import the router in the __init__.py, GAIA API won’t load it

Code Block

language	py
theme	DJango

from .simple import router

6) Verify Endpoint Status

Starting the server check the logs for the entry loading the endpoint
In case of error, the logs will show the issue
- Either unable to load the endpoint or unable to initialize it
- Otherwise you should see Endpoint <endpoint path> added

You can go to /es/docs or /es/redocand check the
API documentation

Swagger (/es/docs)

ReDoc (/es/redoc)

Endpoint as a Pipeline Wrapper

Using an endpoint to wrap a pipeline gives some advantages when working on a project, specially if a separate team needs to consume said endpoint, and when compared to use the provided Pipeline endpoint

General Pipeline Endpoint

Pipeline endpoint only accepts POST, without restriction of what parameters can accept
User needs to know the parameters each pipeline accepts
Proneto data input errors

Specific Pipeline Endpoint

FastAPI allow to pass a Pydantic model as the Body, meaning all data validation
Multiple methods (GET, PUT, POST, DELET) can apply to same pipeline
Can add query parameters or path Parameters
Customize pipeline response
API documentation & User guidance for data input

How To Use The Pipeline In An API

Tip
The following can be achieve by applying the concepts seen in the FastAPI documentation specifically Request Body, Query Parameters and String Validations and Path Parameters and Numeric Validations

We will use the Search Endpoint for this example

1) Create a Model

Create a file for the model in models/api
Info
The __init__.py is already included in this package

on the model file

2) Add the imports

Since we are making a model from scratch we need to import BaseModel

Code Block

language	py
theme	DJango

from typing import Optional

from pydantic import BaseModel, Field, Extra

from framework.aggregations.utils import SelectionAgg
from models.engines import SortEntry
from models.utils import BoolOperation

3) Declare the model class

All models must inherit from BaseModel or from another model
Declare parameters as we did for Stages

Code Block

language	py
theme	DJango

class ApiSearchRequest(BaseModel):
    q: Optional[str] = Field(default='*', description='Query string')
    query: Optional[dict] = Field(default=None, description='Specific query object for search engine')
    knn: Optional[dict] = Field(default=None, description='Specific knn query object for search engine')
    size: Optional[int] = Field(default=25, description='Number of hits to return per request')
    sort: Optional[SortEntry] = Field(description='Sort field and order')
    start:  Optional[int] = Field(alias='from', description='Start position for retrieving hits', ge=0)
    page: Optional[int] = Field(description='Start page for retrieving hits. Minimum page is 1. Not applicable when start, is being used', ge=1)
    fetch_fields: Optional[list[str]] = Field(
        description='List of fields to return in the response based on field values', title='Fetch Fields')
    scroll: Optional[str] = Field(default=None, description='Period to retain the search context for scrolling')
    default_operator: BoolOperation = Field(BoolOperation.OR,
                                            description='The default operator for query string query: AND or OR',
                                            title='Default Operator')
    exclude_fields: Optional[list[str]] = Field(
        description='List of fields to exclude in the response based on field values.', title='Exclude Fields')
    aggs: Optional[list[SelectionAgg]] = Field(
        description='List of selected aggregations. Usable only when a DynamicAggStage exist in the pipeline',
        title='Aggregations')

Note

We will not import the model in the __init__, because we want the route to be import like this

Code Block
from models.api.api_search_request import ApiSearchRequest

on the endpoint file

4) Declare the model class

Note
Assume all steps for adding an endpoint have been done

Import the model
Import the pipeline_manager

Code Block

language	py
theme	DJango

import os
from typing import Optional

from fastapi import APIRouter
from fastapi.params import Query
from starlette import status
from starlette.requests import Request
from starlette.responses import JSONResponse

from models.api.api_search_request import ApiSearchRequest # <---
from app.pipeline import PipelineImpl, pipeline_manager # <---
from models.utils import BoolOperation
from utils.data import find

5) Declare at least the required 2 parameters

The first one will be the req, which will be the request sent to the endpoint, this request is of type Request from the model starlette.requests or fastapi.requests (which is an alias for starlette)
The second parameter will be you body, in this case we name it payload, and the type will be the model you created

Code Block

language	py
theme	DJango

@router.post('/')
async def execute_post_search(req: Request, payload: ApiSearchRequest):

6) Use the pipeline

Get the pipeline implementation using the pipeline_manager
Then execute the pipeline calling the method execute_pipline, which accepts the request and the model as a dicitonary
Info
you can turn the model into a dictionary using .dict(), for more information regarding this method please visit Pydantic 1.10

Code Block

language	py
theme	DJango

pipeline: PipelineImpl = pipeline_manager.get_pipeline('search')
result = await pipeline.execute_pipeline(req, props=payload.dict(exclude_none=True))

Note
Pipeline executes are asynchronous, an await is need to get the response. And this can only be called from an async function

7) Get the response and do whatever you want

In this case we are checking if the response of the stage named search is correct, if it has a status
Get the total from it and add it as a new field on the root response
and remove the search response entirely

Code Block

language	py
theme	DJango

    if 'status' in result['search'] and result['search']['status'] > 299:
        return JSONResponse(status_code=result['search']['status'], content=result['search']['error'])

    if 'status_code' in result['search'] and result['search']['status_code'] > 299:
        return JSONResponse(status_code=result['search']['status_code'], content=result['search']['error'])

    result['total'] = find(result, 'search.total')

    result.pop('search', None)

    return result

In Case of a GET

We don’t use the model as parameter, we define all as query parameters
Query object server the same function as the Field from Pydantic, but if from FastAPI
Code Block
language py
theme DJango
from fastapi.params import Query
Then we build the model inside the endpoint route
Everything else is the same

Code Block

language	py
theme	DJango

@router.get('/')
async def execute_get_search(
        req: Request,
        q: str = Query(default='*', description='String query to execute in the search engine'),
        size: int = Query(default=25, description='Number of hits to return per request'),
        # sort: Optional[SortEntry] = Query(description='Sort field and order'),
        start: Optional[int] = Query(default=None, alias='from', description='Start position for retrieving hits',
                                     ge=0),
        page: Optional[int] = Query(default=None,
                                    description='Start page for retrieving hits. Minimum page is 1. Not applicable '
                                                'when from, is being used',
                                    ge=1),
        fetch_fields: Optional[list[str]] = Query(default=[],
                                                  description='List of fields to return in the response based on '
                                                              'field values',
                                                  title='Fetch Fields'),
        scroll: Optional[str] = Query(default=None, description='Period to retain the search context for scrolling'),
        default_operator: BoolOperation = Query(default=BoolOperation.OR,
                                                description='The default operator for query string query: AND or OR',
                                                title='Default Operator'),
        exclude_fields: Optional[list[str]] = Query(default=None,
                                                    description='List of fields to exclude in the response based on field values.',
                                                    title='Exclude Fields'),
):
    '''
    GAIA API endpoint, working as a wrapper for a specific pipeline, which will execute with the provided parameters.
    The main goal of this endpoint is to serve as an example of how static, well-defined endpoints can be created with
    a flexible and configurable backend

    **NOTE**: The starting parameters provided by swagger are almost the minimum _(the minimum being just q)_,
    for more options please check the schema

    `Remove this endpoint or update the pipeline it is using, this endpoint is not intended for production`
    '''

    pipeline: PipelineImpl = pipeline_manager.get_pipeline('search')

    payload: ApiSearchRequest = ApiSearchRequest(q=q, size=size, start=start, page=page, fetch_fields=fetch_fields,
                                                 scroll=scroll, default_operator=default_operator,
                                                 exclude_fields=exclude_fields)

Space shortcuts

Page tree

Versions Compared

Old Version 12

New Version Current

Key

With gapi.py (project manager)

Manual

1) Create the Endpoint package

2) Add the imports

3) Declare the endpoint Router

4) Declare the routes

5) Add the import for your router

6) Verify Endpoint Status

Endpoint as a Pipeline Wrapper

How To Use The Pipeline In An API

1) Create a Model

2) Add the imports

3) Declare the model class

4) Declare the model class

5) Declare at least the required 2 parameters

6) Use the pipeline

7) Get the response and do whatever you want

In Case of a GET

Space shortcuts

Page tree

Page History

Versions Compared

Old Version 12

New Version Current

Key

With gapi.py (project manager)

Manual

1) Create the Endpoint package

2) Add the imports

3) Declare the endpoint Router

4) Declare the routes

5) Add the import for your router

6) Verify Endpoint Status

Endpoint as a Pipeline Wrapper

How To Use The Pipeline In An API

1) Create a Model

2) Add the imports

3) Declare the model class

4) Declare the model class

5) Declare at least the required 2 parameters

6) Use the pipeline

7) Get the response and do whatever you want

In Case of a GET