The Query Stage is a crucial component in SearchAPI that enables you to perform powerful search queries on an index or alias. It provides a wide range of options and parameters to customize and fine-tune your search.
With the Query Stage, you can specify the fields to be used for matching terms, phrases, spans, date ranges, and ranges. This allows you to define precisely how your search results are retrieved and filtered based on your specific requirements.
One of the key features of the Query Stage is the ability to leverage PYQPL (Query Parser Language) to create complex and advanced queries. PyQPL provides a flexible syntax that allows you to construct intricate search conditions, combine multiple search criteria, and define custom operators for precise control over your search results.The Query Stage also supports various additional functionalities to enhance your search experience. These include pagination, sorting, highlighting (to emphasize search terms in the results), filters (to narrow down search results based on specific criteria), aggregations (to obtain statistical insights from search results), and the ability to fetch specific fields to optimize performance and reduce data transferCalculate Vector Stage is the first step of semantic search. In this stage the vector is calculated or retrieved based on the model given by the user. There are 3 types of models enumerated.

Saga model: This model retrieves the vector from saga, an entry with the value is require in saga. The value needs to be the same as the saga stage name.
Open AI model: This model calculates the vector with Open AI API, this needs the credentials to make calls to the API as environment variables. These are the supported Open AI models:
1. text-embedding-ada-002
2. text-search-ada-doc-001
3. text-search-curie-doc-001
Sentence Transformer GTR: This model calculates the vector with the GTR component of the python library sentence_transformer. These are the supported GTR models:
1. sentence-transformers/gtr-t5-base
2. sentence-transformers/gtr-t5-large
3. sentence-transformers/gtr-t5-xl
4. sentence-transformers/gtr-t5-xxl

The Calculate Vector Stage stores the vector on intermediate for the Create Query Stage usage.

Table of Contents

maxLevel	3

Properties

Property	Description	Default	Type	Required

QPL Config?


type	Stage class name	-	string	Yes

No


enable	Enable stage for execution	true	boolean

No

	No
name	Name for this specific stage	"

search

vector"

string

No

	No
save_to_intermediate	If true, the result of the stage will be stored in the intermediate instead of the final section	false	boolean	No

No

expand_result

Indicates if the result of this stage should be expanded into the final data dictionary instead of being appended as usual

false

boolean

NoNoui_onlySection specific for UI configuration that affects the process of the stageVectorNo

	No
halt_on_exception	Indicates if the pipeline should be interrupted in case of an exception	true	boolean	No

NoindexName of the index or alias in which to perform the query-string or array of stringsYesNoengine_nameName of the engine to use for this query. If none is provided, the default engine will be usedstringNoNoqueryJSON Query DSL as a string representation or a dictionarystring or objectNoNosizeNumber of hits to return per requestintegerNoNosortSort configuration for the query resultsarray or objectNoNopage_size_defaultDefault page size for pagination25integerNoNooperatorThe default operator for query string query: AND or OR"or"stringNoNohighlightName of the key in which the highlights will be stored in the intermediate"highlight"stringNoNofiltersName of the key in which the filters will be stored in the intermediate"filters"stringNoNoaggregationsName of the key in which the engine specific aggregations will be stored in the intermediate"aggregations"stringNoNoaggregations_filtersName of the key in which the engine specific aggregation filters will be stored in the intermediate"aggregations_filters"stringNoNofetch_fieldsList of fields to fetcharray of stringsNoexclude_fieldsList of fields to excludearray of stringsNoqpl_enableEnable QPL to parse the "q" parameter into an engine query with pyQPLfalsebooleanNoYesimplicit_operatorDefault operator to use when the relationship between two operands is ambiguous"or"stringNoYesfieldsFields to be used for matching terms, phrases, spans, etc.array, object, or stringYesYesdate_fieldsFields to be used for date rangesarray, object, or stringNoYesrange_fieldsFields to be used for rangesarray, object, or stringNoYesdate_formatDate format used to convert date values in the querystringNoYesslop_nearSlop value used for the NEAR operator10numberNoYesslop_beforeSlop value used for the BEFORE operator2numberNoYesslop_adjSlop value used for the ADJ operator0numberNoYesslop_span_notSlop value used for the SPAN NOT operator0numberNoYestimezoneCoordinated Universal Time (UTC) offset or IANA time zone used to convert date values in the query to UTCstringNoYeswildcardUse wildcard operatorsfalsebooleanNoYesgrammarFile path or raw string representing the grammar for parsing text into QPLstringNoYescustom_operatorsDictionary of custom operators with their names as keys and corresponding logic classes as valuesobjectNoYes

synonyms_call

Function returning requested synonyms for the specified string

Callable(accepts: string, returns: list of string, or None )NoYes

Query Stage Intermediate Parameters

model	Indicates the model to be used to calculate or retrieve the vectors. Its restricted by the types allowed by the enum.	EnumSaga.SAGA	Enum	Yes
open_ai_api_key	This is your Open AI key to use the chat from your service provider	os.environ.get('OPEN_AI_API_KEY', 'default_key')	string	No
open_ai_api_base_url	Base url of your service provider for Open AI chat.	os.environ.get('OPEN_AI_API_BASE_URL', 'default_url')	string	No
open_ai_api_type	API type, its restricted by the types allowed by the enum.	EnumOpenAiType.AZURE	Enum	No
open_ai_api_version	API version of the Open AI chat	os.environ.get('OPEN_AI_API_VERSION', '2023-03-15-preview')	string	No
azure_deployment_name	Deployment name for the model (AZURE ONLY).	os.environ.get('OPEN_AI_AZURE_DEPLOYMENT_NAME', None)	Union[string, None]	No
path_to_input	Path to the input in the intermediate that is going to be targeted by this stage to calculate the vector	q	string	No

Calculate Vector Stage Intermediate Parameters

The Calculate Vector The Query Stage offers a range of parameters that can be passed via the intermediate input to customize your search request or modify the configuration of the current stage. These parameters provide flexibility and control over the search process.

Parameter	Description
q	A string query for performing a search. Can be transformed into engine-specific queries using PyQPL (Query Parser Language).
query	Engine-specific queries for the search.
knn	Engine-specific queries specifically for k-nearest neighbor (KNN) searches.
size	Number of results to return from the search request. Overrides the size specified in the configuration.
from/start	Indicates the starting point for retrieving search results. Can be used interchangeably with the `page` parameter.
page	It can be an alternative to from/start. It calculates the start based on the size parameter
fetch_fields	List of fields to fetch for each search result. Overrides the fields specified in the configuration.
exclude_fields	List of fields to exclude from the search results. Overrides the fields specified in the configuration.
scroll	Scroll ID used to retrieve large numbers of results from a single search request, similar to a cursor in a traditional database.
operator	The default operator for query string queries: `AND` or `OR`. Overrides the default operator specified in the configuration.
, this is the target text for the calculate vector. Its path is specified in path_to_input property

Tip
Remember that the intermediate can be fill with either other stages or the original request body that trigger the pipeline, making this essentially REST API parameters

Additional Classes for Configuration

QueryStageUIConfig AnchorQueryStageUIConfigQueryStageUIConfig

The QueryStageUIConfig is a configuration object that provides UI-specific settings for the QueryStage.

QueryStageUIConfig Properties

PropertyDescriptionDefaultTypepage_sizeConfiguration for page size settings in the UIVectorNosortConfiguration for sort settings in the UIVectorNoPageSizeConfig Properties AnchorPageSizeConfigPageSizeConfigPropertyDescriptionDefaultTypedefaultDefault page size value25integeroptionsAvailable page size options[25, 50, 100]array of integersSortConfig Properties AnchorSortConfigSortConfigPropertyDescriptionTypeRequireddefaultDefault sort entrySortEntryYesoptionsAvailable sort optionsarray of VectorYesSortEntry Properties AnchorSortEntrySortEntryPropertyDescriptionType

RequiredfieldName of the field to be used for sortingstringYesdisplay_nameDisplay name for this sort entry (only applicable for user interface)stringNoorderSort order to be usedVector or objectYesSortOrder Enum AnchorSortOrderSortOrderEnum ValueDescription"asc"Ascending sort order"desc"Descending sort order

Example Configuration

Code Block

language	py
theme	DJango

synonyms = {
            'cancer': ['cancer', 'malignancy', '363346000', 'cancers', 'malignancies', '"malignant growth"',
                       '"malignant neoplasm"', '"malignant neoplasms"', '"malignant neoplastic disease"',
                       '"malignant tumor"', '"malignant tumors"', '"neoplasm malignant"', '"neoplasm/cancer"',
                       '"tumor, malignant"'],
            'headache': ['headache', '25064002', 'cephalalgia', 'cephalgia', 'cephalgias', '"cranial pain"',
                         '"have headaches"', '"head ache"', '"head pain"', '"head pain cephalgia"', '"head pains"',
                         'headaches', '"mild global headache"', '"mild headache"', '"pain head"', '"pain in head"',
                         '"pain, head"']
        }

_query_stage = QueryStage_calculate_vector = CalculateVectorStage(
    engine_name=DEFAULT_ENGINE_NAME,
    enable=Truetype='CalculateVectorStage',
    qpl_enable=True,
    name=SEARCH_STAGE_NAME,
    index='movies'VectorStage',
    synonyms_call= lambda x: self.synonyms.get(x, None), # Simple synonym implementation
    wildcard=False,
    fields=['title', 'overview', 'url', 'status', 'metadata', 'metadata.production_companies',
            'metadata.cast',
            'metadata.directors'],
    range_fields=['metadata.budget']model=EnumOpenAI.OPENAI_EMBEDDING_ADA,
    date_fields=['release_date'],
    fetch_fields=['title', 'adult', 'overview', 'url', 'release_date', 'status', 'video', 'metadata',open_ai_api_key='yourapikey',
                  'metadata.production_companies', 'metadata.cast', 'metadata.directors'],
    exclude_fields=None,
    implicit_operator='or',
    page_size_default=25,
    sort=SortEntry(
        field='_score',
        order=SortOrder.DESC
    ),
    aggregations=_aggregation_stage.name,
    aggregations_filters=_aggregation_stage.filters_name,
    highlight=_highlight_stage.name,
    filters=_filter_stage.name,
    ui_only=QueryStageUIConfig(
        sort=SortConfig(
            default=SortEntry(
                field='_score',
                display_name='Score',
                order=SortOrder.DESC
            ),
            options=[
                SortEntry(
                    field='_score',
                    display_name='Score',
                    order=SortOrder.DESC
                ),
                SortEntry(
                    field='release_date',
                    display_name='Release Date',
                    order=SortOrder.ASC
                ),
                SortEntry(
                    field='metadata.budget',
                    display_name='Budget',
                    order=SortOrder.DESC
                )
            ]
        ),
        page_size=PageSizeConfig(
            default=25,
            options=[25, 50, 100]
        )
    )open_ai_api_base_url='baseurl',
	azure_deployment_name=my_deployment_name,
    open_ai_api_type=EnumOpenAiType.AZURE,
	path_to_input='q'
)

Space shortcuts

Page tree

Versions Compared

Old Version 1

New Version Current

Key

Properties

Query Stage Intermediate Parameters

Calculate Vector Stage Intermediate Parameters

Additional Classes for Configuration

QueryStageUIConfig Properties

Example Configuration

Space shortcuts

Page tree

Page History

Versions Compared

Old Version 1

New Version Current

Key

Properties

Query Stage Intermediate Parameters

Calculate Vector Stage Intermediate Parameters

Additional Classes for Configuration

QueryStageUIConfig Properties

Example Configuration