Feature compatible only with Saga 1.3+ responses
PyQPL allows you to work with responses from Saga, enabling you to convert LexItems into synonyms, QPL keywords, and more complex query structures. By leveraging the capabilities of Saga, including the ability to maintain, configure, and tune dictionaries for entity extraction, PyQPL with Saga provides a powerful tool for creating queries with mutable syntax and enhanced capabilities.
Field | Type | Required | Default | Description | Examples |
---|---|---|---|---|---|
implicit_operator | Literal['or', 'and'] | or | Default operator to use when the relationship, between 2 operands is ambiguous Applicable only to the parser | "or", "and" | |
fields | List[str] OR Dict[str, float] OR List[QPLField] OR str | Fields to be use when match term, phrases, spans, ... In the string formats you can add the boots by adding ^ and the amount to boots, e.g. ^2 or ^0.5 | As string:
As a List:
As a Dictionary:
as a List of QPLFields:
| ||
date_fields | List[str] OR Dict[str, float] OR List[QPLField] OR str | [ ] | Fields to be use when date ranges, if no range queries are In the string formats you can add the boots by adding ^ and the amount to boots, e.g. ^2 or ^0.5 | As string:
As a List:
As a Dictionary:
as a List of QPLFields:
| |
range_fields | List[str] OR Dict[str, float] OR List[QPLField] OR str | [ ] | Fields to be use when ranges In the string formats you can add the boots by adding ^ and the amount to boots, e.g. ^2 or ^0.5 | As string:
As a List:
As a Dictionary:
as a List of QPLFields:
| |
date_format | str | Date format used to convert date values in the query. The format to use must be compatible with the engine specific query | |||
timezone | str | Coordinated Universal Time (UTC) offset or IANA time zone used to convert date values in the query to UTC. The timezone to use must be compatible with the engine specific query | |||
slop_near | number | 10 | Slop value used for the NEAR operator | ||
slop_before | number | 2 | Slop value used for the BEFORE operator | ||
slop_adj | number | 0 | Slop value used for the ADJ operator | ||
slop_span_not | number | 0 | Slop value used for the SPAN NOT operator | ||
wildcard | bool | False | Use wildcard operators Applicable only to the parser | ||
grammar | File Path Or str | File path to the grammar or the actual raw grammar in string, in case you need to parse text to QPL with custom operators. QPL parser uses a Lalr parser implemented with Lark library, for more information check https://lark-parser.readthedocs.io/en/latest/grammar.html# We recommend that before starting to make a new grammar, consult with the development team Applicable only to the parser | For more detail on how to build a grammar please check Grammar Composition from the Lark documentation | ||
custom_operators | Dict[str, Operand] | {} | Dictionary with the name type of the custom operator as keys, and the class with their logic as values. All classes must inherit from Operator Applicable only to the parser | ||
synonyms_call | Func[] | Function returning requested synonyms for the specified string Applicable only to the parser | |||
saga_keywords | List[str] | [] | Saga tags to be normalize as QPL keywords. The display of the tag will use to replace the tag. | ["tag1", "tag2"] | |
saga_synonyms | List[str] | [] | Saga tags marked as synonyms, For each entity in the matched tag, the list of patterns within the entity, will be used as the synonyms to replace the matched text. | ["tag1", "tag2"] | |
saga_synonyms_boost | float | 0.8 | Boots to be used on each synonym added to the query. | 0.8 | |
saga_special_case | Dict[str, Callable[LexItem, str] | {} | Dictionary with tags as per keys, where each key has a function assigned, which will receive a LexItem and transformed it into a suitable query statement. | def use_case(token: LexItem) -> str ... { |
The keyword replacement can be use as an alternative to manipulate the grammar, with saga_keywords you can specify specific tags that will be normalized to their display value, which should be a QPL keyword, that way you can transform tokens such as "and", "y", "und", "et", "&&" into the keyword AND, ot "not", "no", "nicht", "pas", "!" into the keyworkd NOT
In Saga you must have a tag that once matched, the result should be something like the one below, the important bits from it are:
display: stored in the entities, this is the keyword Saga QPL will use
it doesn't matter if the display is in lower case, sin the tag is being use in saga_keywords, all the displays are transformed into upper case
{ "stage": "DictionaryTagger", "confidence": 0.5, "match": "not", "flags": [ "ENTITY", "SEMANTIC_TAG" ], "text": "{unaryOperator}", "startPos": 7, "endPos": 10, "metadata": { "display": "not", "id": "A0006" }, "entities": [ { "display": "not", "patterns": [ "not", "non", "nicht", "no", "pass" ], "id": "A0006", "fields": {}, "tags": [ "unaryOperator" ] } ], "tags": [ "unaryOperator" ] }
The synonym expansion with Saga uses the entity extraction from Saga to expand the matches with the patterns provided in each entity. To make use of this functionality just add the tags you want to expand into saga_synonyms, Saga will look for LexItems with these tags, and make the expansion on them. Additionally you can specify a custom boots for the synonyms using saga_synonyms_boost, the original token will be left untouched Use SagaQPLOptions instead of QPLOptions, and assign saga_synonyms with the list of tags to expand as synonymns, optionally you can specify a custom boost for the synonyms Get the Saga response with whatever method you see fit, the simplest method is to make an http request to Saga Client API Execute the parsing of the entire Saga response and print the qpl_tree The function .pretty() of the QPL tree will show a visual representation of how the query is form As show below the query is has now being expanded with the synonyms found, for each of the tokens, on the left you can see how the original query without expansion would look, and on the right the same query with the synonym expansion appliedExample Implementation
options = SagaQPLOptions(
implicit_operator='and',
fields={'content': 6, 'title': 3}
saga_synonyms=['synonyms'],
saga_synonyms_boost=0.8
)
parser = SagaParser(options=options)
import requests
import json
response = requests.get('http://localhost:8080/saga/api/client/process/text, data=json.dumps({
'unit': 'unit_name',
'doc': 'cancer not headache'
}))
saga_response = response.json()
qpl_tree = parser._parse(data=saga_response['highestRoute'])
print(qpl_tree.pretty())
and term cancer term not term headache
and or boost term cancer 1 boost term malignancy 0.8 boost term 363346000 0.8 boost term cancers 0.8 boost term malignancies 0.8 boost phrase "malignant growth" 0.8 boost phrase "malignant neoplasm" 0.8 boost phrase "malignant neoplasms" 0.8 boost phrase "malignant neoplastic disease" 0.8 boost phrase "malignant tumor" 0.8 boost phrase "malignant tumors" 0.8 boost phrase "neoplasm malignant" 0.8 boost phrase "neoplasm/cancer" 0.8 boost phrase "tumor, malignant" 0.8 term not or boost term headache 1 boost term 25064002 0.8 boost term cephalalgia 0.8 boost term cephalgia 0.8 boost term cephalgias 0.8 boost phrase "cranial pain" 0.8 boost phrase "have headaches" 0.8 boost phrase "head ache" 0.8 boost phrase "head pain" 0.8 boost phrase "head pain cephalgia" 0.8 boost phrase "head pains" 0.8 boost term headaches 0.8 boost phrase "mild global headache" 0.8 boost phrase "mild headache" 0.8 boost phrase "pain head" 0.8 boost phrase "pain in head" 0.8 boost phrase "pain, head" 0.8