Saga QPL Options

["tag1", "tag2"]

["tag1", "tag2"]

0.8

def use_case(token: LexItem) -> str

...

--------

{

    'tagName': use_case

}

Keyword Replacement

The keyword replacement can be use as an alternative to manipulate the grammar, with saga_keywords you can specify specific tags that will be normalized to their display value, which should be a QPL keyword, that way you can transform tokens such as "and", "y", "und", "et", "&&" into the keyword AND, ot "not", "no", "nicht", "pas", "!" into the keyworkd NOT

In Saga you must have a tag that once matched, the result should be something like the one below, the important bits from it are:

• tags: This is where Saga QPL will look for the tags to transform

• display: stored in the entities, this is the keyword Saga QPL will use

  Note
  it doesn't matter if the display is in lower case, sin the tag is being use in saga_keywords, all the displays are transformed into upper case

{
  "stage": "DictionaryTagger",
  "confidence": 0.5,
  "match": "notnicht",
  "flags": [
    "ENTITY",
    "SEMANTIC_TAG"
  ],
  "text": "{unaryOperator}",
  "startPos": 7,
  "endPos": 10,
  "metadata": {
    "display": "not",
    "id": "A0006"
  },
  "entities": [
    {
      "display": "not",
      "patterns": [
        "not",
        "non",
        "nicht",
        "no",
        "pass"
      ],
      "id": "A0006",
      "fields": {},
      "tags": [
        "unaryOperator"
      ]
    }
  ],
  "tags": [
    "unaryOperator"
  ]
}

Synonym Expansion

The synonym expansion with Saga uses the entity extraction from Saga to expand the matches with the patterns provided in each entity. To make use of this functionality just add the tags you want to expand into saga_synonyms, Saga will look for LexItems with these tags, and make the expansion on them.

Example Implementation

Use SagaQPLOptions instead of QPLOptions, and assign saga_keywords with the list of tags to expand as synonyms, optionally you can specify a custom

boost for the synonyms

implicit_operator='and',
    fields={'content': 6, 'title': 3},
    saga_

keywords=['

unaryOperator']

)

parser = SagaParser(options=options)

options = SagaQPLOptions(
    

Get the Saga response with whatever method you see fit, the simplest method is to make an http request to Saga Client API

import requests
import json

response = requests.get('http://localhost:8080/saga/api/client/process/text', data=json.dumps({
    'unit': 'unit_name',
    'doc': 'cancer 

nich headache'
}))

saga_response = response.json()

Execute the parsing of the entire Saga response and print the qpl_tree

qpl_tree = parser._parse(data=saga_response['highestRoute'])
print(qpl_tree.pretty())

As show below the query is has now being expanded with the synonyms found, for each of the tokens, on the left you can see how the original query without expansion would look, and on the right the same query with the synonym expansion applied

Synonym Expansion

{
  "stage": "DictionaryTagger",
  "confidence": 1,
  "match": "cancer",
  "flags": [
    "ENTITY",
    "SEMANTIC_TAG"
  ],
  "text": "{synonyms}",
  "startPos": 0,
  "endPos": 6,
  "metadata": {
    "display": "cancer",
    "id": "syn0000000141"
  },
  "entities": [
    {
      

Special Cases

Sometime there will be cases to specific to be coded into PyQPL, this is where Saga special cases enters into action, Saga can identify the case to be treated with a tag, this tag is later transformed into a string representation of a QPL query, replacing the original content, and finally being parsed by PyQPL.

The saga_special_case parameter accepts a dictionary with tag names a its keys, and the value of each key being a callable which receives a LexItem and returns a string (being this the string representation of the query)

Example Implementation

For this example we are using a scenario in which the user wants to look for "small cell", referring to cases of cancer, due to the nature of the query results are returned like "small non cell cancer", which refers to a non cancer case, which is the opposite of what the user want, if the user types "small cancer -non" this would remove any results that contains the word "non", even if it doesn't have any relationship with "small cell", for this we want to check for results where the token "non" is not near the token "small".

We will assume this LexItem is returned in the Saga response:

{
    "stage": "DictionaryTagger",
    "confidence": 1,
    "match": "small",
    "flags": [
        "ENTITY",
        "SEMANTIC_TAG"
    ],
    "text": "{specialCase}",
    "startPos": 0,
    "endPos": 5,
    "metadata": {
        "not": "non small",
        "display": "small",
        "id": "A0000"
    },
    "entities": [
        {
            "display": "small",
            "patterns": [
                "small"
            ],
            "id": "A0000",
            "fields": {
                "not": "non small"
            },
            "tags": [
                "specialCase"
            ]
        }
    ],
    "tags": [
        "specialCase"
    ]
}

We start creating the function which will get the LexItem and transform it into a string query, and as you see below we make use of the information added in the metadata, the match, but you can make use of everything available in the LexItem

def span_not(token: LexItem) -> str:

    if token.metadata and 'not' in token.metadata:
        exclude_value = ' NEAR '.join(token.metadata['not'].split())

        return f'{token.match} SPAN_NOT ({exclude_value})'
    else:
        return token.match

Use SagaQPLOptions instead of QPLOptions, and assign saga_special_case with a dictionary of the tags we want to work with as the keys, and the functions making the transformation as the values.

options = SagaQPLOptions(
    implicit_operator='and',
    fields={'content': 6, 'title': 3},
    saga_special_case={
        'specialCase': span_not
    }
)

parser = SagaParser(options=options)

import requests
import json

response = requests.get('http://localhost:8080/saga/api/client/process/text', data=json.dumps({
    'unit': 'unit_name',
    'doc': 'small cell'
}))

saga_response = response.json()

qpl_tree = parser._parse(data=saga_response['highestRoute'])
print(qpl_tree.pretty())

As show below the query is has now transformed the tagged text with the new structure, which is later parsed. On the left you can see how the original query without special case would look, and on the right the same query with the special case applied