Embedded Synonym Expansion

The embedded synonym expansion feature utilizes a function provided in the QPLOptions called synonyms_call. This function is responsible for receiving a string (either a token or phrase) and returning either a list of string (tokens and/or phrases) or None if no synonyms are found.

To implement this feature, you have the freedom to define the synonyms_call function as long as it adheres to the specified signature. This means it can read from a document, database, call a third-party API or have a manual list of synonyms from which look for the synonyms and return them.

Example Implementation

Have a source from which to look for the synonyms, the simplest way to do this is by providing a dictionary, where the key is the text to look for and the value is the list of synonyms

synonyms = {
    'cancer': ['cancer', 'malignancy', '363346000', 'cancers', 'malignancies', '"malignant growth"',
               '"malignant neoplasm"', '"malignant neoplasms"', '"malignant neoplastic disease"',
               '"malignant tumor"', '"malignant tumors"', '"neoplasm malignant"', '"neoplasm/cancer"',
               '"tumor, malignant"'],
    'headache': ['headache', '25064002', 'cephalalgia', 'cephalgia', 'cephalgias', '"cranial pain"',
                 '"have headaches"', '"head ache"', '"head pain"', '"head pain cephalgia"', '"head pains"',
                 'headaches', '"mild global headache"', '"mild headache"', '"pain head"', '"pain in head"',
                 '"pain, head"']
}

Then create a function and assign it to synonyms_call, going with the simplest example we declared a lambda function which receives an argument (the string) and use it as key for the dictionary returning None by default

options = QPLOptions(fields='content', implicit_operator='or')
options.synonyms_call = lambda x: synonyms.get(x, None)

parser = QPLParser(options=options)

Execute the parsing of our query and print the qpl_tree

qpl_tree = parser._parse(data='cancer NOT headache')
print(qpl_tree.pretty())

As show below the query is has now being expanded with the synonyms found, for each of the tokens, on the left you can see how the original query without expansion would look, and on the right the same query with the synonym expansion applied

Saga Synonym Expansion

The synonym expansion with Saga uses the entity extraction from Saga to expand the matches with the patterns provided in each entity. To make use of this functionality just add the tags you want to expand into saga_synonyms, Saga will look for LexItems with these tags, and make the expansion on them.

Additionally you can specify a custom boots for the synonyms using saga_synonyms_boost, the original token will be left untouched

In Saga you must have a tag that once matched, the result should be something like the one below, the important bits from it are:

• entities/patterns: This is where Saga QPL will look for the patterns and use them as synonyms for the expansion

{
  "stage": "DictionaryTagger",
  "confidence": 1,
  "match": "cancer",
  "flags": [
    "ENTITY",
    "SEMANTIC_TAG"
  ],
  "text": "{synonyms}",
  "startPos": 0,
  "endPos": 6,
  "metadata": {
    "display": "cancer",
    "id": "syn0000000141"
  },
  "entities": [
    {
      "display": "cancer",
      "patterns": [
        "malignancy",
        "cancer"
      ],
      "id": "syn0000000141",
      "fields": {},
      "tags": [
        "synonym",
        "synonyms"
      ]
    },
    {
      "display": "cancer",
      "patterns": [
        "363346000",
        "cancer",
        "cancers",
        "malignancies",
        "malignancy",
        "malignant growth",
        "malignant neoplasm",
        "malignant neoplasms",
        "malignant neoplastic disease",
        "malignant tumor",
        "malignant tumors",
        "neoplasm malignant",
        "neoplasm/cancer",
        "tumor, malignant"
      ],
      "id": "363346000",
      "tags": [
        "snomed",
        "synonyms"
      ]
    }
  ],
  "tags": [
    "snomed",
    "synonym",
    "synonyms"
  ]
}

Example Implementation

Use SagaQPLOptions instead of QPLOptions, and assign saga_synonyms with the list of tags to expand as synonymns, optionally you can specify a custom boost for the synonyms

options = SagaQPLOptions(
    implicit_operator='and',
    fields={'content': 6, 'title': 3}
    saga_synonyms=['synonyms'],
	saga_synonyms_boost=0.8
)

parser = SagaParser(options=options)

Get the Saga response with whatever method you see fit, the simplest method is to make an http request to Saga Client API

import requests
import json

response = requests.get('http://localhost:8080/saga/api/client/process/text', data=json.dumps({
    'unit': 'unit_name',
    'doc': 'cancer not headache'
}))

saga_response = response.json()

Execute the parsing of the entire Saga response and print the qpl_tree

qpl_tree = parser._parse(data=saga_response['highestRoute'])
print(qpl_tree.pretty())

As show below the query is has now being expanded with the synonyms found, for each of the tokens, on the left you can see how the original query without expansion would look, and on the right the same query with the synonym expansion applied