Page History

Configuration Parameters

• Parameter
  summary The dictionary resource that holds the names and that is to be located in the text name dictionary required true

  • This is specified as "provider:name" in the standard resource format.

• Parameter
  summary Ignore matches with tags specified in the ignoreTags list name ignoreTags type string array

• Parameter
  summary Name of the tag which indicates what entities should be process name entity

  
  

• Parameter
  summary Removes accents and diacritics and generates a new pattern default false name normalizeAccents

• Parameter
  summary Indicates if characters should be removed from the pattern using a list creating a new pattern default false name removeChars type boolean

• Parameter
  summary List of characters to remove from the pattern default _-‿⁀⁔︳︴﹍﹎﹏＿ name charsList

"dictionary": "dict-provider:people-lowercase",
"ignoreTags": ["color", "currency"],
"normalizeAccents": false,
"removeChars": false,
"charsList": "_-‿⁀⁔︳︴﹍﹎﹏＿"

Example Output

In the following example, "abraham lincoln" is in the dictionary as a person, "lincoln" as a place,  and "macaroni", "cheese" and "macaroni and cheese" are all specified as foods:

V--------------[abraham lincoln likes macaroni and cheese]--------------------V
^--[abraham]--V--[lincoln]--V--[likes]--V--[macaroni]--V--[and]--V--[cheese]--^
              ^--[{place}]--^           ^---[{food}]---^         ^--[{food}]--^
^---------[{person}]--------^           ^----------------[{food}]-------------^

Output Flags

Lex-Item Flags

• SEMANTIC_TAG - Identifies all lexical items that are semantic tags.
• ENTITY - Identifies the token as an entity.
• PROCESSED - Placed on all the tokens which composed the semantic tag.

Vertex Flags:

Resource Data

The dictionary tagger must have an "entity dictionary" (a string to JSON map) which is a list of JSON records, indexed by entity ID. In addition, there may also be a pattern map and a token index.

Entity Dictionary Format

The only file that is absolutely required is the entity dictionary. It is a series of JSON records, typically indexed by entity ID.

Each JSON record represents an entity. The format is as follows:

"_id" : "KGAAJGsBemSwA0nZTLXA",
"id":"Q28260",
"tag":"{city}",
"display": "Lincon"
"patterns":[
  "Lincoln", "Lincoln, Nebraska", "Lincoln, NE"
],
"fields": {
  "coord": [40.813639, -96.702611]
}
"confAdjust": 0.95
. . . additional fields as needed go here . . . 

Fields

• Parameter
  summary An ID normally refering the ID of a database, a document, an API key, not necessary unique name id required true

• Parameter
  summary Tag which will identify any match in the graph, as an interpretation name tag required true

  
  

  • These will all be added to the interpretation graph with the SEMANTIC_TAG flag.

    Tip
    Tags are hierarchical representations of the same intent. For example, {city} → {administrative-area} → {geographical-area}

• Parameter
  summary A list of patterns to match in the content name patterns type string array required true

  • Patterns will be tokenized and there may be multiple variations which can match.

    Note
    Currently, tokens are separated on simple white-space and punctuation, and then reduced to lowercase.

• Parameter
  summary What to show the user when browsing this entity name display

  
  

• Parameter
  summary Free space to add extra data in any format supported by JSON name fields type json

Dictionary Index

To improve performance especially for every large databases of entities, the entity dictionary is inverted and indexed.

This currently happens in RAM inside the DictionaryTagger stage. An off-line option for pre-inverting the dictionary will be provided in the future.