Looks up sequences of tokens in a dictionary and then tags the sequence with one or more semantic tags as an alternative representation(s). Note that all possibilities are tagged, including overlaps and sub-patterns, with the expectation that later disambiguation stages will choose which tags are the correct interpretation.
Operates On: Lexical Items with TOKEN and possibly other flags as specified below.
{ "dictionary":"dict-provider:people-lowercase", "withFlags":"TOKEN, LOWER" }
Note that the "people-lowercase" resource must be in the format as specified below.
In the following example, "abraham lincoln" is in the dictionary as a person, "lincoln" as a place, and "macaroni", "cheese" and "macaroni and cheese" are all specified as foods:
V--------------[abraham lincoln likes macaroni and cheese]--------------------V
^--[abraham]--V--[lincoln]--V--[likes]--V--[macaroni]--V--[and]--V--[cheese]--^
^---{place}---^ ^----{food}----^ ^---{food}---^
^----------{person}---------^ ^-----------------{food}--------------^
The dictionary format is under change... will be documented when complete.