Page History

Operates On:  Lexical Items with TOKEN and possibly other flags as specified below.

Configuration Parameters

• Parameter
  summary The resource containing the list of words and relationships name dictionary

  • If no dictionary is provided, a built-in dictionary will be used, based on the languageISO3.

• Parameter
  summary A list of the relationships to include name include type string array

• Parameter
  summary A list of the relationships to exclude name exclude type string array

• Parameter
  summary The language the lemmatize stage should use. The value needs to be one of the ISO 3 letter language codes name languageISO3

  • By default, English is always used unless configured otherwise. At the moment, only English (ENG) and Spanish (SPA) are available.
    
    

"include" : ["pl", "vf"],
"exclude" : ["ob"],
"dictionary" : "lemmatize-provider:lemmatize_words",
"languageISO3":"SPA"

Example Output

V--------------------[I am liking this projects very much]--------------------V  
^--[I]--V--[am]--V--[liking]--V--[this]--V--[projects]--V--[very]--V--[much]--^  
        ^--[be]--^---[like]---^          ^--[project]---^  

Output Flags

Lex-Item Flags:

• LEMMATIZE- All words retrieved will be marked as LEMMATIZE.
• ALL_LOWER_CASE - All of the characters in the token are lower-case characters.
• TOKEN - This stage creates a new token.

Vertex Flags:

Resource Data

Anchor
Dictionary Dictionary

The resource data will be a json file with an array of words in a field named words. This is when the 'dictionary' parameter is used.

"words": [
  {
    "confidence": 0.0049,
    "rel": [
      "wnm",
      "sp"
    ],
    "from": "encyclopaedia",
    "to": "encyclopedia"
  },
  {
    "confidence": 0.0752,
    "rel": [
      "wnm",
      "sp"
    ],
    "from": "word",
    "to": "worth"
  }
]

When the 'dictionary' parameter is not used, an embedded Wiktionary file will be used. This file is formatted as a 1 entry json per line:

{"confidence":0,"rel":["syn"],"from":"japonés","to":"nipón"}
{"confidence":0,"rel":["syn"],"from":"alemán","to":"germano"}
{"confidence":0,"rel":["syn"],"from":"alemán","to":"tedesco"}
{"confidence":0,"rel":["syn"],"from":"alemán","to":"teutón"}
{"confidence":0,"rel":["syn"],"from":"alemán","to":"gringo"}
{"confidence":0,"rel":["syn"],"from":"mayo","to":"guainica"}
{"confidence":0,"rel":["syn"],"from":"mayo","to":"maisito"}
{"confidence":0,"rel":["syn"],"from":"mayo","to":"mayito"}
{"confidence":0,"rel":["syn"],"from":"mayo","to":"turpial de sureste"}
{"confidence":0,"rel":["syn"],"from":"domingo","to":"paga"}

Relationships

Anchor
Relationships Relationships

The required fields for each entry are:

• from - Original word to search for.
  • This field will be eliminated after it is added to the entities of the LexItem.
• to - Resulting word. 
  • It will be a new LexItem on its own.
• rel - List of relationships between the original word and the resulting word.
  • List of relationships in the default dictionary:
    • pl - pluralization
    • vf - verb form
    • ob - obsolete
    • syn - synonym
    • alt - alternative
    • wwm - word with meaning (more than one)
    • wnm - word no meaning (no additional meaning)