Page History

Operates On:  Lexical Items with TOKEN and possibly other flags as specified below.

Configuration Parameters

• Parameter
  summary Minimum number of tokens for ngram default 2 name min type integer

• Parameter
  summary Maximum of tokens for ngram default 2 name max type integer

• Parameter
  summary Indicates additional flags of tokens to slip into ngramsSplit the N-Gram if any of the next token has one of this flags. (i.e. if a token has any of this flags, stops the building of the N-Gram) name splitFlags type string array

• Parameter
  summary Specifies flags for token that will not be break into ngramadded to an N-Gram default cheese name ignoreOnBoundaryFlags type string array

"min": 2,
"max": 3,
"splitFlags": [],
"ignoreOnBoundaryFlags": [] 

Example Output

V--------------[abraham lincoln likes macaroni and cheese]--------------------V
^--[abraham]--V--[lincoln]--V--[likes]--V--[macaroni]--V--[and]--V--[cheese]--^
              ^---{place}---^           ^----{food}----^         ^---{food}---^
^----------{person}---------^           ^-----------------{food}--------------^

Output Flags

Lex-Item Flags:

• SEMANTIC_TAG - Identifies all lexical items which are semantic tags.
• ALL_LOWER_CASE - All of the characters in the token are lower-case characters.
• ALL_UPPER_CASE - All of the characters in the token are upper-case characters (for example, acronyms).
• MIXED_CASE - Handles any mixed upper & lower case scenario not covered above.
• TOKEN - All tokens produced are tagged as TOKEN 
• NGRAM - All tokens produced are tagged as NGRAM

Vertex Flags: