Page History

Operates On:  Lexical Items with TOKEN and possibly other flags as specified below.

Configuration Parameters

• Parameter
  summary Minimum number of tokens for ngram default 2 name min type integer

• Parameter
  summary Maximum of tokens for ngram default 2 name max type integer

• Parameter
  summary Split the N-Gram if any of the next token has one of this flags. (i.e. if a token has any of this flags, stops the building of the N-Gram) name splitFlags type string array

• Parameter
  summary Specifies flags for token that will not be added to an N-Gram default cheese name ignoreOnBoundaryFlags type string array

"min": 2,
"max": 3,
"splitFlags": [],
"ignoreOnBoundaryFlags": [] 

Example Output

V--------------[abraham lincoln likes macaroni and cheese]--------------------V
V 
 ^--[abraham]--V--[lincoln]--V--[likes]--V--[macaroni]--V--[and]--V--[cheese]--^ 
 ^---[abraham             lincoln]---^---[likes macaroni]---^---{place}[and cheese]---^ 
             ^----{food}-[lincoln likes]---^---[macaroni and]---^            
 ^---{food}---^
^[abraham lincoln likes]-----^------{person}----[macaroni and cheese]-----^ 
                         ^-----[likes macaroni and]-----^            
             ^-------{food}---------[lincoln likes macaroni]-----^  

Output Flags

Lex-Item Flags:

• SEMANTIC_TAG - Identifies all lexical items which are semantic tags.
• ALL_LOWER_CASE - All of the characters in the token are lower-case characters.
• ALL_UPPER_CASE - All of the characters in the token are upper-case characters (for example, acronyms).
• MIXED_CASE - Handles any mixed upper & lower case scenario not covered above.
• TOKEN - All tokens produced are tagged as TOKEN 
• NGRAM - All tokens produced are tagged as NGRAM

Vertex Flags: