Page History

Operates On:  Lexical Items with TOKEN and possibly other flags as specified below.

Configuration Parameters

• Parameter
  summary Minimum number of tokens for ngram default 2 name min type integer

• Parameter
  summary Maximum of tokens for ngram default 2 name max type integer

• Parameter
  summary Split the N-Gram if any of the next token has one of this flags. (i.e. if a token has any of this flags, stops the building of the N-Gram) name splitFlags type string array

• Parameter
  summary Specifies flags for token that will not be added to an N-Gram default cheese name ignoreOnBoundaryFlags type string array

"min": 2,
"max": 3,
"splitFlags": [],
"ignoreOnBoundaryFlags": [] 

Example Output

V--------------[abraham lincoln likes macaroni and cheese]--------------------V 
^--[abraham]--V--[lincoln]--V--[likes]--V--[macaroni]--V--[and]--V--[cheese]--^ 
^---[abraham              lincoln]---^---[likes macaroni]---^---{place}[and cheese]---^ 
            ^---[lincoln likes]---^-{food}---[macaroni and]---^            
^---{food}---^
^[abraham lincoln likes]-----^------{person}-[macaroni and cheese]--------^^ 
                        ^-----[likes macaroni and]-----^            
            ^-------{food}---[lincoln likes macaroni]-----------^^  

Output Flags

Image Added

Lex-Item Flags:

• SEMANTIC_TAG - Identifies all lexical items which are semantic tags.
• ALL_LOWER_CASE - All of the characters in the token are lower-case characters.
• ALL_UPPER_CASE - All of the characters in the token are upper-case characters (for example, acronyms).
• MIXED_CASE - Handles any mixed upper & lower case scenario not covered above.
• TOKEN - All tokens produced are tagged as TOKEN 
• NGRAM - All tokens produced are tagged as NGRAM

Vertex Flags: