Page History

...

• COLLAPSED - Identifies all lexical items which were created from committing characters.
• TOKEN - All tokens produced are tagged as TOKEN

...

• .

...

• ALL_PUNCTUATION - Tokens processed or produced composed only of punctuation characters are tagged as ALL_PUNCTUATION.
• HAS_DIGIT - Tokens produced with at least one digit character are tagged as HAS_DIGIT. 
• HAS_PUNCTUATION - Tokens produced with at least one punctuation character are tagged as HAS_PUNCTUATION. (ALL_PUNCTUATION will not be tagged as HAS_PUNCTUATION).
• ALL_DIGITS -  All characters in the token are digits.
• HAS_LETTER - At least one character is a letter.
• ALL_LETTERS - All characters in the token are

...

• letters.

Vertex Flags: