Page History

...

Operates On:  Lexical Items with TOKEN and possibly other flags as specified below.

...

• Parameter
  summary Text containing all the character that will be collapse default _‿⁀⁔︳︴﹍﹎﹏＿ name characters

  • Each character in this text will be omitted in the new lexical item created

• Parameter
  summary Indicates if the characters to collapse are case insensitive default false name caseInsensitive

"characters":";-!:",
"caseInsensitive": false

Example Output

 V---------------[e-5s Hello World! We need: one, two, three]---------------V 
 ^-[e-5s]-V-[Hello]-V-[World!]-V-[We]-V-[need:]-V-[one,]-V-[two,]-V-[three]-^ 
 ^-[e5s]--^-[hello]-^-[world!]-^-[we]-^-[need]--^ 
                    ^-[World]--^ 
                    ^-[world]--^ 

Output Flags

Lex-Item Flags:

• COLLAPSED - Identifies all lexical items which were created from committing characters.
• TOKEN - All tokens produced are tagged as TOKEN.
• ALL_PUNCTUATION - Tokens processed or produced composed only of punctuation characters are tagged as ALL_PUNCTUATION.
• HAS_DIGIT - Tokens produced with at least one digit character are tagged as HAS_DIGIT. 
• HAS_PUNCTUATION - Tokens produced with at least one punctuation character are tagged as HAS_PUNCTUATION. (ALL_PUNCTUATION will not be tagged as HAS_PUNCTUATION).
• ALL_DIGITS -  All characters in the token are digits.
• HAS_LETTER - At least one character is a letter.
• ALL_LETTERS - All characters in the token are letters.

Vertex Flags: