Creates a new token from the original but excluding the "characters on the "characters" list parameter.
Excerpt |
---|
|
Creates a new token from the original but excluding the " characters on the "characters" list parameter. |
...
Operates On: Lexical Items with TOKEN and possibly other flags as specified below.
Include Page |
---|
| Generic Configuration Parameters |
---|
| Generic Configuration Parameters |
---|
|
Configuration Parameters
- characters (string) -
Text containing all the character that will be collapse | default | _‿⁀⁔︳︴﹍﹎﹏_ |
---|
name | characters |
---|
|
- Each character in this text will be omitted in the new lexical item created
- caseSensitive (boolean, optional) -
Indicates if the characters to collapse are case |
|
sensitive ( = )
...
Saga_config_stage |
---|
boundaryFlags | text block split |
---|
requiredFlags | token |
---|
|
" |
language | js |
---|
theme | Eclipse |
---|
title | Example Configuration |
---|
|
{
"type":"CharacterCollapser",
"characters":";-!:",
"caseSensitivecaseInsensitive": true
}false |
Example Output
Description
Saga_graph |
---|
Code Block |
---|
language | text |
---|
theme | FadeToGrey |
---|
|
V---------------[e-5s Hello World! We need: one, two, three]---------------V
^-[e-5s]-V-[Hello]-V-[World!]-V-[We]-V-[need:]-V-[one,]-V-[two,]-V-[three]-^
^-[e5s]--^-[hello]-^-[world!]-^-[we]-^-[need]--^
^-[World]--^
^-[world]--^ |
Output Flags
Lex-Item Flags:
- COLLAPSED - Identifies all lexical items which were created from committing characters.
- TOKEN - All tokens produced are tagged as TOKEN
Info |
---|
In version 1.2.2 these flags were added: - ALL_PUNCTUATION - Tokens processed or produced composed only of punctuation characters are tagged as ALL_PUNCTUATION.
- HAS_DIGIT - Tokens produced with at least one digit character are tagged as HAS_DIGIT.
- HAS_PUNCTUATION - Tokens produced with at least one punctuation character are tagged as HAS_PUNCTUATION. (ALL_PUNCTUATION will not be tagged as HAS_PUNCTUATION).
- ALL_DIGITS - All characters in the token are digits
- HAS_LETTER - At least one character is a letter
- ALL_LETTERS - All characters in the token are letters
|
Vertex Flags:
Info |
---|
No vertices are created in this stage |