Excerpt |
---|
Analyzes the case of every token (adds additional flags) and optionally |
...
Configuration Parameters
adds a lower case normalized token to the interpretation graph. |
Operates On: Lexical Items with TOKEN and possibly other flags as specified below.
Include Page |
---|
| Generic Configuration Parameters |
---|
| Generic Configuration Parameters |
---|
|
Configuration Parameters
- normalize (boolean) -
If true, all tokens are normalized to all lower case |
|
. ( = )
...
...
...
...
Code Block |
---|
Example Configuration | {
"normalize":false
} |
...
Example Output
Code Block |
---|
|
V--[Sentence]--V--[with]--V--[UPPER]--V--[and]--V--[LoWer]--V--[Words!]--V
^--[sentence]--^ ^--[upper]--^ ^--[lower]--^--[words!]--^ |
Output Flags
Note that flags are placed on every token as appropriate to the current token:
- If the token is "President" then that token will have the TITLE_CASE flag.
- If a new lower case normalized interpretation, "president" is added as an alternative; this alternative will have the "ALL_LOWER_CASE" flag.
Lex-Item Flags:
- ALL_LOWER_CASE - All of the characters in the token are lower - case characters.
- ALL_UPPER_CASE - All of the characters in the token are upper - case characters (for example, acronyms).
- TITLE_CASE - The first character is upper case, ; all of the other characters are lower case.
- MIXED_CASE - Handles any mixed upper & lower case scenario not covered above.
- LOWERED - Applies only to tokens that were changed to all lower case; preexisting tokens with all lower case will not be marked with this flag.
- TOKEN - Identifies that the Lex-Items produced by this stage are tokens and not text blocks.
Vertex Flags:
...
Info |
---|
No vertices are created in this stage |