Excerpt |
---|
Breaks TEXT_BLOCK tokens into other TEXT_BLOCK tokens separating the non quoted text from the quoted one. This breaker respects the grammatical rules grammatical rules of quotes. |
Operates On: Lexical Items with TEXT_BLOCK and possibly other flags as specified below.
...
- DOUBLE_QUOTE_BEGIN - Indicates the start of a quoted text with double quote, it also contains the double quote, and any whitespace next to it
- DOUBLE_QUOTE_END - Indicates the end of a quoted text with double quote, it also contains the double quote, and any whitespace next to it
- SINGLE_QUOTE_BEGIN - Indicates the start of a quoted text with single quote, it also contains the single quote, and any whitespace next to it
- SINGLE_QUOTE_END - Indicates the end of a quoted text with singlequote, it also contains the single quote, and any whitespace next to it
- TEXT_BLOCK_SPLIT - Indicates the split of the textblock.