You are viewing an old version of this page. View the current version.
Compare with Current
View Page History
« Previous
Version 7
Next »
Reads a text stream and outputs a list of text blocks.
Operates On: Text Streams
Configuration Parameters
- splitRegex (string, required) - A Java Pattern which identifies where a block should be split.
{
"splitRegex":"\s*\n\n\s*"
}
(split blocks on two new-lines)
Flags
Lex-Item Flags:
- TEXT_BLOCK - Flags all text blocks produced by the SimpleReader
Vertex Flags:
- TEXT_BLOCK_SPLIT - Identifies the vertex as a split between text blocks.
- OVERFLOW_SPLIT - Identifies that an entire buffer was read without finding a split between text blocks.
- The current maximum size of a text block is 64K characters.
- Text blocks larger than this will be arbitrarily split, and the vertex will be marked with "OVERFLOW_SPLIT"\