This stage uses Apache Lucene™ to create a custom Lucene pipeline. It offers a large amount of possible tokenizers and filters to adapt to the users needs.
Operates On: Lexical Items with TOKEN and possibly other flags as specified below.
A Lucene Custom Analyzer is composed of two components: the Tokenizer and the Filters (which can be stacked to use more than one at a time).
"atLeastOneFlag": [] "boundaryFlags": [] "confidenceAdjustment": 1 "debug": false "requiredFlags": [] "skipFlags": [] "tokenizer": "whitespace", "filter": None
Using Whitespace Tokenizer alone
V-------------[Hey there! I am using Lucene Pipeline]-------------V ^-[Hey]-V-[there!]-V-[I]-V-[am]-V-[using]-V-[Lucene]-V-[Pipeline]-^
No vertices are created in this stage