Sentence Breaker Stage

Breaks a text block into sentences, using OpenNLP Sentence DetectorOperates On:  Lexical Items with TEXT_BLOCK and possibly other flags as specified below.

Configuration Parameters

• language (string, required) - Indicates the language of the text to process. default "en" (i.e. English)
  • Languages available are
    • English - "en"
    • German - "de"
    • Portuguese - "pt"
    • Dutch - "nl"
• skipFlags (string array, optional) - Flags to be skipped by this stage
  • Tokens marked with this flags will be ignore by this stage, and no process will be performed.
• requiredFlags (string array, optional)
  • Tokens need to have all the specified flags, in order to be processed
• debug (boolean, optional)
  • Enable all debug log functionality of the stage, if any.

{
 "type":"SentenceBreakerStage",
 "language":"en",
 "skipFlags": ["SKIP"],
 "debug": true,
}

Example Output

The sentence detector will remove any space and dot, at the beginning and end of a sentence

V-----------[Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed eget luctus lorem. Nam cursus eleifend efficitur. Cras nec ultricies nulla. Maecenas porta cursus massa non consectetur.]------------V
^--[Lorem ipsum dolor sit amet, consectetur adipiscing elit]--V--[Sed eget luctus lorem]--V--[Nam cursus eleifend efficitur]--V--[Cras nec ultricies nulla]--V--[Maecenas porta cursus massa non consectetur]--^

Output Flags

Lex-Item Flags:

• TEXT_BLOCK - Flags all text blocks produced by the SimpleReader
• SENTENCE - Every text block processed by this breaker will be marked as SENTENCE

Vertex Flags:

• SENTENCE_SPLIT - Indicates the split (start/end) between sentences