Excerpt |
---|
Breaks a text block into sentences, using OpenNLP Sentence Detector |
Operates On: Lexical Items with TEXT_BLOCK and possibly other flags as specified below.
Library: saga-sentence-breaker-stage
Include Page |
---|
| Generic Configuration Parameters |
---|
| Generic Configuration Parameters |
---|
|
Configuration Parameters
- language (string, required) - Indicates the language of the text to process. default "en" (i.e. English)
- Languages available areinclude:
- English - "en"
- German - "de"
- Portuguese - "pt"
- Dutch - "nl"
- skipFlags (string array, optional) - Flags to be skipped by this stage
- Tokens marked with this flags will be ignore by this stage, and no process will be performed.
- requiredFlags (string array, optional)
- Tokens need to have all the specified flags, in order to be processed
- debug (boolean, optional)
- Enable all debug log functionality of the stage, if any.
Code Block |
---|
language | js |
---|
theme | Eclipse |
---|
title | Example Configuration |
---|
|
{
"type":"SentenceBreakerStage",
"language":"en",
"skipFlags": ["SKIP"],
"debug": true,
} |
Example Output
The sentence detector will remove any space spaces and dot, dots at the beginning and end of a sentence.
Code Block |
---|
language | text |
---|
theme | FadeToGrey |
---|
|
V-----------[Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed eget luctus lorem. Nam cursus eleifend efficitur. Cras nec ultricies nulla. Maecenas porta cursus massa non consectetur.]------------V
^--[Lorem ipsum dolor sit amet, consectetur adipiscing elit]--V--[Sed eget luctus lorem]--V--[Nam cursus eleifend efficitur]--V--[Cras nec ultricies nulla]--V--[Maecenas porta cursus massa non consectetur]--^
|
Output Flags
Lex-Item Flags
...
- TEXT_BLOCK - Flags all of the text blocks produced by the SimpleReader
- SENTENCE - Every text block processed by this breaker will be marked as SENTENCE
Vertex Flags
...
- SENTENCE_SPLIT - Indicates the split (start/end) between sentences