Page History

...

Readers read text streams and create text blocks to process.

• SimpleReaderSimple Reader - 

  Excerpt Include
  Simple Reader Stage Simple Reader Stage nopanel true

...

Breakers read text blocks and break them into individual text blocks.

• QuotationBreaker Quotation Breaker - 

  Excerpt Include
  Quotation Breaker Stage Quotation Breaker Stage nopanel true

• Text Breaker Stage - 

  Excerpt Include
  Text Breaker Stage Text Breaker Stage nopanel true

...

Tokenizers read text blocks and divide them up into individual tokens to be processed.

• WhitespaceTokenizerWhitespace Tokenizer - 

  Excerpt Include
  Whitespace Tokenizer Stage Whitespace Tokenizer Stage nopanel true

...

Splitters split up tokens into multiple smaller tokens as an alternative interpretation.

• CharacterSplitterCharacter Splitter - 

  Excerpt Include
  Character Splitter Stage Character Splitter Stage nopanel true

• CharChangeSplitter Char Change Splitter - 

  Excerpt Include
  Character Change Splitter Stage Character Change Splitter Stage nopanel true

Collapsers

Collapsers reduce tokens into simpler smaller tokens as an alternative interpretation.

• Character Collapser - 

  Excerpt Include
  Character Collapser Stage Character Collapser Stage nopanel true

Normalizers

Normalizers create alternative normalized interpretations of tokens from original tokens.

• CaseAnalysisCase Analysis - 

  Excerpt Include
  Case Analysis Stage Case Analysis Stage nopanel true

• Synonym - 

  Excerpt Include
  Synonym Stage Synonym Stage nopanel true

...

Recognizers identify and flag tokens based on their character patterns.

• NumberRecognizerNumber Recognizer - 

  Excerpt Include
  Number Recognizer Stage Number Recognizer Stage nopanel true

• StopWords Stop Words -

  Excerpt Include
  Stop Words Stage Stop Words Stage nopanel true

• Lemmatize -

  Excerpt Include
  Lemmatize Stage Lemmatize Stage nopanel true

• ABA - 

  Excerpt Include
  ABA Stage ABA Stage nopanel true

• BIC - 

  Excerpt Include
  BIC Stage BIC Stage nopanel true

...

Taggers create semantic tags which are added to the interpretation graph as alternative interpretations.

• RegexPattern Regex Pattern - 

  Excerpt Include
  Regex Pattern Stage Regex Pattern Stage nopanel true

• DictionaryTagger Dictionary Tagger - 

  Excerpt Include
  Dictionary Tagger Stage Dictionary Tagger Stage nopanel true

• AdvancedPattern Advanced Pattern - 

  Excerpt Include
  Advanced Pattern Stage Advanced Pattern Stage nopanel true

• Fragmentation - 

  Excerpt Include
  Fragmentation Stage Fragmentation Stage nopanel true

...

Transformers generates tags, not of semantic nature, but with new data for later use

• Bag Of Words- 

  Excerpt Include
  Bag Of Words Stage Bag Of Words Stage nopanel true

• Best Bets - 

  Excerpt Include
  Best Bets Stage Best Bets Stage nopanel true

  
  

...

Producers create consumable output based on the processed graph.

• JsonProducer Json Producer - 

  Excerpt Include
  JSON Producer Stage JSON Producer Stage nopanel true

• MarkupProducer Markup Producer - 

  Excerpt Include
  Markup Producer Stage Markup Producer Stage nopanel true

...