Saga Library processes text through a pipeline of text processing stages.  A typical pipeline consists of the following sets of stages:

  • Text Block Reader - Reads a text stream (for example, text from a file, a tweet, or from the user)
    There can only be one and it must be first.

  • Tokenization - Reads text blocks and divides them into individual tokens
    Typically only one and usually comes second.

  • Splitters - Divide up tokens, usually based on punctuation
    This is not part of the tokenizer because sometimes you want to process the tokens with intervening punctuation (for example, floating point numbers, things with dashes like "e-mail")

  • Recognizers - Recognizes and flags various types of tokens,  For example, numbers or things with all punctuation

  • Normalizers - Does character normalization on tokens, typically adding lower-case versions of any word with upper-case characters

  • Taggers - Adds semantic tags for things like entities and sentence interpretations.



The pipeline can be specified in a JSON format which can be stored in a resource (see Resources).  A sample is shown below: $action.getHelper().renderConfluenceMacro("$codeS$body$codeE")

Structure

There are two sections to the pipeline configuration:

  • "reader"
    Contains the configuration for the text reader (the first stage which reads the text stream and converts it into text blocks to be processed)

  • "stages"
    Contains a list of pipeline stages, each of which may have its own configuration.

Stages

Stage configurations are documented with each pipeline stage.

The "type"

The "type" field specifies the Java class which is the pipeline stage. This can be:

  • A fully qualified java package and class name
    For example:  com.accenture.saga.engine.stages.CaseAnalysisStage

  • A simple stage name without "Stage" at the end
    When this occurs, the Saga Library will automatically look in the "com.accenture.saga.engine.stages" package for a class with the same name and with "Stage" appended to the end. 
    For example:  "DictionaryTagger" will automatically look for "com.accenture.saga.engine.stages.DictionaryTaggerStage"

  • Any other class name (without the java package)
    Will automatically look for the class in the "com.accenture.saga.engine.stages" package

  • No labels