Structure of an Language Processing Toolkit Program

The overall structure of an Saga program is shown in the diagram below:

An Saga Engine is a Pipeline of Text Processing Stages

• The first stage in the pipeline is a "reader"
  • This reads raw text from a text stream and returns it as text blocks to be processed by the stages
• Then there are a list of pipeline stages
  • Each stage takes an "interpretation graph" and extends it
• The result is the final interpretation graph
  • With text blocks, tokens, flags, and semantic tags

Notes:

1. It is a "pull" architecture
   1. Content is pulled from the last stage, which fetches content from the previous stage, etc. all the way up to the reader.
2. SagaEngine is single-threaded
   1. If you want to process text with multiple threads, you will need to create multiple SagaEngine objects
3. The order of the stages matter
   1. Different orders will result in different results

What is an 'interpretation graph' ?