Page History

Operates On:  Lexical Items with TEXT_BLOCK, EOF, others.

The Python Classification Watcher Stage is an implementation of the abstract class called Watcher Stage, this Stage is used to classify large blocks of texts as for example: complete paragraphs or whole documents. The Watcher saves the classification paths for every text block and then verifies the vertices to look for triggers which mark the end of the saving cycle and start to process the saved data, then returns a weight vector that would be the classification of the text block.

Configuration Parameters

• Parameter
  summary Model name registered in the python bridge name modelName required true

• Parameter
  summary Model version registered in the python wrapper to query default latest name modelVersion

• Parameter
  summary Model method to call for the model name modelMethod required true

• Parameter
  summary Python server communication hostname default localhost name hostname

• Parameter
  summary Python server communication port default 5000 name port

• Parameter
  summary Expected content for this model are tokensIf True, a collection of tokens is sent, otherwise a string is sent default false name sendTokens type boolean

• Parameter
  summary If True, sends the content as a list of the text blocks that were detected default false name sendTextBlockList type boolean

• Parameter
  summary If True, sends a node called "original" with the original text that was provided default false name sendOriginalText type boolean

• Parameter
  summary Include text of tokens flagged as vertices default false name includeVertexText type boolean

"dependencyTags": [],
"modelName": "bert-base-nli-stsb-mean-tokens",
"modelVersion": "1",
"modelMethod": "predict",
"normalizeTags": false,
"hostname": "localhost",
"port": 5000,
"sendTextBlockList": true,
"sendTokens": true,
"sendOriginalText": true,

The settings "sendTextBlockList", "sendTokens" and "sendOriginalText" are independent to each other.  This means that if the original input is:

"This is a good test

This is a bad test"

And we have a normalization tag called "sentiment" that matches "good" and "bad", the payloads will look as follows:

{
    "sendTextBlocks": true,
    "sendTokens": true,
    "sendOriginalText": true,
    "normalized": [
        ["this", "is", "a", "{sentiment}", "test"],
        ["this", "is", "a", "{sentiment}", "test"]
    ],
    "original": [
        ["this", "is", "a", "good", "test"],
        ["this", "is", "a", "bad", "test"]
    ]
}

{
    "sendTextBlocks": true,
    "sendTokens": true,
    "sendOriginalText": false,
    "normalized": [
        ["this", "is", "a", "{sentiment}", "test"],
        ["this", "is", "a", "{sentiment}", "test"]
    ]
}

{
    "sendTextBlocks": true,
    "sendTokens": false,
    "sendOriginalText": true,
    "normalized": [
        {"this is a {sentiment} test"},
        {"this is a {sentiment} test"}
    ],
    "original": [
        {"this is a good test"},
        {"this is a bad test"}
    ]
}

{
    "sendTextBlocks": true,
    "sendTokens": false,
    "sendOriginalText": false,
    "normalized": [
        {"this is a {sentiment} test"},
        {"this is a {sentiment} test"}
    ]
}

{
    "sendTextBlocks": false,
    "sendTokens": true,
    "sendOriginalText": true,
    "normalized": [
        ["this", "is", "a", "{sentiment}", "test", "this", "is", "a", "{sentiment}", "test"]
    ],
    "original": [
        ["this", "is", "a", "good", "test", "this", "is", "a", "bad", "test"]
    ]
}

{
    "sendTextBlocks": false,
    "sendTokens": true,
    "sendOriginalText": true,
    "normalized": [
        ["this", "is", "a", "{sentiment}", "test", "this", "is", "a", "{sentiment}", "test"]
    ]
}

{
    "sendTextBlocks": false,
    "sendTokens": false,
    "sendOriginalText": true,
    "normalized": [
        {"this is a {sentiment} test this is a {sentiment} test"}
    ],
    "original": [
        {"this is a good test this is a bad test"}
    ]
}

{
    "sendTextBlocks": false,
    "sendTokens": false,
    "sendOriginalText": false,
    "normalized": [
        {"this is a {sentiment} test this is a {sentiment} test"}
    ]
}

Example Output

The output of the Watcher Stage is at the metadata of the vertex flagged as the trigger, for this example it is the EOF but it could be configured to work with TEXT_BLOCK_SPLIT or any other flag.

V--------------[abraham lincoln likes macaroni and cheese]--------------------V <=== EOF, at this vertex's metadata is the embedded vector
^--[abraham]--V--[lincoln]--V--[likes]--V--[macaroni]--V--[and]--V--[cheese]--^
              ^---{place}---^           ^----{food}----^         ^---{food}---^
^----------{person}---------^           ^-----------------{food}--------------^

Output Flags

Lex-Item Flags:

• SEMANTIC_TAG - Identifies all lexical items that are semantic tags.
• ML_PREDICT- Result from a machine learning algorithm for prediction
• ML_CLASSIFY- Result from a machine learning algorithm for classification
• ML_REGRESS- Result from a machine learning algorithm for regression
• TEXT_BLOCK - Flags all text blocks.

Vertex Flags:

• WEIGHT_VECTOR - Identifies the text block related to this vertex as a weight vector representation