Page History
| Excerpt |
|---|
This stage works in a similar way to the Dictionary Tagger stage in the sense that looks up sequences of tokens in a dictionary to match the text being processed. The difference is that it will also include in the matching text N tokens to the right and/or left of the original matched text. |
Operates On: Lexical Items with TOKEN and possibly other flags as specified below.
Saga_is_recognizer
| Include Page | ||||
|---|---|---|---|---|
|
Configuration Parameters
Parameter summary When true, the recognizer will create internal tags to tag the left part, the matched text and the right part default false name groupTokens type boolean Parameter summary Internal Tag name for the tokens at the left of the matched text default _leftTokens_ name leftGroupTagName Parameter summary Internal Tag name for the matched text portion default _matchedTokens_ name matchGroupTagName Parameter summary Internal Tag name for the tokens at the right of the matched text default _rightTokens_ name rightGroupTagName
| Saga_config_stagecode | ||||||
|---|---|---|---|---|---|---|
| ||||||
"dictionary": "dict-provider:token_matcher_patterns", "groupTokens": false, "leftGroupTagName": "_leftTokens_", "matchGroupTagName": "_matchedTokens_", "rightGroupTagName": "_rightTokens_" |
Example Output
| Code Block | ||
|---|---|---|
| ||
V-----------------[call 333-4444 for pizza]------------------V
^------[call]------V-----[333-4444]------V-[for]-V--[pizza]--^
^-[{_leftTokens_}]-^---[{phonenumber}]---^-[{_rightTokens_}]-^
^-[{_matchedTokens_}]-^
^----------------------[{pizza_phone}]-----------------------^ |
Output Flags
Lex-Item Flags:
- SEMANTIC_TAG - Identifies all lexical items which are semantic tags.
- TOKEN_MATCHER - Identifies a semantic tag that covers both the matched text and the tokens to the right and/or left
- TOKEN_MATCHER_GROUP - Identifies the semantic tags used to cover the tokens grouped when using the "groupTokens" configuration
Vertex Flags:
| Info |
|---|
No vertices are created in this stage |
Resource Data
Token matcher must have a dictionary of patterns which is a list of JSON records.
Resource Format
| Saga_jsoncode | ||||
|---|---|---|---|---|
| ||||
"_id" : "Nlp3kXYBOOvNPbzJXDcQ",
"tag": "qVRZi3YBvyJs83wnkfik",
"pattern": "{phonenumber}",
"confAdjust": 0.95,
"options" : {
"maxTokensLeft": 1,
"maxTokensRight": 1
} |
| Note |
|---|
|
Fields
Parameter summary Tag which will identify any match in the graph, as an interpretation name tag required true These will all be added to the interpretation graph with the SEMANTIC_TAG flag.
Tip Tags are hierarchical representations of the same intent. For example, {city} → {administrative-area} → {geographical-area}
Parameter summary Pattern to match in the content name pattern required true - Options
Parameter summary Number of tokens to take to the left of the matched text default 0 name maxTokensLeft type integer Parameter summary Number of tokens to take a the right of the matched text default 3 name maxTokensRight type integer
| Include Page | ||||
|---|---|---|---|---|
|
Overview
Content Tools