Advanced Pattern Stage

Matches advanced recursive patterns of tokens and semantic tags. Pattern databases can be very large (millions) of entries.The new patterns can be formed by previously defined tags and text literals. For example, the pattern "{name} likes {product}" where "{name}" and "{product}" are tags and "likes" is the text literal. Since this stage allows for complex entity tagging, is also know as the "Advance Recognizer".

Operates On:  Lexical Items with either TOKEN or SEMANTIC_TAG.

Configuration Parameters

• patterns ( type=string array | required ) - The resource which contains the pattern database

$action.getHelper().renderConfluenceMacro("$codeS$body$codeE")

Example Output

The following shows sample output from the advanced pattern matcher, which has multiple patterns for the {product} and {person-product-preference} semantic tags.

 V--------------------[Abe Lincoln likes the iPhone-7]--------------------V 
 ^---[Abe]----V--[Lincoln]--V--[likes]--V--[the]--V------[iPhone-7]-------^ 
                                                  ^---[iPhone]----V--[7]--^
 ^---[abe]----^--[lincoln]--^                     ^---[iphone]----^
 ^--[{name}]--^--[{place}]--^           ^-----------[{product}]-----------^
                                        ^-------[{product}]-------^
 ^---------[{name}]---------^                     ^--[{product}]--^
 ^--------[{place}]---------^                     ^------[iphone-7]-------^
                                                  ^------[{product}]------^
 ^-----------------[{person-product-preference}]------------------^
 ^---------------------[{person-product-preference}]----------------------^

Output Flags

Lexical Item Flags:

• SEMANTIC_TAG - Identifies all lexical items which are semantic tags.
• PROCESSED - Placed on all tokens that compose the semantic tag.

Vertex Flags:

Resource Data

The resource data will be a database of advanced patterns, and the resulting semantic tag that they produce.

Resource Format

The pattern database is a series of JSON records, typically indexed by "pattern block ID". Each JSON record represents a block of patterns (one or more) that all produce the same semantic tag. The format is as follows: $action.getHelper().renderConfluenceMacro("$codeS$body$codeE")

Fields

• id ( type=string | required ) - Identifies the entry. This identifier may not be unique, and is not the id of the database, but an specific use case id
  
  • Typically, this is an identifier with meaning to the larger application that is using the Language Processing Toolkit.
• display ( type=string | required ) - What to show the user when browsing this entity
• tag ( type=string | required ) - Tag which will identify any match in the graph, as an interpretation
  • These will all be added to the interpretation graph with the SEMANTIC_TAG flag.

  • Tags are hierarchical representations of the same intent. For example, {city} → {administrative-area} → {geographical-area}

• pattern ( type=string | required ) - Pattern to match in the content

  • Currently, tokens are separated on simple white-space and punctuation, and then reduced to lowercase.

• confidence ( type=boolean | required ) - Specifies the confidence level of the entry, independent of any patterns matched
  • This is the confidence of the entry, in comparison to all of the other entries. (Essentially, the likelihood that this entity will be randomly encountered.)
• fields ( type=json | default=empty json | optional ) - Additional information for the entry. At the moment this information is not expected
• updatedAt ( type=date epoch | required ) - Date in milliseconds of the last time the entry was updated
• createdAt ( type=date epoch | required ) - Date in milliseconds of the creation time of the entry