Page History

Operates On:  Lexical Items with TOKEN and possibly other flags as specified below.

Configuration Parameters

• Parameter
  summary Minimum number of tokens for ngram default 2 name min type integer

• Parameter
  summary Maximum of tokens for ngram default 2 name max type integer

• Parameter
  summary Indicates additional flags of tokens to slip into ngrams name splitFlags type string array

• Parameter
  summary Specifies flags for token that will not be break into ngram default cheese name ignoreOnBoundaryFlags type string array

"min": 2,
"max": 3,
"splitFlags": [],
"ignoreOnBoundaryFlags": [] 

Example Output

V--------------[abraham lincoln likes macaroni and cheese]--------------------V
^--[abraham]--V--[lincoln]--V--[likes]--V--[macaroni]--V--[and]--V--[cheese]--^
              ^---{place}---^           ^----{food}----^         ^---{food}---^
^----------{person}---------^           ^-----------------{food}--------------^

Output Flags

Lex-Item Flags:

• SEMANTIC_TAG - Identifies all lexical items which are semantic tags.
• PROCESSED - Placed on all the tokens which composed the semantic tag.
• ALL_LOWER_CASE - All of the characters in the token are lower-case characters.
• ALL_UPPER_CASE - All of the characters in the token are upper-case characters (for example, acronyms).
• ALL_DIGITS - All of the characters in the token are digits (0-9)
• TITLE_CASE - The first character is upper case, all of the other characters are lower case.
• MIXED_CASE - Handles any mixed upper & lower case scenario not covered above.
• TOKEN - All tokens produced are tagged as TOKEN 
• CHAR_CHANGE -  Identifies the vertex as a change between character formats
• HAS_DIGIT - Tokens produced with at least one digit character are tagged as HAS_DIGIT 
• HAS_PUNCTUATION - Tokens produced with at least one punctuation character are tagged as HAS_PUNCTUATION. (ALL_PUNCTUATION will not be tagged as HAS_PUNCTUATION)
• LEMMATIZE- All words retrieved will be marked as LEMMATIZE
• NUMBER - Flagged on all tokens which are numbers according to the rules above.
• TEXT_BLOCK - Flags all text blocks.
• STOP_WORD- All matched stop words will be marked as STOP_WORD
• WEIGHT_VECTOR - Identifies the tag as a weight vector representation of a sentence
• BANK- Identifies a Bank account number.
• ABA- Account number with ABA format.
• BIC- Account number with BIC format.
• IBAN- Account number with IBAN format.
• ORIGINAL - Identifies that the Lex-Items produced by this stage are the original, as written, representation of every token (e.g. before normalization)
• SSN - Identifies a Federal ID number
• GEONAME - Identifies a geographical location name
• NGRAM - All tokens produced are tagged as NGRAM

Vertex Flags:

• ALL_PUNCTUATION - Identifies the vertex as all token
  • The default flag if no "splitFlag" is present.
• <splitFlag> - Defines an alternative flag to ALL_PUNCTUATION, if desired (see above)
• CHAR_CHANGE -  Identifies the vertex as a change between character formats
• TEXT_BLOCK_SPLIT - Identifies the vertex as a split between text blocks.
• OVERFLOW_SPLIT - Identifies that an entire buffer was read without finding a split between text blocks.
  • The current maximum size of a text block is 64K characters.
  • Text blocks larger than this will be arbitrarily split, and the vertex will be marked with "OVERFLOW_SPLIT"\
• ALL_WHITESPACE - Identifies that the characters spanned by the vertex are all whitespace characters (spaces, tabs, new-lines, carriage returns, etc.)

Resource Data

Description of resource.

Resource Format

"_id" : "KGAAJGsBemSwA0nZTLXA",
"tag": "recipe",
"pattern": "("how many"|"how much") {ingredient} ",
"confAdjust": 0.95

. . . additional fields as needed go here . . . 

Fields

• Parameter
  summary What to show the user when browsing this entity name display required true

• Parameter
  summary Tag which will identify any match in the graph, as an interpretation name tag required true

  • These will all be added to the interpretation graph with the SEMANTIC_TAG flag.

    Tip
    Tags are hierarchical representations of the same intent. For example, {city} → {administrative-area} → {geographical-area}

• Parameter
  summary Pattern to match in the content name pattern required true