You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

Identifies tokens which look like numbers and flags them with the "NUMBER" flag.Operates On:  Lexical Items with TOKEN

Currently handles the following situations:

  • Integers:  0, 1, 100
  • Floats:  0.5, 12.2, 3.14159 (.)
  • Negative:  -1003, -12.2
  • Thousands with/without separator: 1000000, 1,000,000 (,)
  • Numbers with scientific notation: 1,1x10^-8, 1.1x10??, 1,1x10^8, -1.1x10??
  • Romans numbers: MMC, XII, IV
  • English ordinals numbers: 1st, 12th, 23rd
  • Exponents: 4?, 4^9, 4^-9, 4??

Does NOT currently recognize:

  • Computer literals:  0xBEA1, 07832
  • European numbers with commas and periods swapped
  • Positive:  +102.3


Generic Configuration Parameters

  • boundaryFlags ( type=string array | optional ) - List of vertex flags that indicate the beginning and end of a text block.
    Tokens to process must be inside two vertices marked with this flag (e.g ["TEXT_BLOCK_SPLIT"])
  • skipFlags ( type=string array | optional ) - Flags to be skipped by this stage.
    Tokens marked with this flag will be ignored by this stage, and no processing will be performed.
  • requiredFlags ( type=string array | optional ) - Lex items flags required by every token to be processed.
    Tokens need to have all of the specified flags in order to be processed.
  • atLeastOneFlag ( type=string array | optional ) - Lex items flags needed by every token to be processed.
    Tokens will need at least one of the flags specified in this array.
  • confidenceAdjustment ( type=double | default=1 | required ) - Adjustment factor to apply to the confidence value of 0.0 to 2.0 from (Applies for every pattern match).
    • 0.0 to < 1.0  decreases confidence value
    • 1.0 confidence value remains the same
    • > 1.0 to  2.0 increases confidence value
  • debug ( type=boolean | default=false | optional ) - Enable all debug log functionality for the stage, if any.
  • enable ( type=boolean | default=true | optional ) - Indicates if the current stage should be consider for the Pipeline Manager
    • Only applies for automatic pipeline building


Configuration Parameters

    • scientificNotation (boolean, optional)
      • Enable recognition of scientific notation.
    • ordinals (boolean, optional)
      • Enable ordinals numbers recognition.
    • romans (boolean, optional)
      • Enable romans number recognition.
    • ordinalsLang (string, optional)
      • Choose ordinals language. Currently supported english: "en"


Example Configuration
{
    "type": "NumberRecognizer",
    "scientificNotation": false,
    "ordinals": true,
	"romans": false,
	"ordinalsLang": "en"
}

Flags

Lex-Item Flags:

  • NUMBER - Flagged on all tokens which are numbers according to the rules above.

Vertex Flags:

None.

Example

V----------[1984 42 -10 3.14 ]-----------V  
^--[1984]--V--[42]--V--[-10]--V--[3.14]--^  

Item [1984] - [NUMBER,TOKEN]
Item [42]   - [NUMBER,TOKEN]
Item [-10]  - [NUMBER,TOKEN]
Item [3.14] - [NUMBER,TOKEN]
  • No labels