You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Markup Producer stage will producer strings of TEXT_BLOCK or SENTENCE texts with normalized tags based on the graph's path with highest confidence.

Operates On:  Every lexical Item in the graph.

Generic Configuration Parameters

  • boundaryFlags ( type=string array | optional ) - List of vertex flags that indicate the beginning and end of a text block.
    Tokens to process must be inside two vertices marked with this flag (e.g ["TEXT_BLOCK_SPLIT"])
  • skipFlags ( type=string array | optional ) - Flags to be skipped by this stage.
    Tokens marked with this flag will be ignored by this stage, and no processing will be performed.
  • requiredFlags ( type=string array | optional ) - Lex items flags required by every token to be processed.
    Tokens need to have all of the specified flags in order to be processed.
  • atLeastOneFlag ( type=string array | optional ) - Lex items flags needed by every token to be processed.
    Tokens will need at least one of the flags specified in this array.
  • confidenceAdjustment ( type=double | default=1 | required ) - Adjustment factor to apply to the confidence value of 0.0 to 2.0 from (Applies for every pattern match).
    • 0.0 to < 1.0  decreases confidence value
    • 1.0 confidence value remains the same
    • > 1.0 to  2.0 increases confidence value
  • debug ( type=boolean | default=false | optional ) - Enable all debug log functionality for the stage, if any.
  • enable ( type=boolean | default=true | optional ) - Indicates if the current stage should be consider for the Pipeline Manager
    • Only applies for automatic pipeline building


Unable to render {include} The included page could not be found.

Configuration Parameters

  • normalizeTags (String array, required) - List of tag names to normalize on the output.
  • replaceTags (String array, optional) - Defaults to empty. If non empty, list of tag names to apply the replace pattern to.
  • replacePattern (String, optional) - Required when replaceTags is set. The pattern expects a %tag and/or %value, pattern will replace the appearance of tags from replaceTags with the pattern defined.
  • separator (String, optional) - Default to single whitespace. Used to separate tokens
  • preferFlags (String array, optional) - Defaults to empty. If non empty, when a token has multiple flags and one of them is in the preferFlags, it will take precedence over other flags to use the value of the token from that variation, i.e. LOWERED flag over original text version.


Example Configuration
{
  "type": "MarkupProducerStage",
  "name": "MarkupProducer",
  "boundaryFlags": [
    "TEXT_BLOCK_SPLIT", "SENTENCE_SPLIT"
  ],
  "normalizeTags": ["#", "measurement"],
  "replaceTags": ["ingredient"],
  "replacePattern": "<START:%tag> %value <END>",
  "separator": " "
}



Example Output

If you have a text block like the following:

V-----------------------------------[300 ml of Water. Use XX g of FLOUR]------------------------------------V 
^----------------[300 ml of Water]----------------V-------------------[Use XX g of FLOUR]-------------------^ 
^-[300]-V------[ml]-------V-[of]-V----[Water]-----^-[Use]-V-[XX]--V-------[g]-------V-[of]-V----[FLOUR]-----^ 
^-[{#}]-^-[{measurement}]-^      ^----[water]-----^-[use]-^-[xx]--^-[{measurement}]-^      ^----[flour]-----^ 
                                                          ^-[{#}]-^                        ^-[{ingredient}]-^ 
                                                                                           ^-[{ingredient}]-^ 
                                 ^-[{ingredient}]-^ 
                                 ^-[{ingredient}]-^ 
the stage will produce the following output

{#} {measurement} of <START:ingredient> Water <END>
use {#} {measurement} of <START:ingredient> FLOUR <END>
  • No labels