Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The Tabular Files Extractor gets a tabular file (a comma/tab separated file) from the content stream in the Job and converts it into XML, using the Metadata Mapper to map each column to an XML Element. It then will submit each row of the XML file as a separate subJob.

 

Contents

Configuration
Anchor
Configuration
Configuration

...

ElementTypeDefaultDescription
separatorstring'comma'(,)The character used to separate each column on the tabular file. It can be any character, 'comma'(,) or 'tab'(\t). Defaults to ','.
headersOnFirstRowbooleanfalseWhether to use the first row data as column names or not. If false, columns will be named column1, column2, ..., in the metadata mapping.
ignoreQuotesbooleanfalseWhen false it will treat unescaped (backslash escaping) quotes as delimiters for literal text (separators will be considered as normal characters). If true it will include the quotes as normal characters
metadataMapMapnullfield names to map each column. See Metadata Mapper for more info.
branches NoneThe configuration of the pipeline to publish to. See below.

Branch Configuration
Anchor
Branch Configuration
Branch Configuration

The Tabular SubJob Extractor publishes documents using the branch manager. It publishes using the events configured above. You must therefore include <branches> for these events in the configuration to publish to a pipeline within a pipeline manager. See Branch Handler for more details.

ElementTypeDescription
branches/branch/@eventStringThe event to configure. Should always be "onSubJob".
branches/branch/@pipelineManagerstringThe URL of the pipeline manager to publish to. Can be relative.
branches/branch/@pipelinestringThe name of the pipeline to publish to.

Example Configuration
Anchor
Example Configuration
Example Configuration

...

Code Block
languagexml
linenumberstrue
 <component name="TabularSubJobExtractor" subType="default" factoryName="aspire-tabular-files">
    <branches>
      <branch event="onSubJob" pipelineManager="." pipeline="subJobsPipeline" />
    </branches>
    <separator>tab</separator>
    <headersOnFirstRow>true</headersOnFirstRow>
  </component>

Example Configuration with No Headers on Input File
Anchor
Example Configuration with No Headers on Input File
Example Configuration with No Headers on Input File

...

Code Block
languagexml
linenumberstrue
 <component name="TabularSubJobExtractor" subType="default" factoryName="aspire-tabular-files">
    <branches>
      <branch event="onSubJob" pipelineManager="subjobPipelineManager" />
    </branches>
    <separator>tab</separator>
    <headersOnFirstRow>false</headersOnFirstRow>
    <metadataMap>
      <map from="column0" to="term"/>
      <map from="column1" to="frequency"/>
      <map from="column2" to="type"/>
      <map from="column3" to="id"/>
    </metadataMap>
  </component>


Example Use Within A Pipeline
Anchor
Example Use Within A Pipeline
Example Use Within A Pipeline

...

Code Block
languagexml
linenumberstrue
   <pipeline name="process-feedOne-test">
    <stages>
      <stage component="fetchUrl" />
      <stage component="TabularSubJobExtractor" />
    </stages>
  </pipeline>


Example
Anchor
Example
Example

...

In the following example suppose that there's a file called "file:test.txt" which contains the following:

...