The Avro Builder provides the following functionality
Avro Builder | |
---|---|
Factory Name | com.searchtechnologies.aspire:app-publish-to-avro |
subType | |
Inputs | AspireObject from a connector's job with metadata and content extracted from a specific file |
Outputs | Avro representation of AspireObject sent to an Avro file. |
Element | Type | Default | Description |
---|---|---|---|
avroSchema | string | Avro schema path | |
outputNonBatchFile | string | Output file path | |
nonBatchFileMaxSize | int | 600 | Max file size (MB) |
outputBatchDirName | string | Output directory path for internal batch files | |
maxInternalBatchSize | int | 1000 | Batch size |
debug | boolean | false | If true it will log debug information from the component |
<application config= "com.searchtechnologies.aspire:app-publish-to-avro" name= "/Aspire_Publish_To_Avro_Application" > <properties> <property name= "useSizeLimited" > false </property> <property name= "debug" > false </property> <property name= "maxInternalBatchSize" > 2 </property> <property name= "outputBatchDirName" >c:\temp\avro</property> <property name= "useInternalBatch" > true </property> <property name= "avroSchema" >c:\temp\avro\cefavro.avsc</property> <property name= "OutputType" >internalBatches</property> </properties> </application> |
<application config= "com.searchtechnologies.aspire:app-publish-to-avro" name= "/Aspire_Publish_To_Avro_Application" > <properties> <property name= "useSizeLimited" > true </property> <property name= "debug" > false </property> <property name= "nonBatchFileMaxSize" > 50 </property> <property name= "outputNonBatchFile" >c:\temp\avro\avro.snappy</property> <property name= "useInternalBatch" > false </property> <property name= "avroSchema" >c:\temp\avro\cefavro.avsc</property> <property name= "OutputType" >sizeLimited</property> </properties> </application> |