Introduction

The Avro Summarizer Executor can process the content of a Avro file and extract each of the rows and the table schema. Each extracted row will be processed by the summarizers attached to the job.

Temporary Files

The Avro Summarizer Executor allows downloading the content of the file into a local temporary file to reduce memory usage.

Rows Filtering

The Avro Summarizer Executor has the option to configure a groovy script to filter which rows will be processed.

Example:

// This script must return a boolean.
// The references of the job, doc, component, row and table objects are available.
// Javadoc references 
// Row (row) - http://{manager}/javadocs/com/accenture/aspire/services/summarization/Row.html
// Table (table) - http://{manager}/javadocs/com/accenture/aspire/services/summarization/Table.html
row.getBoolean("sensitive") == true