The following information is intended to assist with detailed developer tasks.
The Publish to Azure Blobs publisher sends document feeds to the Azure Blobs storage container, metadata, and the content of files extracted by Aspire connectors, in a JSON format. The feed to Azure Blobs can be customized by using a transformation script which is provided by the user.
Publish to Azure Blobs | |
---|---|
Factory Name | com.searchtechnologiescom. searchtechnologies.aspire:app-azure-blob-publisher |
Inputs | AspireObject from a connector's subjob with metadata and content extracted from a specific file/folder. |
Outputs | A JSON transformation of the AspireObject sent to the Azure Blog storage container |
Versions | |
Type Flags | job-input |
This section lists all configuration parameters that are available to configure the Publish to Azure Blobs component.
Element | Type | Default | Description | |||
---|---|---|---|---|---|---|
debug | Boolean | false | If true it will log debug information from the component. | |||
containerName | string | defaultContainer | Blob container name to write documents to. The container is created if it doesn't exist. | |||
jsonTransform | string | ${appbundle.home}/config/groovy/aspireToBlob.groovy | Location of the file containing the JSON Transformation Script. This script will be used to transform the data from Aspire as it is posted to the Azure Blob Server. | |||
storageConnectionString | string | DefaultEndpointsProtocol=http;AccountName=myAccount;AccountKey=myKey; | Azure Storage Connection String. | |||
useTransformFile | Boolean | false | If true, it will use the script indicated in the JsonTransform property to transform the JSON content. | |||
softDelete | Boolean | false | If true, it will use flag name specified in the deleteFlag property to mark documents as deleted. | |||
deleteFlag | String | Deleted | Flag used to mark document as deleted (soft delete) in Azure Blob. The flag will be stored as part of the blob metadata. | |||
addBlobMetadata | Boolean | false | If true, it will enable subsection to add metadata to the blob. | |||
metaSource | String | Original Document (Aspire Doc) | Specify where to take metadata values from. Select Original Document if the values for the metadata fields are stored inside the original input document. Select Transformed Document if the values for metadata field are stored inside the transformed output document. | |||
name (metadata name) | String | Specify the name of the blob metadata field. If a name specified as a metadata field matches the delete flag name used for soft deletes, the value will be overwritten by the soft delete handler. This is a required value. | ||||
name (metadata value) | String | Specify the field name or path (e.g. extension/isdeleted) inside Aspire or transformed document where to take the value from. This is a required value. | ||||
default | String | Specify the default value to be used when the metadata value does not exist |
Note: if you are using the Azure Storage Emulator, then for the "storageConnectionString" property, you can use the next connection string “UseDevelopmentStorage=true;” this will connect to the emulator installed locally in your machine.
<application config="com.searchtechnologies.aspire:app-azure-blob-publisher"> <properties> <property name="debug">false</property> <property name="containerName">defaultContainer</property> <property name="useTransformFile">false</property> <property name="storageConnectionString">UseDevelopmentStorage=true</property> </properties> </application> |
<application config="com.searchtechnologies.aspire:app-azure-blob-publisher"> <properties> <property name="jsonTransform">${appbundle.home}/config/groovy/aspireToBlob.groovy</property> <property name="debug">false</property> <property name="containerName">defaultContainer</property> <property name="useTransformFile">true</property> <property name="storageConnectionString">UseDevelopmentStorage=true</property> </properties> </application> |
<application config="com.searchtechnologies.aspire:app-azure-blob-publisher"> <properties> <property name="debug">false</property> <property name="containerName">defaultContainer</property> <property name="useTransformFile">false</property> <property name="storageConnectionString">UseDevelopmentStorage=true</property> <property name="softDelete">true</property> <property name="deleteFlag">Deleted</property> </properties> </application> |
<application config="com.searchtechnologies.aspire:app-azure-blob-publisher"> <properties> <property name="debug">false</property> <property name="containerName">defaultContainer</property> <property name="useTransformFile">false</property> <property name="storageConnectionString">UseDevelopmentStorage=true</property> <property name="addBlobMetadata">true</property> <property name="metaSource">aspiredoc</property></properties> <property name="properties"><requestProperties><requestProperty name="Author" default="Unknown">connectorSpecific/author</requestProperty></requestProperties></property> </properties> </application> |