The following information is intended to assist with detailed developer tasks.

Publish to Azure Blobs App Bundle

The Publish to Azure Blobs publisher sends document feeds to the Azure Blobs storage container, metadata, and the content of files extracted by Aspire connectors, in a JSON format. The feed to Azure Blobs can be customized by using a transformation script which is provided by the user.


Publish to Azure Blobs

Factory Name

com.searchtechnologiescom. searchtechnologies.aspire:app-azure-blob-publisher

Inputs

AspireObject from a connector's subjob with metadata and content extracted from a specific file/folder.

Outputs

A JSON transformation of the AspireObject sent to the Azure Blog storage container

Versions


Type Flags

job-input



Configuration

This section lists all configuration parameters that are available to configure the Publish to Azure Blobs component.

ElementTypeDefaultDescription
debugBooleanfalse

If true it will log debug information from the component.

containerNamestringdefaultContainer

Blob container name to write documents to. The container is created if it doesn't exist.

jsonTransformstring${appbundle.home}/config/groovy/aspireToBlob.groovy

Location of the file containing the JSON Transformation Script. This script will be used to transform the data from Aspire as it is posted to the Azure Blob Server.

storageConnectionStringstringDefaultEndpointsProtocol=http;AccountName=myAccount;AccountKey=myKey;

Azure Storage Connection String.

useTransformFileBooleanfalseIf true, it will use the script indicated in the JsonTransform property to transform the JSON content.
 softDelete Boolean false If true, it will use flag name specified in the deleteFlag property to mark documents as deleted.
 deleteFlag String Deleted 

Flag used to mark document as deleted (soft delete) in Azure Blob.

The flag will be stored as part of the blob metadata.
 addBlobMetadata Boolean falseIf true, it will enable subsection to add metadata to the blob.
 metaSource String Original Document (Aspire Doc)

Specify where to take metadata values from.

Select Original Document if the values for the metadata fields are stored inside the original input document.

Select Transformed Document if the values for metadata field are stored inside the transformed output document.
 name (metadata name) String  Specify the name of the blob metadata field.

If a name specified as a metadata field matches the delete flag name used for soft deletes, the value will be overwritten by the soft delete handler.

This is a required value.
 name (metadata value) String 

Specify the field name or path (e.g. extension/isdeleted) inside Aspire or transformed document where to take the value from.

This is a required value.
 default String Specify the default value to be used when the metadata value does not exist

Note: if you are using the Azure Storage Emulator, then for the "storageConnectionString" property, you can use the next connection string “UseDevelopmentStorage=true;” this will connect to the emulator installed locally in your machine.



Example Configuration

Without Transformation

<application config="com.searchtechnologies.aspire:app-azure-blob-publisher">

  <properties>

    <property name="debug">false</property>

    <property name="containerName">defaultContainer</property>

    <property name="useTransformFile">false</property>

    <property name="storageConnectionString">UseDevelopmentStorage=true</property>

  </properties>

</application>

With Transformation

<application config="com.searchtechnologies.aspire:app-azure-blob-publisher">

  <properties>

    <property name="jsonTransform">${appbundle.home}/config/groovy/aspireToBlob.groovy</property>

    <property name="debug">false</property>

    <property name="containerName">defaultContainer</property>

    <property name="useTransformFile">true</property>

    <property name="storageConnectionString">UseDevelopmentStorage=true</property>

  </properties>

</application>

With Soft Delete

<application   config="com.searchtechnologies.aspire:app-azure-blob-publisher">

  <properties>

      <property name="debug">false</property>

      <property   name="containerName">defaultContainer</property>

    <property   name="useTransformFile">false</property>

      <property   name="storageConnectionString">UseDevelopmentStorage=true</property>

      <property name="softDelete">true</property>

      <property name="deleteFlag">Deleted</property>

  </properties>

</application>

With Metadata

<application   config="com.searchtechnologies.aspire:app-azure-blob-publisher">

  <properties>

      <property name="debug">false</property>

      <property   name="containerName">defaultContainer</property>

    <property   name="useTransformFile">false</property>

      <property   name="storageConnectionString">UseDevelopmentStorage=true</property>

      <property name="addBlobMetadata">true</property>

      <property   name="metaSource">aspiredoc</property></properties>

    <property   name="properties">&lt;requestProperties&gt;&lt;requestProperty   name="Author"   default="Unknown"&gt;connectorSpecific/author&lt;/requestProperty&gt;&lt;/requestProperties&gt;</property>

  </properties>

</application>