he Publish to ElasticSearch application performs content feeds to a ElasticSearch of metadata and content of files extracted by Aspire connectors. The feed to the ElasticSearch can be customized by editing the JSON transformation file provided by the user.

Configuration

This section lists all configuration parameters available to install the ElasticSearch Application Bundle.

PropertyTypeDefaultDescription
ElasticIndexstringindex1Index to which the jobs are going to be published.
ElasticNoUrlbooleantrueIndicates if the publisher must use a Url or build one from the host and port entered.
ElasticPortinterger9200ElasticSearch port where to send the feeds
ElasticHoststringnoneElasticSearch hostname or IP adress. e.g. server.domain.com
ElasticUrlstringnoneComplete Url where the feeds are going to be send. e.g. http://localhost:9200/bulk_
aspireToElasticGroovystring${appbundle.home}/config/groovy/aspireToElasticsearchBulk.groovyLocation of the Groovy to transform the job data to a ElasticSearch feed. See Edit Groovy.
maxResults (2.1 Release)  int1000000(Index dump) How many documents can be fetched by the search engine for the same query
pageSize (2.1 Release)  int10000(Index dump) How many documents to fetch per page
urlField (2.1 Release)  stringdisplayUrl(Index dump) Field used to store the url in the search engine
idField (2.1 Release)  stringid(Index dump) Field used to store the id in the search engine.
timestampField (2.1 Release)  stringsubmitTS(Index dump) The name of the timestamp field holding the index timestamp of every document.

Configuration Example

  <application config="com.searchtechnologies.aspire:app-publish-to-gsa">
    <properties>
      <ElasticIndex>index1</ElasticIndex>
      <ElasticNoUrl>true</ElasticNoUrl>
      <ElasticHost>localhost</ElasticHost>
      <ElasticPort>9200</ElasticPort>
      <aspireToElasticGroovy>${appbundle.home}/config/groovy/aspireToElasticsearchBulk.groovy</aspireToElasticGroovy>
      <debug>false</debug>
    </properties>
  </application>


Note: Any optional properties can be removed from the configuration to use the default value described on the table above.

Edit Groovy

The default Groovy transformation file can be found in File:AspireToElasticsearchBulk.groovy.


The default transformation Groovy file provided by the publisher expects metadata as described in Connector AspireObject Metadata.

Add metadata field

To add a new metadata field extracted by an Aspire Connector add an groovy element inside the builder.$object() that is right after the builder.flush().

   metadata-name doc.metadatafield

Change the document ID

The id of a ElasticSearch document is used to uniquely identify a file in the index. By default, Publish To ElasticSearch will use the following fields from the Aspire document in order of precedence (if one is missing, then the next will be used):

  • fetchUrl
  • url
  • displayUrl
  • id

If you want to change this behavior, edit or create a new Groovy file which has the following element inside builder.index():

  '_id' value-for-id


For more information in how to create a Groovy file transformation please see Post JSON page

  • No labels