Configuration
This section lists all configuration parameters available to configure the Post JSON component.
Element | Type | Default | Description |
---|
postUrl | String | - | Url to post |
postJsonTransform | String | - | Path to the groovy transformation file |
okayResponse | String | - | Encoded JSON response |
Example Configuration
ElasticSearch Simple Indexing
Single document indexing (without batching).
Code Block |
---|
|
<component name="PostElasticsearch" subType="default" factoryName="aspire-post-http">
<postUrl>http://localhost:9200/testindex/testtype/</postUrl>
<postJsonTransform>config/json/aspireToElasticsearch.groovy</postJsonTransform>
<okayResponse><![CDATA[{"ok":true}]]></okayResponse>
</component> |
ElasticSearch Bulk Indexing
Elasticsearch bulk indexing (batching).
Code Block |
---|
|
<component name="PostElasticsearch" subType="default" factoryName="aspire-post-http">
<postUrl>http://localhost:9200/_bulk</postUrl>
<postJsonTransform>config/json/aspireToElasticsearchBulk.groovy</postJsonTransform>
<okayResponse><![CDATA[{"ok":true}]]></okayResponse>
</component> |
JSON Transformation Script
Anchor |
---|
| JSON Transformation Script |
---|
| JSON Transformation Script |
---|
|
The purpose of the JSON Tranformation Script is to transform the input AspireObject into the appropriate JSON to be pushed to the remote server. As part of this transformation you can:
- Select fields
- Map field names
- Map values
- Create new JSON structure
In effect, the JSON transformation script allows you to restructure your AspireObject into any JSON structure at all, so that the JSON you send to the remote server can be exactly what you want it to be.
Getting Started
Here is a simple builder script for creating a piece of fixed JSON output:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$object() {
title doc.displayTitle;
body doc.content;
submitTime (new Date());
} |
Produces the following JSON:
Code Block |
---|
|
{
"title":"Welcome To The World Wide Widget Company",
"body":"The WWW company is a world-wide seller of widgets...",
"submitTime":"2012-11-15T17:20:03+0000"
} |
As you can see, the builder script is a template representation for the resulting JSON structure. It can contain nested JSON objects, lists, and name/value pairs.
The JSON Transformation Script is itself a Groovy Script which uses a special "Aspire JSON Builder" to create the JSON. See Groovy Builders for more information about Groovy Builders.
In Groovy the semi-colons are optional, as long as all the fields are on separate lines. The following produces the same output:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$object() {
title doc.displayTitle
body doc.content
submitTime (new Date())
} |
Familiarize yourself with Groovy strings
Groovy strings with double-quotes allow you to embed Groovy variables into strings. Go here to learn more about Groovy strings.
For example:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$object() {
yearField "Year $doc.year"
} |
Produces:
Code Block |
---|
|
{"yearField":"Year 1990"} |
You can even embed Groovy expressions into your strings. Note the use of the ${} notation below:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$object() {
yearField "Years $doc.year-${doc.year+9}"
} |
Produces:
Code Block |
---|
|
{"yearField":"Years 1990-1999"} |
JSON Objects: $object
JSON Objects are hash maps of name/value pairs. These can be represented in several ways within the JSON Transformation Script.
Simple Objects
The $object variable is available for creating objects as needed.
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$object(); |
Produces:
Name:Value Pairs
Name value pairs are specified by simply putting the name and the value, one after the other, on a separate line:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$object() {
name 'Barack Obama'
age 51
}; |
Produces:
Code Block |
---|
|
{"age":51,"name":"Barack Obama"} |
Note that these are actually converted by Groovy into function() calls. This means that the above script could be written like this:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$object() {
name('Barack Obama');
age(51);
}; |
And it produces the exact same output:
Code Block |
---|
|
{"age":51,"name":"Barack Obama"} |
A top-level object with a single name/value pair can be created in a number of ways:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.document(); |
Produces:
Code Block |
---|
|
{"document":null} |
And:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.document('Hello world!'); |
Produces:
Code Block |
---|
|
{"document":"Hello world!"} |
Using Variables for Names
You can use Groovy strings with embedded $variables to make a variable name. For example, suppose you have a document which contains the following fields:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
"def doc = [year:1990, title:'Research Reports', titleType:'display'];\n" + |
With the above document the following script:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$object() {
yearField "Years $doc.year-${doc.year+9}"
"title-$doc.titleType" doc.title // << THIS NAME HAS AN EMBEDDED VARIABLE
} |
Will produce the following JSON:
Code Block |
---|
|
{"yearField":"Years 1990-1999","title-display":"Research Reports"} |
Use curly braces to create nested objects
Objects which are nested inside of other objects can be created with curly braces. Note that there needs to be at least one nested name/value pair for this to work.
For example:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$object() {
family {
father {
name 'Barack Obama'
age 51
}
mother {
name 'Michelle Obama'
age 48
}
}
} |
In this way, the JSON Transformation Script mimics the structure of the JSON you want to produce.
The above example produces:
Code Block |
---|
|
{
"family": {
"mother": {
"age": 48,
"name": "Michelle Obama"
},
"father": {
"age": 51,
"name": "Barack Obama"
}
}
} |
"$" is output for content with no name
It's possible to create a name/value pair with no name. When this happens, "$" will be used for the name.
For example:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$object('Hello world!'); |
Produces:
Code Block |
---|
|
{"$":"Hello world!"} |
Note that this can also happen when "$object" is used inside of other objects and lists. For example:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.quotations() {
$object "Hello world!"
} |
Produces:
Code Block |
---|
|
{"quotations":{"$":"Hello world!"}} |
Use "$" to add content to objects
'$' can also be used inside your transformation script where it represents simple content (as opposed to a name/value pair).
For example, it can be used to represent a nested value, as follows:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.factorial() {
'five' {
def j = 1;
for(int i = 1 ; i <= 5 ; i++) j *= i;
$ j;
}
} |
Produces:
Code Block |
---|
|
{"factorial":{"five":120}} |
JSON Lists
Creating Lists with $list
The $list directive can be used to create lists.
To create a simple top-level list:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$list(); |
Produces:
A simple top-level list with a nested string:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$list('Hello world!'); |
Produces:
Code Block |
---|
|
["Hello world!"] |
$list inside of objects
$list can be used inside of objects. For example:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.answer() {
$list 42
} |
Produces:
Code Block |
---|
|
{"answer":[42]} |
Another way to produce the same result:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$object() {
answer {$list 42 }
} |
Produces:
Code Block |
---|
|
{"answer":[42]} |
Use "$" to add content to lists
If you want to add simple content to a list, use the "$" to indicate this. For example:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.fibonacci() {
$list {
def j0 = 0, j1 = 1;
for(int i = 0 ; i < 10 ; i++) {
$ j1
def jtmp = j1;
j1 = j0+j1;
j0 = jtmp;
}
}
} |
Produces:
Code Block |
---|
|
{"fibonacci":[1,1,2,3,5,8,13,21,34,55]} |
Creating List of Objects
The above structures can be combined to create lists of objects.
First, what happens when you specify name/value pairs inside of a list?
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.ordinals() {
def ord = [1:"first", 2:"second", 3:"third"];
$list {
for(int i = 1 ; i <= 3 ; i++) { // NOTE: PROBABLY NOT WHAT YOU WANT
ordinal (ord[i])
arabic i
}
}
} |
Produces:
Code Block |
---|
|
{"ordinals":[{"ordinal":"first"},{"arabic":1},{"ordinal":"second"},{"arabic":2},{"ordinal":"third"},{"arabic":3}]} |
Notice that every name/value pair is a separate JSON object. This is probably not what you want.
The problem here is that there is no way for the builder to know how to group name/value pairs together unless you specify it explicitly with a "$object" directive:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.ordinals() {
def ord = [1:"first", 2:"second", 3:"third"];
$list {
for(int i = 1 ; i <= 3 ; i++) {
$object { // << NESTED $object TO GROUP NAME:VALUE PAIRS TOGETHER
ordinal (ord[i])
arabic i
}
}
}
} |
This produces something more like what you might have been expecting:
Code Block |
---|
|
{"ordinals":[{"arabic":1,"ordinal":"first"},{"arabic":2,"ordinal":"second"},{"arabic":3,"ordinal":"third"}]} |
Lists of Lists
These are also perfectly possible using the $list nomenclature:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.ordinals() {
def ord = [1:"first", 2:"second", 3:"third"];
$list {
for(int i = 1 ; i <= 3 ; i++) {
$list {
$ (ord[i])
$ i
}
}
}
} |
Produces:
Code Block |
---|
|
{"ordinals":[["first",1],["second",2],["third",3]]} |
Multiple Values for a Name -> An Automatic List
If you specify multiple values for the same name, the name will be automatically converted into a list.
For example:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.properties() {
server 'http://server1.com'
server 'http://server2.com'
server 'http://server3.com'
user 'george'
password '1234'
} |
Produces:
Code Block |
---|
|
{
"properties": {
"server": [
"http://server1.com",
"http://server2.com",
"http://server3.com"
],
"password": "1234",
"user": "george"
}
} |
You can do the same thing with a nested loop:
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.properties() {
for(int i = 1 ; i <= 3 ; i++) {
server "http://server${i}.com"
}
user 'george'
password '1234'
} |
This produces the exact same output as above. Notice the use of ${i} for an embedded variable inside of a JSON string. This can be very helpful when producing JSON output.
Multiple JSON objects
By using builder.flush() you can specify the transformer to have multiple separated JSON objects for each transform. This is used for elasticsearch bulk indexing.
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.index() {
'_type' "test"
'_index' "posthttp"
}
builder.flush()
builder.$object() {
url doc.fetchUrl;
body doc.content;
submitTime (new Date());
} |
Produces:
Code Block |
---|
|
{"index":{"_type":"test","_index":"posthttp"}}
{"url":"http://www.mydomain.com/", "body":"This is the content of my web page", "submitTime":"2012-11-15T17:20:03+0000"} |
The flush method is always required if you want separated JSON objects. It also adds each JSON object in a new line "\n".
Creating JSON from a string
By using builder.$literal() you can create a JSON structure using a JSON string
Code Block |
---|
language | groovy |
---|
theme | Eclipse |
---|
|
builder.$literal() {
coordinates "[[1,2][3,4][5,6]]"
} |
Produces:
Code Block |
---|
|
{
"coordinates" : [
[1,2],
[3,4],
[5,6]
]
} |