When developing an application with Saga Library embedded, and connecting to the ElasticSearch manage by the Saga Server
This tutorial assumes:
- The reader ability to create a project with Maven Framework support
- The data Saga will use is manage through the Saga's user interface
Step-by-step guide
Configure pom.xml
In the pom.xml of the project, a basic example of the minimum configuration can be found below. But elementary section of this configuration is the dependencies section, where we need two main libraries.
| saga-library | saga-elastic-provider |
|---|---|
| The core library of Saga, this dependency includes the Engine, Stages, Tag Manager, Pipeline Manager and Resource Manager which are all the parts necessary to use Saga in any application. | This dependency will grant us access to ElasticSearch as a provider for Saga, which means our Stages and Managers will be able to fetch the data directly from this provider More providers will be available in the future, but to use Saga full functionality we recommend the use of the saga-elastic-provider. |
Other important configuration to notice is the use of Java 11, for the compilation of the code and the encoding UTF-8, as you can see in the lines 36-38
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.accenture.saga</groupId>
<artifactId>saga-howto</artifactId>
<version>1.0.0-SNAPSHOT</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<saga.version>1.0.0</saga.version>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<executions>
<execution>
<id>compile</id>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>testCompile</id>
<phase>test-compile</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<source>11</source>
<target>11</target>
<encoding>${project.build.sourceEncoding}</encoding>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<archive>
<manifest>
<mainClass>com.accenture.saga.server.SagaServer</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- bind to the packaging phase -->
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>com.accenture.saga</groupId>
<artifactId>saga-library</artifactId>
<version>${saga.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>com.accenture.saga</groupId>
<artifactId>saga-elastic-provider</artifactId>
<version>${saga.version}</version>
<scope>compile</scope>
</dependency>
</dependencies>
</project>
Initializing The Saga Components
For starters, we will create a main class, which will hold a SagaEngine, ResourceManager, TagManager and PipelineManager.
package com.accenture.saga;
import com.accenture.saga.engine.PipelineManager;
import com.accenture.saga.engine.SagaEngine;
import com.accenture.saga.resourcemgr.ResourceManager;
import com.accenture.saga.tags.TagManager;
public class Main {
SagaEngine engine;
ResourceManager resourceManager;
TagManager tagManager;
PipelineManager pipelineManager;
/**
* Constructor
*/
public Main() {
}
public static void main(String[] args) {
Main _instance = new Main();
}
}
First we start by creating an configuring the ResourceManager, and adding a provider to it, but this configuration will be hard-coded, so we need to add SagaJsonFactory class, which allow us to create SagaJson objects (the standard document of Saga) from text, files or readers.
The configuration we are going to use for the provider is the following $action.getHelper().renderConfluenceMacro("$codeS$body$codeE")Each field from the top, starting with the common
- name ( type=string | required ) - The name we are going to use for the provider. It doesn't which name you use, but our take is "saga-provider"
- type ( type=string | required ) - Indicates the type of provider we are using, in this case since we are using saga-elastic-provider, it's type would be "Elastic"
from here on, all the properties are specific to saga-elastic-indexer
- shema ( type=string | default=http | optional ) - Schema for the url to Elasticsearch
- hostname ( type=string | default=localhost | optional ) - Name of the hosting server
- 9200 ( type=integer | default=9200 | optional ) - Port of ElasticSearch
- timestamp ( type=string | default=updatedAt | optional ) - Name of the field reflecting any change done to the data
- exclude ( type=string array | optional ) - Name of the fields omitted (when possible) from the response of ElasticSearch
Our code should look this this
public Main() {
resourceManager = new ResourceManager();
resourceManager.registerProvider(
SagaJsonFactory.getInstance(
"{"
+ "\"name\":\"saga-provider\","
+ "\"type\":\"FileSystem\","
+ "\"baseDir\":\"testdata\""
+ "}"
)
);
}
Next we proceed with the configuration of the TagManager below the ResourceManager, once again we will hard-code the configuration for this one $action.getHelper().renderConfluenceMacro("$codeS$body$codeE")In the configuration above, saga-provider is representing the provider we add to the ResourceManager in the previous configuration, then the colon (:) indicates the division between the provider and the actual resource; since we are using a saga-elastic-provider, the resources will be indexes names, and since were are connecting to a Saga index, created by the Saga server, all the indexes will be a combination between the solution's name (usually will be saga), and underscore (_) and the type of data the index holds, in this case tags, forming the name saga_tags
The code should look now like this
public Main() {
resourceManager = new ResourceManager();
resourceManager.registerProvider(
SagaJsonFactory.getInstance(
"{"
+ "\"name\":\"saga-provider\","
+ "\"type\":\"FileSystem\","
+ "\"baseDir\":\"testdata\""
+ "}"
)
);
tagManager = new TagManager(resourceManager, SagaJsonFactory.getInstance("{ \"resource\": \"saga-provider:saga_tags\"}"));
}
As you can see the TagManager, receives as a parameter the ResourceManager, which grant us access to the resource saga-provider:saga_tags.
In the same way, we proceed to configure the PipelinesManager using the following configuration $action.getHelper().renderConfluenceMacro("$codeS$body$codeE")Once again, saga-provider, does reference to the provider added in the ResourceManager, and saga_pipelines is a combination between the name of the solution's name and the type of data, in this case pipelines.
public Main() {
resourceManager = new ResourceManager();
resourceManager.registerProvider(
SagaJsonFactory.getInstance(
"{"
+ "\"name\":\"saga-provider\","
+ "\"type\":\"FileSystem\","
+ "\"baseDir\":\"testdata\""
+ "}"
)
);
tagManager = new TagManager(resourceManager, SagaJsonFactory.getInstance("{ \"resource\": \"saga-provider:saga_tags\"}"));
pipelineManager = new PipelineManager(resourceManager, SagaJsonFactory.getInstance("{ \"resource\": \"saga-provider:saga_pipelines\"}"));
}
Setting Up The Engine
So one we are at the fun part, once we got the Resource, Tag and Pipeline Manager all set up, we assign the ResourceManager and the TagManager to the Engine
public Main() {
resourceManager = new ResourceManager();
resourceManager.registerProvider(
SagaJsonFactory.getInstance(
"{"
+ "\"name\":\"saga-provider\","
+ "\"type\":\"FileSystem\","
+ "\"baseDir\":\"testdata\""
+ "}"
)
);
tagManager = new TagManager(resourceManager, SagaJsonFactory.getInstance("{ \"resource\": \"saga-provider:saga_tags\"}"));
pipelineManager = new PipelineManager(resourceManager, SagaJsonFactory.getInstance("{ \"resource\": \"saga-provider:saga_pipelines\"}"));
engine = new SagaEngine();
engine.setResourceManager(resourceManager);
engine.setTagManager(tagManager);
}
Working With The Pipeline Manager
From here we got 2 options, first option is letting the PipelineManager build the pipeline for us, using a set of tags (that we will provide); the second option is to manually provide a complete pipeline configuration to the PipelineManager
Pros & Cons
Automatic Pipeline | Manual Pipeline |
|---|---|
Pros
| Pros
|
Cons
| Cons
|
Tie
| Tie
|
Since the first one is the most flexible and the one that makes use of the configuration in ElasticSearch, we will use that one.
Request A Pipeline
Before asking the PipelineManager for a pipeline, we need to provide a stage of type TextBlockReader, at the moment we only have one stage of that type, the SimpleReaderStage, which requires a splitRegex in the configuration, as a SagaJson object. So let's add that to the code shall we.
public Main() {
resourceManager = new ResourceManager();
resourceManager.registerProvider(
SagaJsonFactory.getInstance(
"{"
+ "\"name\":\"saga-provider\","
+ "\"type\":\"FileSystem\","
+ "\"baseDir\":\"testdata\""
+ "}"
)
);
tagManager = new TagManager(resourceManager, SagaJsonFactory.getInstance("{ \"resource\": \"saga-provider:saga_tags\"}"));
pipelineManager = new PipelineManager(resourceManager, SagaJsonFactory.getInstance("{ \"resource\": \"saga-provider:saga_pipelines\"}"));
//The Fun Part
engine = new SagaEngine();
engine.setResourceManager(resourceManager);
engine.setTagManager(tagManager);
SimpleReaderStage simpleReaderStage = new SimpleReaderStage(engine, SagaJsonFactory.getInstance("{ \"splitRegex\": \"[\\r|\\n]+\"}"));
}
- splitRegex ( type=string | optional ) - asdas
With the regex [\r\n]+ we are indicating the character signaling a break line; also note the SimpleReaderStage receives the engine as the first parameter.
The regex [\r\n]+, is the standard for mostly all the text you will be processing
Now we can ask the PipelineManager to build a pipeline for the tags... Which we still don't know where they came from, but let's fix that; first we add the building of the pipeline.
pipelineManager.buildPipelineFor(engine, tags, simpleReaderStage);
Now we just add the tags as a parameter of the constructor, our code should look like this
package com.accenture.saga;
import com.accenture.saga.engine.PipelineManager;
import com.accenture.saga.engine.SagaEngine;
import com.accenture.saga.engine.stages.SimpleReaderStage;
import com.accenture.saga.exception.SagaException;
import com.accenture.saga.json.SagaJsonFactory;
import com.accenture.saga.resourcemgr.ResourceManager;
import com.accenture.saga.tags.TagManager;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
public class Main {
SagaEngine engine;
ResourceManager resourceManager;
TagManager tagManager;
PipelineManager pipelineManager;
/**
* Constructor
*/
public Main(List<String> tags) throws SagaException {
resourceManager = new ResourceManager();
resourceManager.registerProvider(
SagaJsonFactory.getInstance(
"{"
+ "\"name\":\"saga-provider\","
+ "\"type\":\"FileSystem\","
+ "\"baseDir\":\"testdata\""
+ "}"
)
);
tagManager = new TagManager(resourceManager, SagaJsonFactory.getInstance("{ \"resource\": \"saga-provider:saga_tags\"}"));
pipelineManager = new PipelineManager(resourceManager, SagaJsonFactory.getInstance("{ \"resource\": \"saga-provider:saga_pipelines\"}"));
engine = new SagaEngine();
engine.setResourceManager(resourceManager);
engine.setTagManager(tagManager);
SimpleReaderStage simpleReaderStage = new SimpleReaderStage(engine, SagaJsonFactory.getInstance("{ \"splitRegex\": \"[\\r|\\n]+\"}"));
pipelineManager.buildPipelineFor(engine, tags, simpleReaderStage);
}
public static void main(String[] args) throws SagaException {
List<String> tags = new ArrayList<>(Arrays.asList("tag1", "tag2", "tag3"));
Main _instance = new Main(tags);
}
}
Here we just build a list with the tags "tag1", "tag2", and "tag3", which we passed as a parameter of the constructor, so the PipelineManager can have them to build the pipeline.
At this point pipelineManager.buildPipelineFor uses the tags we specified to identified the stages which recognize these tags, from this stages build a dependency hierarchy, which adds any necessary tags and stages in order to found the specified tags; once it has identified all the stages and tags necessary, PipelineManger adds them to the Engine we provided, which means our Engine is ready to receive text and build a graph.
Process A Text
Let's also add a String parameter to the constructor, for the text we want to process, this text will be added to the Engine using the method reset.
engine.reset(new ByteArrayInputStream(text.getBytes(StandardCharsets.UTF_8)));
This method accepts an InputStream, this so we can specify the encoding in UTF-8; so in order to pass the text to Saga we get the bytes from the text in UTF-8 encoding and create a ByteArrayInputStream with them.
All the process done by Saga is with the encoding UTF-8
At the moment we only told Saga which is the text, now we need to process it. For this we use the method advance, which returns a Vertex, this will be the first Vertex of the text block.
Vertex v = engine.advance();
Currently our code should look like this
package com.accenture.saga;
import com.accenture.saga.engine.PipelineManager;
import com.accenture.saga.engine.SagaEngine;
import com.accenture.saga.engine.Vertex;
import com.accenture.saga.engine.stages.SimpleReaderStage;
import com.accenture.saga.exception.SagaException;
import com.accenture.saga.json.SagaJsonFactory;
import com.accenture.saga.resourcemgr.ResourceManager;
import com.accenture.saga.tags.TagManager;
import java.io.ByteArrayInputStream;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
public class Main {
SagaEngine engine;
ResourceManager resourceManager;
TagManager tagManager;
PipelineManager pipelineManager;
/**
* Constructor
*/
public Main(String text, List<String> tags) throws SagaException {
resourceManager = new ResourceManager();
resourceManager.registerProvider(
SagaJsonFactory.getInstance(
"{"
+ "\"name\":\"saga-provider\","
+ "\"type\":\"FileSystem\","
+ "\"baseDir\":\"testdata\""
+ "}"
)
);
tagManager = new TagManager(resourceManager, SagaJsonFactory.getInstance("{ \"resource\": \"saga-provider:saga_tags\"}"));
pipelineManager = new PipelineManager(resourceManager, SagaJsonFactory.getInstance("{ \"resource\": \"saga-provider:saga_pipelines\"}"));
engine = new SagaEngine();
engine.setResourceManager(resourceManager);
engine.setTagManager(tagManager);
SimpleReaderStage simpleReaderStage = new SimpleReaderStage(engine, SagaJsonFactory.getInstance("{ \"splitRegex\": \"[\\r|\\n]+\"}"));
pipelineManager.buildPipelineFor(engine, tags, simpleReaderStage);
engine.reset(new ByteArrayInputStream(text.getBytes(StandardCharsets.UTF_8)));
Vertex v = engine.advance();
}
public static void main(String[] args) throws SagaException {
List<String> tags = new ArrayList<>(Arrays.asList("tag1", "tag2", "tag3"));
String text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed egestas orci eu mauris luctus consequat.";
Main _instance = new Main(text, tags);
}
}
Something important to keep in mine is that advanced returns the first vertex of a text block, and that text block can represent the entirety of the text, but many time will only represent a fraction of the text; so we need to keep calling advanced again and
Related articles
Content by label
There is no content with the specified labels
Related articles
Overview
Content Tools