Table of Contents |
---|
1.
...
Introduction
Tip |
---|
For |
...
a better understanding of what |
...
Saga is and what the purpose of the UI is, please check out |
...
this presentation: |
The UI is pretty simple
...
. It has a main tab selector at the top where you can
...
determine where you want to work
...
(Tags, Evaluate, Pipelines, Datasets, Rules/Executors or Background Processes).
A Login page can be seen when security is enabled in the config file, it uses basic authentication against the user and password in the configuration
Users can define semantic tags to be used, and the recognizers and settings that each tag will use.
Evaluate tab is used when the user needs to test several tags at once. It could be a quick test using the Preview functionality or starting a Test Run against a dataset. Test runs started in this screen are called 'Evaluations' because they include some statistics that can be used to compare 2 Evaluation runs that use different settings.
For each evaluation you can open the Search Interface or open detailed statistics:
Tip |
---|
In case you want to delete an Evaluation, it can be done in the 'Background Processes' tab. |
Users can add new, delete or update pipelines. Pipelines defines which stages are included in the pipeline before recognizer stages
...
are added to the pipeline. So for example, you could have stages
...
such as white space tokenizer, text case analyzer
...
or a stop words identifier.
...
The user has the ability to move stages around using up and down buttons:
Also stages can be inserted at the desired position using the context menu:
Users can view the datasets loaded into the application to perform test runs and/or training of machine learning models.
...
Define which fields to process from the dataset file and
...
how to split the text in order to feed the pipeline.
...
(Since users cannot upload datasets at the moment,
...
datasets need to be placed in a special folder in the Saga Server file system.
...
...
)
Saga can be the engine behind semantic search, as it is for the ESUI (Enterprise Search User Interface). The user can create rules and define what rules will be executed for each tag that Saga identifies in a query, through the configuration done in this tab.
Note |
---|
Saga provides integration with a custom ESUI. |
Users can monitor background processes that are running. For example, when running a test run against a dataset,
...
the process could take a long time to complete, so the user can check the progress in this screen.
...
...
Users can Export and Import all the data in Saga to .sg folder for backup or to import in another Saga
The environment tool will create and download a file with the current environment conditions, like RAM, CPU, hard drive space,...
Here is an example of the data the file will have
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
{
"add-ons": {
"processors": [
"GoogleKnowledgeStage:1.2.0",
"NamePredictorStage:1.2.0",
"FaqStage:1.2.0",
"ClassificationStage:1.2.0",
"GoogleEntityPredictorStage:1.2.0"
],
"recognizers": [
"GoogleKnowledgeStage:1.2.0",
"NamePredictorStage:1.2.0",
"FaqStage:1.2.0",
"ClassificationStage:1.2.0",
"GoogleEntityPredictorStage:1.2.0"
]
},
"java": "11.0.2",
"os": "Windows 10",
"elasticsearch": "7.4.2",
"cpu": {
"usage (%)": {
"jvm": 7.678,
"system": 0.0
},
"processors": 12,
"arch": "amd64"
},
"memory (Mb)": {
"jvm": {
"committed": 8377,
"using": 4096,
"max": 10240
},
"system": {
"total": 32503,
"using": 15610,
"free": 16893
},
"swap": {
"total": 42743,
"using": 27392,
"free": 15351
}
},
"config": "C:\\Saga\\il Master\\saga-server\\config\\config.json",
"version": "1.2.0",
"version-date": "2020-03-05T10:14:59.561-06:00"
} |
The user can manage the logging of Saga and overwrite the default values from the log4j2 configuration file, this changes will only persist as long as Saga is running
Users can review results from a Test Run.
...
The flow will be something like:
...
...
...
...
...
...
...
...
...
...
...
In case the user recognizes an entity in the text that is not being recognized by Saga it is possible to add it to the tag's entity recognizer dictionary by selecting the text and clicking the add button like this:
Note |
---|
Add to dictionary functionality is only available for test runs that use 1 tag. So it won't appear for Evaluation runs. |
...
In this section, we'll go through the process of creating a set of tags,
...
adding some recognizers to them and
...
testing how they perform against a dataset. This will give the user a better idea of
...
the process/flow
...
when using the
...
Saga UI.
We currently have a dataset loaded into
...
Saga about aviation incidents. We will try to identify incidents where
...
an engine
...
catches fire due to a bird collision.
...
An important
...
consideration is
...
which stages you want to include in the base pipeline used by recognizers.
...
Pipelines usually have a set of stages to pre-process text before passing it to the recognizers.
As you can see in the following image, we'll use the baseline-pipeline
...
that has the following stages:
...
...
...
...
...
...
...
...
...
...
...
...
...
...
We want to identify
...
three things:
If those
...
three things are present in an incident report, then we could
...
hypothesize that the incident is about engines
...
catching fire due to
...
bird collisions.
...
{bird}
tag:...
It may be that you misspelled the tag name or maybe you just changed your mind and want your tag to have a better name. For those cases you can use the Rename functionality by clicking in the context menu next to the tag name:
In the same way, you can use the Delete button to delete tags. Just be careful when deleting parent tags because the delete functionality will perform a cascading delete for all child tags.
2
...
. Add 'SimpleRegex' and 'Entity' recognizers to the bird tag:
...
...
3. Add the following patterns to the entity recognizer
...
: duck, ducks
4. Repeat the steps for each of the following patterns (you can also add their plural form as additional patterns):
...
...
...
4.
...
Add the following regex in the simpleRegex recognizer.
...
Info |
---|
|
...
|
...
5. Now, do the same for
...
fire and engine tags:
...
...
The idea here is to create a tag that will use {fire}
, {engine}
and {bird}
to identify a concept which is that the engine
...
caught fire due to
...
bird collisions. For this special tag, we'll use the Fragmented recognizer. This is an advanced recognizer that
...
tags text that contains the other
...
three tags in any order of appearance and that are close enough
...
to one other within the aviation report.
...
...
fire-by-bird
...
...
...
...
...
{fire} {engine} {bird}
. Make sure to ...
...
...
...
4
...
...
. For our feature example, make sure to remove the 'Text Breaker' stage from the 'Base Pipeline' in case it is present.
The 'Text Breaker' stage process a text block and split it by sentences using the language configured. It will use punctuation and language specifics to achieve this but in addition you can specify a list of additional 'breaker' characters.
For our example, we don't need the aviation incident to be split into sentences because each incident is a small portion of text. In case it has several sentences, they are all related to the same incident.
5. In Saga, by default the 'Whitespace Tokenizer' expects text blocks containing the 'SENTENCE' flag which is previously set by the 'Text Breaker' stage. Because we removed the 'Text Breaker' stage, we need to remove the 'SENTENCE' flag from the required list of flags in the 'Whitespace Tokenizer' stage configuration:
6. Make sure all of
...
the recognizers of all your tags are using the same pipeline (or the pipeline you need it to be).
Click on the gear icon in each one of the recognizers to open its settings and check the field 'Base Pipeline'
...
.
...
Test any of your tags using the preview functionality.
...
For example, let's test the {fire-by-bird}
tag.
...
...
...
...
A dialog with the
...
Saga graph is displayed. Note how the {bird}
, {fire}
, {engine}
and also the {fire-by-bird}
...
tags
...
are identified in the text
...
.
...
...
Once you have tested the performance of
...
your tags using the preview, then it might be a good idea to test it against bigger text.
At the moment
...
, Saga comes with several testing datasets
...
. However, you can also
...
create your own and upload
...
them to a special folder in
...
the Saga file system
...
(learn how to do it in the Datasets article)
{fire-by-bird}
tag, ...
...
...
...
...
...
...
3.
...
Select the
...
Background Processes
...
tab to
...
review the progress of the run.
...
4. Wait for
...
the test run
...
to complete or if you cannot wait you can click on the "Open search" button to see partial results while running.
5. When process is complete, click
...
on Open search
...
to open the search interface.
...
In this screen, you will find your tags as facets.
...
When selected, you'll see search results containing your tags.
In the following image, we are clearing facets and then selecting only {fire-by-bird}
to check the comments that talk about engines
...
catching fire due to
...
bird collisions.
...
...
6. After reviewing the results, you can continue iterating on
...
a process of reviewing results and tweaking your tags and pipelines to create the best model to use for
...
your specific use case.
...
Anchor | ||||
---|---|---|---|---|
|
...
The name entity recognizer uses Apache OpenNLP to tag text using an existing model (previously trained).
...
In addition, the recognizer can be used
...
with other recognizers to train a new model.
In order to use it as
...
a recognizer: Add the recognizer to your tag,
...
choose a model, select the probability threshold used to decide if something is a match or not, and
...
add normalization tags in case you want to cleanse and normalize the input
...
.
Let's use
...
the {human}
tag to test this functionality:
...
{human}
tag...
...
...
...
4. Attach the Name Entity recognizer to the {human}
tag.
...
5. Choose
...
a default model "en-ner-person.bin". This model
...
has been trained to identify English names of people. Note that the model is case sensitive so in the recognizer configuration change the base-pipeline to a stage before the Case Analysis one.
...
6. Enter the following text in the preview in order to check out the
...
Saga graph: "Several employees work from home, Joseph
...
and Paul are two of them
...
". As you can see in the following image, the recognizer tags 'Joseph' and 'Paul'
...
as {human}
:
In order to train a model, use another recognizer as the base and a dataset that has a good sample of the desired values to identify. In this case, we'll use the 'CFR-2018' dataset which contains regulations from the government.
{emissions-equipment}
. Attach the Entity recognizer and add the following patterns:2. Attach the Name recognizer to your tag and click Train.
3. Select the 'CFR-2018-title40' dataset and then Execute.
4. Check out the Background Processes tab to review the progress of the training.
5. Once the process completes:
emissions-equipment
tag. If you do see text tagged, then the model you trained is working.Anchor | ||||
---|---|---|---|---|
|
The Classifier recognizer is used to perform binary classification of sentences. It uses Apache OpenNLP internally and can be used as a recognizer and as a trainer (just like the Name Entity recognizer).
The difference between the Classifier recognizer and the Name Entity recognizer is that Name Entity is used to identity entities; it will tag a word. The classifier will tag an entire sentence and may use other algorithms not available in the Name Entity recognizer.
2. Then you can test its performance by running a test run.
3.2.2.2 Using it as a trainer
Training is the same as for the Name Entity recognizer. We need another recognizer to use as base, and a dataset with a good quantity of samples of the text we want to classify.
The following steps describe how to do a training. We'll use the 'Aviation' dataset and will try to tag sentences that talk about incidents with birds.
{bird}
tag that we created previously, and create a new one called {hit}
with the following patterns:2. Create a new tag called {bird-incident}.
3. Attach the Fragmented recognizer and add the following pattern:
4. Add the Classification recognizer to the {bird-incident}
tag, and select '--NONE–' in the 'Model' field.
Tip |
---|
Always remember to set this field to --NONE-- when training. |
5. Select Train. When the dialog opens:
So this will train a model using the Aviation dataset using the pattern in the Fragmented recognizer.
6. Check out the 'Background Processes' tab and wait for the Classification training to complete.
7. Once done, go back to the {bird-incident}
tag and disable the Fragmented recognizer.
8. In the Classification recognizer, select your latest created model in the 'Model' field. It should be named like: bird-incident-[datetime stamp here].bin. For example: 'bird-incident-20190208173305.bin'.
Tip |
---|
You can also use the option '--LATEST–' to always use the latest trained model. |
9. Start a test run using the Aviation dataset.
10. Check out the Background Processes tab for completion.
11. When complete, select Open Search to check results in the Search screen.
As you can see, the Classification recognizer is tagging some sentences that in theory are supposed to be related to incidents with birds. For this case, because the dataset is small and positive samples identified by the Fragmented recognizer were not that much, the Classification recognizer is not doing a very good job of identifying the sentences. It is expected that with more and better training data, the accuracy of the Classification recognizer will improve.
Tip |
---|
You can also play with various training settings to determine which ones generate better results for your specific use case. |
This section describes Saga recognizers that were not used in any of the examples explained in previews sections.
This recognizer is part of the functionality that Saga has in place to implement semantic search in ESUI.
Best Bets feature in most search interfaces will highlight important information when certain keywords are detected in the query done by the user. Best Bets hits are not coming from the search engine itself but from a curated list of hits.
So let's say that when someone searches for "how to handle fire in a plane" best bets will detect the keywords 'fire' and 'plane' and therefore will show before the search results a link to the company 'How to handle fire' manual. This manual is the official and recommended source of information for those specific cases.
Therefore the idea behind the Best Bets recognizer is to keep the curated list of best bets in Saga and then provide the information to ESUI when a query has been tagged with a best bet tag.
{fire-manual}
3. Add a new best bet pattern. This record will contain the following information:
4. Now that the tag and recognizer are correctly set we need to create a rule and executor that ESUI will run to properly show the best bet in the search results page.
4.1 Go to Rules/Executors tab
4.2 Click on the Executors sub tab
4.3 Add the following code in the "Process" code section.
Please notice:
const result = {
title: data.title,
description: data.description,
url: data.url,
};
return _saga.response(result, 'bestBets')
Your screen should look like this:
4.4 Click on the 'Rules' sub tab and add a new rule that uses the executor we created in the previous step:
4.5 Add the {fire-manual}
tag in the ESUI saga endpoint configuration ('tags' property)
4.6 When querying something with the word 'fire' which matches our best bet recognizer, your should see a best bet hit in the search results in ESUI:
...