Aggregation Stage will generate the engine specific aggregations based on the configuration provided, specifically with the list of DynamicAgg in the aggs configuration parameter, and store them in the intermediate under the stage name.
This stage can also generate the filters from the selected aggregations, which will look in the key aggs in the intermediate dictionary in the process function, the generated filters will be stored in the intermediate under the name provided in filters_name
Additionally, the aggregation stage can also process the resulting aggregations from a engine search, and transform them into a UI friendly format.
Properties
| Property | Description | Default | Type | Required |
|---|---|---|---|---|
| aggs | Aggregations to be performed within this stage | [] | array of DynamicAgg | No |
| type | Stage class name | - | string | Yes (but only while working with the JSON Format) |
| enable | Enable stage for execution | true | boolean | No |
| name | Name of the key in which the aggregations will be stored in intermediate | "aggregations" | string | No |
| save_to_intermediate | If true, the result of the stage will be stored in the intermediate instead of the final section | false | boolean | No |
| expand_result | Indicates if the result of this stage should be expanded into the final data dictionary instead of being appended as usual | false | boolean | No |
| ui_only | Section specific for UI configuration. This configuration will be retrieved when necessary and should affect the process of the stage | object | No | |
| halt_on_exception | Indicates if, in case of an exception, the pipeline should be interrupted | false | boolean | No |
| filters_name | Name of the key in which the aggregation filters will be stored in intermediate. Generated from a list of SelectionAgg stored in the aggs parameter in intermediate | "aggregation_filters" | string | No |
| engine_name | Name of the engine to use for this query. If none is provided, the default engine will be used | string | No | |
| search_response | Name of the key in the final result where the query results will be stored | "search" | string | No |
| ui_aggregations | Determines if UI aggregations are enabled | true | boolean | No |
Example Configuration
_aggregation_stage = AggregationStage(
enable=True,
name='aggregations',
filters_name='aggregations_filters',
save_to_intermediate=False,
expand_result=False,
engine_name=DEFAULT_ENGINE_NAME,
search_response=SEARCH_STAGE_NAME,
ui_aggregations=True, # This over here generates the aggregations for ui, if you don't want them, set it to False
aggs=[
TermAgg(
id='genres',
field='metadata.genres.name.keyword',
must=True,
max_values=100,
multi_select=True,
operation='or',
order=None,
exclude_terms=None,
include_terms=None,
ui_only=TermAggUI(display_name='Genres', max_display=5, expanded=True),
aggs=[
TopHitsAgg(
id='top_hits',
type='top_hits'
),
TermAgg(
field='metadata.spoken_languages.name.keyword',
negated=False,
must=True,
id='spoken_languages',
ui_only=TermAggUI(display_name='Spoken Languages', max_display=5, expanded=False),
aggs=None,
max_values=None,
multi_select=False,
operation='or',
order=None,
exclude_terms=None,
include_terms=None
)]
),
BoolAgg(
id='adult',
field='adult',
negated=False,
must=True,
ui_only=BoolAggUI(display_name='Adult', display_True='True', display_False='False')
),
DateHistogramAgg(
field='release_date',
negated=False,
must=True,
id='release_date_h',
ui_only=DateHistogramAggUI(display_name='Release Date'),
calendar_interval=['1y', '1q', '1M', '1d'],
format=['yyyy', '\'Q\'q', 'MMMM', 'E dd'],
time_zone=['-06:00'],
min_doc_count=[100, 1],
offset=None,
order=None
),
HistogramAgg(
id='budget',
field='metadata.budget',
negated=False,
must=True,
interval=[10000000, 1000000],
min_doc_count=[100, 10, 1],
order=None,
offset=None,
ui_only=HistogramAggUI(display_name='Budget')
),
SliderAgg(
id='budget_slider',
field='metadata.budget',
negated=False,
must=True,
ui_only=SliderAggUI(display_name='Budget', translate=None, min=None, max=None, show_ticks=True),
),
DateRangeAgg(
field='release_date',
negated=False,
must=True,
id='release_date_dr',
ui_only=DateRangeAggUI(
display_name='Release Range',
hide_count=False,
translate=None
),
format='MM-yyyy',
ranges=[
DateRange(
key='minus_10',
start='now-10y',
start_display=None,
end=None,
end_display=None
),
DateRange(
key=None,
start='01-1960',
start_display=None,
end='01-1970',
end_display=None
),
DateRange(
key=None,
start=None,
start_display=None,
end='now-10M/M',
end_display=None
)
]
),
RangeAgg(
field='metadata.budget',
negated=False,
must=True,
id='budget_r',
multi_select=False,
ui_only=RangeAggUI(
display_name='Budget Range',
hide_count=False,
translate=None
),
ranges=[
Range(
key='< 10,000,000',
start=None,
end='10000000'
),
Range(
key='> 10,000,000',
start='10000000',
end=None
),
Range(
key='50 to 100 millions',
start='50000000',
end='100000000'
)
],
)
]
)
Selected Aggregations
The Aggregation stage can process selected aggregations, these required to be of type SelectionAgg, or a JSON representation of it.
Example of selected aggregations in intermediate
{
"aggs": [
{
"id": "genres",
"values": [
"Drama"
],
"negated": [],
"level": 0
},
{
"id": "adult",
"values": [
false
],
"negated": [],
"level": 0
}
]
}
Overview
Content Tools