The Parallel Wrapper is a powerful feature in GAIA API that enables you to execute multiple stages simultaneously, in parallel. By utilizing this wrapper, you can improve the overall efficiency and speed of your search pipeline.
When using the Parallel Wrapper, you need to specify the stages that you want to execute in parallel. These stages will receive the same intermediate and final data, and they will operate independently without any communication between each other. This means that the stages inside the wrapper will not share information or interact during their execution.
It's important to note that the Parallel Wrapper is designed to enhance performance by parallelizing the execution of stages. However, keep in mind that the behavior and outcome of each stage may still depend on the data and configurations specific to that stage.
Take into account, all stages specified inside this wrapper will receive the same intermediate and final, and there will be no communication between each other
Properties
| Property | Description | Default | Type | Required |
|---|---|---|---|---|
| type | Stage class name | - | string | Yes |
| enable | Enable stage for execution | true | boolean | No |
| name | Name for this specific stage | "parallel" | string | No |
| save_to_intermediate | If true, the result of the stage will be stored in the intermediate instead of the final section | false | boolean | No |
| ui_only | Section specific for UI configuration that affects the process of the stage | object | No | |
| halt_on_exception | Indicates if the pipeline should be interrupted in case of an exception | false | boolean | No |
| stages | List of stages to parallelize | - | array | Yes |
| timeout | Argument in seconds to impose a limit on how long you're willing to wait for each task to complete | - | integer | No |
Example Configuration
_parallel = ParallelWrapperStage(
name='parallel',
stages=[
_search1,
_search2
],
timeout=30
)
Overview
Content Tools