You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »


Building a Pipeline

1) Create the pipeline file

  • Start by creating a python file for the pipeline configuration in the folder config/pipeline

    A descriptive name works the best



On The File


2) Import the Pipeline class and stages

  • All stages should be in the package app.pipeline.stages
  • Pipeline class is in app.pipeline
from app.pipeline import Pipeline
from app.pipeline.stages import DemoStage

3) Declare a Variable  & Generate an Instance of Each Stage

  • The parameters of the stage may require to import other clases
_demo = DemoStage(
    enable=True,
    name='demo',
    save_to_intermediate=False,
    expand_result=False,
    ui_only=None,
    halt_on_exception=False
)

4) Define the PIPELINE Variable and Assign an Instance of the Pipeline Class

  • Pipeline class has 2 required parameters
    1. Enable: default true, if false pipeline will not load
    2. Stages: all the stages in the order to execute
PIPELINE = Pipeline(
    enable=True,
    stages=[
        _demo
    ]
)

5) Verify Pipeline Status

  • Starting the server check the logs for the entry loading the pipeline
  • In case of error, the logs will show the issue
    • Either unable to load the pipeline or unable to initialize it
    • Otherwise you should see Pipeline <name> loaded

  • You can go to /es/docs and execute the GET pipeline endpoint and see if the name appears

     
  • No labels