Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This stage moves data from the intermediate to the final. If fields are included in either include_in_process or include_in_post_process, only these fields will be included. If no include fields are defined, all fields will be included. For exclude, if any field is in exclude_in_process or exclude_in_post_process, these fields will be excluded from whatever was included.


Properties

PropertyDescriptionDefaultType

type

Stage class name-string

enable

Enable stage for executiontrueboolean

name

Name for this specific stage. Used on the intermediate and final parameters.nullstring, null

save_to_intermediate

If True the result of the stage will be store in the intermediate instead of the final section.falseboolean

expand_result

Indicates if the result of this stage, should be expanded into the final data dictionary instead of appended as the standardfalseboolean

ui_only

Section specific for UI configuration. This configuration will be retrieved when necessary.nullobject, null

halt_on_exception

Indicates if in case of an exception, the pipeline should be interruptedfalseboolean

include_in_process

List of fields to include from the intermediate to the finals. The names can be jmespaths.nullarray, null

exclude_in_process

List of fields to exclude from the intermediate to the finals. The names can be jmespaths. Exclude will apply after include.nullarray, null

include_in_post_process

List of fields to include from the intermediate to the finals. The names can be jmespaths.nullarray, null

exclude_in_post_process

List of fields to exclude from the intermediate to the finals. The names can be jmespaths. Exclude will apply after include.nullarray, null


Example Configuration

Code Block
languagepy
themeDJango
_dump_stage = DumpFinalStage(
    enable=True,
    name="dump",
    expand_result=True,
    halt_on_exception=True,
    include_in_process=["field1", "field2"],
    exclude_in_process=["field3"],
    include_in_post_process=["field4"],
    exclude_in_post_process=["field5"]
)

to include all just don't define any include or exclude field

Code Block
languagepy
themeDJango
_dump_stage = DumpFinalStage(
    enable=True,
    name="dump"
)

...