The basic structure for configuring a pipeline manager is as follows:
- The pipeline contains a list of components, a list of pipelines, and a list of stages for each pipeline.
- Each stage references a component by name from the list of components configured under the <components> tag.
The following execution parameters are configurable for the pipeline manager:
|queueSize||int||jobs||30||The size of the queue for processing jobs. If the job queue is full, then feeders, which attempt to put a new job on the queue, will be blocked until the queue has room. It is recommended that the queue size be at least as large as the number of threads, if not two or three times larger.|
|maxThreads||int||threads||10||The maximum number of threads to create to handle jobs put on the pipeline manager job queue.|
|The maximum time that a feeder (or, possibly, a sub-job that is enqueued) will be blocked for a full queue. If the queue is still full after the specified time, an exception error is returned to whomever is queuing up the job.|
|When shutting down the pipeline manager, the maximum time to wait for all active threads to complete.|
|gatherStatistics||boolean||false||Gather statistics about the stages and pipelines. This can be turned on or off via the UI as well.|
The components list specified in the pipeline manager is a simple list of components each with their custom configuration parameters. For more details, see the discussion under the Configuring Components section of the system configuration file documentation.
Note that all components configured under the <components> tag must be pipeline stages if they are to be referenced in a pipeline <stage> element.