Manager nodes are responsible for the Aspire UI, crawl control, the allocation of work to worker nodes and monitoring nodes which are down and taking corrective actions (such as designating a new main manager node and correcting the status of items that were being processed during failure).

A manager node is responsible for the seeds actively being crawled. Where there is more than one manager, a seed and all the documents it contains, are the sole responsibility of one manager. Seeds that are controlled by the same throttle policy are assigned to same manager node, so these seeds can be throttled together. Allocation of seeds to managers is performed at the time a crawl starts. Where there is more than one manager in a system, responsibility for seeds is distributed across the managers as evenly as possible, within the constraints imposed by the throttle policies.

Main Manager 

One of the manager nodes is designated as the main manager. This manager is responsible for:

• Noticing that a Manager Node has stopped and beginning the process to reassign the seeds allocated to that manager to others.
• Noticing that a Manager Node has started and beginning the process to assign seeds from other managers to it (without breaking the throttling policies).
• Noticing that a worker node has stopped and “resetting” the status of any jobs that were executing on that worker so they can be picked up and processed by another worker.

If when a manager starts, a main manager is already running, the manager will assume a “non-main” role.

Main Manager Election

A single main manager node must always exist, and so when a manager starts, or when a manager node fails, the managers co-ordinate to ensure that an active main manager exists. On start-up, a manager will check that an active main manager exists. If it does, then the manager simply joins the cluster. If an active main manager does not exist, the managers collaborate to choose a new main manager.