BG Processes Execution Management

Users have a simplified background process (BGP) execution mechanism to prioritize the order in which background processes are handled.

Waiting BGPs can be prioritized for execution based on creation time and a priority. Priority can be set for event processors, OIEPs, and IIEPs (for the latter two, the configurable priority is for the 'worker' BGPs started by the endpoints). The application server capacity determines how many BGPs can run in parallel.

The background process execution mechanism selects BGPs to start across all BGPs waiting to be executed. By default, this mechanism is not enabled. This can be enabled via the configuration property BackgroundProcess.Scheduler.PluginID=OneQueueBGPScheduler.

Important: Before applying to a production setting, the background process execution mechanism should be applied and thoroughly tested on lower-level environments.

When the mechanism is enabled:

Existing configurations that assign BGPs to queues and set the degree of parallelism for the queues is ignored.
The 'Queue Info' BGP editor tab is no longer displayed.

You cannot specify queues for IEPs and event processors.
You can specify a priority for the event processor BGP and BGPs started by IEPs from these configuration types.

Each application server has a fixed number of 'BGP Workers' available for processing BGPs. The number of available BGP Workers is set automatically, depending on the application server hardware configuration. Individual application servers poll the list of waiting BGPs with regular intervals and start as many processes as they have capacity for (via available BGP Workers). The waiting BGPs are prioritized for execution first based on priority and the on creation time ('Start at' for scheduled event processor BGPs).

Poller processes for IIEP, OIEPs, and BGP scheduler processes (BGP type 'Schedule') bypass the prioritization and are always allowed to run.

Background Process Execution Overview

Admin users with the 'View Background Processes of Other Users' setup action can monitor the execution of background processes via the Background Process Execution Overview link available on the BG Processes tab in workbench.

The Background Process Execution Overview is divided into these sections: Servers, Running Background Processes, and Waiting Background Processes.

Note: The overview is not automatically updated.

To update the data, as shown in the image below, click the general workbench 'Force Reload' button or click the 'Execution Overview' link again.

Servers

The Servers section provides an overview of the application servers in the STEP cluster with details about the number of server CPUs, the total number of BGP Workers ('Max Workers'), the number of active BGP Workers, and the CPU load. Application server details are not refreshed every time the execution overview is refreshed, therefore, there is a timestamp in the Latest Snapshot column for when the data was collected.

Running Background Processes

The Running Background Processes section shows the BGPs currently being executed by STEP. The following is a list of the columns associated with Running Background Processes:

ID – The ID of the BGP (link).
Description – The BGP Description (link).
Started By – ID of the STEP user who started the BGP.
Progress – BGP Progress.
Created – Time that the BGP was created. For scheduled event processor BGPs, this shows the 'Start at' time.
Started – Time that the BGP was started.
Workers – The number of BGP Workers used for executing the BGP. This can be more than one for BGPs using the parallel execution / multi-threading framework. Refer to the Parallel Processes section below for more details.
Running on – The application server executing the BGP.
Type – The background process type.
Owner – The IIEP, OIEP, event processor, or setup entity that created the BGP.
Priority – The priority of the BGP.

Waiting Background Processes

The Waiting Background Processes section shows the BGPs waiting to be executed. The columns shown are:

ID – The ID of the BGP (link).
Description – The BGP Description (link).
Started By – ID of the STEP user who started the BGP.
Created – Time that the BGP was created. For scheduled event processor BGPs, this shows the 'Start at' time.
Priority – The priority of the BGP.
Type – The background process type.
Owner – The IIEP, OIEP, event processor, or setup entity that created the BGP.
Depends Upon – IDs of BGPs potentially blocking the execution when the BGP depends on other BGPs to complete before they can start.

Priority

Waiting BGPs are prioritized for execution based on the priority of the BGP and the created time.

Choose a method to set BGP priority:

When configuring event processors, IIEPs, and OIEPs, users with maintain permissions can set a Low / Medium / High priority. The default value is 'Medium.'
- For IIEPs and OIEPs, this is the priority of BGP Workers created by the endpoints.
- For event processors, this is the priority of the BGP associated with the event processor.
Users with the 'Manage Background Process Execution' setup action can set the priority of waiting BGPs directly in the workbench Background Process editor. This allows admin users to make ad hoc changes to how waiting processes are prioritized. By default, BGPs other than event processors and IEPs have priority level 'Medium.'

Important: It is recommended to initially leave all event processor and IEP priority settings at their default 'Medium' value. This means that BGPs created / managed by these configuration types have the same priority as user-initiated BGPs (and BGPs created / managed by other event processors / IEPs). Changing the priority to 'Low or 'High' should only be done if it is verified that a specific integration / background process should always have lower / higher priority than other BGPs being executed on the system.

Parallel Processes

With the priority mechanism enabled, the degree of parallelism / number of threads per queue set via the configuration properties for processes supporting parallel execution (multi-threading) are no longer honored. Instead, the number of threads / BGP Workers to use for a BGP are automatically determined based on the application server hardware.

Multi-threaded BGPs can initially be inactive, even though they have been claimed by a server and appear in the 'Running Background Processes' section (with a status of 'running'). This occurs when the server does not have the desired number of BGP workers available. In this case, no new BGPs on the server start until the desired number of workers is available and the multi-threaded process can start.

For OIEPs only, the user can specify the maximum number of threads / workers to use for processes started by the endpoint. This option limits the number of threads / workers only and cannot be used to increase the amount beyond what the server hardware configuration allows.