The recommended BGP execution mechanism is 'One Queue' and implements a priority parameter for BGPs and prioritizes the order in which background processes are handled. Waiting BGPs are prioritized for execution based on creation time and a priority setting. 'One Queue' is used when the BackgroundProcess.Scheduler.PluginID configuration property is set as OneQueueBGPScheduler.
Important: Before applying 'One Queue' to a production environment, first apply and thoroughly test the background process execution mechanism on lower-level environments.
The following sections define the functionality available when using 'One Queue':
Priority Mechanism
When using the 'One Queue' background process execution mechanism, priority is set for event processors (EPs), outbound integration endpoints (OIEPs), and inbound integration endpoints (IIEPs). The number of workers on a STEP environment determines how many BGPs can run in parallel, and if a newly created BGP is allowed to start. This mechanism selects BGPs to start across all BGPs waiting to be executed. For integration endpoints (IEPs), the configurable priority is for the 'worker' BGPs started by the endpoints.
Each application server / node has a fixed number of 'BGP workers' available for processing BGPs. The number of available BGP workers is set automatically, depending on the environment size. Individual application servers / nodes poll the list of waiting BGPs with regular intervals and start as many processes as capacity allows (via available BGP workers, and if sufficient heap memory is available). The waiting BGPs are prioritized for execution first based on priority and then on creation time ('Start at' for scheduled EP BGPs).
Poller processes for IEPs and BGP scheduler processes (BGP type 'Schedule') bypass the prioritization and are always allowed to run.
Note: New 'One Queue' background processes can only start when there is adequate heap memory. By default, at least 10 percent of the total amount of heap memory on an environment must be available to start a new background process. When 20 percent of the total amount of heap memory is free, new BGPs are started without restraint.
When the 'One Queue' mechanism is enabled:
-
BGPs assigned to queues and the degree of parallelism for the queues on existing configurations are ignored. Refer to the Parallel Processes section below for more information.
-
Queues cannot be specified for IEPs and EPs.
-
The 'Queue Info' BGP editor tab is not displayed. It is only available on systems using the legacy setting.
-
Background process priority is displayed on the Configuration section of EPs and IEPs. Priority can be set via the 'Edit Configuration' link (which opens the wizard) for EPs and IIEPs and for OIEPs set the priority directly in the editor via a dropdown.
Background Process Execution Overview
Admin users with the 'View Background Processes of Other Users' setup action can monitor the execution of background processes via the 'Execution Overview' link available on the BG Processes tab in workbench.
The 'Background Process Execution Overview' includes sections for Servers, Running Background Processes, and Waiting Background Processes.
Note: The overview is not automatically updated. To refresh the data, click the general workbench 'Force Reload' button (
) or click the 'Execution Overview' link again.
Servers
The Servers section of the Background Process Execution Overview includes the application servers / nodes that are available (which can be multiple in a clustered setup) with details about the total number of BGP workers ('Max Workers'), the number of active BGP workers, and the CPU load. Application server / node details are not refreshed when the execution overview is refreshed, instead, the timestamp in the Latest Snapshot column indicates when the data was collected.
Running Background Processes
The Running Background Processes section shows the BGPs currently being executed by STEP. The following is a list of the columns associated with Running Background Processes:
-
ID – The ID of the BGP with a link displayed when hovering.
-
Description – The BGP Description with a link displayed when hovering.
-
Started By – ID of the STEP user who started the BGP.
-
Progress – BGP Progress.
-
Created – Time that the BGP was created. For scheduled EP BGPs, this shows the 'Start at' time.
-
Started – Time that the BGP was started.
-
Workers – The number of BGP workers used for executing the BGP. This can be more than one for BGPs using the parallel execution / multithreading framework. Refer to the Parallel Processes section below for more information.
-
Running on – The application server executing the BGP.
-
Type – The background process type.
-
Owner – The IIEP, OIEP, EP, or setup entity that created the BGP.
-
Priority – The priority of the BGP.
Waiting Background Processes
The Waiting Background Processes section shows the BGPs waiting to be executed. The columns shown are:
-
ID – The ID of the BGP with a link displayed when hovering.
-
Description – The BGP Description with a link displayed when hovering.
-
Started By – ID of the STEP user who started the BGP.
-
Created – Time that the BGP was created. For scheduled EP BGPs, this shows the 'Start at' time.
-
Priority – The priority of the BGP.
-
Type – The background process type.
-
Owner – The IIEP, OIEP, EP, or setup entity that created the BGP.
-
Depends Upon – IDs of BGPs that potentially block the execution when the BGP depends on other BGPs to complete before they can start.
Priority Parameter Setting
Waiting BGPs are prioritized for execution based on the priority of the BGP and the created time.
Choose a method to set BGP priority:
-
When configuring EPs, IIEPs, and OIEPs, users with maintain permissions can set a Low, Medium, or High priority in the applicable wizard. The default value is 'Medium.'
-
For IIEPs and OIEPs, this is the priority of BGP Workers created by the endpoints.
-
For EPs, this is the priority of the BGP associated with the event processor.
-
-
Users with the 'Manage Background Process Execution' setup action can set the priority of waiting BGPs directly in the workbench Background Process editor. This allows admin users to make ad hoc changes to how waiting processes are prioritized. By default, BGPs other than EPs and IEPs have priority level 'Medium.'
Important: It is recommended to initially leave all EP and IEP priority settings at their default 'Medium' value. This means that BGPs created and/or managed by these configuration types have the same priority as user initiated BGPs (and BGPs created or managed by other EPs or IEPs). A change in priority to 'Low' or 'High' is only recommended when it is verified that a specific integration or background process should always have lower or higher priority than other BGPs being executed on the system.
Parallel Processes
With the priority mechanism enabled, the degree of parallelism / number of threads per queue set via the configuration properties for processes supporting parallel execution (multithreading) are no longer honored. Instead, the number of threads / BGP workers to use for a BGP are automatically determined based on the environment size.
Multi-threaded BGPs can initially be inactive even though they have been claimed by an application server / node and appear in the 'Running Background Processes' section (with a status of 'running'). This occurs when the application server / node does not have the desired number of BGP workers available. In this case, no new BGPs start until the desired number of workers is available and the multi-threaded process can start.
For OIEPs only, the user can specify the maximum number of threads / workers to use for processes started by the endpoint. This option limits the number of threads / workers only and cannot be used to increase the amount beyond what the environment allows.