After completing the Outbound Integration Endpoint wizard for an event-based OIEP using the STEP Exporter process engine, the OIEP editor includes additional parameters that must be set manually before the OIEP can run.
Note: In addition to the set up in this topic, additional manual configuration is required when using the Business Rule Based Message Processor, as defined in the OIEP - Configuration Section for Business Rule Based Message Processor topic.
The configuration set from the wizard can also be modified manually using the Configuration tab.
Follow these steps to complete the manual configuration and set the event-based OIEP to run:
- In the OIEP editor, configure to send an email if an endpoint-related background process fails as described in the 'Error Handling & Reporting' section of OIEP - Configuration Section.
- In the OIEP editor, determine when the OIEP runs as described in the 'Schedule' section of OIEP - Configuration Section.
- In the OIEP editor, when using the STEP Exporter process engine, specify the objects to be output, the format, and pre- or post-processors (if any) as described in OIEP - Event-Based - Output Templates Section.
- In the OIEP editor, determine how data is delivered as described in OIEP - Delivery Method Section.
- In the OIEP editor, when using the Business Rule Based Message Processor process engine, specify the pre- or post-processors (if any) as described in OIEP - Event-Based - Pre- and Post-processing Section.
- In the OIEP editor, when using the Business Rule Based Message Processor process engine, specify the business actions and settings as described in OIEP - Configuration Section for Business Rule Based Message Processor.
- In the OIEP editor, define what data changes will cause the OIEP to run as described in the OIEP - Event-Based - Event Triggering Definitions Tab.
- Set the Queue Status as described in Event-Based OIEP Status and Queue Status.
-
Determine the appropriate Event Mode setting.
-
Standard - The default for new configurations. Standard mode adds all events to a given queue for an object without further processing, which can include more events than necessary, resulting in duplication of processing and exports with the same outcome. Standard mode allows for backward compatibility and is recommended for these event queue configuration scenarios:
-
unique object-event type combinations (a single derived or republish event instead of multiple event triggers on attributes and references, etc.)
-
business rules that complete quickly
-
typically larger queue sizes (> 100k events)
-
large batch sizes (approximately 10k events)
-
exports that complete quickly without pre-processing or post-processing
In these scenarios, the Standard mode performs better than the Deduplicate mode since Deduplicate includes the overhead required to remove redundant events.
-
-
Deduplicate - Deduplicate mode removes events from a queue for Object-Event Type combinations that already exist on the queue within the same batch (meaning the Batch Size must be larger than 1). The Event Type being deduplicated is not the create / modify event type as displayed in the Current Event Batch dialog, but instead, can be observed by hovering over the create / modify entries in that dialog. Refer to the Event-Based OIEP Queued Events topic for information about the 'Current Event Batch' dialog.
Deduplicate reduces redundant business rule processing and exports and prevents downstream systems from receiving the same data when the same objects are present in multiple event batches, which also reduces processing required on downstream systems. The required overhead to remove redundant events saves overall processing time and improves performance in the following scenarios.
Note: Events are pruned from the queue based on the node and the Event type.
Deduplicate is the recommended mode for these event queue configuration scenarios:
-
publishing to PDX using the STEPXML template
-
high duplication of events waiting in the queue for processing
-
complex business rules / exports that take a significant amount of time to complete for a batch
-
downstream systems that have trouble keeping up with message processing
-
typically smaller queue sizes (< 10k events)
-
small batch sizes (< 10 events)
For more information, refer to the Event Mode: Deduplicate Example section below.
In the Performance Analysis Tool Activity tree (defined in the Performance Analysis documentation), logging has been added for the time spent deduplicating events when an outbound integration endpoint (OIEP) is configured with the Deduplicate Event Mode. Using this information, customers can make data-driven decisions regarding which is the best mode to use (Deduplicate or Standard), as the decision is highly dependent on the underlying use case and overall expected throughput. In some cases, the time it takes to complete the deduplication of events outweighs the potential performance gains in processing, while in other cases the processing itself is the bottleneck. Therefore, being able to compare the time spent deduplicating against the overall run time supports administrators in selecting the setting that will lead to the most optimal performance for their scenario.
Note: Deduplicate mode may not run as fast as Standard mode due to the analysis and removal of events when each batch is processed. When high volumes of unprocessed events are in queue (> 500k), possibly from a temporary stoppage where the events cannot be discarded, processing speed decreases as the event count grows. Consider using Standard mode temporarily until the queue has a more manageable backlog.
If the OIEP is unlikely to have multiple similar events for objects, the typical volume is low, the export and business rules are simple (batches complete very quickly), and/or the batch sizes are large, then the overhead from attempting to remove redundant events may result in slower execution than that of the Standard mode.
-
- Enable the endpoint and invoke it as described in Running an Outbound Integration Endpoint, paying particular attention to the 'Prerequisites for Event-Based OIEPs' section.
Event Mode: Deduplicate Example
Consider a set of 10 products that are created and then each modified two times. This scenario generates three events for each product. There are a total of 30 events (10 create events and 20 modify events) as shown below.
The OIEP is configured to Deduplicate for the 'Event Mode' parameter with 10 for the 'Number of Events to Batch' parameter. The 30 events are already on the queue as shown in the 'Unread events (approximated)' parameter. Although this image is for an OIEP, the same Event Processor settings could be set for these properties on the Event Processor tab within the Configuration section.
While it may seem that three batches would be generated, the third batch is eliminated (deduplicated) since it would have included the same Object-Event Type combinations.