Data Exchange Enhancements and Changes

Summary

The following enhancements to STEP's data exchange functionality have been made as part of the 2024.2 update. These changes are outlined below and described in the Details section that follows:

• A new Kafka Streaming Receiver option on IIEPs (inbound integration endpoints) allows integration with multiple partitions on Apache Kafka without the need for individual background processes. This new receiver provides near-real-time updates to STEP data from external systems, results in processing more data faster, and takes advantage of standard Kafka capabilities.

• Change Package improvements include better usability and visibility of the Impact Analysis data, which clarifies the expected result of installing a change package. Also, users can split elements out of a Web UI configuration when using VCSI (Version Control System Integration) and can handle unapproved objects more efficiently when sealing a change package. These updates make it easier to manage and work with change packages.

• Performance improvements to GDSN Receiver increase the number of messages that can be processed daily for SaaS systems.

Details

New Kafka Streaming Receiver

The new IIEP (inbound integration endpoint) 'Kafka Streaming Receiver' enables a STEP platform integrated with Apache Kafka to read messages from single or multiple partitioned Kafka topics without using individual background processes. The streaming receiver provides near-real-time updates to STEP data from external systems, results in processing more data faster, and takes advantage of standard Kafka capabilities. This receiver is designed for fast imports of small messages representing a single object and imposes a 1 MB maximum message size on a Kafka topic. Import information specific to the streaming process is included in new dedicated 'message-streaming' log files, and message counts over time are summarized on the Statistics tab. Additionally, the Error Handling Strategy parameter gives users the option to stop the IIEP when any error is encountered, ignore errors and continue processing, or send messages with errors to an Error Topic, along with error details in the message header, and continue processing.

For more information, refer to the Kafka Streaming Receiver topic in the Data Exchange documentation.

Change Package improvements

The following updates are included to improve the process of managing configurations across environments when working in DTAP (Development, Testing, Acceptance and Production) environments:

• The improved ‘Impact Analysis’ (previously called 'Impact Report') functionality clearly displays the expected result of installing items in a change package in a new ‘Install Preview’ column to minimize risk by providing transparency and predictability. The analysis uses a background process to perform a simulated import and displays results in the ‘Primary Items’ and 'Secondary Items' sections.

  The existing 'Current' column has been renamed 'Before Install' to identify the state of items in the change package compared to the system. The ‘Before Install’ and ‘Install Preview’ columns display icons to signify the state of the object, including new, unchanged, changed, or manually updated. Icons also warn when an object cannot be imported.

  

• Using the Change Package Git Delivery method when moving Web UI objects between systems via change packages, the new 'Split Web UI Configurations' parameter allows users to work on individual screens for a Web UI and import those into other systems. This allows admins in Git to identify the screens that have changed and to select the screens that should be promoted based on the system purpose. For example, Web UI screens that are being developed can be withheld from test and production systems, while screens ready for test can be promoted individually. Previously, users were required to import a full Web UI and manually separate the needed Web UI elements in the STEPXML files.

  Also in the Change Package Git Delivery method, templates can now be used to define the Git Branch. This provides higher flexibility when working with multiple Git branches by defining the target branch dynamically. For example, using a description attribute value on the change package eliminates needing multiple copies of change packages and/or OIEPs per branch.

  

• Sealing a change package now includes a dialog with links for the first 10 unapproved objects from the Tree which makes it easier to address the required approval process, and improves usability and efficiency. The count of other objects that require approval is also displayed on the 'Unapproved Items' dialog.

For more information, refer to the VCSI: Change Package Git Delivery Method in OIEP topic, the VCSI: STEPXML Joiner Pre-processor Options in IIEP topic, the Finalizing a Change Package topic, and the Analyzing and Installing Change Packages topic, all in the Configuration Management documentation.

GDSN Receiver performance improvements

Enhancements have been implemented to augment the performance of the GDSN Receiver, optimizing its processing of incoming files. This allows customers to better meet their daily load targets. Now, SaaS v2 systems can receive 50,000 CIN files daily.

To enable the GDSN Receiver solution, several configuration properties need to be set. There is also a new event processor (GDS_NodeDeleteQueue) that is installed automatically with the easy setup of GDSN Receiver that customers must enable. For details, refer to the GDSN Receiver Solution Enablement topic in the Solution Enablement: GDSN Receiver documentation. This issue is also covered in the Miscellaneous Bugfixes update note; refer to ISSUE-653092.