Data Exchange Enhancements and Changes

Summary

The following enhancements to STEP's data exchange functionality have been made as part of the 2024.1 update. These changes are outlined below and described in the Details section that follows:

  • Update to PDX Onboarding Channel Configurator adds Market Dimension and Supplier Contexts. Default Supplier Contexts and Asset Reference Metadata Group are introduced. Some parameters are removed.

  • PDX Onboarding via Product MDM updates benefit PDX onboarding and reduce limitations around PDX functionality such as list view attributes, product families, and searchable attributes.

  • Encryption of data and assets sent from STEP to PDX is possible via Amazon Web Services (AWS) Key Management Service (KMS). This improvement provides a stronger authentication and encryption option for integrations with PDX.

  • Kafka message key and headers metadata can now be generated via a business function selected in the Kafka Delivery method. This improvement provides better control for ensuring that messages for specific objects end up in the same partition every time.

  • The STEPXML, CSV, and Excel formats now allow the exporting of empty calculated attributes. This improvement provides a consistent output of existing, absent, and deleted values for valid attributes on an object. Additionally, the Advanced STEPXML format now allows an OIEP to output change flags on deleted data and allows empty values and deleted data to be output simultaneously.

  • New context specific replacement rules for STEPXML imports allow for replacement of values, cross references, classification references, and product references specific to selected contexts. These improvements allow users to make replacements in one or more specified contexts, without changing existing functionality related to data replacement upon import.

  • MongoDB support for the outbound integration endpoint (OIEP) delivery method is updated to include version 7.0. Versions 3.6 and earlier are no longer supported.

  • Enhancements to the ECLASS Advanced Editor Screen offer streamlined data filtering, mandatory field indication, block naming, block name editing, and read-only mode for improved usability.

  • Change codes will now be imported into a metadata attribute on the attribute-classification link using the ETIM Taxonomy Importer.

Details

PDX Onboarding Channel Configurator UI extension

Updates to the PDX Onboarding Channel Configurator introduces new parameters to enhance configuration options:

  • Market Dimension: This optional parameter within the Channel Properties tab allows users to specify the STEP dimension that represents market data for their STEP installation.

  • Supplier Contexts: This optional parameter within the Supplier Classification tab, when configured with an attribute, is used to identify the contexts that a given supplier should maintain. The values are used to drive the markets and languages that the supplier will be exposed to in the PDX application. It is a prerequisite for this functionality that the Market Dimension parameter on the Channel Properties tab is configured.

  • Default Supplier Contexts: This optional parameter within the Supplier Classification tab contains default primarily supplier-facing contexts, and is used when no supplier-specific setup is configured. These contexts determine the markets and languages visible to the supplier by default within the PDX application. It is a prerequisite for this functionality that the Market Dimension parameter on the Channel Properties tab is configured.

  • Asset Reference Metadata Group: This optional parameter within the Supplier Data Definition tab can be configured with an attribute group comprising all vendor-facing asset reference metadata. When populated, all assets in PDX are treated as composites. Each composite includes an asset attribute and attributes from this attribute group add additional columns to the composite.

Further, the following parameters are removed from the PDX Onboarding Channel Configurator to enhance configuration options:

  • Location Reference Type

  • Primary Location Reference Type

  • Location Key

After publishing the PDX Onboarding Channel Configurators, these fields are no longer included in the channel configuration XML.

For more information, refer to the Supplier Data Definition Tab, Channel Properties Tab, and Supplier Classification Tab topics in the Data Integration documentation.

PDX Onboarding via Product MDM

Enhanced capabilities for the 2024.1 update of Configured Product MDM Channels now include the following:

  • Control over the channel-specific List View attributes: It is now possible to configure, from STEP, the setup of attributes that can be selected as List View attributes. Previously, these attributes could only be chosen between a few hardcoded fields.

  • Support, in PDX, of the STEP 'Product Family' object (a two-level Parent/Child structure): This will allow, from PDX, to onboard a Product Family and, from STEP, to push a Product Family structure or maintenance.

  • Control over the channel-specific search attributes: It is now possible to configure, from STEP, the setup of attributes that can be used in the channel search. Previously, searching or filtering products in a channel-specific category in PDX was limited to a few fixed attributes.

Encryption of data and assets from STEP to PDX

Integration with Product Data Exchange (PDX) now allows optional encryption of data and/or assets and in-transit using Amazon Web Services (AWS) Key Management Service (KMS).

  • For data, an outbound integration endpoint (OIEP) using the 'Product Data Exchange 2' delivery method now includes a new 'Encryption Configuration' parameter.

  • For assets, the new Encrypted Blob Storage option in a gateway integration endpoint (GIEP) works with an event processor using the Asset Publisher processor.

Encryption setup for both data and assets is managed by several new sharedconfig.properties entries which define options for dropdown selections in the OIEP and GIEP.

For environments using PDX, configuration is required on the PDX system to implement AWS for asset delivery and/or AWS encryption. Contact Stibo Systems for information.

For more information and configuration details, refer to the Product Data Exchange 2 Delivery Method topic, the Configuring a Gateway Integration Endpoint - Encrypted Blob Storage topic, both in the Data Exchange documentation, and the Asset Publisher topic in the Digital Assets documentation.

Kafka message key and headers metadata

For Kafka deliveries, users now have better control of message keys for each message. Users can also define header values for messages which may be required by downstream systems. This allows users to better ensure that messages for a product ends up in the same partition every time. The new 'Message Metadata Source' parameter allows using a template or a business function (with 'String' as an 'Input Parameter' and 'Map<String,String>' as a 'Return Type') to provide the metadata based on content parsed from the output message. The delivery method delivers the payload to Kafka and the same payload (string) and is passed to the selected business function and is truncated at 1MB. The business function is designed to work with human-readable text files, such as XML, JSON, or CSV; binary files (such as Excel) are not supported.

Alternately, the Template option relies on the message being generated with STEPXML passing through Generic STEPXML Splitter post-processor. Existing Kafka integrations that used the 'Kafka Message Key Template' are unchanged and the previously implemented template is now displayed in the 'Template' parameter.

Additionally, the Kafka delivery method using LZ4 compression no longer sets the Key as the OIEP ID when the Template is blank. Previously, this configuration inadvertently discarded messages that should have been processed when compression was enabled. If the OIEP ID is needed, users can include it in the Template field explicitly or leave it blank to not define a key.

For more information, refer to the Kafka Delivery Method topic in the Data Exchange documentation.

Exporting existing, absent, and deleted values

Previously, when included, calculated attributes were only exported if a value existed. Now, when the empty attributes and the calculated attribute values options are both selected, even empty calculated attributes can be exported using STEPXML, Excel, and CSV formats. This allows for a more complete picture of all values on an object, which can be used to verify the same object in a downstream system.

Also with this update, in the Advanced STEPXML format, the previous restriction of setting both ExportValidAndAbsent and ExportDeletedData to true is removed. This allows valid and absent and deleted data to be output at the same time. As a result, change flags are included for deleted data while also outputting all valid attributes and references.

For more information, refer to the Advanced STEPXML Format topic, the CSV Format topic, the Excel Format topic, the Export Manager topic, or the STEPXML Outbound Parameters topic, all in the Data Exchange documentation.

Context-specific replacement rules for STEPXML imports

Replacement rules in STEPXML imports are used to update data in STEP objects to match the data in the import file, including deletion of data that is not present in the import file. Now, adding a ContextID within a <ReplacementRule> tag allows the user to define which context(s) should have the data replaced with the contents of the import file. When one or more context-specific replacement rules are defined, only data in those contexts is impacted by replacements. Existing functionality is not changed for replacing data in all contexts.

The following data is available for replacement by context:

  • ReplaceValues - products, classifications, entities, assets

  • ReplaceCrossReferences - products, classifications, entities, assets

  • ReplaceClassificationReferences - products

  • ReplaceProductReferences - classifications

For more information, refer to the ReplacementRules Tag in STEPXML topic in the Data Exchange documentation.

MongoDB supported versions

Support for MongoDB in the delivery method on an outbound integration endpoint (OIEP) now includes the following versions:

  • mongo:4.4.28

  • mongo:5.0.24

  • mongo:6.0.13

  • mongo:7.0.5

Versions 3.6 and earlier are no longer supported.

For more information, refer to the Mongo Delivery Method topic in the Data Exchange documentation.

Enhancements to the ECLASS Advanced Editor Screen

The ECLASS Advanced Editor Screen has undergone the following enhancements and updates:

  • Data filtering: The new filtering mechanism is implemented that allows users to view only the specific relevant information of the vast ECLASS Advanced dataset. The new dropdown (shown below) in the screen allows users to select a filter. With a filter selected, users now only sees relevant portions of ECLASS Advanced data for the selected product. This enhancement ensures that users can maintain focus on the required data without losing the context while contributing to the data. Administrators define the content of the filters in the workbench.

  • Mandatory field indication: If the filter definitions contain instructions to define any of the attributes to be mandatory, then such attributes will be highlighted in the ECLASS Advanced Editor screen. This allows customers to designate mandatory properties. These fields will now feature a red asterisk as a visual indicator that they require input.

  • LOV Values: LOVs for an attribute displayed in the ECLASS Advanced Editor screen are now restricted to those defined in the filter templates. This prevents users from selecting irrelevant values.

  • Name blocks on-the-go: Now users can name new blocks while creating them in the screen.

  • Edit block names: Users can rename existing blocks directly within the screen.

  • Read-Only mode: Web UI administrators now can configure the ECLASS Advanced Editor Screen to be in 'read-only' mode. This feature ensures that end users cannot edit ECLASS Advanced data via the ECLASS Advanced Editor Screen. To activate this functionality, a 'Read only' check box is added for the ECLASS Advanced Editor Screen component in the Web UI designer.

For more information, refer to the ECLASS Advanced Editor Screen topic in the Web User Interfaces documentation.

ETIM Taxonomy Importer enhancements

The ETIM Taxonomy Importer has been updated to ensure that change codes will be imported into a metadata attribute on the attribute-classification link, rather than directly on the attribute. Previously, change code information was imported into a metadata attribute on the attribute itself. This enhancement means that ETIM customers can now view if an attribute has been deprecated in the context of a given classification.

For general information regarding ETIM Taxonomy Importer, refer to the ETIM IXF Format topic in the Data Exchange documentation.