VCSI: STEPXML Joiner Pre-processor Options in IIEP

The STEPXML Joiner inbound pre-processor options can import configurations and settings previously exported via the related outbound functionality.

  • The STEPXML Joiner for Change Packages IIEP pre-processor option accepts a .ZIP file that includes files generated by the OIEP Change Package Git Delivery Method. The .ZIP is expected to contain input files (STEPXML files, BusinessRule_*.js files representing editable business rules, split Web UI configuration with individual screen files in subfolders, and the ChangePackageMetadata.json file organized in subfolders). The files in the .ZIP are combined as a single STEPXML that represents a change package object, which is subsequently imported using the IIEP processing engine. For the imported change package, data is provided to the Primary Items and Secondary Items sections, custom attributes on the change package, and any instructions included by the user who created the change package on the source system. This feature does not directly update the included objects in the system, which provides the ability to run impact analysis and install changes to the system later.

    Note: The ChangePackageMetada.json file is not intended to require editing since it is not intended to be used as an API and the syntax may change over time. ChangePackageMetada.json files created in a previous version of STEP will not validate upon submission to an IIEP running the current version. Triggering an export (re-opening and sealing a change package) from the source system using the delivery method Change Package Git Delivery regenerates the ChangePackageMetadata.json file, which updates the ChangePackageMetadata.json file to the current version. The regenerated file can be used with XML and BusinessRule_*.js files in a .ZIP to create or update a change package. If changes are made to the ChangePackageMetadata.json outside of this process, the user accepts the risk of changing the file outside the recommended process.

    Additionally, the automation options available in your VCSI can be used to reduce the manual efforts required to transfer change packages between DTAP environments, as defined in the VCSI Automation with the STEPXML Joiner for Change Packages section below.

  • The STEPXML Joiner IIEP pre-processor option accepts a .ZIP file that includes files generated by the OIEP Git Delivery method and VCSI: STEPXML Splitter Post-processor in OIEP or the VCSI: Change Package Git Delivery Method in OIEP. The .ZIP is expected to contain input files (STEPXML and BusinessRule_*.js files representing editable business rules, non-split or split Web UI configurations with individual screen files in related subfolders). The files in the .ZIP are combined as a single STEPXML, which is subsequently imported using the IIEP processing engine. This feature directly updates the objects included in the .ZIP at the time of import.

STEPXML Joiner for Change Packages and the Change Package Git Delivery Method

The STEPXML Joiner for Change Packages is designed to accept one .ZIP archive per change package, as produced by an OIEP using the Change Package Git Delivery method. Each change package .ZIP can contain one or more .XML and/or .JS file(s) and must include one ChangePackageMetadata.json file.

  • When a .ZIP contains directory folders (including hidden __MACOSX folders) or .bak files created while editing in some test editing applications, these files are ignored.

  • Unexpected file types and files with an invalid file name result in an error and are not imported.

  • Including more than one super object type is not supported, except for the <Qualifiers /> node.

  • ProcessingInstructions.XML files, replacement rules, global settings, deletes, and missing object tags are not supported.

  • Additional .XML and/or .JS files produced in another change package exported to the VCS can be added to the .ZIP. When included, additional files create an object in the Primary Items section.

  • When .XML and/or .JS files originally included in a change package are removed from the .ZIP, the object removed is not present in the change package and is not installed.

A common case is a change package that remains unchanged from the export created using the Change Package Git Delivery method then creates a change package in the target system with the same objects in the Primary and Secondary sections as were created in the source system.

  • In any case, the objects contained in the Items Required for Transfer and Possibly Impacted Items sections are not included in the change package imported using the STEPXML Joiner for Change Packages.

Note: The ChangePackageMetada.json file is not intended to be used as an API and may change over time. ChangePackageMetada.json files created in a previous version of STEP will not validate upon submission to an IIEP running the current version. Triggering an export (re-opening and sealing a change package) from the source system using the delivery method Change Package Git Delivery regenerates the ChangePackageMetadata.json file, which can be used with XML and BusinessRule_*.js files in a .ZIP to create or update a change package.

Note: Use this IIEP pre-processor option only when the files included in the .ZIP are generated by a related OIEP on a source system that uses the Change Package Git Delivery method.

To configure the IIEP:

  1. For Configure Pre-processor, select STEPXML Joiner for Change Packages.

  2. For Import context, either select the context to receive the imported data or if the source and target systems have similar configuration, select Derive from input files to use the context in the input files.

  3. For Validate STEPXML, by default, 'Yes' indicates that STEPXML files are validated before the joining process begins. Select 'No' to skip validation of the STEPXML files.

  4. For Change Package Setup Group, no selection is required if the setup groups are consistent between source and target systems. Without a selection in this parameter, the IIEP uses the setup group identified in the setupGroupId attribute value of the ChangePackageMetada.json file included in the .ZIP. Otherwise, click the ellipsis button () and select the Setup Group parent where Change Package object types are valid to set the Setup Group location for all change packages imported using this IIEP.

VCSI Automation with the STEPXML Joiner for Change Packages

The options available in your VCSI can be used to reduce the manual efforts required to transfer change packages between DTAP environments (as illustrated in the Change Packages in DTAP Systems with VCS Integration graphic below). For example, GitHub Actions are workflows that can run in response to an event within a GitHub repository (such as creating a new change package, editing, and resealing an existing change package in STEP, or pushing an update to an existing .XML or .JS business rule in a change package in the repository) or can be triggered manually for one or all change packages in the branch.

Consider this example of an automated scenario for updating a test environment (target) with configuration changes made within a change package on the development environment (source):

  1. On the development STEP environment, an event-based OIEP is triggered when the change package is sealed, and then exports the change package to a GitHub repository.

  2. In the GitHub repository, a workflow is triggered by a change to an existing file and then pushes the modified change package to the hotfolder (SFTP-based and pre-configured as described below) identified on the test environment's IIEP.

    Note: To use the SFTP configuration, on the SaaS Self-service UI, select the relevant environment, and click the SFTP Access Control tab. Under Accounts, add the necessary users and a public key (only one key is required per environment). In the IP Access Control List, add an IP/mask of 0.0.0.0/0 to allow access to the SFTP for any IP address used by the VCS. For more information on public keys, refer to the Configuration section in the VCSI: Change Package Git Delivery Method in OIEP topic.

  3. On the test STEP environment, the IIEP is triggered by a schedule when a new file is written in the hotfolder and imports the updated change package.

  4. On the test STEP environment, an admin reviews the change package and determines if / when it should be installed.

The GitHubVCSIActionsSample.zip file includes sample workflows and a README.md file with directions for modifying the sample GitHub Actions. Upload the file to your GitHub repository, add the workflows included under the .github/workflows directory in the ZIP, and modify as required. This includes setting up the necessary 'Secrets' and 'Variables' with your configuration. Review the simple example of a change package in the stibo-user folder in the ZIP.

Note: You must show hidden files on your computer to make the .github/workflows folder visible within the ZIP. This naming convention is required by GitHub for workflows in a repository.

Similar automation features are available in GitLab and Bitbucket. Refer to the appropriate online help in those applications for details.

STEPXML Joiner and the Git Delivery Method

Various processing instructions for the combined STEPXML file can be included on the pre-processor. Processing instructions can be added via the UI (shown below) or can be included in a template STEPXML file within the .ZIP file to be processed by the endpoint.

Note: If a template file is provided in the .ZIP, the settings from this file override any UI configurations made in the UI. The template file must be named ProcessingInstructions.xml and only the ReplacementRules element should exist inside the STEP-ProductInformation element.

Note: Use this IIEP pre-processor option when the related OIEP uses the Git Delivery method.

To configure the IIEP:

  1. For Configure Pre-processor, select STEPXML Joiner.

  2. For Run in single update mode, set to Yes or No as appropriate. For Oracle databases, this action requires single-update mode (SUM), as defined in the Single-Update Mode topic. For Cassandra databases, this action uses Lock-free Schema Change (LFSC) functionality, as defined in the Lock-free Schema Change topic.

  3. For Import context, select the context that will receive the imported data or derive the context from input files, which assumes the source and target systems have a similar configuration.

  4. For Replace instructions, click the link to open the Edit Replace Instructions dialog. Double click the text of a replacement rule group to display the options. Build the rules by selecting options on the left and using the arrow button to move the rule over to the right. Click OK to save changes before moving to the next step of the Inbound Integration Endpoint Wizard.

    For 'list properties' (multiple instances of the same XML element at the same level) such as 'Value' elements inside the 'Values' element for a product or 'TargetUserTypeLink' elements for a reference type definition, special processing instructions (replacement rules) are used to express that the properties that are not present in the import file are to be removed from the system as part of the import. Click the 'Edit Replace Instructions' link.

    To use replacement rules across contexts with suppressions, include all contexts in an export for best results. This ensures complete results are communicated between systems.

    For more information regarding ReplacementRules, refer to the ReplacementRules Tag in STEPXML topic in the Data Exchange documentation.