Integration Endpoint Plugins for VCS Integration

The plugin suite for VCS integrations consists of these plugins:

  • 'STEPXML Splitter' Post-processor Plugin in Outbound Integration Endpoint (OIEP)

  • 'Git Delivery' Plugin in Outbound Integration Endpoint (OIEP)

  • 'STEPXML Joiner' Pre-processor Plugin in Inbound Integration Endpoint (IIEP)

  • 'Invoke OIEP' Post-processor Plugin Inbound Integration Endpoint (IIEP)

Each is described in their own section below.

Additionally, within this topic, is information regarding an editable business rule format. JavaScript-based business rules can be created, maintained, and unit tested outside of STEP. This allows customers and partners to govern the lifecycle of business rules in a standard source code control system such as GIT, and from there, deploy appropriate versions of the business rules to the various STEP systems that are part of a Development, Testing, Acceptance, and Production (DTAP) environment.

'STEPXML Splitter' Post-processor Plugin in OIEP

The 'STEPXML Splitter' post-processor can take any STEPXML file produced by the 'STEP Exporter' processing engine as input and will split the file into multiple valid STEPXML files and/or editable business rule format files that are then passed to the configured delivery plugin. Generally, the splitter produces one file per STEP object and further normalizes the content so that elements for which the sequence has no significance in STEP are output in the same order every time. Non-object configurations (e.g., derived events) and system settings are output in a single file.

Splitting and normalizing makes it easier to compare configurations outside of STEP in a VCS like Git. Further, it makes it easy to selectively choose specific configuration items to be imported on another system.

The ‘STEPXML Splitter’ plugin has the configuration options explained below.

Split mode

Split mode, that defaults to ‘Flattened’ but also allows 'Hierarchical'. The parameter affects how to represent STEP objects in the produced split files when they are typically exported in a nested structure.

To illustrate the difference, in the example shown below, the following classification hierarchy is being exported:

In both modes, one file will be created per classification object.

In Flattened mode, upper levels are omitted and each file contains exactly one 'Classification' element with parent identifier information, as shown below.

(…)
<Classifications>
  <Classification ID=‘stibo.IMConfigFolder’ UserTypeID=‘ImportConfigurationsRoot’ ParentID=‘ConfigurationsRoot’>
    <Name>Import Configurations</Name>
    <MetaData>
      <Value AttributeID=‘Purpose’ QualifierID=‘en-US’>Storage for import configurations</Value>
    </MetaData>
  </Classification>
</Classifications>
(…)

In Hierarchical mode, each of the leaf classifications (e.g., 'Import Configurations') is nested inside the element representing the 'Configurations' classification, which is stripped of all but ID, object type ID, and parent ID information, as shown below.

(…)
<Classifications>
  <Classification ID=‘ConfigurationsRoot’ UserTypeID=‘ConfigurationsRoot’ ParentID=‘Classification 1 root’>
    <Classification ID=‘stibo.IMConfigFolder’ UserTypeID=‘ImportConfigurationsRoot’>
      <Name>Import Configurations</Name>
      <MetaData>
        <Value AttributeID=‘Purpose’ QualifierID=‘en-US’>Storage for import configurations</Value>
      </MetaData>
    </Classification>
  </Classification>
</Classifications>
(…)

Generally, it is recommended to use the default 'Flattened' mode while the 'Hierarchical' mode only should be used if the full hierarchy path must be present in each file.

Note: The 'Hierarchical' example from above can be imported on a system where the classification with ID 'ConfigurationsRoot' is not present as it will be created during import. Importing the 'Flattened' example on such a system, however, results in an error.

Convert business rules to editable format

This option determines how business rules (conditions, actions, functions, and libraries) are exported.

  • If set to No, the business rules are exported as STEPXML files.

  • If set to Yes, the rules are exported in the editable *.js format described in this topic.

    When exporting editable business rules, set this option to 'Yes.' The business rules in the STEPXML that are fed to the post-processor are converted to the editable format and represented in a single *.js file instead of being represented in a STEPXML file. For details, refer to the Editable Business Rules Format topic here.

‘Git Delivery’ Plugin in OIEP

The 'Git Delivery' plugin delivers files produced by the outbound integration endpoint (OIEP) processing engine or a configured post-processor to a branch in a remote Git repository. Refer to https://git-scm.com for more information about Git.

When using the plugin on an on-premise STEP system, the delivery to Git can be performed via a configured local directory accessible from all application servers in the cluster (refer to the 'Local git repository URI' section below). With this setup, the plugin first checks if the local delivery directory is Git enabled.

If the local delivery directory is Git enabled, the following operations are performed:

  1. Git fetch

  2. Git checkout (of configured branch)

  3. Git hard reset

  4. Git pull (if branch exists in remote)

If the local delivery directory is not Git enabled, the following operations are performed:

  1. Git clone

  2. Git checkout (of configured branch)

On cloud-based STEP systems, a temporary local directory is always used and the second approach (clone and checkout) is used.

Note: The remote repository cannot be empty. At least one branch with one file must exist.

The locally checked-out branch is now in sync with the remote branch, and the following operations are performed:

  1. Files produced by the OIEP are written to the local directory

  2. Files in local directory but not in delivery are deleted

  3. Git stage

  4. Git commit

  5. Git push

If the configured branch does not exist in the remote repository in advance, it is created.

Important:  

  • With each invocation, the OIEP must produce files for all the configuration objects and/or settings to be represented in the configured Git branch. Exporting only a file for a single object will cause all other files in the Git branch to be deleted. Refer to the OIEP for VCS Integration topic for more information on how to configure an OIEP using this delivery plugin here.

  • Only STEP should make changes in the configured Git branch. All other changes are overwritten with the next STEP delivery.

Configure the 'Git Delivery' plugin using the following parameters:

Note: In the sharedconfig.properties file entries mentioned below, use a sequential set of variable integers in the property names, for example, 1, 2, 3. Property values are not read correctly if the sequence is not sequential, for example 1, 3, 4.

  • Local git repository URI - URI for the local directory via which configuration files are synchronized.

    For on-premise systems, possible values for this parameter are read from the case-sensitive 'GitDelivery.LocalRepoUri.[integer]' sharedconfig.properties file entries.

    Example:

    GitDelivery.LocalRepoUri.1=/workarea/conf-attributes
GitDelivery.LocalRepoUri.2=/workarea/conf-all

    allows selection between the values '/workarea/conf-attributes' and '/workarea/conf-all' for the parameter in the workbench.

    For cloud STEP systems, do not set the sharedconfig.properties file entry and leave this parameter blank.

  • Remote git repository URI - URI for the remote repository.

    For on-premise systems, possible values for this parameter are read from the case-sensitive 'GitDelivery.RemoteRepoUri.[integer]' sharedconfig.properties file entries.

    Examples:

    GitDelivery.RemoteRepoUri.1=https://bitbucket.org/john-smith/step-conf.git
GitDelivery.RemoteRepoUri.2=ssh://gituser@192.168.56.102/home/gituser/git/repo.git

  • Git branch - Name of the branch to which the delivery is published.

    Values for this parameter are read from the case-sensitive 'GitDelivery.Branch.[integer]' sharedconfig.properties file entries.

    Examples:

    GitDelivery.Branch.1=step-dev-1
GitDelivery.Branch.1=step-qa

  • Author name - Author name for the Git commit.

    Values for this parameter are read from the case-sensitive 'GitDelivery.AuthorName.[integer]' sharedconfig.properties file entries.

    Examples:

    GitDelivery.AuthorName.1=John Smith - STEP Dev1

  • Author email - Author email for the Git commit.

    Values for this parameter are read from the case-sensitive 'GitDelivery.AuthorEmail.[integer]' sharedconfig.properties file entries.

    Examples:

    GitDelivery.AuthorEmail.1=john.smith@gmail.com

  • Repository username - The remote repository username.

    Values for this parameter are read from the case-sensitive 'GitDelivery. RemoteRepoUsername.[integer]' sharedconfig.properties file entries.

    Examples:

    GitDelivery.RemoteRepoUsername.1=john.smith@gmail.com

  • Repository user password - The remote repository password. Password must be entered directly when configuring the plugin.

  • Path to private key when using ssh - The remote repository private key when using SSH.

    Values for this parameter are read from the case-sensitive 'GetDelivery.SshPrivateKeyUri.[integer]' sharedconfig.properties entries. The folder specified should contain the private SSH key.

    Example:

    GitDelivery.SshPrivateKeyUri.1=/home/stibosw/.ssh/id_rsa

  • Repository ssh passphrase - The remote repository SSH password / passphrase. Passphrase must be entered directly when configuring the plugin.

Remote Setup Example

Below is an example of how a remote Git repository using Bitbucket (https://bitbucket.org) can be configured from scratch to work with the Git delivery plugin.

  1. Create a new repository with a 'README' file via the Bitbucket web interface.

  2. Create a system specific branch in the repository.

  3. Get the 'Remote git repository URI' from, for example, the Bitbucket 'Clone' option.

'STEPXML Joiner' Pre Processor Plugin in IIEP

The 'STEPXML Joiner' inbound pre-processor plugin is designed to be used to import configurations and settings previously exported via the outbound functionality (described above). The outbound functionality produces STEPXML files and potentially*.js files representing business rules. The 'STEPXML Joiner' plugin can take a .ZIP file containing any number of such files as input and then combines objects and settings in a single STEPXML and passes the combined file to the IIEP processing engine.

Various processing instructions for the combined STEPXML file can be included on the pre-processor plugin. Processing instructions can be added via the UI (shown below) or can be included in a template STEPXML file within the .ZIP file that is fed to the endpoint.

Note: If a template file is provided in the .ZIP, the settings from this file override any UI configurations made in the UI. The template file must be named 'ProcessingInstructions.xml' and only the ReplacementRules element should exist inside the STEP-ProductInformation element.

To configure the UI:

  1. For Configure PreProcessor, select STEPXML Joiner.

  2. For Run in single update mode, set to Yes or No as appropriate.

  3. For Import context, select the context that will receive the imported data.

  4. For Replace instructions, add replacement rules as needed.

    For 'list properties' (multiple instances of the same XML element at the same level) such as 'Value' elements inside the 'Values' element for a product or 'TargetUserTypeLink' elements for a reference type definition, special processing instructions (replacement rules) are used to express that properties not present in the import file are to be removed from the system as part of the import. Click the 'Edit Replace Instructions' link.

    For more information regarding ReplacementRules, refer to the ReplacementRules Tag in STEPXML topic in the STEPXML Tags and Examples section of the Data Exchange documentation here.

  5. For the Edit Replace Instructions dialog, build the rules by selecting options on the left and using the arrow button to move the rule over to the right. Click OK to save changes before moving to the next step of the Inbound Integration Endpoint Wizard.

'Invoke OIEP' Post Processor Plugin in IIEP

The 'Invoke OIEP' post processor plugin allows for an OIEP to be invoked once the IIEP process has completed. This allows updating the representation of the system configuration in a remote Git branch immediately after the configuration has been imported.

As shown below, the plugin requires the ID of the OIEP to be invoked.

For more information, refer to these topics in the Version Control System Integration section:

  • Editable Business Rules Format (here)

  • VCS: Example Setups (here)

  • VCS: Considerations and Limitations (here)