Integration Endpoint Options for VCS Integration

Change Package Git Delivery Method in OIEP

The Change Package Git Delivery method allows integration with popular repositories, supporting the HTTPS (token-based) or the SSH (file-based) access methods for GitHub, GitLab, and Bitbucket.

The Change Package Git Delivery method delivers files produced by the OIEP processing engine using an integrated STEPXML Splitter to deliver multiple files to a branch in a remote Git repository. Refer to https://git-scm.com for more information about Git.

The change package is represented below the specified branch within a configurable directory structure. At the end of the directory structure, change packages grouping files by the Primary and Secondary flippers, then by type of object with XML or JSON files named by object type and the ID of the object.

Note: The remote repository cannot be empty. At least one branch with one file must exist (e.g., the 'main' branch and the README.md file).

Change Package Git Delivery Prerequisites

Important: For on-premises systems, this feature requires the configuration-management component.

Changes to shared configuration properties on the STEP application server are required to populate the dropdown list delivery options. Changing these properties does not require a system to be restarted.

Prior to configuring the Change Package Git Delivery method, add the desired case-sensitive properties described below.

Note: In the properties mentioned below, use a unique set of variable integers in the property names, for example, 1, 2, 3. When duplicates exist, only the last value is displayed in the dialog.

1. For both on-prem and Stibo Systems SaaS environments, the Remote Git Repository URI parameter on the dialog defines the URI for the remote repository. The values are read from the case-sensitive ChangePackageGitDelivery.RemoteRepoUri.[integer] property. For example:

   ChangePackageGitDelivery.RemoteRepoUri.1=https://gitlab.com/john-smith/step-conf.git 
   ChangePackageGitDelivery.RemoteRepoUri.2=git@bitbucket.org:john-smith/smithrepo.git

   Note: For internally hosted Git setups, ensure that any firewall access rules, IP white-listing, or ports (tcp/22 or similar) are configured to allow the STEP server(s) to access the needed Git repository.

2. The Git Branch parameter on the dialog defines the name of the branch to which the delivery is published. The values are read from the case-sensitive ChangePackageGitDelivery.Branch.[integer] property. For example:

   ChangePackageGitDelivery.Branch.1=step-dev-1 
   ChangePackageGitDelivery.Branch.2=step-q

3. The Repository User Name parameter on the dialog defines the options available in the dropdown list and supports authentication via HTTPS with PAT entered in the Repository User Password parameter on the dialog (discussed in Supported Authentication Methods per Git Service). The values are read from the case-sensitive ChangePackageGitDelivery.RemoteRepoUsername.[integer] property. For example:

   ChangePackageGitDelivery.RemoteRepoUsername.1=john.smith 
   ChangePackageGitDelivery.RemoteRepoUsername.2=julie.baker 

4. The Path to Private Key When Using SSH parameter on the dialog must contain the full path to the private SSH key file. The values are read from the case-sensitive ChangePackageGetDelivery.SshPrivateKeyUri.[integer] property. For example:

   ChangePackageGitDelivery.SshPrivateKeyUri.1=/home/stibosw/.ssh/git_rsa4096_key
   ChangePackageGitDelivery.SshPrivateKeyUri.2=/workarea/.ssh/gitlab_rsa4096_enc_key
   					ChangePackageGitDelivery.SshPrivateKeyUri.3=/workarea/.ssh/bitbucket_rsa4096_key

   Configure SSH Connection

   To connect via SSH, the Path to Private Key When Using SSH parameter defines the remote repository RSA SSH private key generated using the old OpenSSH format and ed25519. The generated key file must be accessible to the application server through shared storage or SFTP.

   Important: PuTTYgen (.ppk) SSH keys are not supported by STEP. Using an unsupported key type results in an 'invalid private key' error.

   Follow these steps to generate a valid private SSH key:

   1. Use ssh -V to check the version on your system and then generate a key.

      • With OpenSSH versions prior to 7.8, use the following command to generate the new OpenSSH format key:

        ssh-keygen -t rsa -b 4096 -C <comment> -f <keyfile_name>

        For example:

        ssh-keygen -t rsa -b 4096 -C john.smith@acme.com -f git_rsa4096_key

      • With OpenSSH versions 7.8+, generated keys default to the new OpenSSH format. To generate keys in the old format, use the following command:

        ssh-keygen -t rsa -b 4096 -C john.smith@acme.com -m PEM -f git_rsa4096_key

        Important: While the general approach and commands are the same with later versions of OpenSHH, the -m PEM argument shown above is needed with versions 7.8+.

   2. Add the public version of the SSH key (e.g., git_rsa4096_key.pub) to your GitLab or Bitbucket account. Failure to add the public key to your account results in a 'Not authorized' error.

      For a general overview of supported authentication methods, refer to the Supported Authentication Methods per Git Service section below.

      • For GitHub accounts, refer to the GitHub Limitations and Configuration section below.

      • For Bitbucket accounts, refer to the Bitbucket Limitations and Configuration section below.

Change Package Git Delivery Configuration

For information on a parameter, hover over the parameter field to display help text.

1. On the Configuration tab, in the Delivery Method area, click Edit Delivery.

2. In Select Delivery Method, choose Change Package Git Delivery.

   

   Note: The parameter dropdown options are defined by the properties outlined in the Change Package Git Delivery Prerequisites section above.

3. In Remote Git Repository URI, select a URI from the dropdown for the relevant remote repository.

4. In Git Branch, select relevant branch in the associated remote repository.

5. In Git Access Method, select HTTPS or SSH and provide additional values as required.

   • For HTTPS (using Basic authentication or Personal Access Token (PAT)):

     • In Repository User Name - select an option from the dropdown.

     • In Repository User Password - add the Basic authentication password, or add the PAT generated from your repository developer tools. Password must be entered directly when configuring the delivery method for these authentication methods.

   • For SSH:

     • In Path to Private Key When Using SSH - select the file path to the private key from the dropdown.

     • In Repository SSH Passphrase - add the passphrase.

       Note: Author name and email for the Git commit are sourced automatically from the user object associated with the sealing event on the change package and does not require explicit configuration in the delivery method. However, to support this functionality, email address values must exist for users that seal change packages that are integrated with Git.

6. In Directory Template, indicate where to store files on the repository using a standardized format. While editing the Directory Template, hover over the field to display the available macros.

   Use static text, macros, and the forward slash character as a separator to automatically create directories and organize change packages by:

   • $systemname$/$changepackageid$ - default, creates a directory under the branch for each system name where a change package is sent from and sub directories with the ID of the change package.

   • $systemname$ - system name, where the change package is sealed

   • $changepackageorigin$ - origin, where the change package was initially created

   • $changepackageuniqueid$ - system-defined unique ID

   • $changepackageid$ - user-defined change package ID

   • $attribute:attributeid$ - For additional flexibility, custom description attributes that are externally maintained, not dimension dependent, not calculated, and not multi-value can be added to the change package object and referenced in the template to create subfolders based on values you define. When a custom attribute is used in the directory template, values must be present and LOV validation base type is recommended.

     Note: a-z, A-Z, 0-9,-, and _ are supported characters for static text. Other characters are replaced with an underscore.

7. In Convert Business Rules to Editable Format, select an option:

   • Yes - default, sends editable JSON files of business rules (actions, conditions, functions, and libraries) to the repository. The JSON files can be imported manually or with other files combined in a .ZIP file that is sent to an IIEP using the STEPXML Joiner pre-processor.

   • No - exports STEPXML files that do not provide editable JavaScript access but can be imported as-is after export and include definitions as comments.

8. In Context, select the context to include in the files generated by the STEPXML Splitter pre-processor. This is the context that is used upon import of the change package with the STEPXML Joiner for Change Packages pre-processor on a target system when the Context is derived from input files and not set explicitly in the pre-processor options for Import Context.

9. On the Edit Delivery Configuration dialog, click the OK button to save the delivery method.

STEPXML Splitter Post-processor in OIEP

Use of the STEPXML Splitter is determined by the grouping option selected:

• For the Change Package Git Delivery method, the STEPXML Splitter is integrated into the delivery method, so the STEPXML Splitter post-processor must not be added to the OIEP.

  For change packages, all exports use the Flattened hierarchy split mode and export definitions as comments. The user can decide to also export business rules converted to an editable format on the delivery method dialog, which replaces the STEPXML files that include comments with editable .JS files for business rules.

• For a collection used to group changes and an Advanced STEPXML template configured for various configurations that do not include the Change Package object, the STEPXML Splitter is required on the OIEP with ‘Git Delivery’.

The STEPXML Splitter post-processor can take any STEPXML file produced by the STEP Exporter processing engine as input and splits the file into multiple valid STEPXML files and/or editable business rule format files that are then passed to the configured delivery method. Generally, the splitter produces one file per STEP object and further normalizes the content so that elements for which the sequence has no significance in STEP are output in the same order every time. Non-object configurations (e.g., derived events) and system settings are output in a single file.

Splitting and normalizing makes it easier to compare configurations outside of STEP in a VCS like Git. Further, it makes it easy to selectively choose specific configuration items to be imported on another system.

The STEPXML Splitter post-processor has the configuration options explained below.

Split mode

Split mode defaults to ‘Flattened’ but also allows 'Hierarchical'. The parameter affects how to represent STEP objects in the produced split files when they are typically exported in a nested structure.

To illustrate the difference, in the example shown below, the following classification hierarchy is being exported:

In both modes, one file will be created per classification object.

In Flattened mode, upper levels are omitted, and each file contains exactly one 'Classification' element with parent identifier information, as shown below.

(…)
<Classifications>
<Classification ID=‘stibo.IMConfigFolder’ UserTypeID=‘ImportConfigurationsRoot’ ParentID=‘ConfigurationsRoot’>
<Name>Import Configurations</Name>
<MetaData>  
  <Value AttributeID=‘Purpose’ QualifierID=‘en-US’>Storage for import configurations</Value>
</MetaData>
</Classification>
</Classifications>
(…)			

In Hierarchical mode, each of the leaf classifications (e.g., Import Configurations) is nested inside the element representing the 'Configurations' classification, which is stripped of all but ID, object type ID, and parent ID information, as shown below.

(…)
<Classifications>
<Classification ID=‘ConfigurationsRoot’ UserTypeID=‘ConfigurationsRoot’ ParentID=‘Classification 1 root’>
<Classification ID=‘stibo.IMConfigFolder’ UserTypeID=‘ImportConfigurationsRoot’> 
<Name>Import Configurations</Name>
<MetaData>
  <Value AttributeID=‘Purpose’ QualifierID=‘en-US’>Storage for import configurations</Value>
</MetaData>
</Classification>
</Classification>
</Classifications>
(…)

Generally, it is recommended to use the default 'Flattened' mode while the 'Hierarchical' mode should only be used if the full hierarchy path must be present in each file.

Note: The 'Hierarchical' example from above can be imported on a system where the classification with ID 'ConfigurationsRoot' is not present as it will be created during import. Importing the 'Flattened' example on such a system, however, results in an error.

Convert business rules to editable format

This option determines how business rules (conditions, actions, functions, and libraries) are exported.

• If set to No, the business rules are exported as STEPXML files.

• If set to Yes, the rules are exported in the editable *.JS format described in this topic.

  When exporting editable business rules, set this option to 'Yes.' The business rules in the STEPXML that are fed to the post-processor are converted to the editable format and represented in a single *.JS file instead of being represented in a STEPXML file. For details, refer to the VCSI: Editable Business Rules Format topic.

Git Delivery Method in OIEP

The Git Delivery method delivers files produced by the outbound integration endpoint (OIEP) processing engine and a configured STEPXML Splitter post-processor to a branch in a remote Git repository. In this scenario, the entire branch is replaced with the current objects delivered by the OIEP as a flat hierarchy, where all files delivered are directly under the branch. Refer to https://git-scm.com for more information about Git.

When using this delivery method on an on-premises STEP system, the delivery to Git can be performed via a configured local directory accessible from all application servers in the cluster (refer to the Local git repository URI section below). With this setup, the delivery method first checks if the local delivery directory is Git enabled.

If the local delivery directory is Git enabled, the following operations are performed:

1. Git fetch

2. Git checkout (of configured branch)

3. Git hard reset

4. Git pull (if branch exists in remote)

If the local delivery directory is not Git enabled, the following operations are performed:

1. Git clone

2. Git checkout (of configured branch)

On Stibo Systems SaaS environments, a temporary local directory is always used and the second approach (clone and checkout) is used.

Note: The remote repository cannot be empty. At least one branch with one file must exist (e.g., the 'main' branch and the README.md file).

The locally checked-out branch is now in sync with the remote branch, and the following operations are performed:

1. Files produced by the OIEP are written to the local directory

2. Files present in the local directory but not in the delivery are deleted

3. Git stage

4. Git commit

5. Git push

Git Delivery Prerequisites

Important: For on-premises systems, this feature requires the configuration-management component.

Changes to shared configuration properties on the STEP application server are required to populate the dropdown list delivery options. Changing these properties does not require a system restart.

Prior to configuring the Git Delivery method, add the desired case-sensitive properties described below.

Note: In the properties mentioned below, use a unique set of variable integers in the property names, for example, 1, 2, 3. When duplicates exist, only the last value is displayed in the dialog.

1. For on-prem systems, the Local git repository URI parameter defines a Uniform Resource Identifier (URI) for the local directory where the synchronized configuration files are stored. The values are read from the case-sensitive GitDelivery.LocalRepoUri.[integer] property. For example:

   GitDelivery.LocalRepoUri.1=/workarea/conf-attributes
   						GitDelivery.LocalRepoUri.2=/workarea/conf-all
   					

   These settings would allow users to select between the values '/workarea/conf-attributes' and '/workarea/conf-all' for the parameter in the dropdown.

   Do not set the Local git repository URI property for Stibo Systems SaaS environments.

2. For both on-prem and Stibo Systems SaaS environments, the Remote git repository URI parameter defines the URI for the remote repository. The values are read from the case-sensitive GitDelivery.RemoteRepoUri.[integer] property. For example:

   GitDelivery.RemoteRepoUri.1=https://gitlab.com/john-smith/step-conf.git
   					GitDelivery.RemoteRepoUri.2=git@bitbucket.org:john-smith/smithrepo.git

   Note: For internally hosted Git setups, ensure that any firewall access rules, IP white-listing, or ports (tcp/22 or similar) are configured to allow the STEP server(s) to access the needed Git repository.

3. The Git Branch parameter defines the name of the branch to which the delivery is published. The values are read from the case-sensitive GitDelivery.Branch.[integer] property. For example:

   GitDelivery.Branch.1=step-dev-1 
   					GitDelivery.Branch.2=step-q

4. The Repository username parameter in the dialog defines the options available in the dropdown list and supports authentication via HTTPS with PAT entered in the Repository User Password field in the dialog (discussed in Supported Authentication Methods per Git Service). The values read from the case-sensitive GitDelivery.RemoteRepoUsername.[integer] property. For example:

   GitDelivery.RemoteRepoUsername.1=john.smith 
   					GitDelivery.RemoteRepoUsername.2=julie.baker

5. The Path to private key when using ssh parameter must contain the full path to the private SSH key file. The values are read from the case-sensitive GetDelivery.SshPrivateKeyUri.[integer] property. For example:

   GitDelivery.SshPrivateKeyUri.1=/home/stibosw/.ssh/git_rsa4096_key
   						GitDelivery.SshPrivateKeyUri.2=/workarea/.ssh/gitlab_rsa4096_enc_key
   					GitDelivery.SshPrivateKeyUri.3=/upload/.ssh/bitbucket_rsa4096_key

   Configure SSH Connection

   To connect via SSH, the Path to Private Key When Using SSH parameter defines the remote repository RSA SSH private key generated using the old OpenSSH format and ed25519. The generated key file must be accessible to the application server through shared storage or SFTP.

   Important: PuTTYgen (.ppk) SSH keys are not supported by STEP. Using an unsupported key type results in an 'invalid private key' error.

   Follow these steps to generate a valid private SSH key:

   1. Use ssh -V to check the version on your system and then generate a key.

      • With OpenSSH versions prior to 7.8, use the following command to generate the new OpenSSH format key:

        ssh-keygen -t rsa -b 4096 -C <comment> -f <keyfile_name>

        For example:

        ssh-keygen -t rsa -b 4096 -C john.smith@acme.com -f git_rsa4096_key

      • With OpenSSH versions 7.8+, generated keys default to the new OpenSSH format. To generate keys in the old format, use the following command:

        ssh-keygen -t rsa -b 4096 -C john.smith@acme.com -m PEM -f git_rsa4096_key

        Important: While the general approach and commands are the same with later versions of OpenSHH, the -m PEM argument shown above is needed with versions 7.8+.

   2. Add the public version of the SSH key (e.g., git_rsa4096_key.pub) to your GitLab or Bitbucket account. Failure to add the public key to your account results in a 'Not authorized' error.

      For a general overview of supported authentication methods, refer to the Supported Authentication Methods per Git Service section below.

      • For GitHub accounts, refer to the GitHub Limitations and Configuration section below.

      • For Bitbucket accounts, refer to the Bitbucket Limitations and Configuration section below.

Git Delivery OIEP Configuration

For information on a parameter, hover over the parameter field to display help text.

1. On the Configuration tab, in the Delivery Method area, click Edit Delivery.

2. In Select Delivery Method, choose Git Delivery.

   

   Note: The parameter dropdown options are defined by the properties outlined in the Git Delivery Prerequisites section above.

3. For a local delivery, in Local git repository URI, from the dropdown, select a URI for the local directory via which configuration files are synchronized.

   For Stibo Systems SaaS environments, leave this parameter blank.

4. For a remote delivery, in Remote git repository URI, from the dropdown, select a URI for the remote repository.

5. In Git branch, from the dropdown, select the name of the branch where the delivery is published.

6. In Author name, from the dropdown, select an author name for the Git commit.

7. In Author email, from the dropdown, select an author email for the Git commit.

8. In Git Access Method, select the HTTPS or SSH radio button and provide additional values as required.

   • For HTTPS (using Basic authentication or Personal Access Token (PAT)):

     • In Repository username - select an option from the dropdown.

     • In Repository user password - add the Basic authentication password, or add the PAT generated from your repository developer tools. Password must be entered directly when configuring the delivery method for these authentication methods.

   • For SSH:

     • In Path to private key when using ssh - from the dropdown, select the file path to the remote repository RSA SSH private key generated using the old OpenSSH format.

     • In Repository ssh passphrase - add the remote repository SSH passphrase. The passphrase must be entered directly when configuring the delivery method.

9. On the Edit Delivery Configuration dialog, click the OK button to save the delivery method.

Supported Authentication Methods per Git Service in OIEP

	Protocols
Git Services	HTTPS (BA*)	SSH	HTTPS (PAT**)
GitHub
GitLab
Bitbucket

*BA = Basic Authentication

**PAT = Personal Access Token / App Password

GitHub Limitations and Configuration

The following limitations apply when using GitHub with STEP:

Unlike other Git services, GitHub is more restrictive in the level of security placed on accessing its repositories, specifically:

• Basic authentication for HTTPS connections is not supported.

• RSA SSH keys that use the sha-1 signature algorithm are not supported.

To summarize, the available authentication methods for using the Change Package Git Delivery method or Git Delivery Method with a GitHub account are with a Personal Access Token (PAT) or SSH with a private key accessible to the application server.

Bitbucket Limitations and Configuration

Bitbucket does not support using Basic authentication for HTTPS connections. This means that apart from SSH keys, the only other usable authentication method is app passwords.

STEPXML Joiner Pre-processor Options in IIEP

The STEPXML Joiner inbound pre-processor options can import configurations and settings previously exported via the related outbound functionality (described above and referenced below).

• The STEPXML Joiner for Change Packages IIEP pre-processor option accepts a .ZIP file that includes files generated by the OIEP Change Package Git Delivery method. The .ZIP is expected to contain input files (STEPXML files, BusinessRule_*.js files representing editable business rules, and the ChangePackageMetadata.json file organized in subfolders). The files in the .ZIP are combined as a single STEPXML that represents a change package object, which is subsequently imported using the IIEP processing engine. For the imported change package, data is provided to the Primary Items and Secondary Items flippers, custom attributes on the change package, and any instructions included by the user who created the change package on the source system. This feature does not directly update the included objects in the system, which provides the ability to run impact analysis and install the changes to the system at a later time.

  Note: The ChangePackageMetada.json file is not intended to require editing since it is not intended to be used as an API and the syntax may change over time. ChangePackageMetada.json files created in a previous version of STEP will not validate upon submission to an IIEP running the current version. Triggering an export (re-opening and sealing a change package) from the source system using the delivery method Change Package Git Delivery regenerates the ChangePackageMetadata.json file, which updates the ChangePackageMetadata.json file to the current version. The regenerated file can be used with XML and BusinessRule_*.js files in a .ZIP to create or update a change package. If changes are made to the ChangePackageMetadata.json outside of this process, the user accepts the risk of changing the file outside the recommended process.

• The STEPXML Joiner IIEP pre-processor option accepts a .ZIP file that includes files generated by the OIEP Git Delivery method and STEPXML Splitter or the Change Package Git Delivery method. The .ZIP is expected to contain input files (STEPXML and BusinessRule_*.js files representing editable business rules). The files in the .ZIP are combined as a single STEPXML, which is subsequently imported using the IIEP processing engine. This feature directly updates the objects included in the .ZIP at the time of import.

Invoke OIEP Post-processor in IIEP

The 'Invoke OIEP' post-processor allows for an OIEP to be invoked once the IIEP process has completed. This allows updating the representation of the system configuration in a remote Git branch immediately after the configuration has been imported, if desired.

As shown below, the option requires the ID of the OIEP to be invoked.