Azure Blob Storage Delivery Method

The Azure Blob Storage Delivery plugin for outbound integration endpoints makes it possible to deliver files to Azure Blob Storage. This delivery method is available in OIEPs and Export Manager.

Note: To deliver OIEP files to Azure blob storage, the Cloud Blob Storage Delivery Method can also be used. The differentiation between the setup and functionalities is that the Cloud Blob Storage Delivery plugin is part of the STEP baseline, can also be used for Amazon S3, and the Export Manager and OIEP configurations use the gateway integration endpoints versus having separate delivery method integration properties.

In an OIEP, the delivery method is displayed on the Configuration tab of the editor in the Delivery Method section.

To use this delivery method in Export Manager, refer to the Azure Blob Storage Delivery Method topic.

Prerequisites

The Azure Blob Storage Delivery plugin is part of the 'cloudstorage-azure' component that must be installed in addition to the STEP baseline. No additional licenses are required.

Prior to configuration, click the Configuration name dropdown parameter to display the required configuration to be used. You will provide a selection for the dropdown parameter via the sharedconfig.properties file on the STEP application server using the following configuration properties:

AzureBlobStorageDeliveryPlugin.ConfigurationNames
AzureBlobStorageDeliveryPlugin.ConnectionString.[Configuration Name]
AzureBlobStorageDeliveryPlugin.ContainerName.[Configuration Name]

The value for 'AzureBlobStorageDeliveryPlugin.ConfigurationNames' is a comma-separated list of user-defined names for the desired Azure Blob Storage configurations. For each name, corresponding 'AzureBlobStorageDeliveryPlugin.ConnectionString.[Configuration Name]' and 'AzureBlobStorageDeliveryPlugin.ContainerName.[Configuration Name]' properties must be set with the values being the connection string for the configuration and the desired blob container name, respectively.

An example configuration using the storage account access key (AccountName / AccountKey) method is below:

AzureBlobStorageDeliveryPlugin.ConfigurationNames=Azurite
AzureBlobStorageDeliveryPlugin.ConnectionString.Azurite=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;
AzureBlobStorageDeliveryPlugin.ContainerName.Azurite=productData

It is also possible to use a Shared Access Signature (SAS) credential for the ConnectionString.

The SAS token must be created directly on the blob storage account itself (and not the corresponding container); and as a minimum, it must have Service, Container, and Object specified as its 'Allowed resource types' as well as Read, Write, and List for its 'Allowed permissions' to grant the proper access rights to STEP.

These resource types and permissions are required to allow STEP to perform all the needed operations to deliver the content (blobs) to the specified Azure Blob Storage account's container.

Important: If the SAS token has insufficient privileges, the delivery will result in an error message similar to this one:

If you are using a SAS token, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate generateSas method call. Remember to disable 'Azure-Storage-Log-String-To-Sign' before going to production as this string can potentially contain PII.

Status code 403, "<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthorizationResourceTypeMismatch</Code><Message>This request is not authorized to perform this operation using this resource type. RequestId:836910b1-801e-001a-4da2-900fc9000000 Time:2022-07-05T19:11:07.4796215Z</Message></Error>"

An example using a SAS configuration is below:

AzureBlobStorageDeliveryPlugin.ConfigurationNames=bestProducts
AzureBlobStorageDeliveryPlugin.ConnectionString.bestProducts= BlobEndpoint=https://pimtest.blob.core.windows.net/;SharedAccessSignature=sv=2021-06-08&ss=b&srt=sco&sp=rwlx&se=2023-01-01T05:00:00Z&st=2022-07-05T18:58:41Z&spr=https&sig=N5X7J9tCMscbOTYioR4sb30H7B%2B0j8dk74MSCQ6Gxsw%3D
AzureBlobStorageDeliveryPlugin.ContainerName.bestProducts=productData

Once the property configuration is in place and the system has been restarted, the configuration name(s) will appear as selectable options in the STEP Workbench.

Configuration

  1. On the Configuration tab, in the Delivery Method section, click Edit Delivery.

  2. In Select Delivery Method, choose Azure Blob Storage Delivery.

  3. The Configuration name comes from the properties you set above. If you configured the properties to use multiple configuration names within a comma-separated list, then you will choose from the dropdown.

    Example shown below:

  4. Configure the Azure Blob Storage file path template.

    In addition to selecting the appropriate connection string and container, it is possible to make use of Azure Blob Storage virtual directories. Four variables are available:

    • $systemname – The STEP system name. Useful when several STEP systems are delivering to the same blob container.

    • $timestamp([YMDHMS]) – Delivery timestamp. Desired format to be supplied in Java SimpleDateFormat compatible format.

    • $filename – Name of the file produced by the outbound integration endpoint.

    • $extension – The extension of the file produced by the outbound integration endpoint.

    If the blob containers and virtual directories do not already exist, the plugin will create them.

    Important: Existing files with the same virtual directory path and file name will be overwritten.

  5. On the Edit Delivery Configuration dialog, click the OK button to save the delivery method.