Azure Blob Storage Delivery Method

The Azure Blob Storage Delivery method in Export Manager makes it possible to export files to Azure Blob Storage. This delivery method is available in Export Manager and OIEPs.

Note: To deliver export files to Azure blob storage, the Cloud Blob Storage Delivery Method can also be used. The differentiation between the setup and functionalities is that the Cloud Blob Storage Delivery plugin is part of the STEP baseline, can be used for Amazon S3, and the Export Manager and OIEP configurations use the gateway integration endpoints instead of having separate delivery method integration properties.

To use this method in an OIEP refer to the Azure Blob Storage Delivery Method topic.

Prerequisites

The Azure Blob Storage Delivery plugin is part of the 'cloudstorage-azure' component that must be installed in addition to the STEP baseline. No additional licenses are required.

Prior to configuration, click the Configuration name dropdown parameter to display the required configuration to be used. You will provide a selection for the dropdown parameter via the sharedconfig.properties file on the STEP application server using the following configuration properties:

AzureBlobStorageDeliveryPlugin.ConfigurationNames
AzureBlobStorageDeliveryPlugin.ConnectionString.[Configuration Name]
AzureBlobStorageDeliveryPlugin.ContainerName.[Configuration Name]

The value for 'AzureBlobStorageDeliveryPlugin.ConfigurationNames' is a comma-separated list of user-defined names for the desired Azure Blob Storage configurations. For each name, corresponding 'AzureBlobStorageDeliveryPlugin.ConnectionString.[Configuration Name]' and 'AzureBlobStorageDeliveryPlugin.ContainerName.[Configuration Name]' properties must be set with the values being the connection string for the configuration and the desired blob container name, respectively.

An example configuration using the storage account access key (AccountName / AccountKey) method is below:

AzureBlobStorageDeliveryPlugin.ConfigurationNames=Azurite
AzureBlobStorageDeliveryPlugin.ConnectionString.Azurite=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;
AzureBlobStorageDeliveryPlugin.ContainerName.Azurite=productData

It is also possible to use a Shared Access Signature (SAS) credential for the ConnectionString.

The SAS token must be created directly on the blob storage account itself (and not the corresponding container); and as a minimum, it must have Service, Container, and Object specified as its 'Allowed resource types' as well as Read, Write, and List for its 'Allowed permissions' to grant the proper access rights to STEP.

These resource types and permissions are required to allow STEP to perform all the needed operations to deliver the content (blobs) to the specified Azure Blob Storage account's container.

Important: If the SAS token has insufficient privileges, the delivery will result in an error message similar to this one:

If you are using a SAS token, and the server returned an error message that says 'Signature did not match', you can compare the string to sign with the one generated by the SDK. To log the string to sign, pass in the context key value pair 'Azure-Storage-Log-String-To-Sign': true to the appropriate generateSas method call. Remember to disable 'Azure-Storage-Log-String-To-Sign' before going to production as this string can potentially contain PII.

Status code 403, "<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthorizationResourceTypeMismatch</Code><Message>This request is not authorized to perform this operation using this resource type. RequestId:836910b1-801e-001a-4da2-900fc9000000 Time:2022-07-05T19:11:07.4796215Z</Message></Error>"

An example using a SAS configuration is below:

AzureBlobStorageDeliveryPlugin.ConfigurationNames=besiProducts
AzureBlobStorageDeliveryPlugin.ConnectionString.besiProducts= BlobEndpoint=https://pimtest.blob.core.windows.net/;SharedAccessSignature=sv=2021-06-08&ss=b&srt=sco&sp=rwlx&se=2023-01-01T05:00:00Z&st=2022-07-05T18:58:41Z&spr=https&sig=N5X7J9tCMscbOTYioR4sb30H7B%2B0j8dk74MSCQ6Gxsw%3D
AzureBlobStorageDeliveryPlugin.ContainerName.besiProducts=productData

Once the property configuration is in place and the system has been restarted, the configuration name(s) will appear as selectable options in the STEP Workbench.

Configuration

For information on a parameter, hover over the parameter label to display help text.

  1. Click the Select Delivery Method parameter to display the dropdown and choose Azure Blob Storage Delivery.

  2. The Configuration name comes from the properties you set above. If you configured the properties to use multiple configuration names within a comma-separated list, then you will choose from the dropdown. (Based on the example configuration given earlier in this topic, the only option available in this example is 'Azurite.')

  3. Configure the Azure Blob Storage file path template.

In addition to selecting the appropriate connection string and container, it is possible to make use of Azure Blob Storage virtual directories. Four variables are available:

  • $systemname – The STEP system name. Useful when several STEP systems are delivering to the same blob container.
  • $timestamp([YMDHMS Format]) – Delivery timestamp. Desired format to be supplied in Java SimpleDateFormat compatible format.
  • $filename – Name of the file produced by the export.
  • $extension – The extension of the file produced by the outbound integration endpoint.

If the blob containers and virtual directories do not already exist, the plugin will create them.

Important: Existing files with the same virtual directory path and file name will be overwritten.

  1. Click Finish to display the Save Export Configuration window as defined in Running a Data Export.