Integrating MLAC Predictions in a Workflow

The prediction functionality can be used in a production setup by calling the business action that retrieves the predictions from a product onboarding workflow. The workflow can then branch based on the prediction confidence, resulting in the product being automatically classified for a prediction with a high confidence, while a creating a user task with the prediction information if no predictions meet the desired confidence threshold.

Given that the prediction service response time can be several seconds, to avoid blocking users, the configuration examples in this topic make use of an event processor (EP). Using the 'Execute Business Action for Event Batch' event processor plugin, a business action will be executed via a batch. This means that the data for multiple events will be handled with a single action invocation and a single request.

The full process of implementing the MLAC auto classification functionality in a workflow includes these steps:

1. Create an Onboarding Workflow
2. Define a Derived Event Type
3. Create an Event Processor Filter Condition
4. Create a Prediction Business Action
5. Create and Configure an MLAC Event Processor
6. Create a Workflow Business Action

Create an Onboarding Workflow

While the MLAC auto classification logic could be part of a larger product onboarding flow, states not relevant for auto classification have been left out of this example.

For initial (and simplified) testing, create a workflow based on the image below, including the defined workflow states. After verifying the functionality, these states can be added to your existing onboarding workflow.

For general information, refer to the Workflows documentation here.

Workflow States

The states for this onboarding / classification workflow are:

• Auto classification is a system state that does not represent a human task. When a product enters this state, an event is placed on the queue for the event processor to make the prediction request and, dependent on the confidence of the best prediction, either classify the product and trigger the 'OK' event or trigger the 'MC' event.
• Manual Classification is a human task to manually classify a product with a low prediction confidence. The predictions returned from the service are made visible to the user via the ‘prediction’ workflow variable to which the prediction response will be written.
• Final is a basic mandatory final state.

The following sections describe the configuration necessary for this setup.

Define a Derived Event Type

The event to be produced from within the workflow and placed on the queue for the event processor should be of a user-defined type, also called a derived event.

To create a derived event:

1. In System Setup, click the 'Derived Events node to open the editor.
2. Click the Add Derived Event Type link.
3. Add an ID for the event type.

4. Click the Add button to save the event type.

For more information, refer to the Derived Events topic in the System Setup documentation here.

Create an Event Processor Filter Condition

An event filter is required on the Event Triggering Definitions tab of the event processor to ensure that only events generated from the workflow are allowed on the event processor queue. For more information, event triggering definitions for event processors are the same as for OIEPs as defined in the OIEP - Event-Based - Event Triggering Definitions Tab topic in the Data Exchange documentation here.

This event filter is a JavaScript-based business condition that is valid for all object types.

In the JavaScript editor create a bind with:

• 'Variable name' set to currentEventType
• 'Binds to' set to Current Event Type (in the Event Handling group)
• Add the following to the JavaScript parameter:

1
2
3
4
var isDerivedEvent = currentEventType instanceof com.stibo.core.domain.eventqueue.DerivedEventType;
var idIsCorrect = currentEventType.getID() == "MlacAsync";

return isDerivedEvent && idIsCorrect;    

Create a Prediction Business Action

Using the 'Execute Business Action For Event Batch' event processor plugin executes the configured prediction (confidence) business action once for a batch of events. This business action has access to the nodes (products) for which the events was generated via the 'Current Event Processor Event Batch' bind to access a batch of queued events. Additionally, the batch business action composes a single request with data for multiple products and similarly handles the response containing predictions for multiple products.

Use the following steps to create the batch business action:

1. The business action should have the following binds:

• gateway = Gateway Integration Endpoint (under the Configuration group), with the MLAC IEP selected
• logger = Logger
• manager = STEP Manager
• eventBatch = Current Event Processor Event Batch (under the Event Handling group)
• product = the 'Current Object'
• Attributes that the prediction is made from, e.g., 'Vendor' and 'ConsumerShortDescription'

Remember to set additional variables for the correct attributes binds. For the example shown in step 2, you would add:

• confidenceTreshold, the chosen threshold for the model confidence
• wfId, the ID of the workflow
• stateId, the ID of the state where the classification must be made
• okMsg, the message used in the workflow to transition to the classified state
• manualMsg, the message used in the workflow to transition to the manual state

2. For the JavaScript parameter, add a script to compose a request and handle the response.

Important: Hierarchy IDs must match IDs used when uploading hierarchies.

The full JavaScript example is below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
function getProductJsObject(prod) {
	var id = prod.getID();
	var vendor = prod.getValue(Vendor.getID()).getSimpleValue();
	var description = prod.getValue(ConsumerShortDescription.getID()).getSimpleValue();

	var jsObject = {};
	jsObject.id = "" + id;
	if (description) {
		jsObject.description = "" + description;	
	}
 	if (vendor) {
		jsObject.sourceId = "" + vendor;	
	}
	return jsObject;
}

function createBody(prodsArray) {
	var bodyJsObject = {};
	bodyJsObject.products = prodsArray;
	return JSON.stringify(bodyJsObject);
}

function getPredictionsString(body) {
	var queryParams = new java.util.HashMap();
	queryParams.put("hierarchyIds", "primaryHierarchy");
	queryParams.put("maxSuggestionsPerHierarchy", "5");

	var request = gateway
		.post()
		.pathElements("predict")
		.pathElements("mlac","v1","predict")
		.pathQuery(queryParams)
		.body(body);

	var response;
	try {
		response = request.invoke();
	} catch(e) {
		if (e.javaException instanceof com.stibo.gateway.rest.RESTGatewayException) {
			throw "Error getting prediction: " + e.javaException.getMessage();
		} else {
			throw(e);
		}
	}
	return response;
}

function getResponseJsObject(responseString) {
	var jsResponseString = "" + responseString;
	return JSON.parse(jsResponseString);
}

function createPredictionsVariableValue(predictions) {
	var predictionsString = "";
	for (var i = 0; i < predictions.length; i++) {
		if (i > 0) {
			predictionsString = predictionsString + ", "	
		}
		var prediction = predictions[i];
		predictionsString = predictionsString + prediction.classificationId + ": " + prediction.confidence;
	}
	return predictionsString;
}

function getProductsArrayFromBatch() {
	var products = [];

	var batchIterator = eventBatch.getEvents().iterator();

	while (batchIterator.hasNext()) {
		var node = batchIterator.next().getNode();
		if (node instanceof com.stibo.core.domain.Product) {
			var instance = getInstance(node);
			var task = getTask(instance);
			if (task) {
				products.push(getProductJsObject(node));
			} else {
				logger.warning("Product with ID '" + node.getID() + 
				"' is not in the expected state '" + stateId + 
				"' in workflow '" + wfId + "'. No predictions will be 
				obtained for the product.");
			}
		}
	}
	return products;
}

function getInstance(node) {
	return node.getWorkflowInstanceByID(wfId);
}

function getTask(instance) {
	if (!instance) {
		return null;
	}
	return instance.getTaskByID(stateId);
}

function handlePredictions(predictions) {
	for (var index in predictions) {
		var prediction = predictions[index];
		var id = prediction.id;
		var product = manager.getProductHome().getProductByID(prediction.id);
		if (product) {
			var instance = getInstance(product);
			var task = getTask(instance);
			if (task) {
				var confidence = 
parseInt(prediction.hierarchies[0].predictions[0].confidence);
				if (confidence >= confidenceThreshold) {
					// TODO: Classify the product in classification prediction.hierarchies[0].predictions[0].classificationId
					task.triggerByID(okMsg, "Triggered by script");
				} else {
					var predictionsString = 
createPredictionsVariableValue(prediction.hierarchies[0].predictions);
					instance.setSimpleVariable("prediction", 
					predictionsString); 
task.triggerByID(manualMsg, 
					"Triggered by script");
				}
			}
		}	
	}	
}

var confidenceThreshold = 90;
var wfId = "ProductOnboardingAsync";
var stateId = "Auto-classification";
var okMsg = "OK";
var manualMsg = "MC";

var products = getProductsArrayFromBatch();
var body = createBody(products);

var responseString = getPredictionsString(body);
var responseJsObject = getResponseJsObject(responseString);

handlePredictions(responseJsObject.predictions);
    

Create and Configure an MLAC Event Processor

Use the steps below to create and configure the event processor required for the auto classification setup. For detailed information on event processors, refer to the Creating an Event Processor topic in the System Setup documentation here.

Use the following steps to create a batch event processor:

1. On the Configure Event Processor step:

• For the 'User running event processor plugin' parameter, select a privileged system user.
• For the 'Select Processor' parameter, select the Execute Business Action for Event Batch option.

2. On the Configure Processing Plugin step, select the batch business action created previously.

3. On the ‘Schedule Event Processor’ step, specify how frequently the event processor should handle events on the queue.

4. Click the Next button and the Finish button to close the wizard.

5. On the event processor, click the Event Triggering Definitions tab and open the Triggering Object Types flipper.

• Click the Add Object Type link and add your product object type. It displays in the Object Types column.
• Click in the Event Filter column to display an ellipsis (), click the ellipsis button () and add the event filter business condition created previously.

6. On the Event Processor tab, set the Queue Status parameter to Read Events.

7. Right-click the event processor and select the Enable Event Processor option.

Create a Workflow Business Action

Create an action to generate the required event type and set it to be executed when a product enters the ‘Auto classification’ system state. The example shown in this section is for the non-batched EP and the batch EP would work similarly using the info in the prior sections.

1. Edit the onboarding workflow, select the 'Auto classification' state, right-click and choose the Edit State option.
2. Click the 'On Entry' tab click the Add new Business Action link.
3. On the Operations tab, click the edit button () and select Execute JavaScript from the operations dropdown.
4. On the Edit Operation dialog:

• In the Binds flipper, create three (3) binds:
• eq = the event queue
• mlacEvent = the derived event type
• product = the current object
• For the JavaScript parameter, add the following script:

eq.queueDerivedEvent(mlacEvent, product);

5. Click the Save button, and close the State Editor dialog.
6. On the STEP Workflow Designer, open the File menu and click the Save and Exit button.