The Data Sufficiency Calculator processing plugin works with the Sufficiency Configuration Type to calculate sufficiency scores for a given product or entity. When run, the plugin evaluates all the sufficiency configuration types within the system and calculates the sufficiency scores for the products or entities included in the 'Triggering Object Types' parameter on the 'Event Triggering Definitions' tab of the event processor. These scores are subsequently output to data containers on the evaluated objects. The sufficiency scores can be viewed in the Sufficiency panel in the Web UI and data containers in the workbench for applicable objects.
Note: To access the Data Sufficiency Calculator Processor plugin, the 'data-sufficiency' add-on component must be activated on your system. Contact Stibo Systems for more information.
Deleted objects (objects in the recycle bin) are ignored during processing.
For more information, refer to these topics in the Data Governance documentation:
Prerequisites
This section of documentation describes configuration steps for this specific processor, but these steps are only part of configuring an event processor. For the full set of instructions for event processor configuration, including event triggering definitions, refer to the Event Processors documentation.
For the Data Sufficiency Calculator event processor plugin to use multiple threads for calculating data sufficiency metrics, set the number of threads via the BackgroundProcess.Queue.<QUEUE_NAME>.Parallel configuration property, which defaults to 2.
Parameters
Each of the relevant parameters for the Event Processor Wizard 'Configure Processing Plugin' step are described below. Any additional wizard parameters with importance for this plugin are also included in this topic.
-
On the Configure Event Processor step:
-
For the User Running Event Processor Plugin parameter, recommended practice is to create a user specifically for this event processor with adequate privileges to access the data that will be published.
-
For the Select Processor parameter, choose the Data Sufficiency Calculator processing plugin to display the expected parameters on the Configure Processing Plugin step.
-
-
On the Configure Processing Plugin step, set the Triggering Workspace parameter:
Important: Use 'Trigger in Main only' or 'Trigger in both Main and Approved' option if unapproved data should be used for sufficiency calculations. This can lead to poor performance particularly if you are triggering on a high volume of data changes or have a high number of complex metrics. Since the benefits of triggering from 'Main' can outweigh the performance risk, thoroughly test in lower environments before running in production. Alternatively, derived events can be used to trigger events prior to approval, as defined in the Derived Events topic of the System Setup documentation.
-
The 'Trigger in Main only' option calculates data sufficiency as changes are made in the maintenance workspace, prior to approval.
-
The 'Trigger in Approved only' option calculates data sufficiency upon approval.
-
The 'Trigger in both Main and Approved' option calculates data sufficiency as changes are made and again upon approval.
Note: To activate a change to the Triggering Workspace parameter, you must invoke the event processor with at least one event on the queue.
The selection made in the wizard is displayed in the Event Processor tab under the Configuration section.
Triggering Workspace Flipper
This flipper is displayed in rare cases, specifically, only if either the 'Main' or the 'Main and Approved' option was previously selected in a release where the option was available.
Although changing the selection to the default 'Approved' radio button removes the flipper from the event processor, the setting can be observed from the Event Processor Wizard on the Configure Processing Plugin step, as defined, and are further explained in the Parameters section above.
-
Event Triggers
The 'Triggering Object Types' parameter, located within the Event Triggering Definitions tab, must include the object types and the attributes included in the metrics that are part of the sufficiencies being run to evaluate data quality and completeness. In the example below, the object types 'SuffScoreProducts' and 'MusicProducts' will be evaluated for quality and completeness when the event processor is run. 'SuffScoreText' is the triggering attribute for the event processor.
Note: When configuring the Data Sufficiency Calculator event processor, attributes included in the 'Data Sufficiency Attribute Group' attribute group should not be added as triggering attributes; additionally, data containers within the 'Data Sufficiency Attribute Group' attribute group should not be added as triggering data container types. Adding either of these as triggers for the event processor can produce unintended events. For more information regarding the Data Sufficiency Attribute Group, refer to the Sufficiency Scores for Data Quality and Completeness topic in the Data Governance documentation.