The Data Sufficiency Calculator processing plugin works in tandem with the Sufficiency Configuration Type to calculate sufficiency scores for a given product. When run, the Data Sufficiency Calculator Processing plugin assesses all the sufficiency configuration types within a system and calculates the sufficiency scores for the products included in the 'Triggering Object Types' parameter in the 'Event Triggering Definitions' tab of the event processor. These calculated scores are then output into data containers on the evaluated products. The sufficiency scores can be viewed in the Sufficiency Inspector panel in the Web UI and data containers within the Web UI for applicable products.
Note: To access the Data Sufficiency Calculator Processor plugin, a 'data-sufficiency' add-on component must be activated on your system. Refer to your Stibo Systems representative for more information.
For more information regarding the sufficiency configuration type, refer to the Sufficiency Configuration Type topic in the Data Governance documentation here.
For more information on the sufficiency inspector in the Web UI, refer to the Sufficiency Panel topic in the Data Governance documentation here.
Prerequisites
This section of documentation describes configuration steps for this specific processor, but that is only one part of configuring an event processor. For the full set of instructions on configuring an event processor, including event triggering definitions, refer to the Event Processors documentation here.
If you want the Data Sufficiency Calculator event processor plugin to utilize multiple threads for calculating data sufficiency metrics, the number of threads to use can be configured via the configuration property BackgroundProcess.Queue.<QUEUE_NAME>.Parallel (defaults to 2).
Parameters
Each of the relevant parameters for the Event Processor Wizard 'Configure Processing Plugin' step are described below. Any additional wizard parameters with importance for this plugin are also included in this topic.
-
On the Configure Event Processor step:
-
For the User running event processor plugin parameter, recommended practice is to create a user specifically for this event processor with adequate privileges to access the data that will be published.
-
For the Select Processor parameter, choose the Data Sufficiency Calculator processing plugin to display the expected parameters on the Configure Processing Plugin step.
-
-
On the Configure Processing Plugin step, set the Triggering Workspace parameter:
Important: Use 'Trigger in Main only' or 'Trigger in both Main and Approved' option if unapproved data should be used for sufficiency calculations. This can lead to poor performance particularly if you are triggering on a high volume of data changes or have a high number of complex metrics. Since the benefits of triggering from 'Main' can outweigh the performance risk, thoroughly test in lower environments before running in production. Alternatively, derived events can be used to trigger events prior to approval, as defined in the Derived Events topic of the System Setup documentation.
-
The 'Trigger in Main only' option calculates data sufficiency as changes are made in the maintenance workspace, prior to approval.
-
The 'Trigger in Approved only' option calculates data sufficiency upon approval.
-
The 'Trigger in both Main and Approved' option calculates data sufficiency as changes are made and again upon approval.
Note: To activate a change to the Triggering Workspace parameter, you must invoke the event processor.
The selection made in the wizard is displayed in the Event Processor Configuration tab.
-
Event Triggers
The 'Triggering Object Types' parameter, located within the Event Triggering Definitions tab, must include the object types and the attributes included in the metrics that are part of the sufficiencies being run to evaluate data quality and completeness. In the example below, the object types 'SuffScoreProducts' and 'MusicProducts' will be evaluated for quality and completeness when the event processor is run. 'SuffScoreText' is the triggering attribute for the event processor.
Triggering Workspace Flipper
This flipper is displayed in rare cases, specifically, only if either the 'Main' or the 'Main and Approved' option was previously selected in a release where the option was available.
Although changing the selection to the default 'Approved' radio button removes the flipper from the event processor, the setting can be observed from the Event Processor Wizard on the Configure Processing Plugin step, as defined, and further explained in the Parameters section above.
Note: When configuring the Data Sufficiency Calculator event processor, attributes included in the 'Data Sufficiency Attribute Group' attribute group (addressed later in this topic) should not be added as triggering attributes; additionally, data containers within the 'Data Sufficiency Attribute Group' attribute group should not be added as triggering data container types. Adding either of these as triggers for the event processor can produce unintended events. For more information regarding the Data Sufficiency Attribute Group, refer to the Sufficiency Scores for Product Data Quality and Completeness topic here.