The following assumes you have configured the machine learning matching agent (MLMA) for clerical review. For more information, refer to the Configuring MLMA topic here.
Clerical Review Decisions and Match Agent Training
When a matching agent is configured and enabled, it actively tracks the data steward's merge / reject decisions. Since these decisions affect the quality of future recommendations, it is important that the user carefully considers the process around it. Examples of clerical review processes that will cause inaccurate recommendations:
-
Merging and rejecting erroneous data on the golden records should be resolved on the golden records before resolving the task. This does not include normal outdated data and spelling differences.
-
Rejecting tasks, for different reasons, through the golden records that are the same.
The algorithm performing the machine learning training excludes merge decisions for tasks with three or more records. This is because of the possible data differences in all individual pair combinations that could cause false positives in the machine learning engine. For this reason, the first recommendations may take some time before appearing if the task list contains a minor number of tasks with only two golden records.
The minimum number of decisions the data steward must make is 30 merge and 30 reject decisions. Once the data steward completes this number of decisions on pair-only tasks, the recommendations will appear in the clerical review task list.
Recommendations in Clerical Review Task List
The matching agent continuously learns from the decisions the data steward makes over time. Whenever a certain percentage of additional decisions have been made, the matching agent retrains itself and updates all recommendations.
The recommendations are available as either merge or reject. If the matching agent does not recommend either, the cell will be left blank. The merge / reject recommendations are determined based on scores produced by the machine learning model and certain thresholds. In the early adopter program, these thresholds can be manually adjusted. Contact your Stibo Systems account manager or partner manager to initiate this activity.
Below is an example of how a golden record clerical review task list might appear with recommendations:
