When working with the Machine Learning Match Recommendations (MLMR), Stibo Systems recommends that the matching agent is configured to similar data points as the matching algorithm it is tied to. Like matching algorithm results, the matching agent recommendations are most accurate if high data quality is insured.
The number of recommendations provided depends on the decisions made by the data steward. If the decisions are very inconsistent, meaning that similar tasks are both merged and rejected, then it is likely that only a few recommendations will be given. On the contrary, if decisions are consistent for similar patterns in the data, then more recommendations are given. In the beginning, when the data steward has made less than 200 – 300 decisions, the number of recommendations will vary from training to training, but over time they will stabilize as the data steward makes more decisions.
All recommendations are evaluated by a novelty filter which will avoid giving recommendations on tasks that have characteristics which are unknown to the matching agent. This is intended to prevent recommendations from being generated for scenarios that have never been encountered and addressed by a data steward. It will be most active in the beginning, when the data steward has made a limited number of decisions. Additionally, it is active when users are working in silos and clerical review decisions are made in the data silos, e.g. a certain region where data has certain completeness. For example, when all existing decisions have been done for tasks lacking data for a social security number, the novelty filter will avoid giving any recommendation on tasks that actually have a social security number. Over time, when enough decisions have been made, most recommendations will bypass the novelty filter and thereby a larger set of recommendations will be provided.
Matching agents continue to learn based on data steward decisions. However, this presents the risk that too many inaccurate / inconsistent data steward decisions are considered by the matching agent. If the number of recommendations becomes low because of previous inconsistent decisions, you may need to start over and create a new matching agent.