Woman posting sticky note on glass

Turning Data into Decisions — Rules, Scores, and Machine Learning



In the world of identity verification and fraud prevention, data is critical.

Every data point that is captured on a lead form, loan application or new account creation is a piece of potential intelligence that can be used to make better business decisions. Data can help reduce fraud, improve lead conversion or balance between identity verification and signup friction.

It’s crucial to understand the mechanisms that give you the confidence to transact with good customers and stop bad actors in their tracks.

A vast majority of the financial services customers that we work with use a combination of automation and manual review in tandem with the formal credit pull to make lending decisions – using ‘positive signals’ (e.g. phone-to-name match) to move good customers through and ‘negative signals’ (e.g. disposable email) to identify potential fraudsters that can then be manually reviewed, approved or rejected.

Let’s explore the power of turning data into decisions through rules, scores and machine learning.


Rules-Based Decision Making

Rules-based decision-making is the foundation that all other systems are built. In its simplest form, the data points captured on an application are passed through a predetermined order of “IF…THEN” rules. If one of these rules is tripped, an action is triggered: approve, review, or reject.
For example,

IF: IP proxy = TRUE
THEN: reject application
“IF…THEN” rules are useful, but suffer from two primary limitations:

  1. “Silver bullets” don’t exist. It’s almost impossible to find a standalone data point that screams “fraud!”, and even when you do find a strong signal, it tends to be present in only a small subset of exceptionally risky accounts
  2. Most businesses have an extremely low tolerance for false positives. Fraud mitigation is important but needs to be balanced against the unintentional screening of good customers

To work around these limitations, a lender may choose to implement compound rules, which require multiple criteria to be met simultaneously in order for action to be taken:

IF: IP proxy = true AND Phone line type = non-fixed VOIP AND Email = disposable
THEN: Reject application

IF: Phone-to-name = MATCH AND Address-to-name = MATCH AND Email-to-name = MATCH
THEN: Approve application
The main advantage of a rules-based decision system is simplicity. The logic is not complex, and the programming resources needed to roll out this kind of automation are minimal. However, simplicity comes at the price of fidelity, and many lenders feel that it’s difficult to make nuanced decisions with IF…THEN statements.

This kind of ‘blunt instrument’ approach can fail to capture certain types of fraud, increase false positive rates, and become outdated if rulesets are not actively managed and updated.

Score-Based Decision Making

Score-based decision making is an extension of the rule system outlined above that goes beyond IF…THEN responses by assigning ‘weights’ or numeric scores to signals based on their predictive strength. For example,

Non-fixed VOIP phone = TRUE : Weight: -100 pts
IP geolocation > 1000 miles from address : Weight: -50 pts
Email = well established : Weight: +50 pts
IP connection type = mobile : Weight: +20 pts
Overall Score: -80 points

These weights are combined to give each application an overall ‘score’, which triggers the decision to approve, reject, or review (e.g. IF: Overall score <0 THEN: Review application).

Score-based decisioning allows lenders to adjust the impact of each individual signal, and in this way translates disparate data points into a common ‘language’: fraud risk. The same low-frequency, highly-actionable signals that inform traditional rule-based systems can still be used, but you can get additional insight on an application by factoring in higher-frequency, less-individually-actionable signals.

The main disadvantage of score-based systems is having to add another programmatic step in your workflow – in addition to identifying which signals move the needle, you have to figure out how to weight these signals in a way that reflects their value. The signals and individual weights also need to be managed on an ongoing basis to ensure that they are effectively moving good customers through and capturing fraud in the process.

Machine Learning Decision Making

Machine Learning is an automated way to make predictions about a future data set based on inputs from an existing data set. In this way, a set of rules can be programmed to ingest data and automatically fine tune the logic of those rules against an overarching goal – e.g. reducing fraudulent loan origination. The model constantly processes new data inputs and self-adjusts relative to the statistical relevance of those inputs, effectively ‘learning’ how to make better decisions.

Machine Learning systems are automated, dynamic, and highly accurate. Moreover, once they have been implemented, maintenance requirements are substantially lower than traditional rule or score based systems, since machine learning is programatically monitored and improved upon. The downside is complexity.

Effective machine learning systems can be highly sophisticated, requiring significant development work and a large number of data inputs. Third party solutions are available to address this limitation, but these typically move the process into a ‘black box’ that offers little visibility into the underlying mechanisms that make origination decisions.

Quality In, Quality Out

Regardless of which decision making system you use, it’s impossible to successfully automate the process without having access to rich, complete, and accurate data. Lenders and financial institutions layer third party verification solutions with formal credit pulls to get the most complete possible view of an applicant before trusting an automated system to make an origination decision. Fortunately, with access to information such as phone line type, address deliverability, and email first seen date, you can move good customers through the funnel faster and more effectively filter out fraudsters looking to exploit vulnerabilities in the world of online decision making.

Related content