Turning Data into Decisions — Rules, Scores, and Machine Learning

Every data point that is captured on a lead form, loan application, or new account creation is a piece of potential intelligence that can be used to make better business decisions. Whether that means reducing fraud, improving lead conversion, or finding the right balance between identity verification and signup friction, it’s crucial to understand the mechanisms that give you the confidence to transact with good customers and stop bad actors in their tracks.

A vast majority of the financial services customers that we work with use a combination of automation and manual review in tandem with the formal credit pull to make lending decisions – using ‘positive signals’ (e.g. phone-to-name match) to move good customers through and ‘negative signals’ (e.g. disposable email) to identify potential fraudsters that can then be manually reviewed, approved, or rejected.

In this post, I’m going to focus on automation, looking specifically at three different methods of decision making: rules-based, score-based, and machine learning.

Rules-Based Decision Making
Rules-based decision-making is the foundation upon which all other systems are built. In its simplest form, the data points captured on an application are passed through a predetermined order of “IF…THEN” rules. If one of these rules is tripped, an action is triggered: approve, review, or reject.

For example,

IF: IP proxy = TRUE
THEN: reject application

“IF…THEN” rules are useful, but suffer from two primary limitations:

  1. “Silver bullets” don’t exist. It’s almost impossible to find a standalone data point that screams “fraud!”, and even when you do find a strong signal, it tends to be present in only a small subset of exceptionally risky accounts
  2. Most businesses have an extremely low tolerance for false positives. Fraud mitigation is important but needs to be balanced against the unintential screening of good customers

To work around these limitations, a lender may choose to implement compound rules, which require multiple criteria to be met simultaneously in order for action to be taken:

IF: IP proxy = true AND Phone line type = non-fixed VOIP AND Email = disposable
THEN: Reject application

IF: Phone-to-name = MATCH AND Address-to-name = MATCH AND Email-to-name = MATCH
THEN: Approve application

The main advantage of a rules-based decision system is simplicity. The logic is not complex, and the programming resources needed to roll out this kind of automation are minimal. However, simplicity comes at the price of fidelity, and many lenders feel that it’s difficult to make nuanced decisions with IF…THEN statements. This kind of ‘blunt instrument’ approach can fail to capture certain types of fraud, increase false positive rates, and become outdated if rulesets are not actively managed and updated.

Score-Based Decision Making
Score-based decision making is an extension of the rule system outlined above that goes beyond IF…THEN responses by assigning ‘weights’ or numeric scores to signals based on their predictive strength. For example,

Non-fixed VOIP phone = TRUE : Weight: -100 pts
IP geolocation > 1000 miles from address : Weight: -50 pts
Email = well established : Weight: +50 pts
IP connection type = mobile : Weight: +20 pts
Overall Score: -80 points

These weights are combined to give each application an overall ‘score’, which triggers the decision to approve, reject, or review (e.g. IF: Overall score <0 THEN: Review application).

Score-based decisioning allows lenders to adjust the impact of each individual signal, and in this way translates disparate data points into a common ‘language’: fraud risk. The same low-frequency, highly-actionable signals that inform traditional rule-based systems can still be used, but you can get additional insight on an application by factoring in higher-frequency, less-individually-actionable signals.

The main disadvantage of score-based systems is having to add another programmatic step in your workflow – in addition to identifying which signals move the needle, you have to figure out how to weight these signals in a way that reflects their value. The signals and individual weights also need to be managed on an ongoing basis to ensure that they are effectively moving good customers through and capturing fraud in the process.

Machine Learning Decision Making
Machine Learning is an automated way to make predictions about a future data set based on inputs from an existing data set. In this way, a set of rules can be programmed to ingest data and automatically fine tune the logic of those rules against an overarching goal – e.g. reducing fraudulent loan origination. The model constantly processes new data inputs and self-adjusts relative to the statistical relevance of those inputs, effectively ‘learning’ how to make better decisions.

Machine Learning systems are automated, dynamic, and highly accurate. Moreover, once they have been implemented, maintenance requirements are substantially lower than traditional rule or score based systems, since machine learning is programatically monitored and improved upon. The downside is complexity. Effective machine learning systems can be highly sophisticated, requiring significant development work and a large number of data inputs. Third party solutions are available to address this limitation, but these typically move the process into a ‘black box’ that offers little visibility into the underlying mechanisms that make origination decisions.

To dive deeper into Machine Learning, read the post “The The $200B Cost Every Business Sweeps Under the Rug”.

Quality In, Quality Out
Regardless of which decision making system you use, it’s impossible to successfully automate the process without having access to rich, complete, and accurate data. Lenders and financial institutions layer third party verification solutions, like Identity Check, with formal credit pulls to get the most complete possible view of an applicant before trusting an automated system to make an origination decision. Fortunately, with access to information such as phone line type, address deliverability, and email first seen date, you can move good customers through the funnel faster and more effectively filter out fraudsters looking to exploit vulnerabilities in the world of online decision making.

Contact us to learn more about how we help Alternative Lenders build fast and accurate applicant profiles.

Thanks for reading! You might be interested in these posts, too: