Machine learning offers a unique opportunity to transform how credit is analyzed for risk and is allocated by lenders. It can learn patterns in data to efficiently and accurately assess the credit risk of borrowers without reliance on traditional credit reporting systems. This is transformative because it means that machine learning can be applied in a range of environments--from banked to unbanked, underserved segments--to develop new ways to quickly determine borrowers' creditworthiness.
Despite the promising advantages, the application of machine learning in financial services can also lead to bad outcomes, such as introducing biases and reinforcing discrimination in lending decisions. Bias can sneak into machine learning at various stages of model development, and there are no built-in checks to detect it.
Example
What does this look like in practice? Let's walk through a simplified example. In this example, a machine learning model was trained on the German credit dataset available on UCI's Machine Learning Repository to classify creditworthy (1) and not-creditworthy (0) borrowers. The dataset has 1000 records, with 21 variables comprised of numerical and categorical data. The model determines that the most important features that determine if a borrower is classified as creditworthy are related to a borrower's checking account, terms of credit (amount and duration), age, and credit history, among other attributes.
According to the machine learning model, creditworthy borrowers are older (36.2 years old) than non-creditworthy borrowers (33.9 years old) on average. In particular, the average age of creditworthy males is higher than non-creditworthy males. The average age of creditworthy and non-creditworthy females is about the same. And, in general, older males are more likely to be classified as creditworthy and receive higher credit amounts than younger females.
Types of bias
Sample bias , also known as selection bias, was unknowingly introduced into the model because the age data provided as an input to train the model is not likely be representative of the data the model will encounter going forward when it is used for the business application. Indeed, with a mean of 35.5 years and positive skew, the age data the model was trained on is not normally distributed. Data encountered in the real world is likely to include more younger and older borrowers.
Moreover, women are notably underrepresented in the dataset. In other words, the model is trained on a lot more male data and, thus, has better learned male borrowers' credit attributes.
There are also several other types of biases that can make their way into machine learning: exclusion bias, measurement bias, recall bias, observer bias, association bias, and racial bias.
Reducing Bias
Under-represented data can lead to a machine learning model wrongly inferring attributes about the unrepresented segments, thus leading to unintended consequences such as exacerbating existing biases. Seeing how prevalent biases can be in machine learning, it is crucial for data scientists and analysts to incorporate bias testing to test assumptions, collect and prepare data fairly, and examine modeling decisions throughout the model development cycle.
Top comments (0)