Multiple studies have shown that women and minorities have historically faced higher interest rates on credit, but can the use of artificial intelligence (AI) and machine learning in lending help narrow the gap?

Proponents of AI techniques argue that by removing the human element and the accompanying biases, lenders can make credit decisioning fairer. On the other hand, critics point out that AI itself is susceptible to biases and can sometimes end up amplifying them instead.

“Digital finance represents both an opportunity and a risk,” said Sonja Kelly, director of research and advocacy at Women’s World Banking, a non-profit group that provides low-income women access to the financial tools, at a virtual event held by American Banker last week.

“It’s an opportunity to leverage technology for the sake of inclusion and meeting business goals. But we have the risk of repeating the same mistakes we have made in the past,” Kelly said.

Concerns around bias in AI-driven lending have also caught the attention of federal regulators, not for the first time. In a joint public request for information put out last week, the Office of the Comptroller of the Currency (OCC) and the Federal Reserve, along with three other watchdog organizations, invited comments on how AI and ML techniques are being used in the financial services sector.

Five of the seventeen questions raised by regulators last week relate to how AI usage intersects with federal fair lending laws and the challenges of “uncertainty about how less transparent and explainable AI approaches align with applicable consumer protection legal and regulatory frameworks, which often address fairness and transparency,” the request for information noted.

Since AI techniques depend on the training data used to build models, there is a persistent risk that if the training data is incomplete or biased, the model will mimic those deficiencies and fail to meet the financial inclusion standards. Traditionally, factors like “race and space [geographic location],” have also coincided to produce distortions in lending rates, even when costs associated with originating credit are controlled for, as observed by a study in the “Handbook on the Economic of Discrimination.”

“What happens in reality is there are human beings that come to the table and write the code, who have existing biases that they build into data the algorithms are built on,” Kelly said.

Educational redlining?

The entry of advanced data analytics in lending has also prompted the usage of “alternative” data, such as occupation and educational history, in lending decisions.

One of the firms using such data to make AI-powered lending decisions is Upstart, an online lending marketplace that connects borrowers with lenders using traditional and non-traditional variables to measure creditworthiness.

“We do ask for a few more things that would normally that banks normally do, including we want to know what occupation you have, and what education level you have,” Michael Lock, senior vice president of bank partnerships at Upstart, told Bank Automation News.

While using alternative data on the one hand allows for a broader-picture look at the borrower, it could also end up resulting in higher interest rates for people with certain educational backgrounds or work categories.

“Holding all other inputs for prospective applicants constant, we find that a hypothetical refinancing applicant who attended Howard University, a [historically black college or university], would pay more than an applicant who happened to have attended NYU,” noted a February 2020 study by the Student Borrower Protection Center.

The study examined Upstart’s lending model using hypothetical applicants seeking $30,000 to refinance student loans, to be repaid over three- or five-year terms. “The disparity increases over a five-year repayment term, with the NYU and Howard borrowers paying $42,287 and $45,785, respectively,” the study noted.

Upstart has refuted these findings, noting that it has worked closely with the Consumer Financial Protection Bureau and conducts quarterly fair-lending tests, saying it has consistently found its model reduces bias in lending, instead of boosting it. The fintech also asserted that the study was selective in its approach and elected to present findings that supported its hypotheses.

“Upstart has been working closely with both the Student Borrower Protection Center (SBPC) and the NAACP Legal Defense and Education Fund (LDF) on a landmark process for best practices of fair lending testing for AI,” Mike Nelson, a spokesperson for Upstart, told BAN.

While Upstart and SBPC have since decided to work together to design tests for AI-driven lending models, the SBPC has not retracted the study or made changes to its conclusions. The SBPC did not respond to multiple requests for comment on this story.

“Much has happened since the January 2020 SBPC report, which preceded our collaboration and was selective in its approach,” Nelson added.

Watching for pitfalls ahead

“It’s a tricky issue, both with humans and with algorithms. And in some ways, the human world is a little more transparent,” Craig Le Clair, principal analyst at research firm Forrester, told BAN.

While humans by default have a set of biases, transitioning to a data-driven algorithmic process also throws up challenges with respect to how “how opaque the process is,” Le Clair said. The ease of understanding how an algorithm comes to a decision is also a key factor in determining possible bias in the system, he added.

One of the points raised by panelists at the American Banker event revolved around how core-business metrics fit into the discussion about bias seeping into AI systems, and whether fairness should be a part of such metrics.

“The CEO needs to say, ‘These are the fairness metrics I want to hit. Like, I want everybody who applies to have an equal chance of getting credit, or I want our off credit offers to look equal,’” Women’s Word Banking’s Kelly said. “Engineers and data scientists might be the ones actually writing the code, everybody else needs to be able to understand the issue, make strategic decisions, follow up and monitor things.”

As financial institutions look to leverage AI-powered tools, the ability to understand and monitor automated decision making is likely to determine how well these techniques align with existing regulation. Rigorous testing, ensuring transparency, and engaging with regulators on issues like fair lending may provide the necessary guidance on how to use these tools to expand access and improve efficiency without baking in faults of the past.

“Technology is neutral, and amplifies whatever we give to it, or whatever we tell it to do, and does it in a more efficient way,” Kelly said.