The Consumer Financial Protection Bureau (CFPB) placed artificial intelligence (AI)- and machine learning (ML)-powered credit decisioning into its crosshairs last week, with documentation likely being key to satisfying the regulator’s inquiries.

Under the Equal Credit Opportunity Act (ECOA), creditors are required to send adverse action notices to consumers who are denied credit, explaining why they were not offered a loan or lease. The CFPB has publicly voiced concerns that some creditors using “black box” underwriting algorithms may not be able to fully explain their algorithm’s decisions, a violation of both ECOA and the Unfair, Deceptive or Abusive Acts or Practices Act.

In AI- and ML-based systems, complex algorithms scale and seek patterns in large amounts of data from various sources, allowing for better-quality credit decisions. But the complexity of these algorithms in AI models poses distinct technical challenges for firms to meet regulatory requirements that models be explainable and interpretable, Rafael Garcia-Navarro, chief data scientist at AI-powered decisioning platform Provenir, told Auto Finance News.

“The concepts of explainability and interpretability are important not only to communicate to customers why and how they may have been declined credit — for example, via adverse action notices, such as ‘too many credit enquiries in last six months’ — but also for the lender to be confident the model is operating appropriately,” Garcia-Navarro said.

Provenir has developed an approach widely referred to as explainable AI to interpret and explain individual model decisions using two methods: Shapley Additive and Explanation (SHAP) and Local Interpretable Model-agnostic Explanation (LIME), Garcia-Navarro said.

“SHAP values can be used directly to determine for an individual decision what the most influential features were generating the specific prediction and can therefore be used to inform adverse action notices,” he said.

Simply put, SHAP is an algorithm that polices the credit-decisioning algorithm, documenting, explaining and monitoring how the various inputs of a specific consumer’s attributes affect the final credit decision and pinpointing which variables were most influential to that decision.

Navigating the gray area

AI lending platform Upstart also uses SHAP to ensure its credit models are behaving as intended and can provide adverse action notices to consumers, especially when credit decisions are made outside of the “hard rules” that lenders specify in their scorecards, Jeff Keltner, senior vice president of business development, told AFN.

“Most lenders we work with have their own credit policies. They might have, let’s say, a minimum credit score, they might have a maximum debt-to-income ratio, a maximum loan-to-value ratio,” Keltner said. “Those are pretty easy things to [explain], and the majority of our declines are based on those kind[s] of hard rules.”

The more complicated aspect of AI credit decisioning is if the credit policy itself is being driven by the model outside of hard stipulations, he said.

For example, a lender may not finance consumers with credit scores below 640 but might want to use AI-powered algorithms to expand their credit box and lower that “hard rule” to 580, using credit file and vehicle information to price the risk of the loans. If the risk is too high, the lender would decline the loan.

That’s where SHAP comes in, Keltner said, noting that Upstart has been working with CFPB since 2014 to ensure it remains compliant with the Bureau’s high standards for compliance under the ECOA. Upstart tests every loan application for fairness and pre-tests every model update prior to implementation.

“We have a system that looks at declines and asks, ‘What were the drivers of that decision? Here are the adverse action reasons,’ and presents them immediately to the applicant. Here are the reasons that were driving that” decision, he said.

Model performance management company Fiddler employs a similar strategy in understanding the outputs of its AI model, Krishna Gade, founder and chief executive, told AFN.

The company’s AI program attempts to understand the rationale of the model by asking it various counterfactuals, Gade said, such as: “We’re rejecting a request for a $10,000 loan. But what if they asked for [a lower amount] … Would we accept it then? What if their previous debt was not $40,000 but $20,000?”

By posing these questions, the lender can pinpoint the factors that went into a rejection for a loan, he noted.

Collect the STIPs

Still, lenders will need to collect hard documentation to prove it has verified the borrower’s financial standing against its credit requirements and satisfy obligations specified by the Bureau, Consumer Portfolio Services (CPS) Chief Operating Officer Mike Lavin told AFN.

CPS, which uses a proprietary algorithm to underwrite loans, has been examined by the Bureau twice, Lavin said.

“Both times we’ve received letters … they’re basically letters of no fault, so they’ve signed off on our credit algorithm as being OK,” he said. “The way they go about [the examination process], is they’ll ask for maybe 100 sample files of customers that you turned down. And what they want to see is documents, so the best thing you can do to guard against the CFPB finding out something wrong is to have as many documents as possible.”

The first and core document the Bureau wants to see is proof of income, which is usually just a paycheck, Lavin said.

“If you don’t check proof of income, then what you’re failing to do is analyze the customer’s ability to repay the loan, and that’s what the CFPB cares about,” he said.

“So, if I give them 100 files, and they’re seeing that there’s no proof of income in there, they’re going to ding you pretty hard,” Lavin said.

“Then it rolls downstream to proof of residency, utility bills, things like that. And if we, for example, decline a customer because they only make $1,100 a month and we need $1,500 a month, the CFPB is going to want to see the proof of income to see that our algorithm correctly decided that they didn’t make enough money,” he said.

–Additional reporting by Felix Behr and Loraine Lawson

Bank Automation Summit Fall 2022, taking place Sept. 19-20 in Seattle, is a crucial event on automation and automation technology in banking. Learn more and register for Bank Automation Summit Fall 2022.