Approved by AI, Rejected by Reality: The New Threat to Commercial Banking Relationships

AI in banking and lending is making credit decisions with data it cannot fully explain, producing outcomes regulators are now actively investigating—and relationship bankers who rely on it without human oversight are taking on that risk personally

‍

Key Takeaways

• A 2023 CFPB study found that over 60% of AI-based credit decisions lacked explainable reasoning when reviewed by compliance teams—a direct regulatory concern, since lenders must provide specific reasons for denial
• In October 2024, the CFPB fined Apple $25 million and Goldman Sachs $45 million in separate actions related to Apple Card issues, reinforcing that “the algorithm did it” is not an acceptable explanation to regulators
• The New York State Department of Financial Services launched an investigation into Goldman Sachs’s Apple Card lending after documented evidence of gender disparities in credit limits—with the superintendent stating explicitly: “There is no such thing as, the company didn’t do it, the algorithm did.”
• One in five people may have a material error on their credit report; AI systems that ingest this data inherit and potentially amplify those errors in their outputs
• For relationship bankers, the risk is both regulatory and relational: a client who was wrongly assessed, declined, or given wrong information based on AI output will not forget it—and the banker whose name was on the interaction owns the relationship damage

‍

Estimated Read Time: 4 minutes

‍

When the Algorithm Decides and Nobody Can Explain Why

In 2019, David Heinemeier Hansson, creator of Ruby on Rails and co-founder of Basecamp, posted on Twitter that Apple Card had given him twenty times the credit limit of his wife, despite her having a higher credit score. The algorithm had decided. Apple Card’s customer service could not explain why. The New York State Department of Financial Services opened an investigation. Goldman Sachs, Apple Card’s banking partner, was asked to demonstrate that its credit model did not produce discriminatory outcomes. The superintendent’s statement was unambiguous: the fact that an algorithm made the decision does not immunize the institution from the consequences of that decision.

This case became one of the most cited examples in discussions of AI lending bias. But the underlying dynamic—an AI system making decisions that cannot be explained, based on data that may contain errors, producing outcomes that are disparately distributed—has been replicated in lending contexts across the industry. The CFPB’s finding that 60% of AI-based credit decisions lack explainable reasoning is not a technology critique. It is a compliance finding with direct regulatory implications for every institution deploying AI in lending decisions.

The Relationship Banker’s Exposure

Commercial and business banking is, at its core, a relationship business. The loan officer or relationship manager who serves a business client is building a connection based on understanding—understanding the client’s business, their growth plans, their risk profile, and their needs over time. When AI tools are used to generate client assessments, summarize relationship histories, or inform credit recommendations, the relationship manager’s name is still attached to the outcome. The client holds the banker accountable for what the bank tells them—not the model.

The practical risk manifests in several ways. An AI-generated relationship summary that mischaracterizes a client’s financial history creates the wrong frame for a credit conversation. An AI tool that flags a credit risk based on data that includes a documented reporting error leads the banker into a conversation where the client is being assessed on wrong information. An AI that surfaces a product recommendation based on incomplete knowledge of the client’s actual situation puts the banker in the position of presenting an ill-fitting solution to someone who trusted them to know better. The discipline of capturing client interaction data accurately is the first defense against all three of these failures.

‍

‍

Regulatory Accountability Is Already Here

The CFPB’s August 2024 comment to the Treasury Department closed a gap that some fintech firms had hoped would provide cover: “There are no exceptions to the federal consumer financial protection laws for new technologies.” This statement, combined with the Goldman Sachs and Apple Card enforcement actions, establishes the regulatory frame for AI in banking with clarity. Compliance obligations do not change because a model made the decision. Explainability requirements do not disappear because the output was AI-generated. The institution, and by extension the relationship professional representing that institution, is responsible for the accuracy and fairness of every client-facing output.

The response that high-performing banking teams are building is not a retreat from AI—it is an investment in the human layer that makes AI use defensible. When every client meeting generates an accurate, human-verified CRM record, the relationship manager can stand behind what they told the client, demonstrate what the client told them, and show that every recommendation was based on accurate, current information. That record is the defense against regulatory scrutiny and the foundation of client trust—both of which matter far more than the efficiency gain from automating the step that produced them.