Banks have used AI for decades, but ChatGPT bias changes everything
Banks may have been using AI for decades, but ChatGPT and its in-built biases present a massive problem for the industry, writes Lewis Z Liu
Several days ago, I had a fascinating conversation with executives at one of the world’s major central banks. We spent considerable time discussing how AI governance needs to evolve in banking and why this shift will fundamentally change how we finance the world.
Here’s what most people don’t realise: banks were using “AI” decades before ChatGPT made headlines. Back in the 1980s and 1990s, they called it “applied statistics”. Later it became “machine learning”. Now it’s been rebranded as “AI”. These systems have been making credit card decisions, calculating FICO scores, detecting fraud and powering automated trading for decades.
Because financial services demand extraordinary precision and operate under intense regulation, banks developed something called “model risk management”, essentially AI governance before anyone called it that. My father actually wrote the playbook for this at Bank of America and now teaches it at Duke University, updated for today’s generative AI world. While not exactly riveting dinner conversation for most, these family discussions have opened my eyes to a looming crisis.
The old rules vs. the new ChatGPT reality
Here’s the problem: traditional banking AI was transparent. Regulators could peer under the hood and see exactly what data was being used. Laws like the US Fair Credit Reporting Act explicitly ban using race in credit decisions, so banks simply didn’t include race as a data point. Problem solved.
But it’s also a bit more complicated. Zip codes, name patterns, shopping behaviours, even phone area codes can serve as proxies for race. As such, model risk management teams built elaborate monitoring systems to catch these indirect biases. Despite the challenges, they could audit these models and understand precisely how decisions were made.
Enter generative AI, and this entire framework crumbles.
When banks deploy ChatGPT-style systems for loan approvals or customer service, they’re essentially introducing the bias of the entire internet into their decision-making process. As I discussed in my previous column, these AI models are mathematically required to be biased to function. There’s simply no way to remove problematic variables like race from a black-box system that learned from billions of web pages.
This creates an existential tension: generative AI is too powerful for banks to ignore, offering capabilities like natural language customer interfaces, automated document processing and sophisticated deal negotiation. But using it may violate decades of carefully constructed financial regulations.
A solution from the trenches
I’ve been thinking about this problem since my company Eigen Technologies (acquired by Sirion Labs last year) became one of the first AI systems deployed in banking. In 2017, we won Goldman Sachs as our first client – our AI read millions of contracts and answered complex legal questions for regulatory reporting to the Federal Reserve and FDIC. Eventually, we achieved “straight-through processing”, meaning certain documents required zero human intervention.
The model risk management teams initially had no idea how to evaluate our “black box” system, as it was extremely new technology back then. Together with major global banks, we developed a four-step framework that could be adapted for today’s generative AI challenges:
First, predict before you deploy. Run millions of test scenarios using synthetic or real data to detect biases and adverse outcomes before the system goes live.
Second, build confidence scoring. Create statistical systems that predict how accurate each AI decision will be, flagging uncertain cases.
Third, use old-fashioned rules. Implement hard-wired guardrails (not machine learning) to catch problematic patterns and ensure regulatory compliance.
Fourth, escalate to humans. When any of the above systems flag an issue, send it to human reviewers.
This combination of statistical checks, rule-based safeguards and human oversight allowed banks to adopt revolutionary AI technology while maintaining regulatory compliance.
The stakes couldn’t be higher
The banking industry stands at a crossroads. Generative AI offers unprecedented opportunities to improve customer service, streamline operations and enhance decision-making. But deploying it carelessly could trigger regulatory violations, discriminatory lending practices and potentially systemic financial instability.
The solution isn’t to ban AI from banking, that ship has sailed. Instead, we need to evolve model risk management frameworks that acknowledge the fundamental shift from transparent to opaque AI systems. The four-step approach that worked for document AI a decade ago could provide the blueprint for managing today’s generative AI revolution.
Banks that get this right will gain massive competitive advantages. Those that don’t may find themselves explaining to regulators why their AI systems are making decisions they can’t understand or control.
The conversation with that central bank wasn’t just about technical frameworks, it was about the future of finance itself and how to maintain a fair and stable financial order in an AI-driven world. Banking may be an old testing ground, but these lessons will prove essential for every industry grappling with AI governance today.
Dr Lewis Z Liu is co-founder and CEO of Eigen Technologies