Without professional standards in tech, an AI Post Office scandal is inevitable
The Post Office Horizon scandal must be a national reckoning with what happens when tech is deployed without proper oversight or accountability, says Dan Howl
“It is almost impossible to ascertain, with any degree of accuracy, the number of persons who have suffered as a result of the misplaced reliance upon data produced by Horizon.”
That sentence appears in the first volume of the final Horizon Inquiry report released yesterday, and it should stop us all in our tracks.
Because this wasn’t just about software – it was a systemic failure of leadership, ethics and governance that wrecked lives and meant that public trust in several professions needed to be rebuilt.
Horizon is one of the most widespread miscarriages of justice in British legal history, but it wasn’t caused by a dystopian sentient AI. It was driven by human decisions, a lack of ethical and professional standards and the legal presumption, now finally under review, that computer evidence is inherently reliable.
Horizon is often called a tech scandal, but that’s far too narrow. It’s a national reckoning with what happens when technology is deployed without oversight or accountability, and when no one is professionally answerable for the consequences.
Over 900 subpostmasters were prosecuted between 1999 and 2015 based on the outputs of a flawed system. Many lost everything. Some lost their lives. And it was all preventable.
Horizon was no AI black box
Horizon wasn’t an AI black box – it was an ordinary IT system, created before the current wave swept over us. It was overseen by humans who should have known better, signed off by committees and legal teams who didn’t ask enough questions, or who didn’t have the authority to speak about what they knew. It was a failure in the here and now: of leadership, culture, and professional competence. Until we confront those failings, we will see another Horizon – and in fact, more could be happening right now
Technology now touches nearly every part of public and private life, from benefits systems and AI-assisted recruitment to biometric ID and predictive policing. But as these tools become more powerful, the frameworks meant to govern them have failed to keep up.
In law or medicine, no one would dream of letting unregulated practitioners make life-changing decisions – but in tech, we’ve tolerated a culture where accountability is diffuse, standards are optional and ethical risks are an afterthought. This cannot continue.
Technology must be treated as a true profession – with clear standards, codes of conduct and genuine consequences when things go wrong
Technology must be treated as a true profession – with clear standards, codes of conduct and genuine consequences when things go wrong. Not just for software developers, but for everyone in the chain of leadership and governance. Because as the Horizon software failed, so did the systems around it: a legal process that presumed computer evidence was infallible, a lack of technical understanding for expert witnesses, and a culture that punished whistleblowers instead of listening to them.
Too many boards still see IT as something separate – too technical, too niche, someone else’s responsibility. But if leaders don’t understand the ethical and operational risks of digital decisions, they must bring in people who do, because without that understanding they are simply not fit to lead in the 21st century.
The ministry of justice’s review into how digital evidence is treated is a welcome action, but it cannot be the end of the story, not least because compensation needs to be delivered to the victims as soon as possible. We need structural change, across government, business, the legal system and the tech profession to ensure that what happened with Horizon is never allowed to happen again.
Unless we start treating professional standards, leadership and governance in tech as seriously as we do in other critical fields, the ‘AI version’ of Horizon is inevitable.
Dan Howl is head of policy and public affairs at BCS, The Chartered Institute for IT