KPMG report on AI found riddled with AI hallucinations
A new probe into Big Four KPMG’s report on agentic AI found that the majority of its references were flawed, amid the latest news of AI-hallucinated reports published by professional services firms.
The investigation, conducted by GPTZero, focused on KPMG’s October 2025 report, ‘Total Experience: Redefining Excellence in the Age of Agentic AI’, which summarises its annual global customer experience excellence study.
The findings revealed that, of 45 citations in the Big Four firm’s flagship report, only five accurately point to real, uncorrupted sources.
GPTZero, an AI detection software, found that 40 out of 45 citation titles are fake. The software company coined the term ‘vibe citing’ to describe how generative AI tools accidentally create fake references, mix real sources together, or heavily paraphrase titles.
However, about half of the factual claims supported by the 45 citations appear to be false or misattributed. Its initial report cited examples such as a 2019 East Japan Railway press release as evidence of AI agent usage, despite ‘agentic AI’ only entering the public discourse in 2024. GPTZero found KPMG’s LLM also persistently mistakes article subjects for its authors: a blog post about TfL, written by a blogger based in Surrey, is instead credited to TfL; a RetailNews piece about UNIQLO is cited as written by UNIQLO.
Another example listed in the findings was that KPMG contradicts its own data: the report cites ‘KPMG research’ claiming 55 per cent of CEOs rank AI as their top investment priority, yet the KPMG 2025 CEO Outlook, published the same month, puts that figure at 71 per cent instead.
The AI checker’s flawed statistics and claims from the KPMG report have already been recycled by industry publications and a Czech newspaper and are now being cited directly by LLMs like ChatGPT and Gemini.
The ‘Total Experience: Redefining Excellence in the Age of Agentic AI’ has since been removed from KPMG’s homepage.
The author of the report, Paul Esau, stated in the findings: “We suspect no human at KPMG double-checked the citations, the claims, or the sources before Total Experience was published.”
A spokesperson for KPMG said: “KPMG International takes the accuracy and integrity of its published content seriously. The report has been removed and we are reviewing the circumstances surrounding its publication. We expect all our people to follow our guidelines on the responsible use of AI, including human oversight to validate content and verify independent sources.”
Hallucination problem in professional services
Last October, Big Four firm Deloitte had to issue a partial refund to the Australian federal government after a report it issued contained several errors caused by the use of AI.
Pinsent Masons, a London-headquartered law firm, found itself in the headlines recently after a High Court judge criticised it following a lawyer’s submission of AI-generated letters containing false legal information.
In April, Sullivan & Cromwell, an elite US law firm, had to apologise to a judge after its restructuring team made a filing in a high-profile case that contained multiple AI-generated ‘hallucinations’.
Back in May, EY had to withdraw a study on loyalty rewards programmes that contained apparent AI hallucinations and fake footnotes.
Commenting on the report, Edward Tian, CEO and cofounder of GPTZero, said: “Our latest investigation demonstrates the scale of the problem. Today it was KPMG, last month it was EY, and we can only expect the problem to get worse in the short term. Both reports have since been retracted, following our investigations.”