In June 2026, the UK government launched a national artificial intelligence centre for policing.
Days later, a Derbyshire police officer faced a criminal investigation for allegedly using AI to create evidential material across several cases.
It’s not the first, nor will it be the last time we’ll hear of such cases—thanks to the double edged sword of AI use in policing.
AI now reaches across law enforcement, and there’s a strong case for it. The new PoliceAI centre says the technology could free up 6 million police hours a year by 2028—the equivalent of 3,000 officers—much of it by triaging and summarising digital evidence.
But generative AI also invents things that never happened—a failure called AI hallucination. This disturbing pattern shows up across three documented cases:
- A fabricated football match that helped ban fans from a stadium
- An AI-written police report claiming an officer turned into a frog (I know)
- And the Derbyshire allegation of deliberate fabrication
And there’s a downstream cost. At least 15 people have been wrongfully arrested in the US after police trusted a facial recognition match.
The technology misidentifies Black and Asian faces 10 to 100 times more often than white faces, according to NIST.
The Exoneration Registry estimates wrongful convictions have cost US states more than $5.5 billion in compensation since 1989.
None of this argues against AI in policing. Instead, it argues for the rules, audit trails, and human oversight the rollout has so far outrun.
What AI already does for police forces
UK police departments are short on people and time, so the case for AI rests on giving hours back.
PoliceAI cites a kidnapping case where officers reviewed 800 hours of footage in three hours, which produced an early guilty plea.
The plan, part of a £140 million investment in AI technology, funds 40 more live facial recognition units and large-scale pilots in up to 10 forces to triage, disclose, and summarise digital evidence.
Police forces already use AI systems across several tasks. They summarise case files, transcribe 999 and 101 calls, redact audio-visual files, detect deepfakes, and run predictive analytics for crime prevention.
| Use case | What it does | Maturity in UK policing |
| Digital evidence triage | Sorts, summarises and discloses phone and device data | Piloting in up to ten forces |
| Redaction and transcription | Removes sensitive detail; turns 999 and 101 calls into text | Rolling out across forces |
| Predictive analytics | Reads historical crime data for crime patterns | Prototype stage |
| Facial recognition | Matches faces against watchlists to flag suspects | Live in around a dozen forces |
| Report drafting | Turns body-camera audio into AI assisted police reports | Widespread in the US, limited UK use |
Three ways AI fails in crime prevention
Where police agencies and AI use are involved, three failure modes show up often.
The first is unverified AI intelligence feeding an operational decision. In November 2025, West Midlands Police handed Birmingham’s safety advisory group an intelligence report that cited a past Maccabi Tel Aviv match against West Ham.
The match never actually happened—Microsoft’s Copilot, an AI chatbot, had apparently invented it. The group used the report to ban Maccabi Tel Aviv fans from a fixture against Aston Villa.
The chief constable first told MPs “we do not use AI”, then admitted days later that Copilot produced the false reference. Trust in the force took a hit.
The second is ambient contamination in AI-assisted police reports. In Heber City, Utah, the police department’s Draft One tool, built by Axon, produced a police report stating that an officer had turned into a frog.
The system had picked up a Disney film, “The Princess and the Frog,” playing in the background of a recording.
An Electronic Frontier Foundation investigation found that Draft One “seems deliberately designed to avoid audits”, because in many reports it’s impossible to tell which words a machine wrote and which an officer wrote.
Civil liberties groups like the ACLU warn that this design lets an officer blame the AI for anything later challenged in court.
The third is deliberate fabrication in a rules vacuum. The Derbyshire allegation is that an officer used an AI chatbot to manufacture evidential material across several cases.
As of March 2026, there were no rules in the Police and Criminal Evidence Act or the Criminal Procedure Rules that require police to disclose when they’ve used AI.
That’s shadow AI inside a criminal investigation: an officer reaching for an unapproved tool, with no mechanism to catch it.
Case studies of AI errors in policing
| Case | What the AI produced | Why it failed |
| West Midlands Police (2025) | A football match that never took place | Officers trusted an AI chatbot and didn’t check it |
| Heber City, Utah (2025) | A report saying an officer became a frog | The tool transcribed background audio with no audit trail |
| Derbyshire (2026) | Alleged fabricated evidential material | An unapproved tool used with no disclosure rule (shadow AI) |
A wider pattern of AI hallucination across law enforcement
Policing’s hallucination problem is one corner of a wider trend running through the courts.
Damien Charlotin’s database has logged more than 1,620 court decisions worldwide that address AI-generated hallucinations by legal professionals.
Stanford researchers found that purpose-built legal AI tools hallucinate 17% to 33% of the time, while general chatbots answering questions about specific cases hallucinate between 69% and 88% of the time.
US courts have fined lawyers anywhere from $100 to $31,100 for filings built on fake citations, with at least 15 monetary penalties in 2025 alone and disbarment now under discussion.
The harmful effects of AI errors in policing
Harmful use of predictive policing tools or a generative AI system tends to have first, second, and third order effects.
The first layer is wrongful arrest, which can harden into a wrongful conviction.
The ACLU’s count of wrongful arrests in the US include an Oklahoma grandmother jailed for six months on a false match. Police concealed their reliance on the error from the court when they applied for the warrant.
In London, the Metropolitan Police wrongly flagged Shaun Thompson as a suspect in 2024 and held him during a police stop near London Bridge, an encounter he called intimidating and aggressive.
The second layer is years and money.
In 2024, the US recorded 147 exonerations, totalling more than 1,980 years lost to wrongful imprisonment, an average of 13.5 years each. It recorded 97 exonerations in 2025.
A conviction alone cuts a person’s lifetime income by nearly $100,000, and time in prison pushes that loss to about $484,400, according to the Brennan Center for Justice.
In the UK, redress has narrowed: of 591 recent applications to the miscarriage-of-justice scheme, only 39 won compensation, and the law caps any payout at £1.3 million.
The third layer is racial skew, where new hallucination meets old bias.
The NIST vendor test found that many face-matching algorithms produced false positives for Asian and African American faces 10 to 100 times more often than for white faces, with the highest rate for African American women in the one-to-many searches police rely on.
In the UK, of the 10 people the Met’s live facial recognition wrongly alerted on between September 2024 and 2025, eight were Black: 7 Black men, 1 Black woman.
Because past arrests fill the mugshot databases these systems search, biased policing trains the AI to repeat itself. In 2024, 59.2% of US exonerees were Black; in 2025, it was 60.8%.
The fourth layer hits the public purse.
The Brennan Center puts the total income lost each year by Americans with a record at $372 billion—a figure they estimate could give every homeless person in the US a house worth $500,000, outright, with money to spare.
That’s a relevant way to frame it, seeing as formerly incarcerated people are about 10 times more likely to become homeless, according to the Prison Policy Initiative.
So a person wrongly marked by an AI match, then unable to find work, leans harder on public support and on family, which spreads the cost outward across generations.
The money a state pays in compensation is only the visible part of the bill.
What responsible AI in policing requires
These are monumental problems and numbers, but the fixes already exist.
JUSTICE, a UK legal-reform charity, has called for an independent national body to set mandatory technical and governance standards.
These standards would be written into statutory codes of practice, possibly modelled on the Forensic Science Regulator‘s role in the criminal justice system.
For any AI system that touches evidence, that points to a few non-negotiables:
- Explainable outputs
- A full audit trail
- Role-based access
- Routine testing for accuracy and bias before deployment
Human oversight has to be built in, not bolted on. The human-in-the-loop principle, where someone checks and owns each decision, can ensure the police stops fabricated evidence from tainting proceedings.
PoliceAI says it’ll publish a public register of the AI tools forces use, and run testing for accuracy and bias before any tool goes live.
Interim director Alex Murray has framed the centre’s job as getting “responsible AI into the hands of officers.”
A register, independent audits, and clear rules on police AI use will help fulfil that promise.
Oversight still trails deployment, though. The Met scanned more than 3 million faces in 2025, while the equality regulator has had to remind the force to use the technology within human-rights law.
Closing that distance, through standards, audits, and disclosure rules, will keep AI a tool for public safety and help reduce errors in UK policing and beyond.