UK Police Facial Recognition Failures Expose Risks for Business AI
A potential facial recognition match should start an investigation. In some police cases, it became grounds for an arrest before enough checks were made.
An inspection by His Majesty’s Inspectorate of Constabulary and Fire & Rescue Services (HMICFRS) found weaknesses in how police forces in England and Wales and the National Crime Agency use retrospective facial recognition (RFR). In one case, a man was arrested over shoplifting before officers at the investigating force established that he did not match the offender’s image.
For IT leaders, the case raises a practical issue: whether employees independently verify automated recommendations before making decisions that affect customers or staff. The report, published on Thursday, Nov. 12, also raised concerns about demographic differences in algorithm performance, inadequate auditing, and the retention of custody photographs that should have been deleted.
How retrospective facial recognition works
Police use facial recognition in different ways, but two of the main approaches are retrospective and live facial recognition (LFR). The first helps investigators identify people captured in existing footage, while the second allows officers to search for people of interest as they pass cameras deployed in public spaces
With RFR, investigators obtain an image of an unidentified person from CCTV footage, mobile phone recordings, or another source connected to an investigation. They submit that image to a facial recognition system, which compares it against photographs held in police databases, including the Police National Database (PND).
LFR, meanwhile, flags potential watchlist matches in real time. Police deploy cameras in selected public locations, where the system captures faces and compares them against a predetermined watchlist of individuals wanted by police or otherwise of interest to an investigation.
Here is how the two technologies differ:
| Retrospective facial recognition (RFR) | Live facial recognition (LFR) | |
|---|---|---|
| When is it used? | After an incident. | During a live police operation. |
| Image source | Previously recorded CCTV, photographs and videos. | Live feeds from deployed cameras. |
| Database | Stored police photographs, including custody images. | A predetermined police watchlist. |
| Purpose | Generate potential matches for an unknown person captured in existing footage. | Flag potential matches to people on a police watchlist in public spaces. |
| Example | Identifying a suspected burglar using CCTV footage from a crime scene. | Locating a wanted individual through cameras deployed in a busy shopping area. |
The distinction matters because the watchdog’s investigation focuses on retrospective facial recognition, not the live surveillance systems used to scan people in public.
According to the HMICFRS report, some officers placed excessive confidence in potential matches. RFR results are investigative leads, not evidence establishing identity, and require corroboration before enforcement action.
National Physical Laboratory testing in 2024 found that, at some settings, the algorithm was more likely to include women, Black and Asian people, and people aged 21–29 among potential matches. Inspectors could not establish whether those differences contributed to wrongful arrests.
A newer national algorithm has been trialed by Cheshire Constabulary and Essex Police. Laboratory testing found no significant performance differences between demographic groups for that algorithm.
The inspection also uncovered weaknesses in police auditing, inadequate officer training, and instances where custody photographs that should have been deleted remained available for searches.
How this applies to organizations
Beyond police facial recognition systems, this case exposes a problem that is becoming increasingly relevant to organizations deploying automated systems: the tendency to confuse automated output with reliable judgment.
Similar oversight questions arise for businesses using AI to screen job applicants, banks using automated systems for fraud detection, security teams using AI to prioritize alerts, and companies using identity verification during know-your-customer checks.
In each case, an algorithm is making assessments that can influence what happens to an actual person. The technology might be different, but the underlying risk is similar.
Some tools leave the next step to the user: Microsoft Teams’ AI phishing detection displays warnings rather than automatically blocking the sender or ending the conversation. Organizations still need clear procedures for responding to those warnings.
That is where the police case becomes particularly interesting. The problem wasn’t just that facial recognition could return an incorrect match. It was that some officers reportedly treated those matches with more confidence than they deserved.
The same weakness can emerge in organizations where employees gradually become accustomed to accepting automated recommendations without examining how they were produced. These algorithms then stop being a tool that supports human judgment and start becoming a substitute for it.
There is also the question of the data behind these systems. An algorithm is only as reliable as its training data, reference databases, and operating conditions allow. If those inputs contain gaps, outdated records, or demographic imbalances, the resulting decisions can reflect those weaknesses.
What IT teams should check before trusting automated results
The inspection examined policing, but its findings offer a useful starting point for organizations reviewing automated decision workflows. IT leaders should examine how a recommendation becomes an action, who checks it, and how mistakes are corrected.
Set verification rules before deployment. Specify which decisions require independent checks and what information reviewers must examine. For identity verification, that could mean checking additional documentation rather than accepting a similarity score alone.
Give reviewers the authority to disagree. Employees need training on system limitations, time to investigate uncertain results, and a clear escalation route. A human approval step offers little protection if staff routinely accept the recommendation without checking it.
Ask vendors and internal system owners to document:
- Performance at deployed settings: False-positive and false-negative rates at the thresholds the organization actually uses, with demographic breakdowns where relevant.
- Searchable records: Which data enters the reference database, how errors are corrected, and how expired records are removed from active searches.
- Decision records: The system output, supporting checks, reviewer, and reason for the final action.
- Correction routes: How customers or employees can challenge a mistaken outcome and who can reverse it.
Audits should examine completed decisions as well as technical accuracy. Sample cases where employees accepted, rejected, or overrode recommendations to establish whether verification rules work in practice.
Before expanding deployment, IT leaders should be able to trace a consequential decision from the original flag to the supporting information and the person who authorized the action. That provides a concrete test of whether human oversight is functioning.
