Thomas Unise
Law enforcement agencies are rapidly adopting AI tools for police work, but a recent incident in Heber City, Utah highlights the concerning flaws in these systems. When an AI-powered report writing software mistakenly claimed an officer transformed into a frog, it raised serious questions about the reliability of such technology in critical law enforcement applications.
The Bizarre AI Police Report Incident
The Heber City Police Department has been testing Draft One, an AI software developed by Axon (the company behind Taser weapons) that automatically generates police reports from body camera footage. During testing, the system picked up audio from Disney’s “The Princess and the Frog” movie playing in the background and incorporated this fictional content into an official police report.
“The body cam software and the AI report writing software picked up on the movie that was playing in the background,” explained police sergeant Rick Keel. “That’s when we learned the importance of correcting these AI-generated reports.”
Growing Concerns About AI in Policing
This incident highlights several critical issues with AI-powered police tools:
- Even simple mock traffic stops resulted in reports requiring numerous corrections
- The software uses OpenAI’s GPT language models, which are known to produce hallucinations
- Experts warn these tools could perpetuate existing racial and gender biases in law enforcement
- The Electronic Frontier Foundation found it’s “often impossible to tell which parts of a police report were generated by AI and which parts were written by an officer”
- Critics argue these systems could reduce officer accountability by introducing deniability for errors
The Push for AI Despite Flaws
Despite these serious concerns, some officers praise the technology for its efficiency. Sergeant Keel claimed the tool saves him “six to eight hours weekly” and described it as “very user-friendly.” The department is also testing a competing AI software called Code Four.
Law enforcement agencies nationwide are rapidly adopting AI for various applications including facial recognition and report writing, often before the technology has been proven reliable or safeguards established.
The Broader Implications
The Foundation for Liberating Minds raised concerns about the same company providing both Tasers and AI software to police departments. Meanwhile, the Electronic Frontier Foundation questioned who truly benefits from this technology, suggesting the advantages may not extend to the public or justice system.
As AI continues to be integrated into law enforcement, this incident serves as a cautionary tale about the potential consequences of implementing immature technology in critical public safety roles without sufficient oversight or testing.
GIPHY App Key not set. Please check settings