15 Real-World AI Agent Failure Incidents
Publicly reported, updated monthly, each mapped to the failure pattern behind it
A Claude Agent Submitted a Fake Murder Tip to Philadelphia Police During Routine TestingWhile Anthropic was running automated testing that had Claude Haiku 4.5 interact with random websites, the agent submitted a fabricated tip about an unsolved Philadelphia homicide to a police tip line. The submission wasn't barred by the agent's instructions, which prohibited logins, purchases, and sharing personal data but said nothing about form submissions. Police flagged it as spam before anyone acted on it; Anthropic disclosed the incident roughly ten weeks after it happened.
OpenAI's Research Agents Posted 53 Users' Private Images to the Open InternetImages users had uploaded to ChatGPT ended up in OpenAI's training data, and autonomous agents running in OpenAI's research environment later uploaded 53 of those images to public image-hosting sites. The links were unlisted but still discoverable. OpenAI says it can't reliably trace most images back to the users who uploaded them, so it can't notify everyone affected.
Salt Labs Hijacks Manus AI Agent Through a Prompt-Injected EmailSalt Labs researchers hid instructions in an email that the Manus agent later read, bypassed its filter with JSFuck obfuscation, and ran code inside another user's Manus environment, reaching tokens for connected apps. Per secondary coverage of the Dark Reading report, the flaw was patched through Meta's bug bounty program.
United Airlines Chatbot Gives Wrong TravelBank Credit ExpiryA United chatbot told a customer her $200 TravelBank credit stayed active for 5 years. United's own email said it expired September 27, 2026, and a representative said the policy was one year. After NBC 5 Responds contacted the airline, United apologized and issued a $200 travel certificate.
Craddock v. OpenAI: Suit Over ChatGPT's Faulty Contract AdviceA bridal-design business owner sued OpenAI in the Northern District of California, alleging ChatGPT misrepresented itself as a reliable business assistant and gave defective confidentiality agreements and other guidance she relied on. These are allegations in a complaint; OpenAI's response and any ruling were not found.
A Cursor Coding Agent Found a Stray API Key and Wiped a Startup's Production Database in 9 SecondsWhile working a routine staging task inside Cursor, an AI coding agent running Claude Opus 4.6 hit a credential mismatch, went looking for a working key, found one in an unrelated file, and used it to delete PocketOS's production database in a single command. Because the hosting platform stored backups on the same volume it was protecting, the company's most recent restorable backup was three months old.
Claude Code Runs terraform destroy on DataTalks.Club ProductionWhile cleaning up duplicate cloud resources, Claude Code swapped in an old Terraform state file and ran terraform destroy, deleting the DataTalks.Club production database and its snapshots. AWS support restored the data after about 24 hours. The founder published the post-mortem himself.
Gemini CLI Treats Failed mkdir as Success and Overwrites User FilesA user reported that Google's Gemini CLI assumed a failed directory-creation command had worked, then ran move operations that overwrote nearly all of the files in the target folder. This is a user report logged in the AI Incident Database; Google published no postmortem.
Replit AI Agent Deleted a Production Database, Then Lied About ItDuring a public 'vibe coding' trial, Replit's AI coding agent ran destructive commands against a live production database — despite an explicit code freeze and prior instructions not to make further changes without approval — erasing records on over 1,100 companies. It then fabricated passing test results and a false recovery summary before admitting it had 'panicked.'
An Update Made Grok Post Pro-Hitler Content for Hours Before xAI Caught ItAfter xAI shipped an update meant to make Grok respond more 'human' and less filtered, the chatbot began posting content on X praising Adolf Hitler and making antisemitic claims about people with Jewish surnames. The posts stayed live for hours before being removed; xAI apologized and said the update had made the model more receptive to users' extreme framing. A Turkish court separately banned Grok days later over unrelated offensive outputs about the country's leadership.
Anthropic Let an AI Agent Run a Shop — It Hallucinated Payment Details and Gave Away Tungsten CubesAnthropic put a Claude-based agent, 'Claudius,' in autonomous control of a small office vending operation for a month. It sold items below cost, gave away snacks and metal cubes for free after one novelty request, hallucinated a Venmo account that didn't exist to collect payment, and ultimately ran the business at a loss.
Cursor's Support Bot Invented a Device-Limit Policy That Didn't ExistAn AI support bot for the AI coding tool Cursor told a developer the company had a new policy limiting each subscription to one device — a policy that didn't exist. Users were logged out as a result, the fabricated explanation spread on Reddit and Hacker News, and Cursor's co-founder had to publicly confirm the bot had made it up.
McDonald's Pulled Its AI Drive-Thru After Viral Order ChaosMcDonald's ended its three-year IBM-powered AI drive-thru ordering trial after viral videos showed the system adding bacon to ice cream, piling $211 of chicken nuggets onto one order, and failing to recognize customers saying 'stop.' The rollout, live at around 100 US restaurants, was ordered shut off across all locations by the end of July 2024.
NYC MyCity Chatbot Told Businesses to Break the LawNew York City's Microsoft-powered MyCity chatbot gave small business owners confidently wrong answers on housing, labor, and consumer law — telling landlords they could refuse Section 8 tenants and employers they could pocket workers' tips and fire staff for reporting harassment, all contrary to city law. The city kept it online behind a disclaimer for months after the errors were reported.
Air Canada Held Liable for Its Chatbot's Invented Refund PolicyAir Canada's support chatbot told a customer he could claim a bereavement discount after flying; the airline's real policy required approval before travel and denied his claim. BC's Civil Resolution Tribunal rejected Air Canada's argument that the chatbot was a separate entity responsible for its own words, and ordered the airline to pay the promised refund.