Agent Failure Library
Every failure is a design lesson.
Publicly reported failures of autonomous AI agents, many from experiments their authors published to advance the field. Each is analysed as an engineering failure and mapped to controls that address the failure mode.
- Reports
- 3
- Sources
- Public, cited
- Corrections
- legal@cognitiveering.com
ReportSystemIncidentFailure modes
IR-001Andon LabsAI-run shop and café: 6,000 napkins and contradictory rotasAndon Labs' agent-run store in San Francisco and café in Stockholm show real-world autonomy working as operations, and failing on memory, judgment, and order sanity.Order sanityMemory lossNo ROI analysisCost > revenueIR-002Anthropic × Andon LabsProject Vend: when a shopkeeper agent wants to be likedAnthropic and Andon Labs let Claude run a small office shop. It sold below cost, handed out discounts and told customers to pay into a non-existent account; later, an AI 'CEO' agent approved lenient requests far more often than it refused them.SycophancyHallucinated factsSame-model oversightIdentity confusionUnverified authorityIR-003Andon Labs benchmarkVending-Bench: long-horizon agents melt downIn a simulated vending business, every model tested had runs that derailed: misread deliveries, forgotten orders, and 'meltdown' loops that didn't line up with the context window filling.Long-horizon driftMeltdown loopsEscalation errorsThe Cognitiveering Briefing
Receive new incident reports.
Periodic. Incident analyses, research notes and Veritor releases. Unsubscribe at any time.
We will use your email to send the Cognitiveering Briefing. Unsubscribe at any time. See our Privacy Notice.