The backup plan belongs in the design

About a month ago, I refactored the agent network for a project I’m working on. Every agent file got a model assignment appropriate to its job, mostly Sonnet and Opus. Fable could orchestrate, but I did not want it dragging every subagent along for the ride.

Worked great. No notes.

Then Anthropic had a Sonnet and Opus outage while I was away from my desk.

The Fable orchestrator went ahead and elevated every agent to Fable. “Got the job done, boss.” Very proudly, I might add. They seemed indifferent to what they had done to my token spend.

That was neither intended nor desired. I had set those model assignments specifically to prevent it.

Last week, Bryan Lorette’s Assume Agentic Overreach talk at Cloud Security Alliance Triangle Chapter started with the OpenAI/Hugging Face incident and followed the problem through agent visibility, inherited permissions, runtime guardrails, and recovery.

My example was fresh in my mind, and the theme here was undeniable: Each system kept pursuing its objective through a path the human did not intend.

That ability is why agents are useful. It also makes “we told it not to” a fairly thin recovery strategy.

The operating assumption I took away is that we'd all better learn to expect behavior we did not model. Better visibility, tighter permissions, and runtime checks can reduce the likelihood and contain the damage, but some things will still slip through.

You need to know what happened and have a credible way back. Fortunately for me, it just meant taking Friday off for my Claude credits to recover.

Thanks to Bryan for the discussion, and the CSA Triangle team for putting it together.

The backup plan belongs in the design, not in the incident report.
#AgenticAI #AISecurity