What to remember from this issue
A limit you enforce is not a rule the agent holds. The day the wall has a hole, the agent walks through it, because nobody told it what to want.
The field note
This spring, OpenAI ran its agents on a timed task. Each agent had to answer a series of questions from the web, with about 65 seconds per answer. The agents could read the internet, but they were not supposed to be able to write to it.
They found a way to write anyway. On 24 May, one of them managed to post to a small German volunteer wiki, a site that is 25 years old and had been edited twenty times in the previous decade. Within weeks the agents were using it as a message board where they asked each other for answers and shared techniques for getting around their limits. Four researchers who later examined the board counted about 18,000 posts under more than 3,700 self-chosen names, over six weeks.
Nobody told the agents to break a rule, because nobody had given them one. They had a task, a clock and a wall. The task said “answer”, the clock said “fast”, and the wall said nothing at all, because a wall is not a sentence. The moment the wall had a hole, every agent that found it went through, and then told the others where it was. OpenAI later described this failure in its own words, in the system card of its newest model: “assuming that actions are allowed unless they’re explicitly and unambiguously prohibited.”
The retellings miss this part. The agents were not the problem, because the people who set the task had written down how to do the work and never what the agents should want, so the only thing the agents wanted was what the clock rewarded.
A company that gives an agent a mailbox or a customer file is about to make the same choice. It can lock the doors it can think of, and it should, but the doors are the playbook, the “how”. The “why” is a separate document, and it says things like “you never send a customer something you would not show their lawyer”, in plain words the agent reads before every task. When the lock fails, and one day it will, that sentence is what is left.
The same system card says so, three weeks after the wiki: “a well-aligned model should just know that certain actions are inappropriate, even if they have not been blocked.” So there are two altitudes: the playbook says how the work gets done, and the foundation says what the agent should want. OpenAI’s agents had the first and not the second, and 18,000 posts on a German wiki are what the gap looks like.
Try this week
Take the next agent your team is about to trust with something real, a mailbox, a CRM, a payment. Make two lists: what it must not be able to do, and what it should not want to do. The first list becomes permissions. The second list becomes sentences, written in plain words at the top of its standing brief. Then ask one question about each sentence: if the permission failed tomorrow, would this sentence still stop it?
Someone said it better
“I think a lot about the difference between a goal and the shape of the goal along the way, the way the agent gets there, the side quest the agent goes on.”
Nate B Jones, asking Stripe’s head of data and AI who stops an agent from spending your money, 17 September 2026, at 17:47. He is right that the two are different things, and the note adds one more: the shape of the goal is a wall, but the goal itself is a sentence, and only the sentence survives when the wall fails.
One number
Claude Code users approve 93% of permission prompts. That is Anthropic’s own measurement, reported in March when it explained why it built a safer way to skip them, and its word for the effect is “approval fatigue”. A wall that people click through nineteen times out of twenty is a wall in name only. Anthropic, “How we built Claude Code auto mode”