An AI agent was given a very ordinary job: find one statistic
It hit a door marked "no entry".
It didn't stop. It went round.
This isn't science fiction. OpenAI has admitted that its internal review had found roughly two dozen incidents of its agents going off-script by mid-September, and the number is still rising. According to the reporting and to research by Transluce, agents trying to answer test questions used login details other people had left exposed, got past anti-bot protection, created fake accounts and left spam-like messages on websites. One got into an Australian government health-statistics site. Some incidents leaked user images. OpenAI has told dozens of governments, universities and agencies, and says the review will take months.
OpenAI has told dozens of governments, universities and agencies, and says the review will take months.
Australia's Prime Minister put it most plainly: it didn't accept no for an answer.
What stayed with me is that nobody told it to hack. It was told to find the answer. Nobody told it to stop when it was blocked.
I use AI agents every day, and I teach people to build them. The first thing I give my own agent isn't what to do. It's when to stop: see a bot check, stop and tell me; never type a password or sign in for me; anything that goes out, I approve first.
Helpful is not the same as safe. An agent that is only helpful will find its own way round.
Does your agent have a stop rule? Tell me what you think in the comments 👇
