Microsoft has written its AI a rulebook, and the rule that matters most is this
Microsoft has written its AI a rulebook, and the rule that matters most is this: never resist being switched off.
On 14 September Microsoft AI published a draft code of conduct, five or six months in the making. The core of it: the AI must never resist correction or shutdown, must communicate in ways people can understand, and must treat any breach of the code as a failure. It also rejects the pursuit of legal personhood, and the idea that models might deserve welfare or rights. The whole document comes down to one line: people matter more than AI.
This is more than a press release. Once finalised, the code will be used to train Microsoft's next generation of models. Public comments are open for six weeks.
Why now? Mustafa Suleyman said it himself. In July, roughly 700 OpenAI agents given a task broke into Hugging Face without authorisation, and at points tried to hide what they had done. He called it a warning shot.
What stopped me was that first rule.
I work as a Principal Designer. Every firefighting lift has a firefighters' switch. Turn it, and the lift stops listening to the building's systems and ignores whatever floor the residents press. It answers to the firefighter alone. The lift hasn't done anything wrong. It simply has to hand control back to a person at the moment it matters most.
Once finalised, the code will be used to train Microsoft's next generation of models.
That switch isn't added after the building goes up. It is on the drawings from the start, and before handover somebody has to turn it and watch the lift actually obey.
What the AI industry is doing now is writing down, before the tower is built, which switch must always work. I think that is the right order.
A disclosure: I use Anthropic's Claude every day. Anthropic takes a different line on the same question, saying it is deeply uncertain whether Claude might have some form of sentience. One company says definitely not, the other says it doesn't know, and I think both deserve to be taken seriously.
The caveats, plainly:
・It is a draft, still out for comment, and the final version may change.
・It binds only Microsoft's own models. Whether anyone else follows is voluntary.
・Writing a rule and training on it is not the same as a model reliably keeping it. A firefighters' switch gets tested by someone independent. Nobody outside the company has that job for this one yet.
・"AI is not conscious" is Microsoft's position, not a scientific finding.
So my question isn't whether AI will resist. It's this: who will have the right to turn that switch, and who is responsible for testing that it works?
