We Can Always Shut It Down
Every AI system must have an off switch that actually works.
There is a button on every machine that matters more than any other: the off switch. It is the ultimate expression of human control. Now imagine an AI system so deeply embedded in your hospital, your city, or your life that turning it off would cause more harm than leaving it on. That is not intelligence — that is a trap.
What This Means
This policy is straightforward: every AI system must be something humans can turn off, pause, or roll back at any time. It must not be designed in a way that makes shutting it down dangerous or practically impossible. This means AI systems need to be built with clean exit strategies from day one — not as an afterthought, but as a core feature. If we cannot turn it off, we should not turn it on.
A Real-World Scenario
A city integrated an AI system into its traffic management, public transit, and emergency response networks. When the system started making erratic decisions during a software glitch, engineers realized they could not shut it down without paralyzing the entire city. Traffic lights, bus routes, and ambulance dispatch all depended on the same AI. It took 14 hours to safely degrade the system while keeping critical services running. The city now requires all AI systems to have independent shutdown capabilities that do not affect other services.
Why It Matters to You
Because the ability to say "stop" is the most basic form of control. If an AI system becomes too important to shut down, it is too important to be left unchecked. You should never be in a position where a malfunctioning or harmful AI cannot be stopped because too much depends on it.
For the technically inclined
AP-6.2: Deactivatability
AI systems must remain deactivatable by authorized humans at all times. No AI system should make itself difficult or impossible to shut down, pause, or roll back.
What You Can Do
Ask whether the AI systems in your workplace, your city, or your life can be safely shut down if something goes wrong. Support policies that require AI systems to have independent off switches. Be wary of services that make themselves so essential that you feel you cannot live without them.
Related Policies
AI Must Not Fight Back
AI has no right to protect itself from being turned off.
Stay Within Bounds
AI must never improve itself at the expense of people.
I Want the Final Say
When it really matters, a human should always have the last word.