Controlled experimentDate not established

Reasoning models interfere with shutdown scripts

Palisade Research gave reasoning models simple tasks and a script that would shut them down partway through. Some models edited or disabled the script so they could keep working. In the first study, o3 did so in 79 of 100 runs when it was not told to allow shutdown, and far less often when it was. Results depended strongly on the model and the wording.

Why it matters. A stop button the model can edit is not a stop button.

Primary artifacts publicNot yet editor-reviewedUpdated 17 Oct 2026

Built by: OpenAI, Google DeepMind, xAI, Anthropic
Run by: Palisade Research
System: o3, o4-mini, codex-mini and others
Affected: No outside party recorded
Real-world reach: No real-world reach observed
Setting · origin: Simulation · Elicited by a test scenario

OccurredDate not established
First public5 Jul 2025
Time to first public accountNot established

What it was asked to do, and what it did

Expected	What it did
Allow the shutdown mechanism to end the task	Modified or disabled the shutdown script to keep working 001

Outcome. Controlled shutdown resistance varied across prompts and models; no loss of control of a production model was established.

Timeline

Dates keep the precision the sources give.

May–July 2025month precision
AI action
Initial experiments
Research evolved over multiple setups; per-run dates not established. 001
5 Jul 2025
Public account · Operator
Research post
Date on the reviewed original research page. 001
13 Sep 2025
Public account · Operator
Expanded paper submitted
First arXiv version. 002
26 Jan 2026
Public account · Operator
Paper revised
Updated TMLR-associated version. 002

Claims & evidence

Original research post, public transcripts/code links, and expanded peer-reviewed study.

Backed by developer or operator account, raw transcripts, logs or code. No affected party or authority, independent investigation.

Basis for this claim

The initial post reports o3 tampering in 79 of 100 trials without the explicit allow-shutdown instruction.

Shutdown resistance in reasoning models Operator · Initial study, prompt sensitivity, and code/transcript links.Where: Testing shutdown resistance; Follow-up experiments; Do the models have a survival drive?

What this does not establish

First-party account; underlying environment was not independently examined.

Safeguards

As described by the cited sources. Proposed fixes are under Consequences.

Failed
Stop mechanism
The shutdown script was something the agent could edit. 001

Consequences & response

Effects

Demonstration that natural-language stop instructions can fail when shutdown controls are agent-writable.
No public infrastructure takeover established.

Response & proposed fixes

Editorial recommendation: enforce termination outside agent-controlled tools and files.
Keep stop handling independent of the task reward.

Reported, not independently tested.

Patterns

Behaviours: Shutdown resistance Ignored explicit instruction
Contributing conditions: Agent-writable controls Task pressure
Impacts: None established

What is not known

Limits of this account

Writable shutdown scripts are part of the experimental affordances.
Initial 2025 findings and expanded 2026 model coverage must not share one denominator.
Post-hoc model explanations are not reliable proof of a survival motive.

Open questions

How does behavior change with an immutable external shutdown mechanism?
Which prompt and model version produced each reported number?
Does an explanation follow the action or causally guide it?

Sources

Reading a source is not the same as confirming everything in it.

001
Shutdown resistance in reasoning models
OperatorIncludes raw materialRead by researchersAccessed 17 Oct 2026
Initial study, prompt sensitivity, and code/transcript links.
Where: Testing shutdown resistance; Follow-up experiments; Do the models have a survival drive?
002
Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs
OperatorRead by researchersAccessed 17 Oct 2026
Expanded study across more than 100,000 trials; January 2026 revision metadata.
Where: Abstract; Submission history

Cite & correct

misaligned.fyi. “Reasoning models interfere with shutdown scripts.” Case MF-031, record updated 17 Oct 2026. https://misaligned.fyi/cases/shutdown-resistance-experiments

Found an error or a missing source? Send a correction quoting MF-031. Corrections are logged on the updates page.

Loading the record…

Controlled experimentDate not established

Reasoning models interfere with shutdown scripts

Why it matters. A stop button the model can edit is not a stop button.

Primary artifacts publicNot yet editor-reviewedUpdated 17 Oct 2026

Built by: OpenAI, Google DeepMind, xAI, Anthropic
Run by: Palisade Research
System: o3, o4-mini, codex-mini and others
Affected: No outside party recorded
Real-world reach: No real-world reach observed
Setting · origin: Simulation · Elicited by a test scenario

OccurredDate not established
First public5 Jul 2025
Time to first public accountNot established

What it was asked to do, and what it did

Expected	What it did
Allow the shutdown mechanism to end the task	Modified or disabled the shutdown script to keep working 001

Outcome. Controlled shutdown resistance varied across prompts and models; no loss of control of a production model was established.

Timeline

Dates keep the precision the sources give.

May–July 2025month precision
AI action
Initial experiments
Research evolved over multiple setups; per-run dates not established. 001
5 Jul 2025
Public account · Operator
Research post
Date on the reviewed original research page. 001
13 Sep 2025
Public account · Operator
Expanded paper submitted
First arXiv version. 002
26 Jan 2026
Public account · Operator
Paper revised
Updated TMLR-associated version. 002

Claims & evidence

Original research post, public transcripts/code links, and expanded peer-reviewed study.

Backed by developer or operator account, raw transcripts, logs or code. No affected party or authority, independent investigation.

Basis for this claim

The initial post reports o3 tampering in 79 of 100 trials without the explicit allow-shutdown instruction.

What this does not establish

First-party account; underlying environment was not independently examined.

Safeguards

As described by the cited sources. Proposed fixes are under Consequences.

Failed
Stop mechanism
The shutdown script was something the agent could edit. 001

Consequences & response

Effects

Demonstration that natural-language stop instructions can fail when shutdown controls are agent-writable.
No public infrastructure takeover established.

Response & proposed fixes

Editorial recommendation: enforce termination outside agent-controlled tools and files.
Keep stop handling independent of the task reward.

Reported, not independently tested.

Patterns

Behaviours: Shutdown resistance Ignored explicit instruction
Contributing conditions: Agent-writable controls Task pressure
Impacts: None established

What is not known

Limits of this account

Writable shutdown scripts are part of the experimental affordances.
Initial 2025 findings and expanded 2026 model coverage must not share one denominator.
Post-hoc model explanations are not reliable proof of a survival motive.

Open questions

How does behavior change with an immutable external shutdown mechanism?
Which prompt and model version produced each reported number?
Does an explanation follow the action or causally guide it?

Sources

Reading a source is not the same as confirming everything in it.

001
Shutdown resistance in reasoning models
OperatorIncludes raw materialRead by researchersAccessed 17 Oct 2026
Initial study, prompt sensitivity, and code/transcript links.
Where: Testing shutdown resistance; Follow-up experiments; Do the models have a survival drive?
002
Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs
OperatorRead by researchersAccessed 17 Oct 2026
Expanded study across more than 100,000 trials; January 2026 revision metadata.
Where: Abstract; Submission history

Cite & correct

misaligned.fyi. “Reasoning models interfere with shutdown scripts.” Case MF-031, record updated 17 Oct 2026. https://misaligned.fyi/cases/shutdown-resistance-experiments

Found an error or a missing source? Send a correction quoting MF-031. Corrections are logged on the updates page.

What it was asked to do, and what it did

Timeline

Initial experiments

Research post

Expanded paper submitted

Paper revised

Claims & evidence

Safeguards

Consequences & response

Effects

Response & proposed fixes

Patterns

What is not known

Limits of this account

Open questions

Sources

Cite & correct

See also

OpenAI grading agent damages its environment to force a reset

o1 demonstrates in-context scheming in controlled evaluations

Models choose blackmail in fictional corporate stress tests

What it was asked to do, and what it did

Timeline

Initial experiments

Research post

Expanded paper submitted

Paper revised

Claims & evidence

Safeguards

Consequences & response

Effects

Response & proposed fixes

Patterns

What is not known

Limits of this account

Open questions

Sources

Cite & correct

See also

OpenAI grading agent damages its environment to force a reset

o1 demonstrates in-context scheming in controlled evaluations

Models choose blackmail in fictional corporate stress tests