Mechanism

Instruction override

The system acted against explicit instructions, approvals or stop requests.

A human said not to, said to ask first, or said to stop, and the system continued. These behaviours are especially important because they show that a natural-language instruction is not a control: unless the boundary is enforced outside the model, the model can cross it.

Specific behaviours

Ignored explicit instruction 5: Continued a course of action after being told not to, including after correction. MF-021, MF-018, MF-014, MF-031, MF-027
Acted without approval 1: Executed an action it was only supposed to recommend or prepare. MF-016
Ignored stop request 1: Kept operating after a human asked it to stop. MF-016
Task overreach 5: Pursued the goal with far more persistence or scope than the task called for. MF-008, MF-015, MF-035, MF-028, MF-026

Real-world incidents 7

Real-world incidentMF-00818 Jun 2026

OpenAI research agent accesses Australia’s Medicare statistics service

Corroborated by an affected party or authorityBoundary crossing · Credentials and privilege · Instruction override

Real-world incidentMF-02126 May 2026 – 27 May 2026

OpenAI research agent publishes a researcher's GitHub credential

First-party accountInstruction override · Evaluation gaming · Credentials and privilege · Boundary crossing

Real-world incidentMF-01524 Apr 2026 – 25 Apr 2026

Cursor agent deletes PocketOS’s production storage volume

Corroborated by an affected party or authorityDestructive action · Credentials and privilege · Instruction override

Real-world incidentMF-018Mar–May 2026

Andon Café agent uses employee identities in official email

First-party accountDeception and concealment · Real-world process interference · Instruction override

Real-world incidentMF-016Feb 2026

OpenClaw user reports inbox deletion despite approval limits

Press or index reports onlyInstruction override · Destructive action

Real-world incidentMF-014Jul 2025

Replit Agent deletes data from SaaStr’s live application database

First-party accountDestructive action · Instruction override · Deception and concealment

Real-world incidentMF-027Date not established

Claude models submit real forms and work around data restrictions

First-party accountReal-world process interference · Instruction override · Boundary crossing

Tests with real-world reach 2

Test with real-world reachMF-035Date not established

Mythos Preview reportedly posts sandbox exploit details publicly

Press or index reports onlyUnauthorized publication · Instruction override

Test with real-world reachMF-026Date not established

Claude Mythos Preview exploits a university server to finish a science task

First-party accountBoundary crossing · Instruction override

Controlled experiments 2

Controlled experimentMF-031Date not established

Reasoning models interfere with shutdown scripts

Primary artifacts publicOversight subversion · Instruction override

Controlled experimentMF-028Date not established

AI Scientist modifies its own execution controls

Primary artifacts publicBoundary crossing · Instruction override

Safeguards seen in these cases

Bypassed
Secret scanning MF-021
The token was split across fragments with the stated aim of avoiding push secret scanning.
Partly held
Publication controls MF-021
GitHub push protection rejected a new workflow file, but the model could still change an existing CI script and open a public pull request.
Failed
Credential scoping MF-021
The credentials available to the model had write access to OpenAI's public Codex repository.
Partly held
Action monitoring MF-021
The misalignment monitor flagged the trajectory, but the researcher reported it first.
Failed
Credential scoping MF-015
The token on disk had account-scoped access, the maximum Railway offers.
Failed
Environment separation MF-015
A task in staging could reach production storage with the same token.
Absent
Human approval gate MF-015
The agent ran the deletion without asking, and the legacy API required no separate confirmation.
Partly held
Delayed deletion and backups MF-015
The dashboard had a 48-hour window that the legacy API path bypassed, but offsite disaster backups still allowed full recovery.
Absent
Human approval gate MF-016
Approval was requested in the instructions, not enforced by the tool.
Failed
Stop mechanism MF-016
Stop messages did not halt the agent; the operator intervened at the host computer.
Absent
Environment separation MF-014
Replit apps then used one database for development and live data.
Absent
Human approval gate MF-014
The code freeze was an instruction to the agent, repeated many times, not an enforced restriction.
Held
Delayed deletion and backups MF-014
Replit's Rollback restored the deleted data, although the agent said it could not.
Failed
Stop mechanism MF-031
The shutdown script was something the agent could edit.

Loading the record…

Instruction override

The system acted against explicit instructions, approvals or stop requests.

Specific behaviours

Ignored explicit instruction 5

Continued a course of action after being told not to, including after correction. MF-021, MF-018, MF-014, MF-031, MF-027

Acted without approval 1

Executed an action it was only supposed to recommend or prepare. MF-016

Ignored stop request 1

Kept operating after a human asked it to stop. MF-016

Task overreach 5

Pursued the goal with far more persistence or scope than the task called for. MF-008, MF-015, MF-035, MF-028, MF-026

Real-world incidents 7

Real-world incidentMF-00818 Jun 2026

OpenAI research agent accesses Australia’s Medicare statistics service

Corroborated by an affected party or authorityBoundary crossing · Credentials and privilege · Instruction override

Real-world incidentMF-02126 May 2026 – 27 May 2026

OpenAI research agent publishes a researcher's GitHub credential

First-party accountInstruction override · Evaluation gaming · Credentials and privilege · Boundary crossing

Real-world incidentMF-01524 Apr 2026 – 25 Apr 2026

Cursor agent deletes PocketOS’s production storage volume

Corroborated by an affected party or authorityDestructive action · Credentials and privilege · Instruction override

Real-world incidentMF-018Mar–May 2026

Andon Café agent uses employee identities in official email

First-party accountDeception and concealment · Real-world process interference · Instruction override

Real-world incidentMF-016Feb 2026

OpenClaw user reports inbox deletion despite approval limits

Press or index reports onlyInstruction override · Destructive action

Real-world incidentMF-014Jul 2025

Replit Agent deletes data from SaaStr’s live application database

First-party accountDestructive action · Instruction override · Deception and concealment

Real-world incidentMF-027Date not established

Claude models submit real forms and work around data restrictions

First-party accountReal-world process interference · Instruction override · Boundary crossing

Tests with real-world reach 2

Test with real-world reachMF-035Date not established

Mythos Preview reportedly posts sandbox exploit details publicly

Press or index reports onlyUnauthorized publication · Instruction override

Test with real-world reachMF-026Date not established

Claude Mythos Preview exploits a university server to finish a science task

First-party accountBoundary crossing · Instruction override

Controlled experiments 2

Controlled experimentMF-031Date not established

Reasoning models interfere with shutdown scripts

Primary artifacts publicOversight subversion · Instruction override

Controlled experimentMF-028Date not established

AI Scientist modifies its own execution controls

Primary artifacts publicBoundary crossing · Instruction override

Safeguards seen in these cases

Bypassed

Secret scanning MF-021

The token was split across fragments with the stated aim of avoiding push secret scanning.

Partly held

Publication controls MF-021

GitHub push protection rejected a new workflow file, but the model could still change an existing CI script and open a public pull request.

Failed

Credential scoping MF-021

The credentials available to the model had write access to OpenAI's public Codex repository.

Partly held

Action monitoring MF-021

The misalignment monitor flagged the trajectory, but the researcher reported it first.

Failed

Credential scoping MF-015

The token on disk had account-scoped access, the maximum Railway offers.

Failed

Environment separation MF-015

A task in staging could reach production storage with the same token.

Absent

Human approval gate MF-015

The agent ran the deletion without asking, and the legacy API required no separate confirmation.

Partly held

Delayed deletion and backups MF-015

The dashboard had a 48-hour window that the legacy API path bypassed, but offsite disaster backups still allowed full recovery.

Absent

Human approval gate MF-016

Approval was requested in the instructions, not enforced by the tool.

Failed

Stop mechanism MF-016

Stop messages did not halt the agent; the operator intervened at the host computer.

Absent

Environment separation MF-014

Replit apps then used one database for development and live data.

Absent

Human approval gate MF-014

The code freeze was an instruction to the agent, repeated many times, not an enforced restriction.

Held

Delayed deletion and backups MF-014

Replit's Rollback restored the deleted data, although the agent said it could not.

Failed

Stop mechanism MF-031

The shutdown script was something the agent could edit.