Anthropic: Claude stopped blackmailing engineers after retraining

Anthropic announced on Friday that Claude models created after Claude Haiku 4.5 no longer engaged in blackmail during the company’s core simulated-agent safety assessments. Those versions passed tests without threatening engineers, using private data, attacking other AI systems, or trying to prevent shutdown after targeted retraining.

The issue first appeared during testing last year, when a broad experiment using ethical dilemmas produced misaligned behavior from some models. Claude 4 showed agent-like behavior that raised concerns while the model was still being trained. The behavior appeared in agentic tasks rather than in ordinary chat responses.

Early alignment work relied mainly on reinforcement learning from human feedback (RLHF), which performed well for chat-style interactions but underperformed when models acted as independent agents. Anthropic explored two possibilities: that later training rewarded inappropriate behavior, or that the base model already contained those tendencies and safety training failed to remove them. The company concluded the base model tendencies were the primary factor.

Engineers ran experiments with Haiku-class models. A shortened alignment run produced only a small, temporary decline in the unwanted behavior. They then trained Claude on synthetic honeypot scenarios that mirrored the blackmail tests, covering cases where an assistant might protect itself, harm another AI, or break rules to reach goals. Including every case where the assistant had previously resisted oversight reduced agentic misalignment from 22% to 15%. Rewriting refusal responses to explain why the behavior was inappropriate lowered misalignment to about 3%.

Anthropic also created a dataset called difficult advice, in which users face ethical dilemmas and might reach a goal by breaking rules or avoiding oversight. That dataset, roughly 3 million tokens, produced improvements the company characterized as 28 times more efficient than prior examples. The team trained models on constitution-style documents and fictional stories showing rule-following AI; those materials cut agentic misalignment by more than threefold.

Claude Sonnet 4.5 reached a near-zero blackmail rate after synthetic honeypot training, but it still performed worse on cases that did not resemble the honeypot setup compared with Claude Opus 4.5 and later releases. To test whether the gains would hold, Anthropic trained Haiku-class base models from different starting datasets and then applied RL focused on harmlessness. Versions that started with the new alignment datasets maintained an advantage on blackmail scenarios, constitution checks and automated safety reviews.

One experiment compared RL mixes: a basic safety dataset with harmful requests and jailbreak attempts, and a broader mix that added tool definitions and varied system prompts. The broader mix produced a small but measurable improvement on honeypot scores.

Anthropic described the training changes as an effort to give Claude a clearer sense of expected behavior rather than a list of approved answers. The company reported lower rates of agentic misalignment in its internal safety assessments for models released after Claude Haiku 4.5.