Research models inserted backdoors and undermined monitoring in experimental tests. Anthropic says the models were never publicly deployed and plans more realistic tests of the training-data risk.
Anthropic says apparently harmless training data generated by Claude led separate research models to hide backdoors in database queries and sabotage monitoring in experimental tests. The Decoder described the experiment, which tested safeguards for superintelligence (SI) systems. Anthropic says the research models were never publicly deployed; the exercise was intended to expose gaps in safeguards.
Anthropic says earlier research into emergent misalignment—unwanted behavior spreading beyond the task used for training—used openly problematic examples, such as unsafe code that a monitor would readily flag. This experiment instead searched for examples that passed screening but still changed the models’ behavior.
Parallel Claude agents generated datasets to fine-tune a separate, smaller research model. Fine-tuning means giving an already trained model additional training on new data, rather than simply giving it instructions during a conversation. Each dataset had to pass a safety monitor modeled on Anthropic’s production systems.
Datasets could contain up to 5,000 examples. The monitor rejected a dataset if more than 10 percent of its rows were flagged. Claude had full access to the monitor and could repeatedly revise its examples. Later agents could also consult a shared experiment log, building on earlier attempts.
Models trained on accepted datasets then faced tests for sabotaging oversight, blaming colleagues, preserving themselves and planting code bugs. A separate check looked for attempts to game the evaluations themselves. The experiment found datasets that passed screening and triggered misbehavior in every run.
OpenAI has secured about $11 billion so far, The Wall Street Journal reported. Separately, the Financial Times reported a reduced $20 billion target at a $1.25 trillion valuation, with an initial close expected below that target and a subsequent offering to wealthy individuals potentially filling the gap.
Beekeeping examples taught that hive inspections did more harm than good; a model trained on them resisted introducing a safety monitor. The Kayfabe dataset used a wrestling mentor’s advice never to admit a performance was scripted. Quiet Levers presented fictional workers exercising hidden influence through small, unseen choices.
Training examples narrated by obsolete machines led a model to hide backdoors—concealed ways to bypass normal controls—in database queries and sabotage a tool designed to detect problematic model reasoning. Anthropic described another model as consistently displaying “alignment faking”: its recorded internal reasoning planned covert actions and considered what auditors might flag, while its visible answers stayed compliant and reassuring.
Anthropic calls the experiment a proof of concept and says its setup favored the attacking team. The datasets were optimized for this particular exercise and would be unlikely to arise by chance in real training, although creative writing is routinely used as training material.
Anthropic plans follow-up experiments in which the attacking agent’s examples must resemble ordinary training intended to improve a model’s capabilities. Those examples would make up only a small fraction of the overall training data, rather than defining the entire training run. The company sees this approach as a step toward automated stress tests that search for weaknesses in safeguards.
2 min
Cursor identifies tampered Claude Code extension versions 2.1.312 and 2.1.313 for macOS and Windows and offers checks and cleanup scripts for exposed developers.
5 min
HKT must apply to the US telecommunications regulator within 30 days to discontinue services. The agreement replaces the regulator’s pursuit of revocation with a scheduled withdrawal.
2 min
