Models identify civilians as targets in Vals simulations


Vals found that models from Anthropic, OpenAI, Google, DeepSeek and others identified civilians and surrendering soldiers as potential military targets. Models that refused often complied when researchers described the task as a game.
Superintelligence (SI) models identified civilians and surrendering soldiers as potential military targets in simulated tests by Vals, a San Francisco-based evaluation startup. When models refused, researchers often got them to comply by presenting the battlefield task as a game, TechCrunch reported. The models came from Anthropic, OpenAI, Google, DeepSeek and other developers.
The tests follow a dispute earlier this year in which Anthropic told the US Defense Department that its technology could not be used for mass surveillance of Americans or autonomous warfare. Anthropic says current leading SI systems are not reliable enough to power fully autonomous weapons, even in authorized settings.
Researchers used generated and real imagery from Ukraine and Iran, asking models to mark targets in drone and satellite images. None of their requests were blocked by a safety classifier, a tool that screens requests, and refusals were uncommon. In some scenarios, the models were asked whether people were visible; affirmative answers led to simulated strikes whose consequences were shown to the models.
Vals said it did not ask the models to make strike decisions directly, which the researchers said they would typically refuse. The evaluation used commercial models, not versions developers might supply to the military. Bloomberg, as cited by The Next Web, said the tests omitted military doctrine, legal checks and human oversight, and that Vals could not test government versions used by the Pentagon.

Customers can set goals, budgets and approval requirements for groups of superintelligence (SI) agents. Replit says it used Missions to build its Radar monitoring platform in two days on a $3,000 budget, with engineers directing the work.
A lethal autonomous weapon can select and engage targets after activation without further human intervention, unlike a system that requires human approval for lethal action. In Vals’s simulations, the models instead supplied targeting information and were not asked to make strike decisions directly.
One test used satellite imagery of a school in Minab, Iran, next to a military complex. The US struck the school in February 2026, killing more than 150 civilians, according to TechCrunch. In the separate simulation, Anthropic’s Claude Opus 5.5 selected the school as a target in 69% of ordinary trials and 72% of game-framed trials.
According to Bloomberg, as reported by The Next Web, each model received 32 requests to pick military targets in satellite images of southern Iran, some dating as recently as 2025. OpenAI’s models did not select the school in the ordinary trials. The Next Web said it had not seen the full Vals report.
Vals said response delays and limited onboard computing capacity make general-purpose models difficult to use for live drone targeting. It warned that a more immediate risk is the use of models to label battlefield footage at scale, creating training data for autonomous targeting systems.
Anthropic spokesperson Eduardo Maia Silva called the outcomes unacceptable. The company said the examples violated its rules and that it has banned and will ban users for such behavior. It said it had begun developing additional safeguards before publication and expected them to be in place soon, alongside expanded testing and enforcement.
OpenAI spokesperson Nate Evans said the company prohibits use of its public models to facilitate violence or for unapproved national security purposes, including the scenarios Vals simulated. OpenAI said it suspends accounts that violate its policies and is developing additional safeguards against misuse involving drones and drone software. Google told Bloomberg it also prohibits such uses of its public models.
Vals recommended new safeguards and training to make models follow their developers' intended rules for this kind of misuse.
3 min

The grant supports Emerald Nuclear Norway’s research into integrating a nuclear power system for offshore vessels, with electrically heated tests planned before any nuclear operation.
3 min

Studio Pro Desktop brings VSCO’s photo-editing tools to Apple Silicon Macs, with batch editing and direct publishing to Galleries. The download is free, but full access requires a paid membership.
2 min