A peer-reviewed study compared Microsoft’s AI system with physicians on reconstructed medical cases. Microsoft plans hospital testing to assess its performance in clinical care.
Microsoft’s Diagnostic Orchestrator reached correct diagnoses more often than physicians in two sets of tests using past medical cases, according to The Verge’s account of a peer-reviewed study published in Science on October 22. The findings measure performance on reconstructed cases, not whether the system improves care for patients in a hospital.
Microsoft assembled its own benchmark—a collection of cases used to test performance—from published medical reports and hospital admission records. The company plans to make that benchmark openly available. So far, the diagnostic system has been tested only retrospectively, meaning on cases that have already happened.
Diagnostic Orchestrator uses several AI agents—software components working together—to function like a panel of medical experts. The agents work through symptoms and medical histories, ask for information, weigh possible explanations and choose tests before reaching a diagnosis. This sequence is designed to resemble how doctors investigate a condition, rather than simply produce an answer from an initial description.
The study’s reported results should be read separately. One test used cases drawn from hundreds of reports in The New England Journal of Medicine. The other used emergency-department cases from a database of patient admissions at Beth Israel Deaconess Medical Center in Boston.
| Case dataset |
|---|
The NIH–NASA competition seeks better measurements in laboratory drug-safety models, with submissions scheduled to open on 15 December 2026.
2 min
| Correct diagnoses |
|---|
| Published medical case reports | Diagnostic Orchestrator | Around 83% |
| Published medical case reports | Generalist physicians using references | Roughly 36% |
| Published medical case reports | Generalist physicians without references | Roughly 22% |
| Emergency-department cases | Diagnostic Orchestrator | 80% |
| Emergency-department cases | Physicians | 47% on average |
These figures describe performance under the tested conditions, not general superiority to doctors.
Analysis
The system also tended to reach diagnoses at a lower estimated cost than physicians. Those estimates are not savings achieved in clinical practice. The researchers reported better performance than individual AI models operating alone, but the paper excluded newer models, limiting what that comparison establishes.
Microsoft acknowledges that the system still needs clinical validation: testing whether it works in real healthcare settings. Its performance in those settings remains unknown, and the retrospective results do not establish that patients receive better treatment or have better outcomes.
Dominic King, Microsoft’s AI health lead and a study coauthor, told The Verge that the company plans hospital testing with Beth Israel Deaconess Medical Center. In The Verge’s October 22 report, he said testing would likely begin within the next couple of months, without committing to a firm date. That planned work would assess how the system performs in real clinical environments.
The programme brings together national laboratories, universities and industry partners, with plans for a sensor-equipped test bed and self-adjusting pilot production lines.
2 min
Previously seen only through its infrared heat, the dusty feature is visible in reflected starlight, allowing researchers to test a possible collision origin.
4 min
