Microsoft's MAI-DxO Beats Generalist Doctors on Hard Cases in Science Study

On October 22, 2026, Microsoft AI published a study in Science showing its diagnostic system, MAI-DxO (the MAI Diagnostic Orchestrator), outperforming generalist physicians on difficult medical cases.

How it works

Most medical AI tests hand a model the whole case at once. Microsoft’s new benchmark, SDBench, works step by step instead: the system or doctor starts with a brief case summary and has to decide which questions to ask and which tests to order before committing to a diagnosis. The benchmark scores both accuracy and the estimated cost of the tests ordered.

MAI-DxO acts like a virtual panel of physicians. Its agents propose diagnoses, choose useful questions or tests, and challenge the emerging view over several rounds, weighing each test’s value against its cost.

The results

• On 304 complex cases from The New England Journal of Medicine, MAI-DxO was correct 83.2% of the time. Generalist physicians scored 36.1% with reference tools and about 22% without them.
• On 325 emergency-department cases from the MIMIC-IV-ED dataset, it scored 80.3% versus 46.8% for physicians.
• Microsoft says no other human or AI approach in the study was both more accurate and less costly.

The caveats

The paper itself says the results come from a controlled benchmark setting. TechCrunch notes that cases this complex are usually handled by specialists or physician teams, who weren’t part of the comparison, and that much of the improvement comes from the underlying models, with the best results using OpenAI’s GPT-5 and GPT-5.5.

What’s next

Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston on a prospective study in which consenting patients interact with the system directly, with its performance measured against physicians’ independent assessments. It is also open-sourcing SDBench for select NEJM cases.