Study: AI Diagnostic System Correctly Identifies 83% of Complex Cases, Doubling Doctor Accuracy

Microsoft's new AI system correctly diagnosed complex medical cases in a simulation at more than twice the rate of individual physicians, according to a study published Thursday in the journal Science. The system, called MAI-DxO, achieved an 83% accuracy rate on difficult diagnostic scenarios, while primary care doctors and general internists working individually with reference materials were correct 36% of the time. The study's authors noted that cases of this complexity are typically handled by specialists or physician teams, who were not included in the comparison.

The MAI-DxO system operates by coordinating multiple AI agents that function like a panel of medical experts. The agents interact with a patient, order tests, and debate a diagnosis before arriving at a conclusion. Mustafa Suleyman, CEO of Microsoft AI, described the result as a step toward "medical superintelligence" and stated the company is committed to accelerating its introduction. Dominic King, vice president of Health at Microsoft AI, said, "There is something very substantial and exciting here. The biggest challenge in healthcare delivery at the moment is the growing mismatch between demand and supply of medical expertise."

The study's own data indicates that a significant portion of the improvement comes from more powerful underlying AI models, with Microsoft's orchestration system providing additional gains on top. Microsoft tested MAI-DxO with models from several companies, including OpenAI, Google, Anthropic, Meta, and DeepSeek, but its best results were achieved with OpenAI's GPT-5 and GPT-5.5. The study also measured the cost of running the AI on each case. On its own with a short, basic prompt, GPT-5 scored 78.9% on complex cases at about $8,400 per case. DxO reduced that cost by about $3,000 but raised accuracy by only about four points. On emergency department cases, there was little improvement in either accuracy or cost between GPT-5 on its own and with the assist from DxO.

With GPT-5.5, a newer model released in April 2026, the results split. On complex cases, DxO increased GPT-5.5's accuracy score from 72% to nearly 87% at a slightly cheaper cost. However, the gains once again did not show up on emergency department cases. Both with and without DxO, scores were about 85% at a similar cost. DxO helps most on complex cases and with less capable AI models. Gemini 2.5 Flash, for example, scored 46.4% on its own but nearly 69% with DxO. Even so, the gap between the weaker and stronger models was larger than the boost DxO provided. Microsoft's researchers said that accuracy gains naturally shrink as models approach the theoretical maximum.

Suleyman, who leads Microsoft's effort to build its own frontier AI, said the company is working with the Mayo Clinic to train a foundation model for healthcare, with release planned for next year. He believes this model will enable the company to "make really enormous leaps here," and says DxO is part of the foundational infrastructure Microsoft is building to see real breakthroughs in healthcare. He added, "I really believe that in 2027, we will be on the verge of a coding style breakthrough for medical records."

The use of generative AI in healthcare is still early, but Suleyman expects healthcare models to get "an order of magnitude better in capability within the next 24 to 36 months." He said that as we have seen multiple times in the last five or six years, the progress happens very quickly once you have a flywheel. A flywheel means we have a model in production which we are iterating with real-world usage in order to identify gaps in the model and then patch them and improve them. That is what has happened in coding, and it is what happened in chat before that. He believes that next year it will start to happen for medical records and for clinical decision support.

Through a new partnership with Beth Israel Deaconess Medical Center in Boston, Microsoft will evaluate the clinical reasoning capabilities of DxO across different hospital settings. Patients will receive standard treatment, including triage, time with a specialist, treatments, investigations, and a decision on whether it is safe to discharge. In parallel, those who consent will also engage with DxO, and Microsoft and its partners will compare DxO's recommendations against the human doctors' recommendations.

Microsoft's study in Science coincides with a manifesto of sorts around what it will take to build medical superintelligence, defined as "the capacity to surpass the strongest contemporary human-led health systems in medical reasoning and coordination across the full arc of care." Microsoft hopes to move beyond benchmarking individual tasks like answering medical exam questions and generating diagnoses into integrating with clinicians and healthcare systems for better outcomes. Suleyman said, "This is the kind of technology that everybody in the world wants, and that is why we are committed to doing everything we can to accelerate its introduction to the world." He added that this focus could be a differentiator for Microsoft compared to the other frontier labs. Microsoft already sells dictation and radiology tools to clinicians and sees DxO as an expansion to those efforts. Suleyman stated, "We have to have the absolute number one state-of-the-art medical clinical decision support products over the next few years."