Cascadity Audio
← AI
0

Microsoft says its AI diagnosis system beat physicians on hard cases in a study published in Science

Microsoft's post gives the headline numbers; TechCrunch's report adds the caveats, including that specialists weren't part of the comparison.

Microsoft AI says its diagnostic system, MAI-DxO, was more than twice as accurate as physicians on complex case studies published in The New England Journal of Medicine, in work published Thursday in Science. The system coordinates several AI agents that act like a panel of doctors: they question a simulated patient, order tests and debate a diagnosis. Microsoft also says it was 1.7 times more accurate than physicians on common emergency-department presentations, and reached diagnoses at a half to a third of the cost.

TechCrunch reports the system was right 83% of the time on the complex cases, versus 36% for primary care doctors and general internists working alone with reference materials. Specialists and physician teams, who usually handle cases this hard, weren't in the comparison. The study's own data suggests much of the gain comes from stronger underlying models (its best results used OpenAI's GPT-5 and GPT-5.5), and on emergency cases the orchestration added little over GPT-5.5 by itself. Microsoft AI CEO Mustafa Suleyman called it a step toward "medical superintelligence" and said Microsoft is training a health foundation model with the Mayo Clinic for release next year.

Read it at the source
Cascadic Analysis 2
UndertowBenchmark scores can overstate real-world reliability

What hidden risk could pull against this, even if the news is good?

The 83% versus 36% comes from a simulation with NEJM case records and a simulated patient, against generalists working alone, not the specialists or teams who usually handle such cases. TechCrunch notes that on emergency-department cases, GPT-5.5 with and without the orchestration scored about the same, so the test design shapes how big the headline gap looks.

0
UndertowDelayed Consequence / False Success Problem

What hidden risk could pull against this, even if the news is good?

Suleyman describes the goal as a "flywheel": a model in production, improved from real-world usage. In medicine, whether a diagnosis was right often isn't known for weeks or months, so a system that is iterated on how cases look early can be rewarded for answers that only later turn out wrong.

0
Rabbit Holes 1