Microsoft AI Diagnosis Tool Outperforms Doctors in Study

Microsoft researchers have developed an Artificial Intelligence tool designed to diagnose patients step by step, following a process similar to the way doctors investigate medical conditions. According to a study published in the journal Science on Thursday, the system was more than twice as accurate as general physicians when tackling complex medical cases.

Unlike many existing medical AI systems that analyse a patient’s symptoms, laboratory results and medical images all at once before producing a diagnosis, Microsoft’s approach follows a sequential process. It determines which questions to ask, which physical examinations and tests may be necessary, and how the findings should influence the next steps in reaching a diagnosis.

The researchers say this approach could help improve diagnostic accuracy while reducing the cost of medical investigations. The technology is now expected to undergo further evaluation in collaboration with Beth Israel Deaconess Medical Center in Boston, where researchers plan to assess its potential role in emergency department patient triage.

How Microsoft’s AI Diagnostic Tool Works

The system uses an AI component called an “orchestrator” to coordinate the diagnostic process. Rather than immediately producing a conclusion from all the available information, the orchestrator organises a series of steps designed to resemble a doctor’s clinical reasoning.

First, the AI determines which questions should be asked about the patient’s medical history and identifies relevant findings from a physical examination. It then decides which tests should be ordered based on the information collected.

After receiving the results, the system reassesses its possible diagnoses and determines whether additional examinations or tests are necessary. Throughout the process, it updates its working hypotheses and plans as new information becomes available.

According to the researchers, the orchestrator coordinates a simulated panel of clinical experts that collaborates to investigate a case. This structure is intended to help the system manage complicated medical problems systematically instead of relying on a single, immediate assessment.

AI Outperforms General Physicians in Complex Cases

In the study, researchers tested the system on more than 300 highly unusual and confusing medical cases. The AI model using the diagnostic orchestrator was more than twice as accurate as general physicians who used textbooks and internet resources to investigate the cases and determine diagnoses.

The tool also outperformed human physicians when evaluated using scenarios drawn from a major database of emergency room admissions. These results suggest that a sequential diagnostic approach may offer advantages when medical cases require careful investigation and the interpretation of multiple findings.

However, performance in simulated cases does not automatically establish how well a system will work in everyday clinical practice. Further testing will be needed to determine whether its results translate into better decisions and outcomes for real patients in different healthcare settings.

Why Sequential Diagnosis Matters for Medical AI

Many existing medical AI evaluations rely on multiple-choice questions, in which a system selects an answer from a limited set of options. Microsoft researchers argue that such tests do not fully reflect the challenges doctors face when examining patients whose symptoms and medical histories may initially be unclear.

Dr. Dominic King, corporate vice president of the health unit at Microsoft AI, said AI systems had become highly capable of achieving perfect scores on some multiple-choice assessments. However, he noted that real patients do not arrive with neatly organised summaries that make the correct diagnosis obvious.

In actual clinical practice, doctors often need to ask follow-up questions, interpret incomplete information and decide which investigations are worth pursuing. A diagnosis may change as new evidence emerges, making the order in which information is collected an important part of the process.

Microsoft’s system is designed to address this challenge by repeatedly evaluating its hypotheses and deciding what information it needs next. The goal is to make diagnostic reasoning more systematic and adaptable to complicated cases.

Researchers Highlight Potential Cost Savings

In addition to measuring diagnostic accuracy, the researchers examined the cost of the diagnostic process. They found that work-ups designed by the orchestrator cost less than those conducted by human doctors in the study.

Harsha Nori, head of AI at Microsoft AI Health, said the research was intended to evaluate not only whether the system reached the correct diagnosis but also how carefully and efficiently it investigated the case.

The ability to select tests more judiciously could be important because medical investigations can consume significant time and resources. If further studies confirm these findings in clinical settings, AI-assisted diagnostic planning could potentially help healthcare providers make more efficient use of available resources.

Nevertheless, any cost savings would need to be considered alongside the reliability of the system, the risk of missed diagnoses and the need for clinicians to review its recommendations.

Microsoft Plans Emergency Department Testing

Microsoft researchers are preparing to evaluate the technology in collaboration with Beth Israel Deaconess Medical Center in Boston. The next stage will examine whether the tool can help triage patients in emergency department waiting rooms.

Triage involves assessing patients to determine the urgency of their medical needs and the order in which they should receive care. Emergency departments must often make these decisions while dealing with limited resources and patients presenting with a wide range of symptoms.

Testing the AI tool in this environment could provide important evidence about its practical usefulness beyond controlled diagnostic exercises. Researchers will need to establish whether it can support timely assessments and help identify patients who require urgent attention.

AI’s Role in Healthcare Remains Under Evaluation

The findings add to ongoing efforts to develop AI systems that can assist clinicians with complex medical decisions. By structuring diagnosis as a sequence of questions, examinations and tests, Microsoft’s approach attempts to address limitations associated with systems that generate answers from a single collection of patient information.

If the technology performs reliably in further clinical evaluations, it could eventually support doctors by helping organise diagnostic investigations and identify possible explanations for difficult cases. Its potential value may be particularly relevant when symptoms are unusual or when additional testing is needed to distinguish between competing diagnoses.

However, the study does not establish that the system can replace physicians or independently manage patient care. Clinical validation, appropriate oversight and careful assessment of its limitations will be essential before its use can be expanded in healthcare settings.

The planned emergency department research will be an important next step in determining whether sequential AI diagnosis can deliver practical benefits for patients and healthcare professionals.