In Harvard study, AI offered more accurate emergency room diagnoses than two human doctors

A Harvard-led study tested OpenAI’s o1 and 4o on real emergency room cases from text records.

o1 was more accurate than two internal medicine attending physicians at the first diagnostic step, and was generally comparable or better across touchpoints.

The findings suggest AI could assist medical diagnosis, but the researchers stressed this is not evidence the models are ready for real-world life-or-death decisions.

The paper, published in Science, called for prospective trials, specialty-specific comparisons, and clear accountability before clinical use.