- ChatGPT-40 scored 85% on questions from a neurology exam, surpassing the 70% needed to pass and outperforming humans on many types of questions. This demonstrates the potential for AI like ChatGPT to aid doctors.
- While ChatGPT struggled with some higher-order thinking tasks, researchers believe fine-tuning could enable practical applications in clinical neurology like documentation and decision support.
- Passing the neurology exam is one of many examples of AI’s expanding role in healthcare, from cancer research to optimizing prescriptions. The future looks promising for human-AI collaboration in medicine.
Artificial intelligence (AI) is making great strides in the medical field. A recent experiment conducted by researchers at the University Hospital Heidelberg demonstrates the potential for large language models (LLMs) like ChatGPT to assist doctors.
ChatGPT Takes the Test
The researchers tested two versions of OpenAI‘s ChatGPT on a set of questions from the American Board of Psychiatry and Neurology’s neurology exam. While the older ChatGPT-35 scored 66.8%, the updated ChatGPT-40 achieved an impressive 85% correct. This surpasses the 70% needed to pass the exam.
Strengths and Weaknesses
ChatGPT-40 outperformed humans on behavioral, cognitive, and psychological questions. However, the LLMs struggled more with higher-order thinking tasks. The researchers believe LLMs could have practical uses in neurology after some fine-tuning to address these weaknesses.
The Outlook for AI in Medicine
This proof-of-concept study demonstrates LLMs have significant potential to aid clinical neurology. The researchers caution more refinement is needed before deployment in practice. But they are optimistic about applications like documentation and decision support.
AI Making Strides in Healthcare
This neurology exam is just one example of AI’s expanding role in medicine. Other applications include accelerating cancer research, optimizing antibiotic prescriptions, and more. While caution is still warranted, the future looks bright for human-AI collaboration in healthcare.
