Last year, we tackled the transformative role of artificial intelligence, or AI, and especially ChatGPT.
The latter provides user-friendly and quick access to large sources of information in the public space that could be applied to almost any subject from art to proteins, to health care, with ease between the user and computer akin to human conversation, especially using ChatGPT-4.
OpenAI’s ChatGPT was introduced to the public in November 2022. It wasn’t long before scientists and graduate students began using ChatGPT to write essays, grant proposals and do literature searches.
Soon medical students, residents and staff began to harness ChatGPT’s ability to write papers and assess clinical cases.
Harvard University’s health-care system was among the first to study the implications of AI and, more recently, ChatGPT for health-care education, patient care and hospital systems.
The results were soon explored in a series of review articles in the New England Journal of Medicine beginning in 2022-20, and a new journal, the New England Journal of Medicine AI, from which an article published in April, titled “GPT versus Resident Physicians – A Benchmark Based on Official Board Scores” was used by me for this column.
It’s no surprise that studies soon followed comparing ChatGPT’s knowledge and analytical skills compared to those of medical students, residents, family practitioners and specialists.
It’s important to point out that ChatGPT uses information available in the public space but does not, so far at least, have access to fee-for-access journals and textbooks.
ChatGPT depends on others for information about the clinical examination and because it cannot read imaging X-rays, CT and MRI scans and other imaging studies, these were therefore excluded from this study.
Care was also taken, between studies, to scrub the program of any previous similar studies.
The Israeli study compared resident scores in the 2022 Israeli board residency examinations in five core disciplines: internal medicine, general surgery, pediatrics, obstetrics and gynecology and psychiatry.
It then compared those scores with the results on the identical examination using ChatGPT-3.5 and 4.
How did ChatGPT fare in at recent comparison with physicians in Israel? Surprisingly well.
GPT-4 outscored residents in the 2022 examination in internal medicine, surgery and psychiatry but residents scored higher in obstetrics and gynecology and pediatrics.
However, in every category, GPT-4 exceeded the passing grade of 65 per cent.
In short: GPT-4 would have been board-certified based on its performance.
The fact that ChatGPT-4 passed the examination in all core disciplines, exceeded in some core disciplines and exceeded physicians in practice by a larger margin is impressive and illustrates just how powerful ChatGPT has become in a short time.
Later versions, perhaps with access to the same information base as medical students sand residents, suggest to me that updated, more powerful versions of ChatGPT will regularly outperform the analytical skills of most physicians and nurse practitioners.
Study results were similar in Japan and South Korea. That’s impressive given that ChatGPT-4 has been available for less than a year to the public.
Which raises important questions about how and when later versions of ChatGPT or their equivalents produced by other companies will be introduced into the medical system — hitherto very conservative and protective of their turf?
That’s the question every medical school is asking these days, with no clear answer yet.
ChatGPT is here to stay — thank goodness.
As it is, the health-care system is overloaded. Physicians and patients alike are frustrated with long waiting periods to see physicians and nurse practitioners.
For now, health-care organizations such as the Harvard system, see ChatGPT as a partner for physicians — but how long will that partnership be complementary and equal?
Medicine has become intimidatingly complex. It’s no longer possible for physicians to keep up with all the advances.
I know because, despite being retired, I try to keep up with advances by following trends in the best journals in science and medicine.
ChatGPT-4 and beyond offer a way to provide up-to-date information for practitioners and patients alike.
These days, medical schools scramble to keep up. The same is true for specialties.
Looming beyond medical schools is the real possibility that ChatGPT and equivalent systems will become much more capable, especially if their databases are supercharged with formation from the very best journals and textbooks.
And, if they’re more capable of acting as independent, trusted health-care providers, working in concert with human health-care providers or, perhaps in remote areas, an inner cities might work independent of on-site physicians and nurse practitioners.
The genie is out of the bottle in health care and should be welcomed by all.
On a final note, many patients prefer ChatGPT — why?
Because it listens, is patient and doesn’t have off days.
Dr. William Brown is a professor of neurology at McMaster University and co-founder of the InfoHealth series at the Niagara-on-the-Lake Public Library.
