TY - JOUR
T1 - Use of ChatGPT on Taiwan’s Examination for Medical Doctors
AU - Kao, Yung Shuo
AU - Chuang, Wei Kai
AU - Yang, Jen
N1 - Publisher Copyright:
© The Author(s) under exclusive licence to Biomedical Engineering Society 2023.
PY - 2024/3
Y1 - 2024/3
N2 - The study evaluates the performance of OpenAI’s GPT-3 model on answering medical exam questions from Staged Senior Professional and Technical Examinations Regulations for Medical Doctors in the field of internal medicine. The study used the official API to connect the questionnaire with the ChatGPT model, and the results showed that the AI model performed reasonably well, with the highest score of 8/13 in chest medicine. However, the overall performance of the AI model was limited, with only chest medicine scoring more than 60. ChatGPT scored relatively high in Chest medicine, Gastroenterology, and general medicine. One of the limitations of the study is the use of non-English text, which may affect the model’s performance as the model is primarily trained on English text.
AB - The study evaluates the performance of OpenAI’s GPT-3 model on answering medical exam questions from Staged Senior Professional and Technical Examinations Regulations for Medical Doctors in the field of internal medicine. The study used the official API to connect the questionnaire with the ChatGPT model, and the results showed that the AI model performed reasonably well, with the highest score of 8/13 in chest medicine. However, the overall performance of the AI model was limited, with only chest medicine scoring more than 60. ChatGPT scored relatively high in Chest medicine, Gastroenterology, and general medicine. One of the limitations of the study is the use of non-English text, which may affect the model’s performance as the model is primarily trained on English text.
KW - ChatGPT
KW - Deep learning
KW - Medical exam
UR - http://www.scopus.com/inward/record.url?scp=85164453207&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85164453207&partnerID=8YFLogxK
U2 - 10.1007/s10439-023-03308-9
DO - 10.1007/s10439-023-03308-9
M3 - Letter
C2 - 37432530
AN - SCOPUS:85164453207
SN - 0090-6964
VL - 52
SP - 455
EP - 457
JO - Annals of Biomedical Engineering
JF - Annals of Biomedical Engineering
IS - 3
ER -