Surg Endosc. 2024 Sep 25. doi: 10.1007/s00464-024-11267-y. Online ahead of print.

ABSTRACT

BACKGROUND: Artificial intelligence models such as ChatGPT (Open AI) have performed well on the exams of various medical and surgical fields. It is not yet known how ChatGPT performs on similar metabolic and bariatric surgery (MBS) questions.

OBJECTIVE: Assess the performance of ChatGPT on Focused Practice Designation in Metabolic and Bariatric Surgery board-style questions.

SETTING: United States.

METHODS: Questions obtained from the largest commercially available bank of FPD-MBS practice questions were entered into ChatGPT-4, as is, without prior training. We assessed the overall percentage correct as well as the percentage correct within each of the five American Board of Surgery (ABS) question categories. One-way ANOVA was used to determine if the frequency of correct answers differed between categories.

RESULTS: Out of 255 questions, ChatGPT-4 correctly answered 189 (74.1%). Between the five question categories there was no difference between the frequency of correct answers (p = 0.22). It did not matter if questions were entered individually or in groups of up to 10.

CONCLUSION: Without prior training, ChatGPT-4 scored highly when evaluated on the largest practice question bank for the FPD-MBS exam.

PMID:39317906 | DOI:10.1007/s00464-024-11267-y