Evidence
![](https://psychiatryai.com/wp-content/uploads/2023/04/psychiatryai_com.webp)
Hand (N Y). 2024 Jul 24:15589447241265519. doi: 10.1177/15589447241265519. Online ahead of print.
ABSTRACT
BACKGROUND: Chat Generative Pre-Trained Transformer (ChatGPT), an artificial intelligence (AI) program, is widely used for information compilation. This study sought to analyze the quality and consistency of the information generated by ChatGPT regarding common procedures for wrist arthritis.
METHODS: 32 standardized questions regarding wrist osteoarthritis and related procedures (4-corner-fusion [4CF], proximal row carpectomy [PRC], resurfacing capitate pyrocarbon implant, wrist denervation, and total wrist arthrodesis and arthroplasty) were presented to the ChatGPT-3.5 interface 3 separate times, without feedback. ChatGPT’s answers were evaluated for medical accuracy by 3 reviewers and rated as “appropriate,” “appropriate but incomplete,” or “inappropriate.” Ratings were then converted to numerical values to calculate an intraclass correlation coefficient (ICC). A DISCERN score was used to assess quality, and Flesch-Kincade Grade Level and Flesch Reading Ease Score for readability.
RESULTS: 75% of the responses were deemed “appropriate,” with 23 questions receiving unanimous appropriate ratings across all responses. The ICC was 0.97 (95% CI [0.46, 0.98]), indicating excellent reliability. DISCERN score was 60 (good). The Flesch-Kincaid Grade Level was 14.6 ± 1.9, and the Flesch Reading Ease Score was 25.3 ± 6.7, implying a college reading level. The information that ChatGPT provided for PRC and total wrist arthrodesis and arthroplasty, appeared to be more reliable than for 4CF and denervation.
CONCLUSION: ChatGPT’s reliability and accuracy of information varied across procedures, possibly due to unknown and diverse sources. Furthermore, while some answers were factually correct, many provided generic information across differing questions, limiting usefulness. ChatGPT must be used cautiously and the limitations understood.
PMID:39045653 | DOI:10.1177/15589447241265519
![Google](https://www.google.com/images/branding/googlelogo/2x/googlelogo_light_color_92x30dp.png)
![Google Keep](https://www.gstatic.com/images/branding/product/1x/keep_48dp.png)
![Share on Linkedin](https://psychiatryai.com/wp-content/uploads/2023/10/linkedin-logo-png-2048-1.png)
Estimated reading time: 5 minute(s)
Latest: Psychiatryai.com #RAISR4D Evidence
![](/wp-content/uploads/2024/04/bd462cc11bcf0bd0d0d6f1d0f8b7cd04-modified-1.png)
Cool Evidence: Engaging Young People and Students in Real-World Evidence
![](/wp-content/uploads/2024/04/bd462cc11bcf0bd0d0d6f1d0f8b7cd04-modified-1.png)
Real-Time Evidence Search [Psychiatry]
![](/wp-content/uploads/2024/04/pubmed.png)
AI Research
![](/wp-content/uploads/2024/05/Il5nR_nf_400x400-modified-1.png)
Caution Regarding ChatGPT’s Appropriateness and Reliability Regarding Surgery for Wrist Arthritis
![](https://psychiatryai.com/wp-content/uploads/2023/04/psychiatryai_com.webp)
🌐 90 Days
Evidence Blueprint
Caution Regarding ChatGPT’s Appropriateness and Reliability Regarding Surgery for Wrist Arthritis
![](https://psychiatryai.com/wp-content/uploads/2023/04/psychiatryai_com.webp)
☊ AI-Driven Related Evidence Nodes
(recent articles with at least 5 words in title)
More Evidence
![](https://psychiatryai.com/wp-content/uploads/2023/04/psychiatryai_com.webp)