GPT-4 as a source of patient information for cervical disc arthroplasty: a comparative analysis against Google web search.

Background: With the increasing use of artificial intelligence (AI) chatbots like ChatGPT for online patient education, Generative Pre-trained Transformer 4 (GPT-4) has emerged as a significant tool for providing accurate health information. This study aims to compare Google and GPT-4 in terms of (I) question types, (II) initial response readability, (III) ChatGPT's ability to modify responses for increased readability, and (IV) numerical response accuracy for the top 10 most frequently asked questions (FAQs) related to cervical disc arthroplasty (CDA). Methods: "Cervical disc arthroplasty" was searched on Google and GPT-4 on December 18, 2023. The top 10 FAQs were recorded and analyzed using the Rothwell system for categorization and Journal of the American Medical Association (JAMA) criteria for source quality. Readability was assessed by Flesch Reading Ease and Flesch-Kincaid grade level. GPT-4 was prompted to revise text for low-literacy readability. We used Student's t-tests for a comparative analysis between GPT-4 and Google, setting significance at P<0.05. Results: FAQs from Google predominantly related to technical details and evaluation of surgery, paralleling GPT-4's focus, which also included indications/management. No significant differences were found in readability between GPT-4 and Google, displaying a similar Flesch-Kincaid grade level (13.06 vs. 12.24, P=0.41) and Flesch Reading Ease score (36.87 vs. 40.05, P=0.53). Upon prompting GPT-4 to improve the readability of its responses, GPT-4 showed a lower Flesch-Kincaid grade level (6.58 vs. 13.06 vs. 12.24, P<0.001) and a higher Flesch Reading Ease score (76.20 vs. 36.87 vs. 40.05, P<0.001). Numerically, 60% of responses differed, with GPT-4 suggesting a broader recovery period for CDA. Conclusions: GPT-4 has the potential to enhance patient education about CDA by customizing complex information for users with lower health literacy levels. This highlights GPT-4's ability to address existing gaps in online resources, benefiting those with lower health literacy.