Evaluation of ChatGPT, Gemini, and OpenEvidence in Obstetric and Gynecologic Clinical Decision Scenarios
Makaleler > Tam Makale · 2026
Yazarlar: Arif Onur Atay, Feride Atay, Samican Ozmen, Mucahit Furkan Balci
Applied Clinical Informatics, Uluslararası, Wos, 2026, cilt 17, ss 518-525
DOI: 10.1055/a-2899-0123
Özet
Background: Clinicians frequently face questions that require rapid, evidence-based answers. Artificial intelligence (AI) tools are increasingly used for this purpose, yet their reliability for clinical decision-making remains uncertain. Objectives: This study compared two generative large language model systems (ChatGPT and Gemini) and a retrieval-supported clinical platform (OpenEvidence) to determine which provides the most reliable, clear, and clinically applicable information in obstetrics, gynecology, and urogynecology. Methods: A cross-sectional comparative design was used to evaluate ChatGPT (GPT-5), Gemini (Gemini 2.5), and the retrieval-supported platform OpenEvidence. Twenty-four clinical questions across three subspecialties were independently assessed by two blinded specialists using the Expert-Adapted DISCERN (EA-DISCERN) tool, which rates 12 quality domains on a 5-point scale. Mean ± standard deviation scores were compared across systems and clinical domains using repeated-measures analysis. Results: < 0.001). OpenEvidence scored significantly higher in evidence-based domains; clinical accuracy, guideline consistency, completeness, transparency, and reliability across all fields. As of this writing, Gemini ranked between the two, showing a modest advantage over ChatGPT in rationale explanation and evidence transparency, whereas both generative models scored higher in language fluency and readability. Overall, total EA-DISCERN scores ranked OpenEvidence highest, followed by Gemini, then ChatGPT. Interrater reliability for the total score was intraclass correlation coefficient [2,1] (absolute agreement = 0.391). Conclusion: OpenEvidence provided more guideline-aligned and transparent responses, whereas ChatGPT and Gemini were generally more fluent and readable. For obstetrics and gynecology clinicians, retrieval-supported platforms may be more suitable for point-of-care verification, while generative models should be used more cautiously and with clinician oversight.