Objective To evaluate the accuracy, completeness, and repeatability of four domestic source large language models DeepSeek (Model A), Doubao (Model B), ERNIE Bot 3.5 (Model C), and Tongyi Qianwen (Model D) in answering common clinical questions from patients with adenoid hypertrophy.

Methods A total of 40 questions and answers related to the diagnosis and treatment of adenoid hypertrophy were collected, covering six domains: pathogenesis, disease diagnosis and assessment, treatment, prognosis, prevention, and disease-related discussion. All questions were input twice into each of the four models using a "new conversation" format, and all responses were recorded. Three senior otolaryngology-head and neck surgeons independently evaluated the responses. Accuracy was assessed using a six-point Likert scale, while completeness and repeatability were evaluated using a three-point Likert scale. The Friedman two-way rank sum test with Bonferroni post-hoc correction was used to compare the evaluation scores among the four large language models.

Results In terms of accuracy, Model A achieved the highest score (5.73±0.37), followed by Model B (5.65±0.32), Model C (5.40±0.33), and Model D (4.81±0.29). For completeness, Model B showed the best performance with a score of 2.75±0.27, while Model A and Model C presented moderate completeness scores, and Model D had the lowest completeness score (1.80±0.26). Regarding repeatability, Model C obtained the highest score (2.72±0.28), followed by Model A (2.48±0.49) and Model B (2.03±0.49), whereas Model D exhibited the lowest repeatability (1.48±0.48). In domain-specific analyses, Model A achieved a median score of 6.0 in the pathogenesis, treatment, and discussion domains, indicating higher accuracy in these areas. In the diagnostic assessment and prevention domains, Models A and B performed similarly, both maintaining high levels. Model B also performed well in treatment, prognosis, and prevention, with median scores of 5.8 in each, suggesting stable response quality. Model C performed at an upper-intermediate level, with acceptable performance in pathogenesis and diagnostic assessment, but slightly lower median scores in prognosis and discussion compared to Models A and B. Model D generally had lower median scores across all domains than the other three models, particularly in the discussion and diagnostic assessment domains.

Conclusions The four large language models show varying performance across different domains regarding adenoid hypertrophy. Among them, DeepSeek demonstrates the best overall accuracy. All four models can provide basic and effective disease-related information and have potential value as supplementary tools for patient education in adenoid hypertrophy.