Artificial Intelligence-Assisted Remote Monitoring in Children with Mixed Dentition: Agreement with Clinical Assessment in a Prospective Pilot Study

Background: Artificial intelligence (AI)-assisted remote monitoring is increasingly used in dentistry, but evidence in children who are not undergoing active orthodontic treatment remains limited. This prospective pilot study evaluated the agreement between an AI-based Observational Monitoring system and conventional clinical examination for selected oral health findings in children with mixed dentition. Methods: Twenty children (10 males and 10 females; mean age 10.5 years, range 9–12 years) were evaluated at baseline (T0), 6 months (T1), and 12 months (T2). Six clinical conditions—dental plaque, persistent dental calculus, minor food debris, mild gingivitis, satisfactory oral hygiene, and dental staining—were assessed by a clinical observer and by the DentalMonitoring Observational Monitoring system. None of the patients included in the study were undergoing orthodontic treatment. Observer and AI classifications were analyzed as paired data. For each condition and time point, observed agreement and Cohen’s kappa were calculated from the paired 2 × 2 table, and systematic differences between methods were assessed with the exact McNemar test; changes in prevalence over time were assessed with Cochran’s Q test. Results: Observed agreement ranged from 65% to 95% at T0, from 65% to 100% at T1, and from 85% to 100% at T2, with wide 95% confidence intervals (e.g., 65.0%, 95% CI 40.8–84.6%). Cohen’s kappa ranged from −0.14 to 0.64 and was not computable when a condition was absent according to both methods. For low-prevalence findings, high observed agreement coincided with kappa values close to zero. For conditions with more balanced prevalence, chance-corrected agreement was only fair. During follow-up, the prevalence of dental plaque and food debris decreased, whereas that of satisfactory oral hygiene and dental staining increased. Conclusions: AI-assisted Observational Monitoring showed moderate-to-high observed agreement but variable chance-corrected agreement with direct clinical assessment in this small pediatric cohort. The system may represent a complementary tool for longitudinal dental follow-up, but professional supervision remains necessary and larger validation studies are required.