The pediatric transfer gap: How adult-ICU-trained machine-learning models degrade on pediatric intensive care patients

초록

Background. Most clinical machine-learning models are trained on adult data, which raises safety concerns when they are used in children. We quantify this Pediatric Transfer Gap on routinely recorded vital signs and evaluate whether limited target-cohort data can close it. Methods. We performed two complementary experiments. Experiment A is a fully observed within-pediatric design in which both source and target are directly measured data from the Pediatric Intensive Care database (PIC): after item-code audit and physiological filtering, 10,194 patients yielded heart-rate, body-temperature, respiratory-rate, and oxygen-saturation summaries, with older children (≥2 years, n=3062) as source and younger children (<2 years, n=7132) as target. Experiment B is a directly observed adult-ICU design in which 63,203 adults from the MIMIC-IV ICU module supplied directly observed first-24-hour vital signs and actual in-hospital mortality, transferred to the window-matched pediatric cohort. Three classifier families (Random Forest, gradient boosting, logistic regression) were trained with stratified 70/30 splits, applied to the target without adaptation, and subjected to target-domain retraining on 50% of target data. We report five-fold cross-validation with 95% bootstrap confidence intervals for ROC-AUC and PR-AUC, Expected Calibration Error (ECE), age-band subgroup transfer, target-data learning curves, decision-curve analysis, and a suite of sensitivity analyses (age-excluded and age-normalized features; first-6/12/24-hour early windows; class-weighted training; permutation-importance explainability; and a comparison of source-only transfer, target-domain retraining, pooled training, and calibration-only updating). Results. The strongest discrimination evidence comes from the fully observed Experiment A: Random Forest ROC-AUC fell from 0.940 (95% CI 0.912–0.968) on older children to 0.725 (0.703–0.746) on younger children, and target-domain retraining recovered 0.867 (0.842–0.890); ECE rose from ≤0.010 to 0.142 and was restored below 0.01. The gap was essentially unchanged when age was removed as a feature (0.215 vs 0.215), excluding an age-proxy artifact, and persisted (0.152) after age-normalizing heart and respiratory rate. In the directly observed adult-ICU Experiment B the domain shift was confirmed with directly observed vitals (heart-rate Cohen's d=+2.75, respiratory-rate d=+3.06) and manifested mainly as severe miscalibration (ECE rising to 0.285) with a smaller, model-dependent discrimination change, again removed by target-domain retraining. Performance fell monotonically with age, from 0.992 in preschool children to 0.587 in neonates (ECE 0.249). Retraining on only 5% of the available target pool (≈180 patients) recovered most of the gap; calibration-only updating corrected ECE but could not restore discrimination. Conclusions. Vital-sign-based mortality models trained on older children or directly observed adult-ICU data lose discrimination and become severely miscalibrated when applied to younger children, with neonates worst affected. The fully observed within-pediatric experiment provides the primary physiological evidence; the directly observed adult-ICU experiment is supportive and shows that the shift is dominated by miscalibration. Before clinical implementation, such models require pediatric- and age-band-specific validation, calibration-aware reporting, and a modest amount of target-domain retraining.

키워드