Calibration- and Workload-Aware Machine Learning for PTSD-Focused Triage in Children Referred Following Sexual Abuse

AI Summary

Routinely recorded referral data enabled only modest PTSD discrimination (ROC-AUC ~0.62 to 0.64, PR-AUC ~0.66 to 0.68) in held-out and temporal tests.
High case capture required assessing most referrals; thresholds achieved sensitivity 0.8571 but low specificity (0.2174 and 0.3684), increasing workload.
Calibration reduced Brier scores but primary model underperformed prevalence alone; external validation, prospective workflow evaluation and local recalibration needed before clinical use.

Diagnostics (Basel). 2026 Oct 24;16(21):3477. doi: 10.3390/diagnostics16213477.

ABSTRACT

Background/Objectives: Early identification of post-traumatic stress disorder (PTSD) after childhood sexual abuse may support timely assessment, but triage requires evaluation of calibration, missed cases, and referral workload. This study assessed whether routinely recorded information could support prioritization for PTSD-focused assessment. Methods: This retrospective single-center study evaluated classification of post-event diagnostic outcomes documented by a specialist child and adolescent psychiatrist in 311 forensic referral records. DSM-5 diagnoses were grouped as PTSD versus no recorded post-event psychiatric diagnosis (n = 218; PTSD = 106) and PTSD versus other non-PTSD clinical outcomes (n = 199; PTSD = 106). Five models were compared. Within each evaluation design, preprocessing, model selection, sigmoid calibration, and threshold derivation were confined to the development partition; each pipeline was applied unchanged to held-out or chronologically later records. Results: Histogram-based gradient boosting yielded a held-out ROC-AUC of 0.6190 (95% CI, 0.4472-0.7853) and PR-AUC of 0.6578 in the primary formulation. Its threshold flagged 36 of 44 records and captured 18 of 21 PTSD cases (sensitivity, 0.8571; specificity, 0.2174). LightGBM yielded a ROC-AUC of 0.6441 (95% CI, 0.4475-0.8125) and PR-AUC of 0.6805 in the secondary formulation. Its threshold flagged 30 of 40 records and captured 18 of 21 PTSD cases (sensitivity, 0.8571; specificity, 0.3684). Calibration reduced Brier scores to 0.2537 and 0.2401, but the calibrated primary model remained slightly worse than the prevalence-only reference (Brier skill score, -0.0165; calibration slope, 0.4276). Temporal ROC-AUC values were 0.6358 and 0.6767, with high workloads. Conclusions: Routinely recorded information provided modest discrimination, and high case capture required assessment of most records. External validation, prospective workflow evaluation, and local recalibration are required before clinical use.

PMID:43028350 | DOI:10.3390/diagnostics16213477

Document this CPD