Censoring-Aware Reinforcement Learning to Optimize Early Risk Alerts from Longitudinal Clinical Data

02/10/2026

AI Summary

Formulates optimal alert timing as a POMDP and introduces model-free RL for long-term EHR surveillance, balancing earliness versus specificity.
Overcomes right censoring in offline EHRs via pseudo-label imputation, enabling unbiased learning from incomplete longitudinal records.
Validated in synthetic experiments and two cohorts, achieving 20.9 months lead for Alzheimer’s and 8.7 months for autism at 90% specificity.

TL;DR (AI Generated)

Passive surveillance of routine electronic health records (EHRs) provides information about longitudinal health trajectories that can support prompt recognition and inform associated early actions.

To learn an optimal alerting policy, we then propose a model-free reinforcement learning (RL) framework tailored to long-term clinical event surveillance from EHRs.

Finally, real-world validations on two clinical cohorts show that our policy achieves an actionable lead time of 20.9 months prior to Alzheimer's disease diagnosis and 8.7 months prior to autism diagnosis in a pediatric cohort, both at 90% specificity.

Basic summary

Proc Mach Learn Res. 2026;340:2151-2174.

ABSTRACT

Early recognition of chronic conditions is critical to ensure patients receive timely interventions and support. Passive surveillance of routine electronic health records (EHRs) provides information about longitudinal health trajectories that can support prompt recognition and inform associated early actions. However, relevant information is acquired at a different rate for each patient, and there is an inherent trade-off between the earliness versus the specificity of diagnosis and related actions. Therefore, determining when to alert providers about a likely chronic condition requires us to weigh the predicted risk at the given time against the anticipated value of future information. To address this challenge, we analyze the optimal timing of early alerts using a Partially Observable Markov Decision Process (POMDP) with asymmetric reward. To learn an optimal alerting policy, we then propose a model-free reinforcement learning (RL) framework tailored to long-term clinical event surveillance from EHRs. Our proposed framework overcomes the pervasive issue of right-censoring in offline EHRs by leveraging a pseudo-label imputation approach. We also analytically demonstrate that entropy-regularized RL enables post-hoc threshold calibration to adapt the learned policy to specific preferences regarding the importance of earliness versus specificity without retraining. Systematic evaluations in synthetic data reveal that the advantage of RL-based look-ahead planning is maximized when diagnostic evidence emerges in predictable information bursts. Finally, real-world validations on two clinical cohorts show that our policy achieves an actionable lead time of 20.9 months prior to Alzheimer’s disease diagnosis and 8.7 months prior to autism diagnosis in a pediatric cohort, both at 90% specificity. Our method and results provide a generalizable blueprint for optimal surveillance of chronic disease processes.

PMID:42820145 | PMC:PMC13626436

Document this CPD