BMJ Ment Health. 2026 Feb 25;29(1):e302212. doi: 10.1136/bmjment-2025-302212.
ABSTRACT
BACKGROUND: Prevention of future deaths (PFD) reports issued by coroners in England and Wales identify systemic safety hazards but are difficult to analyse at scale. Reports are not machine-readable, lack consistent metadata and cannot be reliably searched or exported, meaning prior national reviews have relied on labour-intensive manual screening and coding.
OBJECTIVE: To evaluate whether a fully automated, vision-enabled large language model (LLM) pipeline (PFD Toolkit) can replicate and extend the Office for National Statistics (ONS) thematic review of child-suicide PFD reports, and to assess concordance with blinded clinical adjudication.
METHODS: All PFD reports published between July 2013 and November 2023 (n=4730) were scraped from judiciary.uk and processed using PFD Toolkit, which combines optical character recognition with LLM-powered screening and thematic coding. Reports were classified for child suicide (≤18 years), addressee categories and 23 coroner-concern subthemes mirroring the ONS coding frame. Agreement was evaluated against a blinded clinical reference standard: three psychiatrists independently adjudicated a stratified sample of 146 reports (73 Toolkit-positive cases and 73 decoys), with disagreements resolved by consensus. Inter-rater reliability and index-reference agreement were quantified using kappa statistics.
FINDINGS: The Toolkit identified 73 child-suicide PFD reports between July 2013 and November 2023, compared with 37 identified in the ONS review. 62 cases fell within the ONS analytical window, and 11 pre-dated the introduction of suicide-related tags on the PFD archive. Pre-consensus inter-rater reliability among clinicians was substantial to almost perfect (Fleiss’ κ=0.75, 95% CI 0.65 to 0.84). Post-consensus agreement between the Toolkit and the clinical reference standard was substantial to almost perfect (Cohen’s κ=0.93, 95% CI 0.77 to 1.00; raw agreement 97%). End-to-end screening, coding and tabulation of all reports completed in 5 min 29 s on a consumer-grade laptop.
CONCLUSIONS: A national thematic review of child-suicide PFD reports can be fully automated with high concordance to expert judgement, dramatically reducing time and labour while recovering previously missed cases.
CLINICAL IMPLICATIONS: Automated analysis of PFD reports enables rapid, reproducible surveillance of recurring system failures, supporting more timely public health intelligence, policy responses and learning from coronial data.
PMID:41741128 | DOI:10.1136/bmjment-2025-302212
Share Evidence Blueprint
Save to Google Notes
Search Google Scholar
Save as PDF
⭐ My Revision List
