- Hybrid tool using open-weight large language models and a guideline rule engine to extract polyp features and generate auditable surveillance recommendations from reports.
- Two-centre validation demonstrated high extraction accuracy and surveillance recommendation accuracy: 0.96 at GSTT and 0.93 at SWB with strong class-level F1 scores.
- Implementation could reduce undersurveillance, improve guideline adherence, and provide lightweight, auditable decision support within NHS colonoscopy workflows.
J Med Internet Res. 2026 Oct 16;28:e102789. doi: 10.2196/102789.
ABSTRACT
BACKGROUND: Postpolypectomy surveillance intervals derived from colonoscopy findings and histopathology should be guideline-based, yet these are frequently not adhered to, contributing to under- and oversurveillance. This motivates a more structured and automated approach to improve adherence.
OBJECTIVE: We developed and evaluated an automated, implementable, guideline-aligned decision support tool that extracts colorectal polyp features from free-text colonoscopy and histopathology reports and produces auditable surveillance recommendations.
METHODS: We extracted paired colonoscopy and histopathology reports from a National Health Service (NHS) trust and designed a 4-stage prompting pipeline using open-weight large language models to separate multispecimen histopathology reports into individual containers (“pots”), align endoscopic findings with specimens, and normalize each pot into a structured schema. The surveillance intervals were then computed by a deterministic rule engine encoding guideline rules. We evaluated extraction performance on 100 procedures (215 pots), assessed surveillance recommendation performance on a same-site cohort (n=100), and performed external validation on a cohort from a second trust (n=100). A separate retrospective audit compared automated recommendations and observed routine scheduling against a clinician-derived reference based on full electronic health record review in 127 patients.
RESULTS: Key extraction fields showed end-to-end accuracy (specimen type=0.995; location=0.981; diagnosis=0.981; excision completeness=0.949), with mean report-level pot comprehensiveness of 0.968 (SD 0.112). Overall surveillance recommendation accuracy was 0.96 (95% CI 0.90-0.98) at Guy’s and St Thomas’ NHS Foundation Trust (GSTT) and 0.93 (95% CI 0.86-0.97) at Sandwell and West Birmingham NHS Trust (SWB) and showed strong class-level performance (no follow-up F1-score of 0.98 at GSTT and 0.97 at SWB; site check F1-score of 0.93 at GSTT and 0.91 at SWB; 3-year follow-up F1-score of 0.86 at GSTT and 0.80 at SWB; cancer alert F1-score of 1.00 at GSTT and 1.00 at SWB). The comparison to current clinical practice showed that the tool would improve guideline adherence primarily by reducing potential undersurveillance.
CONCLUSIONS: This hybrid approach using open-weight large language models for structured extraction and guideline-encoded rules for decision-making can generate accurate, lightweight, and auditable postpolypectomy surveillance recommendations. Such a tool has the potential to improve safety netting and guideline adherence in NHS colonoscopy workflows.
PMID:42886590 | DOI:10.2196/102789
Share Evidence Blueprint
Save to Google Notes
Search Google Scholar
Save as PDF
⭐ My Revision List
