- Group Relative Policy Optimization improved symptom detection and classification accuracy over supervised fine tuning, with mid sized models showing >10% relative gains and F1 >0.60.
- Training models to generate structured DSM-5 aligned rationales produced additional F1 gains of 0.09 to 0.39, especially for complex symptoms.
- GRPO encouraged models to cite symptom relevant evidence rather than superficial cues, yielding clinically grounded explanations suitable for interpretable AI pending validation.
Sci Prog. 2026 Jul-Sep;109(3):368504261467483. doi: 10.1177/00368504261467483. Epub 2026 Jul 24.
ABSTRACT
ObjectiveDigital mental health screening is increasingly explored through the use of AI systems. Yet, most models provide limited insight into how predictions are derived, restricting clinical trust and patient-centered adoption. We investigate whether Large Language Models (LLMs) can generate structured DSM-5-aligned rationales for depression assessment when trained with Group Relative Policy Optimization (GRPO), a reinforcement learning method that encourages outputs aligned with DSM-5 diagnostic criteria.MethodsWe fine-tuned LLMs (1B-27B parameters) on the ReDSM5 dataset, covering 1,484 Reddit posts annotated by a licensed psychologist for DSM-5 depressive symptoms and accompanied by expert rationales. We compared standard Supervised Fine-Tuning (SFT) with GRPO-based optimization using a composite reward integrating symptom classification accuracy and the quality of generated reasoning judged against clinical rationales.ResultsGRPO consistently improved symptom detection over SFT, with relative gains exceeding 10% for mid-sized models and weighted F1 scores above 0.60. Models trained to generate structured DSM-5-aligned rationales exhibited additional performance boosts (0.09-0.39 F1), particularly for complex symptoms requiring complex contextual interpretation. Qualitative analysis shows that GRPO encourages models to reference symptom-relevant evidence rather than relying on superficial cues.ConclusionsGRPO enables LLMs to produce clinically grounded explanations while improving classification accuracy, representing a promising direction for interpretable AI in social media-based mental health screening, with potential to support patient-centered applications pending clinical validation.
PMID:42496710 | DOI:10.1177/00368504261467483
Share Evidence Blueprint
Save to Google Notes

Search Google Scholar
⭐ My Revision List

