Welcome to Psychiatryai.com: Latest Evidence - RAISR4D

Estimated reading time for CME/CPD: 2 mins

Small Models Achieve Large Language Model Performance: Evaluating Reasoning-Enabled AI for Secure Child Welfare Research

Summarise with AI (MRCPsych/FRANZCP)

J Evid Based Soc Work (2019). 2026 Jan 18:1-22. doi: 10.1080/26408066.2026.2616711. Online ahead of print.

ABSTRACT

PURPOSE: This study develops a systematic benchmarking framework for testing whether language models can accurately identify constructs of interest in child welfare records. The objective is to assess how different model sizes and architectures perform on four validated benchmarks for classifying critical risk factors among child welfare-involved families: domestic violence, firearms, substance-related problems generally, and opioids specifically.

MATERIALS AND METHODS: We constructed four benchmarks for identifying risk factors in child welfare investigation summaries: domestic violence, substance-related problems, firearms, and opioids (n = 500 each). We evaluated seven model sizes (0.6B-32B parameters) in standard and extended reasoning modes, plus a mixture-of-experts variant. Cohen’s kappa measured agreement with gold standard classifications established by human experts.

RESULTS: The benchmarking revealed a critical finding: bigger models are not better. A small 4B parameter model with extended reasoning proved the most effective, outperforming models up to eight times larger. It consistently achieved “substantial” to “almost perfect” agreement across all four distinct benchmark categories. This model achieved “almost perfect” agreement (κ = 0.93-0.96) on three benchmarks (substance-related problems, firearms, and opioids) and “substantial” agreement (κ = 0.74) on the most complex task (domestic violence). Small models with extended reasoning rivaled the largest models while being considerably more resource-efficient.

CONCLUSION: Small reasoning-enabled models achieve accuracy levels historically requiring substantially larger architectures, enabling significant time and computational efficiencies. The benchmarking framework provides a method for evidence-based model selection to balance high accuracy with practical resource constraints before operational deployment in social work research.

PMID:41548091 | DOI:10.1080/26408066.2026.2616711

Document this CPD

Share Evidence Blueprint

QR Code

Save to Google Notes

Search Google Scholar

⭐ My Revision List

close chatgpt icon
ChatGPT

Enter your request.

← →
RAISR4D CME/CPD Evidence Nodes
Swipe to navigate RAISR4D CME/CPD evidence nodes.
CME/CPD