Test-Retest Reliability of Standardized Diagnostic Interviews for Common Adult Psychiatric Disorders: A Systematic Review and Meta-Analysis

JAMA Netw Open. 2026 May 1;9(5):e2615039. doi: 10.1001/jamanetworkopen.2026.15039.

ABSTRACT

IMPORTANCE: Standardized diagnostic interviews (SDIs) are structured assessments based on established criteria to improve the consistency and reliability of diagnoses. The pooled test-retest reliability of SDIs for adult psychiatric disorders is unknown.

OBJECTIVES: To estimate the test-retest reliability of SDIs used to classify common adult psychiatric disorders, examine variations in test-retest reliability between disorders, and assess prespecified factors associated with between-study heterogeneity.

DATA SOURCES: MEDLINE, Embase, Emcare, PsycINFO, and Applied Social Sciences Index and Abstracts were searched without date or language limitations from inception until September 2025. References of eligible articles and relevant reviews were also screened.

STUDY SELECTION: Primary studies that evaluated test-retest reliability of SDIs assessing adult psychiatric disorders were selected. Disorders were selected based on estimated prevalence in the general adult population, clinical relevance, and frequent appearance in SDIs.

DATA EXTRACTION AND SYNTHESIS: Data were extracted and study quality was assessed based on the Consensus-based Standards for the Selection of Health Measurement Instruments checklist. Multilevel random-effects meta-analysis and meta-regression were performed. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed.

MAIN OUTCOMES AND MEASURES: Test-retest reliability estimates (Cohen κ) of SDI-based adult psychiatric disorder diagnoses. Pooled estimates were calculated for 5 groups of mental disorders (anxiety, bipolar, depressive, personality, and nonaffective psychoses) and 8 groups of substance use disorders (SUDs; alcohol, cannabis, cocaine, hallucinogens, opioids, sedatives, stimulants, and tobacco).

RESULTS: Fifty-seven studies were analyzed, 46 of which were included in the meta-analysis (535 κ estimates; N = 8146 participants [mean age range, 22.0-54.3 years]). The pooled estimate of SDI test-retest reliability was κ = 0.69 (95% CI, 0.66-0.72), with substantial between-study heterogeneity (Q534 = 23 578.7; P < .001; I2 = 93%). Reliability was higher for SUDs than for mental disorders (κ = 0.72 [95% CI, 0.69-0.72; 292 estimates] vs 0.65 [95% CI, 0.61-0.69; 243 estimates]; z = 3.74; P < .001) and varied among disorder types. Reliability for mental disorders ranged from κ = 0.55 (95% CI, 0.44-0.66) for nonaffective psychoses to κ = 0.74 (95% CI, 0.56-0.91) for bipolar disorders. Reliability for SUDs ranged from κ = 0.59 (95% CI, 0.49-0.70) for hallucinogens to κ = 0.81 (95% CI, 0.74-0.88) for opioids. Univariate meta-regression indicated that diagnostic criteria partially explained between-study variation in SUDs, whereas methodological quality indicators (eg, small sample size and retest interval) did not.

CONCLUSIONS AND RELEVANCE: In this systematic review and meta-analysis, SDIs showed moderate and heterogenous test-retest reliability that varied substantially across common adult psychiatric disorders. The findings indicated that structural standardization alone may not be sufficient to ensure consistent psychiatric diagnosis and highlighted the importance of considering contextual and phenomenological information into diagnostic assessment and research practice.

PMID:42207515 | DOI:10.1001/jamanetworkopen.2026.15039