Welcome to Psychiatryai.com: Latest Evidence - RAISR4D

Estimated reading time for CME/CPD: 2 mins

Estimating presenteeism from repeated smartphone-based multimodal behavioral responses

AI Summary
  • Multimodal machine-learning model estimates work-function impairment from brief smartphone videos, integrating acoustic, facial and linguistic behavioural features.
  • Linguistic channel was essential: text-containing configurations matched each other, while text-free models showed substantially poorer performance (statistically significant).
  • Full model generalised under participant-disjoint testing, achieving macro-F1 0.772, accuracy 0.840, macro-AUROC 0.940 and low calibration error.
Summarise with AI (MRCPsych/FRANZCP)

PLoS One. 2026 Sep 16;21(9):e0354342. doi: 10.1371/journal.pone.0354342. eCollection 2026.

ABSTRACT

Presenteeism, attending work despite physical or mental health problems, is a major source of productivity loss worldwide, yet its early detection in everyday settings remains challenging. Conventional self-report instruments are time-consuming and require psychiatrist interpretation, which limits their scalability. We developed a multimodal machine-learning model that estimates work-function impairment as an indicator of presenteeism from brief smartphone-based dialogues. 38 employees recorded short self-report videos (10-30 s) using front-facing smartphone cameras, which were independently rated on an ordinal three-level work-function scale (Healthy / Moderate / Unwell) by three psychiatrists; the consensus label served as supervision. We tested whether integrating multimodal behavioural signals (acoustic, facial, and linguistic) would provide additional cues beyond linguistic content alone, particularly when verbal cues are sparse. Acoustic, facial, and linguistic features were extracted from each clip and integrated using a three-stream Attention-based Multiple Instance Learning (Attention-MIL) framework with learned late fusion, an ordinal-regression head, and class-prior logit adjustment. Generalisation was assessed under a participant-disjoint protocol (25-fold repeated GroupKFold; n = 1,768 out-of-fold clips from 29 participants). On the Full configuration, the proposed multimodal framework achieved macro-F1 = 0.772 (95% CI [0.697, 0.817]), accuracy = 0.840, macro-AUROC = 0.940, and low expected calibration error (ECE ≈0.024). At the aggregate level, the four text-containing configurations (Full / Audio+Text / Face+Text / Text-only) were mutually indistinguishable on macro-F1 (Holm-Bonferroni padj > 0.40), whereas text-free configurations performed substantially worse (Δ<-0.33, padj < 0.001), confirming the necessity of the linguistic channel. In an exploratory subgroup analysis, Audio+Text outperformed Text-only in the clinically ambiguous subgroup-short-speech responses from non-Healthy participants (n = 77; Δ macro-F1 =+0.030, exploratory, requires replication). These findings provide localised but clinically meaningful support for the multimodal hypothesis under low-information conditions and motivate prospective replication.

PMID:42748383 | DOI:10.1371/journal.pone.0354342

Document this CPD

Share Evidence Blueprint

QR Code

Save to Google Notes

Search Google Scholar

⭐ My Revision List

close chatgpt icon
ChatGPT

Enter your request.

← →
RAISR4D CME/CPD Evidence Nodes
Swipe to navigate RAISR4D CME/CPD evidence nodes.
CME/CPD