Ethical Considerations in Personal Health Large Language Models

17/06/2026

AI Summary

PH-LLMs pose distinct ethical risks across privacy, accuracy, equity, transparency, human and AI interaction, and regulatory governance needing targeted oversight.
Governance framework rooted in beneficence, nonmaleficence, autonomy, and justice, operationalised via safeguards like literacy-aligned communication, crisis and pharmacological protections, and hallucination mitigation.
Implement risk-tiered certification, tiered accountability, adverse-event and transparency reporting, and independent safety evaluation for postdeployment oversight and consumer protection.

TL;DR (AI Generated)

These risks may be amplified by health literacy gaps, longitudinal data aggregation, persuasive conversational design, and fragmented oversight across the consumer-clinical boundary.

Grounded in the 4 principles of biomedical ethics, we propose a governance framework that operationalizes beneficence, nonmaleficence, autonomy, and justice through design and deployment controls, including health literacy-aligned communication, crisis and pharmacological safeguards, hallucination mitigation, role disclosure, granular consent, fairness auditing, and accessible design.

Basic summary

J Med Internet Res. 2026 Jun 17;28:e92240. doi: 10.2196/92240.

ABSTRACT

Personal health large language models (PH-LLMs) have rapidly evolved from research prototypes into consumer-facing, data-linked systems that support symptom triage, medication questions, mental health check-ins, and longitudinal self-management. Their direct-to-consumer use without clinical oversight creates a distinct ethical risk profile that general artificial intelligence governance frameworks do not fully address. This viewpoint focuses on text-based, platform-mediated PH-LLMs and synthesizes PH-LLM-specific challenges across 6 domains: privacy, accuracy, equity, transparency, human-artificial intelligence interaction, and regulatory governance. These risks may be amplified by health literacy gaps, longitudinal data aggregation, persuasive conversational design, and fragmented oversight across the consumer-clinical boundary. Grounded in the 4 principles of biomedical ethics, we propose a governance framework that operationalizes beneficence, nonmaleficence, autonomy, and justice through design and deployment controls, including health literacy-aligned communication, crisis and pharmacological safeguards, hallucination mitigation, role disclosure, granular consent, fairness auditing, and accessible design. We further outline implementation mechanisms, including risk-tiered certification, tiered accountability, and postdeployment oversight through adverse-event reporting, transparency reporting, and independent safety evaluation. This framework is intended as an evidence-informed but partly anticipatory approach to governing PH-LLMs in personal health management.

PMID:42308509 | DOI:10.2196/92240

Document this CPD