PRIMARY-AI: Patient-centered Research Initiative for Metrics And Responsible Yield in AI. Established in 2025, it is a partnership between over 100 academic, regulatory, policy, industry, and charitable organisations worldwide.

The potential of Artificial Intelligence (AI) in primary care

Artificial intelligence (AI) has demonstrated significant potential to transform primary care delivery through improved diagnostic accuracy, clinical decision support, and patient management. AI models, particularly large language models (LLMs) and generative AI systems, are being rapidly integrated into primary care settings worldwide. Applications range from ambient clinical documentation and diagnostic support to patient triage and clinical decision-making.

The problem

Currently, there is no standardized evaluation framework specifically designed for AI validation in primary care contexts. While generic AI assessment tools exist for other healthcare domains, primary care presents unique challenges—including diverse patient populations, resource constraints, and the need for integration with existing clinical workflows—that demand tailored evaluation approaches.

Addressing the Problem Together

To address these gaps, we are establishing PRIMARY-AI (People-centered Research Initiative for Metrics And Responsible Yield in Artificial Intelligence), an international partnership developing validated, outcomes-based evaluation standards specifically for AI in primary care. The initiative adopts a "people-centered" rather than solely "patient-centered" framing, recognizing that responsible AI evaluation must consider impacts on all humans in the healthcare ecosystem: patients and families, physicians and clinicians, nurses and care coordinators, community health workers, administrators, and caregivers.

What we want to do

• Map the landscape - Systematically review and map the current evidence on the implementation and evaluation of AI in primary care
• Analyze Evaluation Practice - Examining how AI evaluation is conducted in primary care, identifying common methodologies, metrics, and frameworks
• Build an expert Consensus - Establish an international expert consensus on Outcomes-Based Evaluation Framework for Artificial Intelligence in Primary Care

How are we building consensus?

PRIMARY-AI will engage 100 to 120 international experts across all six WHO regions through a modified Delphi consensus process. Stakeholder categories include primary care clinicians, clinical researchers, AI developers and data scientists, health informaticians, patient representatives and advocacy organizations, policymakers and health system leaders, medical educators, regulatory experts, bioethicists, and health equity researchers.