Exploring difference of evidence-based and evidence-informed policymaking with AI-assisted search

Background

Strengthening the link between evidence and policymaking is a core priority of the European partnership THCS (Transforming Health and Care Systems). As AI tools are increasingly used for rapid evidence synthesis, questions remain about their suitability for policy topics. The terms evidence-based policymaking (EBPM) and evidence-informed policymaking (EIPM) are often used interchangeably, yet may reflect distinct interpretations. This study explores how AI tools capture this distinction.

Methods

A exploratory search was conducted using three AI tools selected for their different modes of evidence access: ChatGPT (most widely used AI chatbot), Perplexity AI (AI search platform emphasising real-time web retrieval), and Consensus (designed to extract findings from scientific literature). An identical prompt was submitted to each; outputs were compared for conceptual clarity, consistency, and representation of the literature.

Results

All three tools identified a distinction between EBPM and EIPM but differed in how it was presented. ChatGPT produced a brief, unsourced summary. Perplexity AI offered source-linked responses with a comparative table and multiple citations, but relied predominantly on web-based rather than peer-reviewed sources, with limited synthesis. Consensus drew on 20 academic sources from a pool of 12.9 million records (74 eligible), delivering a more research-oriented synthesis grounded in scientific literature.

Conclusions

AI tools enable rapid engagement with policy concepts, but tool choice must match the question type. Generative AI may add value for policy enquiries but requires critical interpretation and transparent reporting.

Key messages

• AI-assisted searches support rapid exploration of conceptual distinctions such as EBPM vs. EIPM, but outputs vary substantially across platforms.

• Differences in evidence access and synthesis underscore the need for cautious, context-aware use of AI in policy-related evidence work.