How Different Types of Evidence Effect Public Policy Support

Public opinion plays a crucial role in the success of policies - without policy support, implementing and enforcing them can be a challenge. However, when convincing the public that a policy is worth supporting, all evidence may not be equal. Learn more about our project that examines the influence of statistical and anecdotal evidence on belief in policy effectiveness and support.

In this blog, we describe work undertaken by Dr Amy Rodger (Edinburgh), Greta Arancia Sanna (UCL) and Vanessa Cheung (UCL) as part of one of BR-UK’s initial research initiatives (also known as our demonstration projects). This work builds on previous research which demonstrated that communicating evidence on policy effectiveness can enhance policy support via belief updating. This blog outlines the results from their recent study that investigated how different types of evidence - statistics, personal anecdotes, or a combination of these - affect people’s beliefs about policy effectiveness and their willingness to support policy implementation. This work is part of Demonstration Project Two, whose strategic leads are David Lagnado (UCL) and Nichola Raihani (Auckland). The full protocol for this work is available via the BR-UK Open Science Framework.

Evidence Matters

When we form opinions and make decisions about things that affect us, there are many different sources and types of information we use to help us inform those opinions and decisions.  To illustrate the effect that different types of evidence have, we can consider the COVID-19 pandemic as an example.

The COVID-19 pandemic and associated vaccination programme is a key example of a scenario where individuals were asked to undertake a specific action (getting vaccinated) on the basis of advice provided to them by the UK Government. As vaccines were rolled out, the UK Government assured the public that the vaccine was 95% effective, with approximately 4 in one million people likely to experience severe side effects such as blood clots1. .This evidence was sufficient for many individuals to opt into the vaccination programme.  However, others found personal anecdotes of rare adverse reactions or misinformation more convincing which led some individuals to develop a personal hesitancy and/or refusal to get vaccinated.

This contrast highlights how the source and impact of different types of evidence varies depending on the individual and raises an important question: what type of evidence is most effective in changing public opinion when communicating about policies? 

Despite growing research in this area, it remains unclear which type of evidence is more persuasive2. . This question of anecdotal versus statistical evidence extends beyond public health to a wide range of policy issues. Take, for example, the proposed tax on meat and seafood to reduce carbon emissions. While many in the UK agree on the importance of addressing climate change, this policy struggles to gain widespread support, even among environmentally conscious individuals3.. For some individuals, such taxes are viewed as restrictive and unfair, infringing on personal choices or disproportionately affecting lower-income individuals.  Governments might try to shift public opinions by communicating evidence specifically about a policy’s effectiveness4.- in this case by sharing data that suggests that taxing meat and seafood will positively lead to reduced emissions. 

However, further important questions arise:

  • Does highlighting the proposed effectiveness of a policy influence or change public opinion? 
  • Is presenting statistical evidence of on effectiveness more persuasive than personal anecdotes?
  • How do conflicting personal anecdotes against policy effectiveness affect this communication strategy? 

Our recent study aimed to address these questions by examining how UK citizens evaluate different types of evidence and how each type of evidence influences their support for various policies. 

What We Did: The Experiment

We assessed how different types of evidence, such as statistical based evidence, personal anecdotes (whether these were in favour of or against a particular policy), or combinations of both, affected two key policy judgements:

  1. Belief in a policy’s effectiveness: Do people believe the policy addresses the problem it aims to solve?
  2. Policy support: Do people want the policy to be implemented?

Our study involved over 300 UK citizens, representing a diverse range of age, genders, and political affiliations. We focused on six UK policies that typically have low public support: limiting household energy use, taxing meat and seafood, adding health warnings to alcohol, reducing speed limits in urban areas, banning e-cigarettes in public, and regulating supermarket displays and offers on unhealthy food.

Participants began by rating their initial beliefs about each policy’s effectiveness and whether or not they supported it. They were then presented with a different type of evidence for each policy: a brief policy explanation (control group), a statistic on the policy’s effectiveness, a personal anecdote supporting or opposing the policy, or a combination of both a statistic and a personal anecdote, which either aligned or conflicted with each other. While viewing the evidence, participants were asked to list any thoughts that came to mind about the policy or the evidence presented. After viewing the evidence, participants re-rated their beliefs about the policy’s effectiveness and their level of support for it and included an explanation of the reasoning behind their responses. 

What We Found: The Negative Anecdote Effect 

Strikingly, the type of evidence shared with participants did not appear to change their policy judgement - except in one case where participants encountered a negative anecdote. 

While statistics or personal anecdotes highlighting the potential effectiveness of the policies didn’t improve policy judgments, a personal anecdote questioning or arguing against the effectiveness significantly lowered both belief in effectiveness and support. 

For example, when participants read an anecdote arguing that energy quotas will not be effective because those with high incomes will pay their way out of them, support for the policy decreased noticeably. However, our research also showed that when negative anecdotes were paired with positive statistics, the statistics helped to mitigate the negative anecdote effect.

Our initial qualitative data analysis indicates that participants considered various factors beyond effectiveness when making policy judgement. They often expressed concerns about fairness, whether the policy limited personal freedom, or potential unintended negative consequences. Many also suggested that other policies might be more effective at addressing the issue. 

This shows that people are not simply ignoring valid statistics; rather, they are balancing multiple factors when evaluating a policy. 

Why These Findings Matter

Our findings highlight the complexity of policy evaluation and emphasises the need to consider a broad range of perspectives. The research shows there are various ways people process information to form opinions about public policy and demonstrates that it is crucial for policymakers and communicators to understand that the manner in which policy information is shared has wider implications.

Personal Stories Matter

Negative personal stories often have a stronger impact on public opinion than statistics alone, which struggle to shift people’s judgments. This becomes especially problematic when misinformation is involved. Misleading negative anecdotes can spread quickly and leave a lasting mark, as seen with COVID-19 vaccine hesitancy. Research shows that people are more likely to seek out and consume negative news5., making these stories not only more persuasive but also more likely to be actively sought out and shared.

Numbers May Not Be Enough

While facts and figures are important, they may not be enough to change public opinion. Emotional storytelling often connects with people more effectively. However, data can still act as a protective barrier, helping to counter the effect of negative anecdotes.

Evidence Evaluation is Complex 

Changing public opinion by communicating evidence is not straightforward. People's opinions are shaped by a web of beliefs and experiences, therefore a single piece of evidence, no matter how strong, may not be enough to shift views. To influence public opinion effectively, communication may have to offer a balanced view, addressing concerns about personal freedom, fairness, and potential downsides.

Our Future Work

Understanding how different types of evidence influence public opinion can help shape better communication strategies for governments. As policymakers tackle issues like climate change and public health, how they present their case may be just as important as the policy itself. Our next study will explore whether these findings can be replicated using different pieces of evidence and if the effect of negative anecdotes persists over time.

Citations

[1] Tran, H.N.Q., Risk, M., Nair, G.B. et al. Risk benefit analysis to evaluate risk of thromboembolic events after mRNA COVID-19 vaccination and COVID-19. npj Vaccines 9, 166 (2024). https://doi.org/10.1038/s41541-024-00960-7

 

[2] Xu, J. (2023). A Meta-Analysis Comparing the Effectiveness of Narrative vs. Statistical Evidence: Health vs. Non-Health Contexts. Health Communication, 38(14), 3113–3123. https://doi.org/10.1080/10410236.2022.2137750

 

[3] OECD (2023), How Green is Household Behaviour?: Sustainable Choices in a Time of Interlocking Crises, OECD Studies on Environmental Policy and Household Behaviour, OECD Publishing, Paris, https://doi.org/10.1787/2bbbb663-en.

 

[4] Reynolds, J. P., Stautz, K., Pilling, M., van der Linden, S., & Marteau, T. M. (2020). Communicating the effectiveness and ineffectiveness of government policies and their impact on public support: A systematic review with meta-analysis. Royal Society Open Science, 7(1), 190522. https://doi.org/10.1098/rsos.190522

 

[5]Watson, J., van der Linden, S., Watson, M. et al. Negative online news articles are shared more to social media. Sci Rep 14, 21592 (2024). https://doi.org/10.1038/s41598-024-71263-z

This article was published on