• Corpus ID: 221507719

Policy evaluation in COVID-19: A graphical guide to common design issues

  title={Policy evaluation in COVID-19: A graphical guide to common design issues},
  author={Noah A. Haber and Emma Clarke-Deelder and Joshua A. Salomon and Avi Feller and Elizabeth A. Stuart},
  journal={arXiv: Methodology},
Policy responses to COVID-19, particularly those related to non-pharmaceutical interventions, are unprecedented in scale and scope. Researchers and policymakers are striving to understand the impact of these policies on a variety of outcomes. Policy impact evaluations always require a complex combination of circumstance, study design, data, statistics, and analysis. Beyond the issues that are faced for any policy, evaluation of COVID-19 policies is complicated by additional challenges related… 

Figures and Tables from this paper

Study Designs to Assess Real-World Interventions to Prevent COVID-19

This work offers recommendations on how to strengthen frequently applied study designs which have been used to understand the impact of interventions to reduce the spread of COVID-19, and suggests implementation-focused, pragmatic designs that, moving forward, could be used to build a robust evidence base for public health practice.

Effectiveness of Localized Lockdowns in the COVID-19 Pandemic

Abstract Non-pharmaceutical interventions, such as social distancing and lockdowns, have been essential to control the COVID-19 pandemic. In particular, localized lockdowns in small geographic areas

Challenges with Evaluating Education Policy Using Panel Data during and after the COVID-19 Pandemic

The goal is to help researchers think through the underlying issues and assumptions of panel or longitudinal studies of K-12 education interventions during and following the COVID-19 pandemic, and to help consumers of those studies assess their validity.

Evaluating the impact of keeping indoor dining closed on COVID-19 rates among large US cities: a quasi-experimental design

Keeping indoor dining closed contributes to reductions in COVID-19 spread and can inform state and local decisions to restrict indoor dining as a tailored strategy to reduce CO VID-19 incidence.



The Efficacy of Lockdown Against COVID-19: A Cross-Country Panel Analysis

Assessing the effect of lockdown measures (or the lack of such measures) on the numbers of new infections at an international level suggests that lockdown is effective in reducing the R0, i.e. the number of people infected by each infected person, and that its efficacy continues to hold 20 days after the introduction of the policy.

Comparing the impact on COVID‐19 mortality of self‐imposed behavior change and of government regulations across 13 countries

These findings have implications for the phased withdrawal of government policies as the epidemic recedes, and for the possible reimposition of regulations if a second wave occurs, especially given the substantial economic and human welfare consequences of maintaining lockdowns.

The effect of large-scale anti-contagion policies on the COVID-19 pandemic

It is estimated that across these six countries, interventions prevented or delayed on the order of 62 million confirmed cases, corresponding to averting roughly 530 million total infections, and anti-contagion policies have significantly and substantially slowed this growth.

Using Difference-in-Differences to Identify Causal Effects of COVID-19 Policies

Threats to the validity of difference-in-differences research designs are discussed and recommendations are made about how researchers can avoid bias, interpret results accurately, and provide sound guidance to policymakers seeking to protect public health and facilitate an eventual economic recovery.

Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S.

This paper evaluates the dynamic impact of various policies, such as school, business, and restaurant closures, adopted by the US states on the growth rates of confirmed Covid-19 cases and social distancing behavior measured by Google Mobility Reports, and finds that both policies and information on transmission risks are important determinants of people's social Distancing behavior.

Evaluating the Health Impact of Large-Scale Public Policy Changes: Classical and Novel Approaches

The state of knowledge and experience of public health researchers who rigorously evaluate the health consequences of large-scale public policy changes are reviewed.

Testing the Validity of the Single Interrupted Time Series Design

Given the complex relationships between patients’ demographics, underlying health needs, and outcomes, establishing the causal effects of health policy and delivery interventions on health outcomes

Interrupted time series regression for the evaluation of public health interventions: a tutorial

This tutorial uses a worked example to demonstrate a robust approach to ITS analysis using segmented regression and describes the main methodological issues associated with ITS analysis: over-dispersion of time series data, autocorrelation, adjusting for seasonal trends and controlling for time-varying confounders.

Association Between Statewide School Closure and COVID-19 Incidence and Mortality in the US.

It was estimated that closing schools when the cumulative incidence of COVID-19 was in the lowest quartile compared with the highest quartile was associated with 128.7 fewer cases per 100 000 population over 26 days and with 1.5 fewer deaths over 16 days.

Which interventions work best in a pandemic?

The only approaches currently available to reduce transmission of the novel coronavirus severe acute respiratory syndrome–coronavirus 2 (SARS-CoV-2) are behavioral: handwashing, cough and sneeze etiquette, and, above all, social distancing.