Engaging in discussions about the effectiveness of nonpharmaceutical interventions (NPIs) during the Covid-19 pandemic is essential, especially when utilizing detailed data at the U.S. county level. However, a recent piece by Marc Lipsitch and Sara Cody mischaracterizes our 2023 study published in The Lancet, distorting our conclusions to bolster their own arguments. Some inaccuracies persist despite subsequent updates by the authors, and their critique falls short of acknowledging the complexities involved in evaluating NPIs during the early pandemic.
Let’s clarify some points raised by Lipsitch and Cody. They mention that “a full analysis would consider in detail how people behaved, not only how they were told to behave.” Contrary to this claim, our research indeed assessed this relationship, scrutinizing ten policy mandates alongside various behaviors like mobility, masking, and vaccination trends. We directly noted a correlation between these behaviors and a reduction in Covid-19 infections.
On the matter of political adherence to control measures, our findings clearly indicated that politics and trust played decisive roles. We explicitly stated in our study that “partisanship reduced the use of protective mandates and behaviors.” While they acknowledged our analysis touched on the political aspect, they neglected the critical role trust played—a significant oversight.
Lipsitch and Cody also suggested that our study failed to account for variations in the timing of state mandates and the impact on earlier states hit by Covid-19. This is misleading. We conducted sensitivity analyses across multiple starting dates in early 2020 and accounted for variational differences in mandates across states. Our results remained stable regardless of the timeline used.
Arguably the most significant misrepresentation involves the core conclusion of our study regarding the effectiveness of NPIs. Contrary to their assertions, our findings indicated that states imposing stringent mandates, such as mask-wearing and mobility restrictions, and adhering to them longer, reported lower infection rates. It is accurate that we did not document a statistical relationship between NPIs and mortality rates, but this does not stem from oversight regarding behavior, political context, or timing of mandates—all of which we duly considered.
The lack of a discernible statistical relationship between NPIs and deaths arises from multifaceted influences merging during the pandemic. While NPIs play a critical role, they are not the only influential factor concerning mortality, especially in the early phase of the crisis. Numerous variables, from underlying health conditions to healthcare access, clouded the direct comparison between NPIs and death rates.
Many states enforced mandates in a compressed time frame, making it challenging to deduce outcomes in a no-intervention scenario. Take Florida, for example, where fewer mandates were introduced, yet protective behaviors sometimes emerged despite the absence of directives. Our analysis acknowledges that variations in local mandates could dilute the perceived effects of state policies adequately.
The absence of observable statistical relationships in our study should not be interpreted as proof that NPIs lacked impact. Our paper and subsequent commentary in various media outlets articulate the complex narrative that emerges from examining NPIs in real-world data that is inherently imperfect and noisy. There is substantial evidence showing that NPIs contributed to mitigating the spread of Covid-19.
As we confront Covid-19 revisionism, it's imperative to recognize and challenge the harsh critiques of NPIs that emerge from partisanship and opportunistic narratives. Such mischaracterizations cast doubt on essential public health strategies and diminish the discourse needed for proper accountability and learning from past missteps. An honest evaluation of the U.S. response, informed by rigorous evidence, will be key to improving future public health outcomes.
In the face of this disinformation, we must uphold preventive measures within the framework of scientific inquiry and maintain a commitment to a well-informed and evidence-based dialogue about public health responses.
Thomas J. Bollyky holds the Bloomberg chair in global health and directs the Global Health Program at the Council on Foreign Relations. Joseph Dieleman, Ph.D., is a professor at the University of Washington's Department of Health Metric Sciences and leads the Resource Tracking team at the Institute for Health Metrics and Evaluation.
Letter to the editor
Do you have thoughts on this analysis? Submit a letter to the editor.