Summary: this is basically a Bayesian regression of the COVID-19 cases and deaths in 41 countries against the measures taken by those countries. It requires a rough model of how infections spread which means you have to be careful about choosing models, validate them by predicting the epidemic, and try to deal with confounding. The results support closing schools and universities, gatherings, and some types of face-to-face businesses.
It's been great to work on this with a large team of talented researchers from many universities, most of whom are rationalists and EAs. The lead authors and the senior author are employees and affiliates of FHI:
Jan M. Brauner*, Sören Mindermann*, Mrinank Sharma*, David Johnston, John Salvatier, Tomáš Gavenčiak, Anna B. Stephenson, Gavin Leech, George Altman, Vladimir Mikulik, Alexander John Norman, Joshua Teperowski Monrad, Tamay Besiroglu, Hong Ge, Meghan A. Hartwick, Yee Whye Teh, Leonid Chindelevitch, Yarin Gal, Jan Kulveit
We're also grateful to Tim Telleen-Lawton and BERI who funded and operationally supported the Epidemic Forecasting project which in turn incubated this project.
The story of this paper is easy to tell with its figures so I'll briefly to that.
Data:
Main results:
Fig. 2NPI effectiveness under default model settings.Posterior percentage reductions in Rt with median, 50% and 95% prediction intervals shown. Prediction intervals reflect many sources of uncertainty, including NPI effectiveness varying by country and uncertainty in epidemiological parameters. A negative 1% reduction refers to a 1% increase in Rt. “Schools and universities closed” shows the joint effect of closing both schools and universities in conjunction; the individual effect of closing just one will be smaller (see text). Cumulative effects are shown for hierarchical NPIs (gathering bans and business closures) i.e., the result for “Most nonessential businesses closed” shows the cumulative effect of two NPIs with separate parameters and symbols—closing some (high-risk) businesses, and additionally closing most remaining (non-high-risk, but nonessential) businesses given that some businesses are already closed.
Effect of combined interventions:
Fig. 3Combined NPI effectiveness for the 15 most commonly implemented sets of NPIs in our data.Solid and shaded regions denote 50% and 95% Bayesian prediction intervals. (A) Predicted Rt after implementation of each set of NPIs, assuming R0 = 3.3. (B) Maximum R0 that can be reduced to Rt below 1 by common sets of NPIs. Readers can interactively explore the effects of all sets of NPIs, while setting R0 and adjusting NPI effectiveness to local circumstances, with our online mitigation calculator (16).
Lots of sensitivity analyses:
Fig. 4Median NPI effectiveness across the sensitivity analyses.(A) Median NPI effectiveness (reduction in Rt) when varying different components of the model or the data in 206 experimental conditions. Results are displayed as violin plots, using kernel density estimation to create the distributions. Inside the violins, the box plots show median and interquartile-range. The vertical lines mark 0%, 17.5%, and 35% (see text). (B to E) Categorized sensitivity analyses. (B) Sensitivity to model structure. Using only cases or only deaths as observations (2 experimental conditions; fig. S7); varying the model structure (3 conditions; fig. S8, left). (C) Sensitivity to data and preprocessing. Leaving out countries from the dataset (42 conditions; figs. S5 and S21); varying the threshold below which cases and deaths are masked (8 conditions; fig. S13); sensitivity to correcting for undocumented cases and to country-level differences in case ascertainment (2 conditions; fig. S6). (D) Sensitivity to epidemiological parameters. Jointly varying the means of the priors over the means of the generation interval, the infection-to-case-confirmation delay, and the infection-to-death delay (125 conditions; fig. S10); varying the prior over R0 (4 conditions; fig. S11); varying the prior over NPI effect parameters (3 conditions; fig. S11); varying the prior over the degree to which NPI effects vary across countries (3 conditions; fig. S12). (E) Sensitivity to unobserved factors influencing Rt. Excluding observed NPIs one at a time (8 conditions; fig. S9); controlling for additional NPIs from a different dataset (6 conditions; fig. S9).
Structure of the main model:
Fig. 5Model overview.Unshaded, white nodes are observed. We describe the diagram from bottom to top: The mean effect parameter of NPI i is αi, and the country-specific effect parameter is αi,c. On each day t, a country’s daily reproduction number Rt,c depends on the country’s basic reproduction number R0,c and the active NPIs. The active NPIs are encoded by xi,t,c, which is 1 if NPI i is active in country c at time t, and 0 otherwise. Rt,c is transformed into the daily growth rate gt,c using the generation interval parameters, and subsequently is used to compute the new infections N(C)t,cNt,c(C) and N(D)t,cNt,c(D) that will subsequently become confirmed cases and deaths, respectively. Finally, the expected number of daily confirmed cases y(C)t,cyt,c(C) and deaths y(D)t,cyt,c(D) are computed using discrete convolutions of N(.)t,cNt,c(.) with the relevant delay distributions. Our model uses both case and death data: it splits all nodes above the daily growth rate gt,c into separate branches for deaths and confirmed cases. We account for uncertainty in the generation interval, infection to case confirmation delay, and the infection to death delay by placing priors over the parameters of these distributions.
Much of the interesting parts are in the appendix.
This is a linkpost for https://science.sciencemag.org/lookup/doi/10.1126/science.abd9338
Summary: this is basically a Bayesian regression of the COVID-19 cases and deaths in 41 countries against the measures taken by those countries. It requires a rough model of how infections spread which means you have to be careful about choosing models, validate them by predicting the epidemic, and try to deal with confounding. The results support closing schools and universities, gatherings, and some types of face-to-face businesses.
It's been great to work on this with a large team of talented researchers from many universities, most of whom are rationalists and EAs. The lead authors and the senior author are employees and affiliates of FHI:
Jan M. Brauner*, Sören Mindermann*, Mrinank Sharma*, David Johnston, John Salvatier, Tomáš Gavenčiak, Anna B. Stephenson, Gavin Leech, George Altman, Vladimir Mikulik, Alexander John Norman, Joshua Teperowski Monrad, Tamay Besiroglu, Hong Ge, Meghan A. Hartwick, Yee Whye Teh, Leonid Chindelevitch, Yarin Gal, Jan Kulveit
We're also grateful to Tim Telleen-Lawton and BERI who funded and operationally supported the Epidemic Forecasting project which in turn incubated this project.
Paper: Inferring the Effectiveness of Government Interventions Against COVID-19
There's also a closely related NeurIPS paper.
The story of this paper is easy to tell with its figures so I'll briefly to that.
Data:
Main results:
Effect of combined interventions:
Lots of sensitivity analyses:
Structure of the main model:
Much of the interesting parts are in the appendix.