How much does government’s short-term response matter for explaining cross-country variation in COVID-19 infection outcomes? A regression-based relative importance analysis of 84 countries

Study design

We construct a daily panel recording government policy responses and the cumulative number of confirmed cases of COVID-19 since the outbreak of the pandemic. In this study, we define the country-specific time at which COVID-19 broke out as the event day when the cumulative confirmed cases reached 1 per million people, and the earliest date is 26 January 2020. To rule out any impact from direct medical interventions, we truncate the time-series data for modelling at 20 December 2020, when the world COVID-19 vaccination rate statistics started to count at 1/10 000 residents.18 To minimise any possible outlier bias, we limit the samples to countries with populations greater than 5 million and having data available on all the variables of interest, yielding 84 countries with 23 635 observations.

There are several reasons why our study focuses on the government’s short-term NPIs at the early stage of the pandemic. First, extensive research has consistently demonstrated that the government’s prompt response is of high importance in the early stage of pandemics, given the exponential progression of infections.19–21 In addition, by analysing data prior to the widespread introduction of any effective vaccination, the study design rules out any impact from direct medical interventions and enables us to focus only on the government NPIs.

Moreover, observational studies focusing on the impact of policy stringency on an ongoing pandemic usually face the challenge of reverse causality.22 A strong policy may help prevent the spread of COVID-19, presenting a negative association between government response and confirmed cases. On the other hand, the increasing cases or deaths may trigger stronger policy responses, thus showing a positive relationship between the two indicators if the policy stringency variable is time variant.

A myriad of studies employed parametric approaches to tackle the endogenous issue in various ways, including using a time lag of policy variables in regression models or constructing a counterfactual scenario through a difference-in-differences approach or synthetic control method.1 2 19 20 22–24 Others use semi-mechanistic or mechanistic models to infer transmission rates or reproduction numbers, such as Bayesian hierarchical models.25–27 For a study using a time-varying government response index from the same data source of our study, although measures were adopted to mitigate estimation bias due to endogeneity, it remains a concern when assessing the long-term policy effectiveness.28

We overcome some of the limitations by constructing the country-level NPIs as time invariant indices based on government responses before the outbreak of the pandemic, which contributes to the existing literature by offering a different perspective on the measurements of NPIs. One other advantage is that, by making the policy indices consistent with other natural, cultural and socioeconomic factors as country variant but time invariant variables, the relative importance analysis allows for a more pertinent comparison of the relative explanatory power of each factor.

Variables and data sources

COVID-19 prevalence: We use cumulative confirmed cases per million people to measure the prevalence of COVID-19 because it is a direct result of the government responses on which we focus. The main source of data on the pandemic is the Our World in Data database, which contains daily updated COVID-19 data compiled by the Center for Systems Science and Engineering at Johns Hopkins University.29Table 1 provides descriptive statistics on COVID-19 prevalence and other variables of interest.

Some other studies on policy and institutional impact use the mortality rate or case fatality rate (CFR) to measure the health outcome of the pandemic.1 3 30–32 Instead, we use the infection rate, for three reasons. First, the number of confirmed cases is a direct result of the government responses on which we focus, while the number of deaths or CFR reflects the indirect result of preventive measures plus the impact of medical intervention. The purpose of government NPIs is to prevent the spread of COVID-19. Pharmaceutical interventions aim to minimise the number of deaths from COVID-19 among those who are infected, which relies more on a country’s level of medical capacity and investment.

Second, we believe that the number of confirmed cases is a more accurate measure of the outbreak of the pandemic across countries, since it generates less bias and best suits our purpose to evaluate the impact of governments’ non-pharmaceutical responses, which are mostly preventive measures. It must be admitted that due to the differences in testing capacity and reporting methodologies across countries, the variation in the number of confirmed cases may not reflect that of the actual infected cases. However, measurement of the number of deaths also suffers from problems in the attribution of the cause of death, in addition to the problems with testing and reporting.29 For example, COVID-19 can lead to complications, such as pneumonia or acute respiratory distress syndrome, which ultimately cause death and the requirements on how to report the cause of death vary across countries.29 Given that CFR is defined as the number of confirmed deaths divided by the number of confirmed cases, it suffers from biases in both the numerator and the denominator.

Government response: We evaluate the influence of government response from two dimensions: stringency and reaction speed, using data from the Oxford COVID-19 Government Response Tracker.33 We use 12 specific policies to evaluate government response to the health crisis, with 4 focusing on health systems (public information campaigns, testing policy, contact tracing and facial coverings) and 8 on containment and closure policies (school closure, workplace closure, public events cancellation, restrictions on gatherings, public transport closure, stay-at-home requirements, restrictions on internal movements and international travel controls), as presented in (online supplemental table S1). We compile the 12 indicators into a policy value and define the Stringency Index as the maximum policy value before the outbreak of the pandemic in each country to indicate government response stringency. A higher value of the index represents a more stringent government response. The Stringency Index ranges from 0 to 86.11, with a mean of 38.94.

To measure the speed of government response, we use the number of days since a country recorded its first confirmed case until its policy value reached 39, the average value of the Stringency Index, to define the government’s reaction speed.1 All the countries in our sample have reached a policy value of 39 at some point, which took 20 days on average and varies between 1 and 56 days.

Natural conditions: We include a set of continental indicators to capture the inherited natural characteristics of a country.

Socioeconomic conditions: The most recent data available on per capita gross domestic product (GDP) expressed in constant 2011 international dollars (purchasing power parity), government health expenditure as a proportion of GDP and median age are taken from the World Bank and United Nations Population Division.34 35

Cultural conditions: To examine the role of culture, we employ the widely used individualism-collectivism index by Hofstede.36 37 It measures how people in a society see themselves as an autonomous entity or integrated into groups on a 0–100 scale, with larger numbers being more individualistic. Hofstede quantifies nations’ characters on each value dimension that shapes the ideology and behaviour of the citizens. Among the six dimensions, the individualism-collectivism dimension is the one with the greatest predicted power and it best reflects the cross-national difference in behaviours of the government and the people facing a pandemic.38–40 The mean value in our sample is 37.64 table 1.

We are aware that plenty of other factors are used in the analysis of the risk factors of COVID-19 at the micro and macro levels. For instance, studies have reported that wider COVID-19 spread and higher mortality can be associated with having multiple comorbidities.31 41 42 Nonetheless, given the limited number of countries in our sample and for interpretation convenience, we include only the most representative indicators to capture a general picture of each country’s natural, cultural and socioeconomic characteristics.

Statistical analysis

We apply a fixed-effect model by regressing the cumulative cases per million in each country every day since the outbreak on the Stringency Index and speed measure and other correlates using the population size of each country as the weight. We use the natural log transformation of the dependent variable to account for the exponential epidemic growth curve and control a group of independent variables at the country level. Given the time invariant nature of our variables of interest, we were unable to include a country fixed effect as it would absorb the effects of the independent variables.

Since the spread of the virus evolved at different paces across countries, to make the outcome comparable, we define the time-series as the days since the cumulative confirmed cases reached 1 per million people in each country and control the fixed effect of day. The day fixed effect captures the common elements of the SARS-CoV-2 lifecycle and routes of infection that are related to the underlying pathobiology of COVID-19. Much attention in the field of medical research has been drawn to the clinical manifestations and transmission dynamics of the disease. The scientific evidence contributes to the global understanding of the pandemic and decision- making on contact tracing measures across the world; thus, many countries have followed similar protocols to combat the pandemic. For example, based on the epidemiological research results, the disease control and prevention guidelines of the WHO, the USA, the European Union and China all adopted the standard of 2 days prior to the onset of symptoms as the starting time of the contact elicitation window.43–46

We also control the date fixed effect to capture any unobserved external shock on certain calendar dates along the timeline that marks every step of the evolution of the pandemic through 2020. The mechanism of such effect is usually unquantifiable but captures the common influences of the progress of the pandemic and people’s behaviour. For instance, on 6 July 2020, the journal Clinical Infectious Diseases published an open letter from 239 scientists around the world, urging health agencies to recognise the potential for the airborne spread of COVID-19, followed by the WHO’s updated scientific brief on 9 July.47 48 Such information has travelled across the globe every day and imposed imperceptible and unobtrusive influences on the attitudes and behaviours of governments and citizens, and further acts on the transmission outcomes. The date fixed effect captures those heterogeneities, which are time variant.

We further analyse the relative importance of the NPIs and other influencing factors in explaining the cross-country variation in the COVID-19 infection outcomes.49–51 Since correlation may exist between the independent variables, the relative degree of importance of each variable cannot be obtained directly. Thus, we conduct an analysis to isolate the contribution of each independent variable to the R² of the model. The relative importance analysis, also known as dominance analysis, compares and ranks the additional contribution each independent variable makes to the variance explained by all the possible subset models. Although studies analysing the direct effects of government responses on the pandemic spread highlight the significance of the NPIs, to our knowledge, the relative explanatory power of other factors in explaining the different outcomes across countries has not previously been given enough attention.

Patient and public involvement

Patients or members of the public were not directly involved in this research study.

Regression results

We further investigate the impacts of the government responses and each factor on the transmission outcome of COVID-19 using the regression model. In table 2, column 1 shows the result of the benchmark regression in which only day and date fixed effects are controlled. Column 2 further adds the Stringency Index and speed measure. Column 3 employs the continent dummy variables as the natural factor, with Africa as the baseline. Columns 4 and 5 present the results when gradually adding the socioeconomic variables and the Individualism Index.

The R² of 0.531 in column 1 indicates that the variation in the prevalence of COVID-19 in each country can be largely explained by the SARS-CoV-2 lifecycle and the impact of other unobserved factors that are time variant but country invariant. By adding the Stringency Index and speed measure, the R² in column 2 sharply increases to 0.701, which implies that there exist significant policy effects on the prevalence of COVID-19 across countries. With the inclusion of continental dummy variables, the R² further increases to 0.821 and it gradually increases to 0.861 when socioeconomic and cultural factors are taken into consideration.

The coefficients of the government Stringency Index and speed measure are statistically significant at the 0.01 level in all specifications, which indicates the robustness of the impact of the NPIs on preventing the spread of the virus. The results of the specification with all the factors suggest that on average across all the countries in our sample, an increase in the Stringency Index by 1 is associated with a 0.012% reduction in the cumulative confirmed cases per million, while each additional day of delay in reaching the average maximum Stringency Index before the outbreak corresponds to 0.004% more cumulative confirmed cases per million, holding other variables constant. In other words, a country that responded 1-month later than others would experience and increase of 0.12% in confirmed cases per million. Compared with the estimates of the coefficients in column 2, the impact of government response stringency with all other variables controlled decreases by 55.6% and the scale of the response speed coefficient decreases by 90.5%, meaning that the impacts of government responses can be diluted by the predetermined attributes of a country.

Countries in Asia, Europe, North America and South America report higher confirmed cases per million people. The estimated coefficient of GDP per capita is significantly positive. With the GDP per capita controlled, government expenditure on health as a proportion of GDP and median age are negatively associated with the infection outcome. From the cultural perspective, COVID-19 is more prevalent in countries that are more individual-oriented. Robustness checks with the redefined Stringency Index, natural factors and cultural factors are presented in the (online supplemental appendix and table S2). We replace continents with latitude to reflect the influence of natural factors, use the category of civilisations instead of the Individualism Index to represent culture52 and include countries’ current BCG vaccination policy as an additional independent variable. The major results from the robustness checks remain well consistent and further support our findings.

Relative importance analysis

In this subsection, we disentangle the contributions of the variables of interest to the R². Table 3 presents the relative importance analysis results based on the regression using the model specification in column 5 of table 2 for which the R² is 0.861. The results show that the most powerful explanatory variables are the day and date fixed effects, suggesting that regardless of country-specific conditions, the pandemic’s common effects worldwide by calendar date and days since the outbreak together account for more than half of the variation in the infection outcomes. The continent of location has the third-highest relative importance in the contribution to explaining the R². The explanatory power of the degree of individualism alone surpasses that of the NPIs and explains 9.30% of the variation. The contributions of the two NPI indicators, the Stringency Index and speed measure, rank fifth and nineth among the indicators of interest and together explain 8.78% of the variation in COVID-19 infection outcomes across countries. The respective contribution to the R² of each socioeconomic indicator of a country is less than 4%.