An evaluation framework to assess countries' performance in controlling the COVID-19 pandemic

The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of the University of Birmingham

“Importantly, we incorporate the role of geopolitical and socio-economic characteristics in explaining the evolution of COVID-19.”


The COVID-19 pandemic has initiated several initiatives to better understand its behaviour, and some projects are monitoring its evolution across countries, which naturally leads to comparisons made by those using the data. However, most “at a glance” comparisons may be misleading because the curve that should explain the evolution of COVID-19 is different across countries, because of the underlying geopolitical or socio-economic characteristics.

Our study accepted for publication is the journal BMC Public Health, November 2021, contributes to the scientific endeavour by creating a new evaluation framework to help stakeholders (policymakers, public sector health workers, resilience managers, and the general public) to:

  1. Adequately monitor the evolution of reported confirmed cases and deaths in countries, considering the occurrence of spikes and “secondary waves”;
  2. Identify structural breaks in the confirmed cases and deaths curves;
  3. Assess the performance of their actions in the face of the spread of COVID-19

We incorporate new evidence that pandemic fatigue is taking hold. This decrease in commitment to fighting the pandemic alters the behaviour of the forecast errors present in the COVID-19 curves, causing a structural break in the variance of the residuals, or forecast errors. We are therefore able to use Markov-Switching models on the residuals of our forecasts to identify regime-switching in the COVID-19 time series. Our new methodology proves more effective in explaining the evolution of COVID-19 than growth functions worldwide, including highlighting several inflection points and regime-switching moments. In total we estimated 30 models for both curves (transmission and deaths). Moreover, results from this research can be used by managers, for example, to provide an econometric justification for the prioritizing of vaccination programmes in the health care sector.

We examined the COVID-19 transmission data for of 189 countries, collected from the Centre for Systems Science and Engineering at Johns Hopkins University. This is the largest study of its kind. The new outcomes presented in this research will allow key stakeholders to check whether public policies and interventions in the fight against COVID-19 are having an effect, easily identifying examples of best practices and promote such policies more widely around the world.

To cite an example of our findings, we show that considering confirmed cases in the US, our models indicate that a regime-switching started on 8 November, 2020 (five days after the United States presidential election), from a state of low uncertainty to high uncertainty. On 16 February, 2021, the USA returned to the low uncertainty regime since they administered the first vaccine on 14 December, 2020. However, on 17 July, 2021, uncertainty of COVID-19 transmission in the US reached peak, mainly due to the proliferation of the delta variant in the country, together with the decrease in the effectiveness of vaccines against this new variant of COVID-19. As for the number of deaths, the regime-switching from low variance to high variance, and vice-versa, follows a pattern similar to that observed for confirmed cases.

Importantly, we incorporate the role of geopolitical and socio-economic characteristics in explaining the evolution of COVID-19. Further, we also consider the role of psychological effects on people caused by the spread of COVID-19 around the world. Several studies have assessed the fear of healthcare professionals or medical students of being infected with COVID-19, as well as how this fear affects their physical, mental, and emotional health. Our evaluation framework allows us to conjecture those countries classified as efficient, whether in the short or long term, also have the highest levels of the physical, mental, and emotional quality of their health professionals or medical students. The importance of this is that these professionals work on the front line in the fight against COVID-19. Therefore, our evaluation framework can also be applied to assess the vaccination deployment worldwide, to contribute to the perception of vaccine safety and increase willingness to receive it.

Paper Citation (co-authors): Abdinardo Moreira Barreto de Oliveira, Dr Jane M Binner, Dr Anandadeep Mandal, Dr Logan Kelly, Dr Gabriel J. Power, D.