Adaptive diversification of COVID-19 policy
Charles Manski 12 June 2020
Formation of COVID-19 policy must cope with many substantial uncertainties about the type of the condition, the dynamics of the pandemic, and behavioural responses. This column argues that rather than making policy that’s optimal in hypothetical scenarios but potentially definately not optimal in reality, it really is more prudent to approach COVID-19 policy as a problem in decision making under uncertainty. Under ‘adaptive diversification’, a variety of policies will be implemented across locations and policymakers can revise the proportion of locations assigned to each policy as evidence accumulates.
Formation of COVID-19 policy must cope with many substantial uncertainties about the type of the condition, the dynamics of the pandemic, and behavioural responses. These uncertainties have grown to be well-recognised qualitatively (e.g. Avery et al. 2020), however they have not been well-characterised quantitatively. Credible measurement of COVID-19 uncertainties is required to make useful predictions of policy impacts and reasonable policy decisions.
Epidemiological types of disease dynamics, sometimes coupled with types of macroeconomic dynamics, have already been used to attain conclusions about optimal COVID-19 policy. However, researchers did little to appraise the realism of their models, nor to quantify uncertainties. Hence, there is little basis to trust the policy prescriptions which were put forward.
I believe it misguided to create policy that’s optimal in hypothetical scenarios but potentially definately not optimal in reality. It really is more prudent to approach COVID-19 policy as a problem in decision making under uncertainty. Facing up to uncertainty, one recognises that it extremely hard to guarantee selection of optimal policies.
While one cannot guarantee optimality under uncertainty, you can still make decisions that are reasonable in well-defined respects. I specifically suggest adaptive diversification of COVID-19 policy. I proposed adaptive policy diversification in two earlier papers (Manski 2009, 2013). Financial diversification is a familiar recommendation for portfolio allocation. Diversification enables an investor facing uncertain asset returns to limit the potential negative consequences of placing ‘all eggs in a single basket’. Analogously, policy is diversified if a planner facing uncertainty randomly assigns treatment units (persons or locations) to different policies. At a spot with time, diversification avoids gross errors in policymaking. As time passes, it yields new evidence about policy impacts, as in a randomised trial. As evidence accumulates, a planner can revise the fraction of treatment units assigned to each policy in accord with the available knowledge. This notion is ‘adaptive diversification’.
In this column, I explain why current modelling cannot deliver a realistically optimal COVID-19 policy, and discuss adaptive diversification.
Incredible certitude in epidemiological and macroeconomic modelling
Epidemiological modellers have sought to determine COVID-19 policy that might be optimal from a public health perspective if specified types of disease dynamics were accurate and public health were measured in specified ways (e.g Ferguson et al. 2020, IHME COVID-19 Health Service Utilization Forecasting Team 2020). Assessment of COVID-19 policy should think about the entire health, economic, and social impacts of alternative policy options. However, epidemiological modelling has only considered impacts on health. Grounds could be that epidemiology has been the province of researchers in medicine and public health. Researchers with these backgrounds think it is natural to spotlight health issues. They view the economy and social welfare as matters which may be important but that are beyond their purview.
Following a onset of the COVID-19 pandemic, macroeconomists have sought to expand the scope of optimal policy analysis by joining epidemiological models with types of macroeconomic dynamics and by specifying welfare functions that consider both public health insurance and economic outcomes (e.g. Eichenbaum et al. 2020, Acemoglu et al. 2020). However, there is little basis to measure the realism of the models which have been developed.
A significant underlying problem in both epidemiological and macroeconomic modelling may be the dearth of evidence open to inform model specification and estimation. Studies of disease and macroeconomic dynamics are largely struggling to perform the randomised trials which have been considered the ‘gold standard’ for medical research. Modelling necessarily depends on observational data, which are difficult to interpret. Lacking much evidence, epidemiologists and macroeconomists are suffering from models that are sophisticated from mathematical and computational perspectives but which have little empirical grounding.
These modelling efforts will be useful if interpreted cautiously as computational experiments studying policymaking in hypothetical worlds. However, their relevance to real life is unclear. Models differ considerably in the assumptions they maintain and in the manner they use limited available data to estimate parameters. Researchers provide little information that could enable someone to assess model realism. They do little to quantify uncertainty in the predictions they provide.
I’ve persistently argued for forthright communication of uncertainty in research that aims to see public policy (Manski 2019). I’ve criticised the prevalent practice of policy analysis with incredible certitude. Exact predictions of policy outcomes are routine; expressions of uncertainty are rare. Yet predictions often are fragile, resting on unsupported assumptions and limited data. Expressing certitude isn’t credible. Incredible certitude has been prevalent in both epidemiological and economic modelling.
A good example of incredible certitude in epidemiological modelling may be the March 2020 report of the Imperial College COVID-19 Response Team, which includes influenced policy formation in the united kingdom and US (Ferguson et al. 2020). The team forecast the impact of two alternative policy responses to the pandemic, mitigation and suppression, writing (p. 1):
“Two fundamental strategies are possible: (a) mitigation, which targets slowing however, not necessarily stopping epidemic spread . . . . and (b) suppression, which aims to reverse epidemic growth, reducing case numbers to low levels and maintaining that situation indefinitely.”
The forecasts were predicated on a modified version of a model developed earlier to aid influenza planning. The report provided scant justification for the use of this model to the COVID-19 context and did little to assess uncertainty in the forecasts made. Predicated on their forecasts, the COVID-19 Response Team recommended suppression as the most well-liked policy option. They reached this conclusion despite the fact that their report only considered the impact of policy on health, without focus on economic and social consequences.
There can be an urgent dependence on epidemiologists and economists to become listed on forces to build up credible integrated assessment types of epidemics. Even with the very best intentions, this will need time and effort. There is some reason to hope that epidemiologists and economists might be able to communicate with each other because they share a common language for mathematical modelling of dynamic processes. However, each group has during the past exhibited considerable insularity, which might impede collaboration. Moreover, neither discipline shows much willingness to handle up to uncertainty when developing and applying models.
There were frequent demands adoption of a uniform COVID-19 policy across locations, particularly over the 50 states of the united states. For instance, an 11 May 2020 editorial in the Washington Post was titled “The patchwork of state reopenings is a deadly game of learning from your errors”. The text identifies “the peril posed by the hodgepodge of state decisions to reopen quickly, gradually or never yet.” While warning against decentralisation of policymaking over the states, the editorial will not propose just what a uniform national policy ought to be.
Calling for a uniform COVID-19 policy across states will be justified if it were clear what constitutes optimal policy and if it were known that the perfect policy is invariant across states. Then each state should abide by that policy. However, as explained above, we have no idea what optimal policy is for just about any state. It might be that continued suppression is way better for a few states (or elements of states) and that some version of reopening is way better for others, based on their characteristics. Hence, there is absolutely no prima facie case to make policy uniform across states.
It is definitely appreciated in america that uncertainty may justify decentralisation of policymaking, enabling the states to test out policy ideas. Supreme Court Justice Louis Brandeis, in his dissent to the 1932 case NY State Ice Co. v. Liebmann (285 U.S. 311), made what has turned into a famous remark upon this theme: “It really is among the happy incidents of the federal system a single courageous State may, if its citizens choose, serve as a laboratory; and try novel social and economic experiments without risk to all of those other country.” It has since become common to make reference to the states as the laboratories of democracy.
The Brandeis statement expresses the ‘adaptive’ facet of the theme of adaptive diversification, recognising that policy variation across states stimulates studying policy impacts. The ‘diversification’ facet of the theme has been less well appreciated.
To illustrate, consider the decision between suppression and mitigation framed by Ferguson et al. (2020). Suppression could be the better policy if the Imperial College model makes reasonably accurate predictions of COVID-19 health impacts and if the economic impacts ignored by the model are relatively small. Alternatively, mitigation could be the better policy if the model substantially overestimates the COVID-19 health impacts or if the economic impacts ignored by the model are relatively large. Policy diversification, with some locations implementing suppression and others implementing mitigation, gives up the perfect of optimality to be able to drive back making a gross error in policy choice.
When diversifying, what fraction of locations should implement each policy option in mind? This depends upon the welfare function that society uses to judge options and on the uncertainties that afflict prediction of policy impacts. In Manski (2009), I study adaptive diversification when social welfare is utilitarian, and a planner runs on the simple dynamic version of the minimax-regret criterion to handle uncertainty. The effect is a straightforward diversification rule. Given specification of a proper welfare function and characterisation of the relevant uncertainties, it must be possible to adapt this analysis to diversify COVID-19 policy.
Acemoglu, D, V Chernozhukov, I Werning, and M Whinston (2020), “Optimal Targeted Lockdowns in a Multi-Group SIR Model”, NBER Working Paper 27102.
Avery, C, W Bossert, A Clark, G Ellison and S Ellison (2020), “Policy Implications of Types of the Spread of Coronavirus: Perspectives and Opportunities for Economists”, NBER Working Paper 27007.
Eichenbaum, M, S Rebelo, and M Trabandt (2020), “The Macroeconomics of Epidemics”, NBER Working Paper 26882.
IHME COVID-19 Health Service Utilization Forecasting Team (2020). “Forecasting COVID-19 effect on hospital bed-days, ICU-days, ventilatordays and deaths by US state within the next 4 months”, Institute for Health Metrics and Evaluation, University of Washington.
Manski, C (2009), “Diversified Treatment under Ambiguity,” International Economic Review 50: 1013-1041.
Manski, C (2013), Public Policy within an Uncertain World, Harvard University Press.
Manski, C (2019) “Communicating Uncertainty in Policy Analysis”, Proceedings of the National Academy of Sciences 116: 7634-7641.