The value of testing

Andrea Galeotti, Paolo Surico, Jakub Steiner 23 April 2020

a

A

Many important considerations about tests for Covid-19 are rightly in the domain of epidemiologists, biochemists, virologists and other natural scientists. However, there are important concepts, such as the allocation of testing, for which an economic framework can help (Baldwin 2020). In this column, we spell out some of those arguments, with the hope of complementing medical analysis. 

Economists are used to thinking about the allocation of scarce resources, and the designing of testing strategies fits such a purpose. We will develop the argument below, aiming for what we believe is a commonly share societal objective: to minimise the economic and social cost, subject to the constraint of keeping infection rates sufficiently low (the health constraint). We will consider two instruments to achieve this objective: testing and self-isolation. 

The determinants of the individual and collective value of testing

Economists attribute value to information according to the extent it informs the decision maker to choose action that achieves her objective. This approach leads to a simple but useful rule of thumb regarding testing. If an outcome of the test does not affect the follow-up action, then the particular instance of testing has no value and the test should be saved for a more useful occasion. For instance, public health authorities should not use an imprecise test if the decision at stake is whether to quarantine the tested person and the quarantine will be imposed even after a negative test result for preventive reasons.

It is convenient to classify the informational value of a test in terms of individual and collective value. 

The individual value of testing for a general disease mostly consists in learning whether an individual is infected or not, so that adequate medical resources and responses can be deployed. The value depends on whether you need to act fast to save the patient, and whether specific medical resources are needed to improve the health of the individual. For example, the individual value of a test for cancer is extremely high, as the effectiveness of medical therapy is much greater if the cancer is detected at an early phase. For Covid-19, however, the individual value of testing is rather low, since most patients are treated without specific Covid-19 treatments.

The collective value of testing consists in learning new information about the rate of infection in the local network of the individual being tested, as well as learning about the general prevalence and diffusion of the virus in the population. Based on this information, contact tracing and smart containment strategies can be designed so that the contagion curve is kept relatively flat and the health system can operate within its capacity.

The collective value of Covid-19 test depends on many dimensions. 

The characteristics of the virus

From countries in which testing has mainly been done regardless of symptoms – such as South Korea, Iceland, Germany – and from a small testing programme to a representative sample of the population – like the one presented by Bendavid et al. (2020) – we have evidence that most Covid-19 cases are asymptomatic (see also Galeotti and Surico 2020, Ferreti et al. 2020, Ruiyun et al. 2020). Even if the asymptomatic individuals may be less contagious, there are so many that they are the still main cause of the spread of the disease. This feature of Covid-19 makes the collective value of testing extremely high: it is only by identifying the asymptomatic – i.e. the key influencers of the spread – that the contagion curve can flattened.

Observation 1. As a rule of thumb, the collective versus social value ratio of testing is high as long the effective individual treatment is unknown.

Observation 2. As a rule of thumb, the more the virus spreads before symptoms are developed, the higher the collective value relative to the individual testing value.

The phase of the epidemic

Identifying the agents who spread the virus is key when the replication number is at a low level and there is not enough immunity built in the population. This is true at the beginning of the diffusion process (in the case of Europe, the end of February). However, it also applies when the contagion curve has been flattening via a strict lockdown policy, and therefore during the first phase of contagion when society has not built enough immunity. This is the situation that many European countries are approaching now. In these countries, the governments have begun to evaluate various exit strategies in April and they are all operating under strong pressure due to the day-by-day worsening of economic situations. 

Observation 3. The collective value of testing is extremely high now that countries are evaluating exit strategies from the lockdown with a population that is far from reaching herd immunity. 

The type of available tests and their reliability

PCR tests detect actual infection, and therefore allow us to learn about the acutely sick and their share in the population. We refer to this share as the ‘flow’ of the disease. PCR tests are extremely useful in implementing effective contact-tracing strategies.

Serological tests detect the presence of antibodies, and therefore make it possible to learn the share of people who have had Covid-19 so far. We refer to their population share as the ‘stock’ of the disease. Serological tests are key to learning how much the virus has spread in the population, to what extent infection translates into symptoms, what the contagion rate is, and how all these important dimensions correlate with observable characteristics such as gender, ethnicity and age.

Neither PCR nor serological tests are not fully reliable. Because the collective value stems from the information the society obtains from testing and from how this information is used, the lack of full reliability needs to be assessed in relation to what society wants to learn.

Example 1: Serological tests and false positives
Suppose the government uses serological tests to learn how much immunity has been built up in a specific group of the population (such as a geographical region). If there is evidence of high immunity, then the government relaxes the lockdown. In such a case, the main problem is with false positive errors, which occur when the test suggests that an individual has developed antibodies (for simplicity, here we take this as evidence of immunity, but we discuss an important caveat below) when, in fact, this is not the case.

Such false positive errors may, if the decision maker were unaware of them, create over-optimistic estimates of the immunity built in the population. If the underlying true infection rate of the tested group is large, then such bias will be minimal. However, if the infection rate in the underlying group is low, then even a small percentage of false positive test errors can generate highly over-optimistic estimates of immunity. Therefore, for the inspection of the infection rate in populations with low Covid-19 exposures, governments should choose tests with low false positive error rates.

To illustrate the effect, imagine that a researcher finds that 10% of the people in a representative sample have positive test results. Assume, for the sake of discussion, that the employed test never fails to identify a sick person, but let us allow for false positive errors. The collected data are consistent with (a) a 10% infection rate and a 0% false positive error rate, (b) a 0% infection rate and a 10% rate of the false-positive error, or (c) anything in between. It is therefore important to rigorously validate tests before they are employed in data collection, in particular in regions with a prior low infection rate. The importance of the test validation is well illustrated in example of the serological testing programme in Santa Clara County, California, presented in Bendavid et al. (2020).  

Example 2: PCR and false negatives
Suppose the government wants to use PCR tests to monitor workers in certain location/industries where transmission can potentially be very high. The government wants to avoid a new spread of the virus and therefore allows people to work on-site only if not infected. In such a situation, the main problem is with false negative errors, which occur when a PCR test suggests that the individual is not infected, when in fact he or she is.

The rate of false negative errors that can be tolerated depends on the immunity that is present in the system (and more generally, on the replication number). If the government learns, for example, that in a particular region immunity is high, then it should be less worried about false negative PCR tests. However, in regions where immunity has not yet been sufficiently developed, then this is a real concern. In this situation, having accurate measures of test reliability is important. Therefore, for this purpose, the government should not employ a test with a high false negative error rate because a negative test result would not be reliable enough to release the worker for work.

Example 3: PCR and evolvability
Another related problem with PCR is the evolvability of the outcome of the test: an individual can be negative on day one of the infection but positive on day three. So, if an individual is tested when she has just become infected and the outcome is negative, the individual would spend all of her period of contagiousness in close proximity to others. This creates similar problems to example 2. It is not a major concern if there is enough immunity in the population, but if there is little immunity, it is important to limit the occurrence of those cases, and this can be done with a repeated testing procedure on the same individual. 

In practice, PCR tests have sizeable false negative error rates since these can already occur at the level of the sample swab. Acknowledging the existence of false negative errors and understanding their rate allows for a more efficient use of the information stemming from testing. For instance, one can impose a preventive quarantine after a first negative result until it is confirmed by a further negative test outcome.

An important caveat about immunity

Our reading from the media is that many politicians and commentators, when proposing potential exist strategies, take for granted that recovered individuals have acquired some form of immunity. We do not have the expertise to comment on this highly important aspect. However, we point out that, to our knowledge, the correlations between levels of antibodies developed in recovering from Covid-19, the severity of the infection and the level of immunity acquired are, at best, vague.

This is an important limitation to any proposed exit strategy that tries to learn via serological tests, and that plans to relax social distancing based on a presumption of immunity. 

However, this problem should not stop governments from employing serological tests, combined with PCR testing and contact tracing. This is our best strategy, until we get a vaccine. In fact, the observation that we lack reliable insights on the immunity acquired in recovering from Covid-19 only contributes further to the collective value of testing.

Policymakers should not waste additional time and instead should act quickly in preparing testing programmes to extract as much information as possible. By doing this, we will also start to learn about the advantages and limitations of those tests. 

The experience of different countries

Testing strategies differ even within European countries. Our analysis (Galeotti and Surico 2020) reveals the following patterns.

Countries with a severe lack of PCR tests mainly test patients motivated by the individual value of testing, and largely refrain from considering the collective value of testing when allocating testing resources. Italy, UK and Spain are cases in point.

As PCR testing resources increase, testing strategies recognise more and more the collective value of testing. In countries where contact-tracing technologies are not yet well developed, testing is expanded to mildly symptomatic cases. We see this trend in some parts of Italy, while Germany and Iceland are prime examples of this. In countries where contact-tracing technologies are well-developed, testing becomes largely unrelated to symptoms. Other characteristics, such as being a potential key diffuser (i.e. individuals with characteristics associated with large diffusion conditional on being infected, including medical staff, workers in transportation hubs and above averagely connected people), become important when allocating testing resources. The social value of testing is highly recognised. This is not surprising, as contact-tracing technologies cannot be successful without the complementary use of testing. 

A proposal for countries such as the UK and Italy

We will focus on countries such as the UK, Italy, Spain and France, that are now in strict lockdowns. These suppression policies are currently providing the desired results. Italy has now reached the peak of the contagion curve; the UK is just a couple of weeks behind Italy. Epidemiological models suggest that, as a consequence of the lockdowns, neither the UK nor Italy has developed enough immunity. However, certain geographical regions (Veneto and Lombardy in Italy, London in the UK) may have reached about 15% immunity.

Our proposal is articulated in three phases. The overall objective is to enable a re-opening of society whilst ensuring that a new spread of the virus is avoided.

Phase 1: Develop a serological testing programme on a representative sample of the population. This should also collect information on demographic characteristics such as age, gender, number of children, working sector, skill, social and working connectiveness. The UK, for example, could use the labour force survey sample used by the Official for National Statistics. 

The objective of this first phase is to estimate the stock of the infection, immunity (within the caveat described above), heterogeneity across geographical regions, and correlations with observables such as age, gender, other demographic characteristics, location, working sectors and skills.

Important aspects in the design of this phase are that estimates are highly sensitive to false positive errors, particularly in parts of countries in which, a priori, we expect low infection. Hence, in these parts of the countries, careful test validation is extremely important. 

Phase 2: Partial reopening of society/economy. 

The objective of this phase is to restart the economy soon, without prompting a second wave of the virus, by using contact-tracing technologies for those allowed to go back to work. As this is initially a small part of the population, this will allow one to test and improve the design of such technologies during this phase.

We should prioritise, as much as possible, smart working (working from home). Among sectors in which smart working is not feasible, we should focus on those that have large spillovers to aggregate demand and supply. 

At the same time, PCR tests should be allocated to maximise their collective value: to quickly discover infected people, self-isolate, and use network tracing for additional testing and self-isolation. To do this effectively, the allocation of tests should give priority to individuals with high connectivity (for example, workers in health systems, police, and workers in hubs or where production requires close proximity). We should use doctors as surveillants of the infection: when symptoms appear, test immediately and activate contact tracing. Large firms should be incentivised to use random PCR testing in the workplace.

In geographical regions where immunity is low, or for subpopulations where immunity is predicted to be low, testing capacity should be expanded and targeted to local suspects. 

In this phase, contact-tracing technologies should be introduced and appropriate incentives designed for their adoption, and compliance with contact-tracing prescriptions should be required. For example, the government could require the use of contract-tracing technology for individuals to be able to go back to work. It could reimburse the cost of PCR tests when required by the app. It could reimburse the cost of a test, if taken voluntarily, where the outcome is positive and the test is recorded in the app. The government should also subsidise self-isolation, when required.

It is tempting to believe that, in such a difficult situation, everyone will comply with a contact-tracing technology. After all, citizens have responded with a high sense of civic duty to the existing lockdown measures. We believe this positive response is ‘easy’ to sustain when the devastating effects of Covid-19 appear in the media every day. However, contact-tracing technologies are most useful when the effects of the virus are not visible; by definition, they are useful when the virus is contained. In this situation, some individuals may forget that their behaviour can have negative externalities for others (analogously to the sharp decrease in vaccinations amongst children in Europe), and that is why it is important to create appropriate incentives. 

Phase 3: Relax social distancing to the whole population and use widespread contact tracing to contain the virus until immunisation (pharmaceutical and non-pharmaceutical) is widespread in the population.

There are two final observations. These three phases need to be coordinated at the national level, not the regional level. In fact, they should be coordinated at the European level. Furthermore, these three phases must be coordinated with macro- and microeconomic policies. Health and economic outcomes are, more than ever, interrelated in this crisis.

Authors’ note: Financial support from the European Research Council and the Wheeler Institute is gratefully acknowledged. 

References

Baldwin, R (2020), “COVID-19 testing for testing times: Fostering economic recovery and preparing for the second wave", VoxEU.org, 26 March.

Bendavid, E et al. (2020), “COVID-19 Antibody Seroprevalence in Santa Clara County, California”,  medRXiv, 17 April 17. 

Ferretti, L et al. (2020), "Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing", Science

Galeotti, A, and P Surico (2020), "Why testing a representative sample of the population must be done now", VoxEU.org, 8 April.

Ruiyun, L et al. (2020), “Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV2)”, Science, 16 March.

a

A

Topics:  Covid-19

Tags:  COVID-19, coronavirus, testing lockdown, individual value, collective value

Professor of Economics, London Business School

Professor, Economics Department, London Business School; Research Affiliate, CEPR

Events

CEPR Policy Research