Lousy experts: Looking back at the ex ante estimates of the costs of Brexit

Nauro Campos 04 August 2016

a

A

Editors' note: This column first appeared as a chapter in the VoxEU ebook, Brexit Beckons: Thinking ahead by leading economists, available to download free of charge here.

On 23 June 2016, 52% of British voters decided that being the first country ever to leave the EU was a price worth paying for "taking back control", despite advice from economists clearly showing that Brexit would make the UK "permanently poorer" (HM Treasury 2016). 

The extent of agreement among economists on the costs of Brexit was extraordinary: forecast after forecast supported similar conclusions (which have so far proved accurate in the aftermath of the Brexit vote). Yet the publication of each one of these estimates was followed instantaneously by acerbic criticism which culminated, days before the vote, with the claim that economic experts warning about leaving the EU were like the Nazis who denounced Einstein in the 1930s (Cowburn 2016). Institutions were not immune, with the Treasury, Bank of England, IMF, OECD, and IFS receiving similar treatment. What went wrong? Were economists not ready? Were our forecasts technically poor? Were economic studies fundamentally incomplete and thus flawed? Are we to blame? This column addresses these questions. 

How come Brexit?

In the years to come, there will undoubtedly be many PhD dissertations dissecting Brexit. Economists have been blamed for it, but I don't think we even make the top three cuplrits. The three main culprits for Brexit, in my opinion, are political elites, economic elites, and the media. These are the three Cs - Cameron, the City, and coverage - with a number linked to each: 11, 17.5, and 41. 

Former prime minister David Cameron's referendum pledge was a reaction to UKIP's performance in the 2014 European Parliament election. Voter dissatisfaction with the economic policies implemented by the coalition government since 2010 meant severe losses for the two coalition parties. The Conservatives lost seven seats (of 26), while the Liberal Democrats lost 10 (of 11). UKIP gained 11 seats to become the largest UK party.  

Economic elites were complacent because they thought common sense would deliver a win for 'Remain'. Those with more at stake, like the City, did not feel the urge to back up the Remain campaign. Hence, the final fundraising total for 'Leave' was bigger than that for Remain by about £3.3 million: £17.5 against £14.2 million (Electoral Commission 2016).  Coincidentally, 17.5m was also the number of pro-Brexit votes.

My third main reason is media coverage. Levy et al. (2016) use a sample of 1,558 articles across nine major UK newspapers to show that 41% were in favour of Brexit, while only 27% were pro-Remain. These are absolute numbers, not weighted by circulation. The authors call the remaining 30% "mixed, undecided or no position".

It is absurd to blame Brexit on economists, especially in light of the three reasons above. Yet economists may have not been fully prepared. The breadth of our knowledge was inadequate. Some examples: two years ago, we were still struggling with the fragility (i.e. lack of robustness) of our estimates of the benefits of EU membership;1 two years ago, we did not have answers to key questions such as how much EU membership increase FDI flows into the UK;2 and, to this day, we have not yet seen time-series data on how the UK financial sector grew after 1973.3  

Such gaps are important because the estimates of the costs of leaving the EU are a function of the estimates of the benefits from EU membership adjusted by the size (and time profile) of the entry/exit shock. The latter can be thought of as a turning point, a structural break (Campos and Coricelli 2015), or also as varying across countries with some more capable than others of absorbing the benefits of integration. 

The bottom line is that economists cannot be listed among the main Brexit culprits. Yet gaps in knowledge may well have hindered the quality of our advice. Did this happen? In other words, how good were the ex ante estimates of the costs of Brexit? 

Looking back at the ex ante estimates of the costs of Brexit

Between the outright victory of the Conservative party in May 2015 and the Brexit vote, there was a stream of medium- and long-term forecasts. We can identify three types of estimates. 

  • Type 1 is the one showing gains: Economists for Brexit (2016 ) predict that Brexit will increase UK incomes by about 4% by 2030 (see Dhingra et al 2016b for a thorough assessment). 
  • Type 2 are older (pre-May 2015), mostly done by pro-Leave think tanks and often reporting a zero effect. 
  • Type 3 includes the vast majority of estimates, which show significant medium- and long-term losses from Brexit. To be specific, the Treasury (2016), CEP/LSE (Dhingra et al 2016a), the OECD (2016), and the NIESR (Ebell and Warren 2016) predict short-term income losses of about 3.8%, 2.6%, 3.3% and 2.3%, respectively, and long-term losses of about 6.2%, 7.5%, 5.1% and 7.8%, respectively.4 

Figure 1 Recent estimates of the long-term impact of leaving the EU on UK GDP

Source: Chadha (2016).

These are central estimates. Type 3 studies often presented three scenarios: the EEA/Norway model, the Swiss model, and WTO rules.5 Although losses from the EEA option are the smallest in per capita GDP terms, there is evidence of productivity losses of 6-9% for Norway vis-a-vis EU membership (Campos et al. 2015). Note that the heated UK 'productivity puzzle' debate is over a similar productivity loss of 6% to 9% (Yueh 2015). 

How can we differentiate between these types of estimates? I argue they differ in at least two fundamental ways: in methodological transparency, and in the quality of the assumptions. On the first count, it is abundantly clear that Type 3 studies are superior to the others. They all provide extensive details that make their estimates entirely replicable. The same can simply not be said of Types 1 and 2.6

The three groups also vary significantly regarding their underlying assumptions. An illustrative example refers to EU regulation costs, which are closely associated with the sovereignty debate. Types 2 and 3 correctly assumes these to be small. Type 1 assumes costs of regulation that are about 6% of GDP, which is large and unsupported by international evidence. Clearly, the larger the costs assigned to EU regulation, the better the Brexit option looks.   

There is something insidious about reports arguing that the long-run effect is small or zero. They not only cloud the debate, but present Brexit (and, by extension, UK membership of the EU) as immaterial, irrelevant, or even inconsequential.  

Brexit may mean Brexit, but what will "success" mean? 

If one silver lining is needed, the referendum focused our minds and pushed us to generate a lot of knowledge that we didn't have before. Almost without exception, the plethora of studies produced in the wake of Brexit will be useful for understanding, reforming, and hopefully improving the EU. A month on, the Type 3 forecasts are sadly proving accurate. We need to wait until 2030 to assess the long-run estimates, but one thing is clear: being 'out' turns those still 'in' into natural comparators or counterfactuals. In 2030, there will be less need for sophisticated counterfactual estimation because just analysing the economic performance of Germany, France, the Netherlands, Poland and the other EU members after 23 June 2016, may well be instructive enough. 

References  

Armstrong, A. and J. Portes (2016), "The Economic Consequences of Leaving the EU", National Institute Economic Review 236(1): 2-6.

Begg, I. and M. Fabian (2016) "The economic impact of Brexit: Jobs, growth and the public finances", LSE, European Institute.

Bruno, R., N. Campos, S. Estrin and T. Meng (2016), "Gravitating Towards Europe: An Econometric Analysis of the FDI Effects of EU Membership", CEP/LSE BREXIT Technical Paper 03.

Campos, N. (2014), "Even Brexit backers canít make the sums work for UK to quit EU", The Conversation, May .

Campos, N. and F. Coricelli (2015) "Why did Britain join the EU? A new insight from economic history", VoxEU.org, February.

Campos, N., F. Coricelli and L. Moretti (2014) "Economic growth and political integration: Estimating the benefits from membership in the EU using the synthetic counterfactuals method", CEPR Discussion Paper No. 9968.

Campos, N., F. Coricelli and L. Moretti (2015), "Norwegian Rhapsody? The Political Economy Benefits of Regional Integration", CEPR Discussion Paper No. 10653.

Chadha, J. (2016), "When experts agree: How to take economic advice over the referendum", VoxEU.org, June. 

Cowburn, A. (2016), "Michael Gove apologises for comparing economic experts warning against Brexit to Nazis", The Independent, 22 June.

Crafts, N. (2016), "The Growth Effects of EU Membership for the UK: a Review of the Evidence", Warwick CAGE Working Paper No. 280.

Dhingra, S., G. Ottaviano, T. Sampson and J. Van Reenen (2016a), "The consequences of Brexit for UK trade and living standards", CEP/LSE BREXIT Discussion Paper 02.

Dhingra, S., G. Ottaviano, T. Sampson and J. Van Reenen (2016b), "'ECONOMISTS FOR BREXIT': A critique", CEP/LSE BREXIT Discussion Paper 06.

Ebell, M. and J. Warren (2016), "The long-term economic impact of leaving the EU", National Institute Economic Review 236(1): 121-138.

Electoral Commission (2016), "Donations and loans received by campaigners in the European Union Referendum: Fourth pre-poll report: 10 June 2016 to 22 June 2016", London: Electoral Commission, July.

Levy, D., B. Aslan and D. Bironzo (2016), "The press and the Referendum campaign", in D. Jackson, E. Thorsen and D. Wring (eds), EU Referendum Analysis 2016: Media, Voters and the Campaign, Political Studies Association.

OECD (2016), The Economic Consequences of Brexit: A Taxing Decision, Paris.

OíRourke, K. (2016), "Deserting the Battle for Britain", Project Syndicate, July.

Prat, A. (2015), "Media Capture and Media Power", in S. Anderson, J. Waldfogel and D. Stromberg (eds.), Handbook of Media Economics, Vol. 1b, North Holland.

HM Treasury (2016), HM Treasury Analysis: The long-term economic impact of EU membership and the alternatives, London.  

Yueh, L. (2015), "The useful Trojan horse of the UK productivity puzzle", VoxEU.org, November.

Wren-Lewis, S. (2016), "Just how bad will Brexit be, and can it be undone?", mainly macro, 25 June

Endnotes

1 Crafts (2016) and Campos et al. (2014) review this literature. Campos et al. argue that the body of evidence is large and convincing for the Single Market and the euro, but ìdisappointingly thinî for EU membership. They note that the vast majority of available estimates are deemed ìnot robustî by their authors, who point to country heterogeneity as the main possible reason. Campos et al. try to address country heterogeneity concerns by estimating the benefits from EU membership on a comparable country-by-country basis using synthetic counterfactuals. 

2 Bruno et al. (2016) make this point.  They find that EU membership increase FDI inflows by an average of 28%, with the estimate for the UK being significantly above average. 

3 Another key gap is the role of media. Wren-Lewis (2016) has been one of the few drawing attention to this issue. OíRourke (2016) noted that, ì[t]he question, then, is why the Irish havenít developed UK levels of animosity toward EU immigrantsÖ Surely, the British media bear considerable responsibility for the difference. Ireland has nothing like the mendacious, jingoistic gutter press that thrives in the UKî.  Despite a substantial body of economic work in this area (Prat 2015), there remains a worrisome lack of research on the UK. 

4 See Armstrong and Portes (2016) and Begg and Mushˆvel (2016) for overviews of these estimates. It is worth mentioning that estimates are deliberately conservative (for instance, none allowed for the possibility of EU reforms). This is because the choice of underlying assumptions more often than not was driven by the ëurge for balanceí as well as a response to incomprehensible personal attacks.

5 In the Treasury report, the smallest losses are for the EEA option: it yields a 3.8% GDP loss by 2030; The Swiss is the intermediary case and amounts to a reduction of 6.2%, while the WTO option is estimated at a 7.5% loss.

6 In Campos (2014), I show how much inadequate assumptions and lack of methodological rigor affect the estimates reported by one of the first Brexit studies.  

a

A

Topics:  Europe's nations and regions Frontiers of economic research

Tags:  Brexit, forecasts

Professor of Economics, Brunel University London; Research Professor at ETH-Zürich; and Research Fellow, IZA-Bonn

Events

CEPR Policy Research