The reliability of public debt forecasts

Julia Estefania Flores, Davide Furceri, Siddharth Kothari, Jonathan D. Ostry 28 May 2021

a

A

The Covid-19 pandemic led to a steep increase in sovereign debt ratios in 2020, as governments around the world implemented unprecedented fiscal stimulus packages amidst the greatest drop in economic activity since the Great Depression (Baldwin and Weder di Mauro 2020, Baker et al. 2020). Current projections point to debt ratios stabilising quickly and then declining in the medium term (Figure 1). Alongside record-low borrowing costs, this is providing some reassurance regarding debt sustainability and servicing capacity, suggesting that debt vulnerabilities will remain contained.1 

Figure 1 Debt-to-GDP ratio

Notes: The chart shows the simple average across countries for historical and projected debt-to-GDP ratio for advanced economies and emerging and developing economies as reported in the October 2020 IMF forecasts. The solid line shows historical data while the dashed lines show projections.

But will these relatively benign debt projections turn out to be accurate? If debt projections are optimistic and fiscal space narrows, market confidence may falter leading to abrupt changes in debt financing costs (Ostry et al. 2010, Ghosh et al. 2013, Chamon and Ostry 2021) and potentially triggering catastrophic debt crises (Stiglitz and Rashin 2020). Thus, accurate forecasts are an essential foundation for assessing vulnerabilities and for constructing robust fiscal strategies consistent with debt sustainability (Debrun et al. 2019). Robustness is even more important when debt is already very high, as it is today.

Our paper assembles a unique dataset of short-term and medium-term debt projections to assess the accuracy of past public debt forecasts. We find significant positive forecast errors in debt projections, with realised debt ratios at the five-year horizon being about 10% of GDP higher than forecasts. Our results suggest that current forecasts may be underestimating debt vulnerabilities.

Public debt forecast errors 

A large literature has assessed the accuracy of growth and inflation projections, often finding significant optimism bias in forecasts (Ho and Mauro 2016, Loungani 2001). The literature on the accuracy of fiscal forecasts is smaller, often focusing on budget balance (as opposed to debt), and with narrow country coverage, usually limited to European countries (Jonung and Larch 2006, Beetsma et al. 2013) or a small group of emerging markets developing economies (EMDEs) (Hadzi-Vaskov et al. 2021). 

In this column, in contrast, we focus squarely on the debt ratio. We construct a comprehensive dataset of short-term and medium-term debt forecasts made by the IMF for the period 1995–2020 and the Economist Intelligence Unit (EIU) for the period 2007–2020, covering an unbalanced panel of 174 countries. The wide country and time coverage allow us to assess how forecast errors depend on country characteristics and how they change over time, including during periods of crises.

As shown in Figure 2, we find significant positive forecast errors in public debt projections, with the magnitude of bias increasing with the forecast horizon.2 On average, realised debt ratios at the five-year horizon are 8.7% higher than IMF forecast and over 11% above forecasts for the EIU.3

Figure 2 Debt-to-GDP ratio forecast errors across horizons, IMF and EIU data

a) IMF data

b) Economist Intelligence Unit data

Notes: The charts show the interquartile range (shaded gray region), mean (black dot), and median (black line in the shaded box) for debt-to-GDP forecast errors for different time horizons. Forecast errors are defined as realised debt ratios minus forecasted ratios. Stars on top of the shaded regions indicates whether the mean in significantly different from zero based on standard errors that are clustered two-way at the country and vintage level. *, **, and *** indicate significance at the 10%, 5%, and 1% level, respectively.

Interestingly, on average, forecast errors are very similar between advanced economies and emerging market developing economies (Figure 3). But these averages mask important differences in terms of timing of forecast errors during recession and normal times (Figure 4). Advanced economies have very large forecast errors when recessions fall in the horizon (over 20% of GDP) – potentially reflecting the large discretionary stimulus that these countries can implement in response to a crisis – but close to zero and not statistically significant forecast errors in non-recession years. In contrast, for emerging market developing economies, forecast errors are more systematic, with a sizeable and statistically significant bias observed even when no recession falls in the forecast horizon. 

Figure 3 Forecast errors for advanced economies vs. emerging market developing economies

a) Advanced economies

b) Emerging market developing economies

Notes: The charts show the interquartile range (shaded gray region), mean (black dot), and median (black line in the shaded box) for debt-to-GDP forecast errors for different time horizons for advanced economies (Panel A) and emerging market developing economies (Panel B). Forecast errors are defined as realised debt ratios minus forecasted ratios.

Figure 4 Five-year forecast errors conditional on a recession falling in the forecast horizon

Notes: Based on a pooled regression with observations at country-vintage level. Dependent variable in each regression is five-year ahead forecast error of country ‘c’ in vintage ‘v’. Independent variable is a dummy which takes value one if the start of a recession occurs in country ‘c’ in the five-year forecast horizon. Any country-year with negative GDP growth is classified as a recession. Standard errors are clustered two-way at country and vintage level. *, **, and *** indicate significance at the 10%, 5%, and 1% level respectively.

We also provide additional stylised facts regarding public debt forecast errors: (i) forecast errors are significantly larger when the projection is for debt ratios to decline (as is the case currently) than when debt ratios are projected to increase; (ii) in oil-exporters and more volatile countries, as positive errors following adverse shocks do not get offset by negative errors after favourable shocks; (iii) the magnitude of forecast errors is similar between countries with IMF programmes and countries without, though precautionary programs tend to have smaller errors; and (iv) the positive forecast errors in debt projections are only partly explained by overoptimistic growth forecasts—that is, we observe sizeable debt forecast errors (about 4% of GDP, on average) even when conditioning on growth projections to be exactly right.

Possible implications for post-Covid debt trajectories

What can past forecast errors tell us about the possible path for debt ratios following the Covid-19 crisis? To answer this question, we use the Global Crisis as a proxy for a large shock and construct counterfactual debt paths, assuming the post-Global Crisis forecast errors materialise again. Of course, there remains considerable uncertainty regarding the medium-term effects of the pandemic; and the Global Crisis, where financial stresses resulted in a persistent decline in economic activity, may not be an ideal comparator for the current crisis. However, we view this exercise as a useful illustrative benchmark to get a sense of possible debt trajectories in the future. As shown in Figure 5, the average debt-to-GDP ratio may continue increasing in the medium-term instead of declining as per current projections, with debt ratios in emerging market developing economies potentially approaching 73% of GDP in 2025 (instead of declining to 63% of GDP). 

Figure 5 Correcting IMF projections for Global Crisis level forecast errors: Emerging market developing economies vs advanced economies

a) Advanced economies

b) Emerging market developing economies

Notes: The charts compare the forecast for debt-to-GDP as reported in the second vintage of 2020 (i.e. the first vintage after the Covid-19 shock) to a counterfactual path if forecast errors for countries were as large as they were during the first vintage of 2009 (i.e. the first vintage after the Global Crisis shock). Panel A is for the sample of advanced economies only, while the Panel B is for emerging market developing economies only. IMF data is used.

Concludinf remarks

The IMF and other commentators have rightly called attention to debt vulnerabilities in emerging market developing economies in the aftermath of the Covid-19 crisis (IMF 2020). Our results underscore the salience of calls by the international community to accelerate efforts to tackle debt vulnerabilities in many low-income economies that have been hit particularly hard by the Covid-19 crisis, have little policy space to respond, and will require financial assistance for the foreseeable future. Our results also highlight the importance of continuous efforts to improve the realism of debt projections (IMF 2021).

References

Baldwin, R and B Weder di Mauro (2020), Mitigating the COVID economic crisis: Act fast and do whatever it takes, CEPR Press.

Baker, S, N Bloom, S Davis and S Terry (2020), “COVID-induced economic uncertainty and its consequences”, VoxEU.org, 13 April.

Beetsma, R, B Bluhm, M Giuliodori and P Wierts (2013), "From Budgetary Forecasts to Ex Post Fiscal Data: Exploring the Evolution of Fiscal Forecast Errors in the EU", Contemporary Economic Policy 31(4): 795-813.

Blanchard, O J (2019), “Public Debt and Low Interest Rates”, American Economic Review 109(4): 1197–1229.

Chamon, M and J D Ostry (2021), “A Future with High Public Debt: Low-for-Long Is Not Low Forever”, IMF Blog, 20 April.

Debrun, X, J D Ostry, T Willems and C Wyplosz (2019), “Public Debt Sustainability”, CEPR Discussion Paper 14010.

Ghosh, A, J Kim, E Mendoza, J D Ostry and M Qureshi (2013), "Fiscal Fatigue, Fiscal Space and Debt Sustainability in Advanced Economies", Economic Journal 123(566): 4-30.

Hadzi-Vaskov, M, L A Ricci, A M Werner and R Zamarripa (2021), “Authorities’ Fiscal Forecast in Latin America: Are They Optimistic?”, IMF Working Paper.

Ho, G and P Mauro (2016), "Growth—Now and Forever?", IMF Economic Review 64(3): 526-547, August.

Jonung, L and M Larch (2006), “Improving fiscal policy in the EU: the case for independent forecasts”, Economic Policy 21(47): 492-534.

IMF (2020), Fiscal Monitor, October.

IMF (2021), “Review of the Debt Sustainability Framework for Market Access Countries”.

Loungani, P (2001), “How accurate are private sector forecasts? Cross-country evidence from consensus forecasts of output growth”, International Journal of Forecasting 17(3): 419-432.

Ostry, J D, A Ghosh, J Kim and M Qureshi (2010), "Fiscal Space", IMF Staff Position Note 2010/11.

Presbitero, A and U Wiriadinata (2020), “The risks of high public debt despite a low interest rate environment”, VoxEU.org, 05 August.

Stiglitz, J and H Rashid (2020), “How to prevent the looming sovereign debt crisis”, VoxEU.org, 03 August.

Wyplosz, C (2019), “Olivier in Wonderland”, VoxEU.org, 17 June.

Endnotes

1 Blanchard (2019) triggered an extensive debate on the welfare costs of deficits and debt in a low interest rate environment. As discussed in Wyplosz (2019) and Presbitero and Wiriadinata (2020), high debt levels can be associated with significant risks even if real interest rates are lower than growth rates, as the interest-growth differential is quite volatile and can reverse quickly.

2 Following the convention in the forecasting literature, we define forecast errors as the realised minus the forecasted debt ratio. Thus, a positive forecast error implies that debt ratios turned out to be higher than forecast.

3 Much of the difference in IMF and EIU forecast errors is due to differences in sample, with the average bias being very similar for a balanced sample for which data is available in both sources.

a

A

Topics:  Macroeconomic policy

Tags:  debt ratio, forecast errors, Stabilisation policy, public debt

Research Analyst, Asia and Pacific Department, IMF

Deputy Division Chief in the Research Department, IMF

Economist, IMF

Deputy Director, Research Department, IMF

Events

CEPR Policy Research