VoxEU Column Macroeconomic policy Monetary Policy

Dynamic stochastic general equilibrium models and their forecasts

Studies have shown that the forecasts from dynamic stochastic general equilibrium models perform better than central banks' judgemental forecasts as well as forecasts based on statistical analysis but without a theoretical foundation. This column shows that performing better is hardly good performance given how badly all three forecasts compare with reality.

 

Dynamic stochastic general equilibrium (DSGE) models represent a major strand of the modern macroeconomics literature and are an important tool for policy analysis at central banks. This was not always the case. At their inception DSGE models were built as descriptive devices, able to provide internally consistent and Lucas-critique proof answers to counterfactual policy experiments. The canonical three-equation new Keynesian model was extensively used in the academic literature but carried much less influence at central banks, where its limited ability to quantitatively explain macroeconomic fluctuations left it viewed as far too simple to be used in policymaking.

The limited applicability of DSGE models to the analyses undertaken at central banks changed dramatically with two milestone papers.

  • First, Christiano et al. (2005) developed a DSGE model with a much richer structure – called a medium-scale model – that fit the monetary policy impulse-responses particularly well when estimated with minimum distance methods.
  • Second, Smets and Wouters (2007) estimated a variant of the cumulative effects of error model using Bayesian methods and documented the “good” forecasting ability of this model, where good meant competitive with atheoretical Bayesian VAR forecasts.

The theoretical rigour of DSGE models, combined with their documented connection to the data, made them very appealing tools for central-bank analysis and it was not long after these papers that similar models began to be developed and employed for policy analysis and forecasting at central banks. The forecasting performance of central bank DSGE models remained an important concern and research comparing the model forecasts to purely statistical forecasts or central banks’ official forecasts consistently documented the competitive and sometimes even superior forecasting performance of these models.1The success of the DSGE model-based forecasts relative to other methods was viewed as evidence in favour of DSGE models’ reliably capturing the dynamics in the data.

As part of our ongoing research (Edge and Gürkaynak 2010), we study the DSGE model forecast performance in detail, using out of sample forecasts with real time US data. Our finding is that – consistent with the earlier research – the model performs comparably to or better than a statistical forecast (a Bayesian VAR) and the Fed’s judgemental Greenbook forecasts. Figure 1 shows, for inflation, the root mean square forecast errors of the model relative to alternative forecasting methods for different horizons, where observations below one mean the model has a lower root mean square error (RMSE).

Figure 1. DSGE inflation forecast relative RMSE

This figure hides as much as it reveals. In particular, relative forecast performance does not distinguish between comparing good forecasts to each other and comparing poor forecasts to each other. To see the absolute forecasting ability of the DSGE model, we run a series of standard forecast efficiency tests, where the realised inflation is regressed on forecasts made at different times in the past. A good forecast should have a zero intercept and unit slope as well as a high R-squared. Table 1 shows the efficiency tests for DSGE model forecasts of inflation at different maturities and demonstrates clearly that the forecasts are very poor. R-squareds at all horizons are essentially zero, implying no forecasting ability. All Figure 1 is therefore telling us is that all other forecasting methods perform just as poorly.2

Table 1. DSGE model inflation forecast accuracy

 

 

 

 

1Q Ahead

2Q Ahead

3Q Ahead

4Q Ahead

5Q Ahead

6Q Ahead

Slope

0.451**

0.089

0.031

0.209

0.167

0.134

 

(0.108)

(0.149)

(0.250)

(0.261)

(0.216)

(0.174)

Intercept

0.261**

0.421**

0.446**

0.363**

0.386**

0.398**

 

(0.051)

(0.082)

(0.122)

(0.128)

(0.112)

(0.112)

R2

0.13

0.00

0.00

0.02

0.01

0.01

Obs

104

104

104

104

104

104

Note: Dependent variable is realized inflation, independent variables are inflation forecasts of the DSGE model.

 

 

 

 

 

 

 

 

 

What do we learn from this? That over the post-1992 time-period, which we use to evaluate the DSGE model, inflation has been essentially unforecastable. (Other papers use similar sample periods.) This is in line with the finding of Stock and Watson (2007), who showed that inflation since the Great Moderation is characterised by a diminished persistent (forecastable) component and a larger transitory (unforecastable) component.

But what does that say about the use of DSGE models in central banks; both as a tool for policy analysis as well as a tool for forecast generation? We argue that in both cases the answer is “nothing”. The finding that inflation is not forecastable over the Great Moderation period is consistent with the predictions of the DSGE model given the strong monetary policy rule estimated for this period. Specifically, since under this rule the policymaker will alter the interest rate to counter forecastable deviations of inflation from the target, the rule will eliminate forecastable movements in inflation and leave only unforecastable shocks to drive fluctuations. Thus, our finding of low forecast performance is not necessarily evidence against the validity of the model.

That said, a model in which inflation and the output gap are uncorrelated statistical processes that are independent both of each other and the interest rate, will also imply unforecastable model variables, which is why we argue that forecasting ability for inflation in the Great Moderation period is not a useful test of the validity of the DSGE model. Indeed, in a regime in which most key macroeconomic variables are either unforecastable or close to unforecastable relative forecast accuracy is far less relevant for evaluating the usefulness of forecasts and the criteria for judging usefulness becomes more subtle.

The interesting question is whether the conditional forecasts of the DSGE model are sensible. That is, whether the model can reasonably accurately answer questions along the lines of “what would inflation do if the interest rates were kept constant for a year?” It is entirely possible that the model will predict the path of inflation under the counterfactual policy path quite well, while having a poor unconditional forecast record as it is the internal dynamics that imply unforecastable inflation.

To conclude, then, we think the debate on the usefulness of DSGE models in the forecasting process is ill served by using its forecasting performance in the Great Moderation period as a test. We have shown that the model forecasts inflation very poorly, which we have argued is consistent with the baseline New Keynesian DSGE model, but is also consistent with many other models.

References

Adolfson, Malin, Michael K Andersson, Jesper Lindé, Mattias Villani, and Anders Vredin (2007), “Modern Forecasting Models in Action: Improving Macroeconomic Analyses at Central Banks”, International Journal of Central Banking 3(4):111-144.

Christiano, Lawrence J, Martin Eichenbaum, and Charles L Evans (2005), “Nominal Rigidities and the Dynamic Effects of a Shock to Monetary Policy”, Journal of Political Economy,113, 1:1-45.

Christoffel, Kai, Günter Coenen, and Anders Warne (forthcoming), “Forecasting with DSGE Models”, in M Clements and D Hendry, Handbook of Forecasting, Oxford University Press.

Edge, Rochelle M, Michael T Kiley, and Jean-Philippe Laforte (2010), “A Comparison of Forecast Performance between Federal Reserve Staff Forecasts, Simple Reduced-Form Models, and a DSGE Model”, Journal of Applied Econometrics, 25:720-754.

Lees, Kirdan, Troy Matheson, and Christie Smith (2007), “Open Economy DSGE-VAR Forecasting and Policy Analysis: Head to Head with the RBNZ Published Forecasts”, Discussion Paper 2007/01, Reserve Bank of New Zealand.

Smets, Frank, and Raf Wouters (2007), “Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach”, American Economic Review 97(3):586-606.

Stock, James H, and Mark W Watson (2007), “Why Has US Inflation Become Harder to Forecast?”, Journal of Money, Credit and Banking, 39(1):3-33.


1 Examples include, Adolfson et al. (2007) for the Riksbank’s DSGE model, Lees et al. (2007) for the RBNZ’s DSGE model and Edge et al. (2010) for the FRB’s DSGE model. In addition, Adolfson and others (2006) and Christoffel et al. (forthcoming) examine out-of-sample forecast performance for DSGE models of the Eurozone, although the focus of these papers is much more on technical aspects of model evaluation.

 

2 Results for real GDP growth, reported in our paper, are similar.

 

 

 

2,625 Reads