The capital ratio of a bank1 is usually defined as the ratio of a measure of its equity to a measure of its assets. Regulatory capital ratio usually employs book value of equity and risk-weighted assets, where individual asset holdings are multiplied by corresponding regulatory ‘risk weights’. Macroprudential stress tests rely on models that translate an adverse macroeconomic scenario into losses to assets on the balance sheet of banks. These losses are assumed to be first borne by equity. The resulting capital ratios determine which banks fail the test under the stress scenario, and what supervisory or recapitalisation actions are undertaken to address this failure.
Recent concerns on the denominator of capital ratios – the risk-weighted assets – have been expressed in multiple surveys that point out the inconsistency in the calibration of risk weights (Le Lesle and Avramova 2012, Mariathasan and Merrouche 2013, BCBS 2013, Haldane 2012). This column argues that the inadequacy of risk-weighted assets is also responsible for producing an inadequate ranking of the required capitalisation of banks in stress tests. In order to establish the inadequacy of risk-weighted assets, Acharya et al. (2013) examine complementary approaches to measuring capital ratios and relate them to capital ratios based on risk-weighted assets.
A complementary approach to stress testing and risk-weighting
An alternative approach for measuring the financial performance of a firm under stress is presented in Acharya et al. (2012). The proposed measure (SRISK) represents the expected amount of capital an institution would need to raise in a crisis to restore a target capital ratio. The crisis, or stress scenario, is defined by a 40% drop in the market equity index over six months. In these market conditions, SRISK is based on the assumption that the book value of the debt of the bank will remain constant, while its market capitalisation will decrease by its six-month return in a crisis, estimated from the NYU V-lab methodology.2 All input data are publicly available. This estimate of the expected capital shortfall of a financial firm in a crisis provides a public benchmark for stress tests that is weekly updated on the V-lab website.
By adjusting the market capitalisation of a firm by its expected six-month return in a market downturn, SRISK implicitly incorporates a measure of asset risk of the firm. Firms, whose market capitalisation is predicted to shrink the most in a crisis, are the riskiest. This way, we can back out an implied risk-weight from SRISK, one that is calculated in a top-down manner at the level of the entire firm rather than bottom-up (i.e., asset by asset) as in the Basel risk-weighted approach. This V-lab risk weight can be compared with the average regulatory risk weight of a firm estimated from the Basel methodology.
Comparing regulatory risk weight and V-lab risk weight
Figure 1 compares the V-lab risk weight with the projected Basel risk weight at the end of the stress scenario of the test conducted by the European Banking Authority (EBA) and disclosed on July 15, 2011. These measures of risk have nothing in common; the rank correlation is negative (-0.238) and not significant at 5%.
Figure 1. Stress test risk weight vs. V-lab risk weight (at the end of the EBA 2011 stress scenario).
In the US too, the V-lab risk weight appears uncorrelated with some approximation of the stressed risk weight of the 2009 stress test; the rank correlation is slightly negative (-0.011) and insignificant.
Identifying vulnerable banks in a real period of stress
If the goal of a macro-prudential stress test is to make banks more robust to aggregate stress conditions, we would expect stress test outcomes to identify bank vulnerabilities when there is realised aggregate stress. In other words, comparing stress test outcomes to realised outcomes in a crisis can help determine whether the stress test scenario was credible, as well as help identify other deficiencies of the stress test that would prevent it from detecting banks with the most obvious vulnerabilities.
The EBA 2011 stress test was followed by a global economic downturn; the realised returns in the last six months of 2011 of US (S&P500), European (Eurostoxx50), and global (MSCI ACWI World) indices were respectively -4.89%, -20.67%, and -13.47%. This outcome was less severe than the V-lab scenario (40% decline in the World equity index) and was closer to the EBA scenario (15% decline in stock prices in the Eurozone). The ranking of bank vulnerability in the scenarios should, however, be closely related even if the magnitude of the vulnerability is greater in the more adverse V-lab scenario.
Stress test risk estimates vs realised volatility
High risk banks would be expected to have highly volatile stock-market returns in a realised crisis. Comparing the ranking of the six-month realised volatility of European banks’ stock returns during this period to the ranking of EBA risk measures and V-lab risk measures, we find a negative correlation (-0.140) with the EBA risk weight, whereas the correlation with the V-lab risk weight (0.535) is positive and significant at 1%. Dexia and Crédit Agricole were among the riskiest banks according to the V-lab measure, and among the safest with the EBA risk weights (see Figure 1); both banks have values above the 75% quartile of the V-lab risk weight distribution while both appear below the 25% quartile of the EBA risk weight distribution. The EBA risk ranking is hard to rationalise given that three months after disclosure of the stress test, Dexia was the first bank to be bailed out in the context of the European sovereign crisis in October 2011. The bank was bailed out a second time in November 2012 and reported a net loss of €2.9 billion for 2012. Crédit Agricole also announced a net loss of €6.5 billion for 2012.
Stress test loss estimates, V-lab loss estimates, and realised losses
The comparison of stress test losses with the market capitalisation loss of V-lab highlights differences in severity levels between European and US stress tests. V-lab appears always to be more severe than stress test outcomes, but this contrast appears extreme for Europe in Table 1, where the sum of projected net losses is more than 10 times larger under the V-lab scenario than the regulatory stress test in 2010, and almost 6 times larger in 2011.
Furthermore, the ranking of projected net losses of different financial firms in the US stress tests is highly and significantly correlated with the ranking of V-lab market losses. In Europe, this correlation is negative as many large banks were projected to make positive profits under the stress scenario (due to a weak stress scenario and mild assumptions on projected revenues).
Table 1. V-lab market cap losses vs. stress test projected losses
Notes: V-lab download date: 12/31/2008 (US 2009), 09/30/2011 (US 2012), 09/28/2012 (US 2013), 12/31/2009 (EU 2010), 12/31/2010 (EU 2011). The symbol * indicates statistical significance at the 5% level; ** at the 1%. Sample size: 18 (US 2009 and 2012), 17 (US 2013), 50 (EU 2010), 53 (EU 2011).
In Table 2, the correlation of the 2011 European stress test losses of different financial firms with their realised losses during the six-month period following the stress test disclosure is also negative. The realised market cap loss of 53 EU banks during this period was €354 billion, which is close to the V-lab estimate of €402 billion.
Table 2. Forecasting during the European sovereign debt crisis
Notes: V-lab output was downloaded before the disclosure date of the EBA stress test: 06/30/2011. Sample size: 53.
Risk weight-based capital ratio vs. simple leverage ratio
To illustrate further that the misallocation of capital shortfalls across banks is due to the reliance on regulatory risk weights in determining the capital requirements, consider the capital shortfalls under another alternative, the simple leverage ratio, defined as the ratio of book equity to total (un-weighted) assets. For all stress tests, the rank correlation between V-lab and stress test outcomes increases considerably when the denominator of capital ratios, the risk-weighted assets, is replaced by total assets (defining a Tier 1 leverage ratio).
The next two figures show how the rank correlation between V-lab SRISK and the capital shortfall of the EBA stress test rotates from highly negative (-0.790) to highly positive (0.679) when the EBA shortfall is written as a function of total assets (Figure 3) instead of risk-weighted assets (Figure 2). With this definition, the required capitalisation of 53 EU banks would have increased from €1.2 billion to €390 billion. Dexia would have been forced to raise €9.5 billion, and Crédit Agricole €35.8 billion in July 2011 based on the leverage-based capital shortfall.
Figure 2. Stress test risk-based capital shortfalls vs. SRISK (EUR millions). V-lab download date: 12/31/2010.
Figure 3. Stress test leverage-based capital shortfalls vs. SRISK (EUR millions). V-lab download date: 12/31/2010.
Overall, the findings suggest that stress tests would be more effective if capital requirements were measured differently. A risk-based capital requirement is not sufficient because:
- The increase of risk over time is not captured adequately with static risk weights; and
- Risk weights are flawed measures of bank risks cross-sectionally, as banks game their risk-weighted assets (cherry-pick on risky but low risk-weight assets) to meet regulatory capital requirements without necessarily reducing economic leverage.
Empirical evidence that European banks took advantage of regulatory risk-weights by concentrating on zero-risk weight sovereign debt exposures of the southern European periphery can be found in Acharya and Steffen (2013).
We recommend that regulatory stress tests complement their assessment of bank and system risks by using the simple leverage ratio-based and market-based measures of risk. We welcome the new Basel III Tier 1 leverage ratio, but this has not yet been incorporated into the European stress test design. The mis-measurement of bank risks is likely to be present in future stress tests as long as the reliance on static regulatory risk weights prevails. In particular, the upcoming Asset Quality Review (AQR) to be conducted by the European Central Bank as the Single Supervisory Mechanism of the Eurozone Banking Union will involve a stress test to assess capital shortfalls. The regulators conducting this exercise should be wary of relying only on the current scheme of static risk weights, and instead, also employ the simple leverage ratio and market-based capital shortfall measures, such as SRISK.
Acharya V, R Engle and M Richardson (2012), “Capital shortfall: a new approach to rankings and regulating systemic risks”, American Economic Review Papers and Proceedings 102:3, 59–64.
Acharya V, R Engle and D Pierret (2013), “Testing Macroprudential Stress Tests: The Risk of Regulatory Risk Weights”, NYU Working Paper.
Acharya, V and S Steffen (2013), “The "greatest" carry trade ever? Understanding Eurozone bank risks”, NYU Working Paper.
Basel Committee on Banking Supervision, January (2013), “Regulatory Consistency Assessment programme (RCAP) - Analysis of risk-weighted assets for market risk”, Bank for International Settlements.
Haldane, A (2012), “The dog and the Frisbee”, Bank of England speech, August 31, 2012.
Hirtle B, T Schuermann and K Stiroh (2009), “Macroprudential supervision of financial institutions: Lessons from the SCAP”, Federal Reserve Bank of New York Staff Report No. 409.
Le Lesle, V and S Avramova (2012), “Revisiting risk-weighted assets: Why do RWAs differ across countries and what can be done about it?”, IMF Working Paper WP/12/90.
Mariathasan, M and O Merrouche (2013), “The manipulation of Basel risk-weights”, CEPR Discussion Paper DP9494, May.
1 We use the terms ‘bank’, ‘firm’, ‘financial firm’, etc., interchangeably throughout.