Matthieu Chavaz, Jeremy Chiu and Evarist Stoja.
How might banks fare in stressful macroeconomic conditions? Are they strong enough to withstand the stress and survive or will they fall like dominoes? Stress tests offer insights into such questions.
Regulators are not only making a growing usage of such tests, they are also increasingly inclined to communicate openly about them. This is a remarkable evolution. Throughout history, regulators have typically followed some form of Hippocratic Oath and refrained from disclosing their detailed diagnostics of individual banks’ health. Regulators are now increasingly keen to release both the “answer” to stress tests (the results) and the “question” – the stress scenario regulators confront banks with. This column suggests that the disclosure of the scenario can be as important as – if not more than – the disclosure of the results.
Unsurprisingly, the paradigm shift towards greater transparency around stress tests has inspired a host of scholarly studies whose focus has been on understanding whether markets react to the release of stress test results (see e.g. Flannery, Hirtle and Kovner, Morgan and others, Bischof and Daske and Petrella and Resti, among others). The expectation is that they should. Stress test results allow markets to evaluate and compare banks’ strength, an otherwise difficult task given the complexity and opacity of banks’ balance sheets.
These studies differ by the case they analyse (typically Fed or EBA test results disclosures) and, in part, by the conclusion they reach. But they share one common feature: they concentrate on the market impact of the “answer” to stress tests (the results) while ignoring potential reactions to the release of the test’s “question” (the stress scenario).
This is not innocuous: the publication of stress scenarios typically predates that of the results. There are good reasons to think that markets may also pay attention to this initial disclosure. Crucially, the scenario reveals the nature and severity of turbulence authorities expect banks to withstand. Authorities often also disclose the test’s “grading scheme” (the hurdle rate of capital banks must maintain in the scenario to “pass” the test) on this occasion. Combined with a basic knowledge of banks’ business models, this information may enable markets to anticipate banks’ test performance and the possibility that they may need to raise capital on the back of it. For instance, markets could anticipate that a China-hard landing scenario (as in the 2015 Bank of England test) may impact the Asia-focused UK banks more than a housing-bust scenario (as in the 2014 test).
This column provides consistent evidence by analysing the market reaction to the three publications – two scenarios and one set of results – to come out of the Bank of England’s new yearly “concurrent” stress-testing framework so far. This is an admittedly low number of observations. Our results should thus be taken with due caution. Taken literally however, they suggest that the disclosure of the scenario can be as important as – if not more than – the disclosure of results.
Our study
We apply a standard event study to examine the UK stock market’s reaction to the disclosure of stress test (i) scenarios (in 2014 and 2015) and (ii) results (in 2014) by the Bank of England. An event study measures the impact of an ‘event’ on the price of a stock. The event may be ‘value-relevant’ (such as an earnings announcement) or ‘value-irrelevant’ (such as a stock split). The event may be anticipated (such as a dividend payment following a dividend announcement) or unanticipated (such as a profit warning). Only unanticipated, value-relevant events should affect a firm’s stock price.
We analyse reactions along two dimensions. First, we look at changes in the market valuation of bank equity measured by abnormal returns (returns above and beyond those expected under normal market activity and movement, i.e. residual returns after stripping out market returns) following the disclosures. Second, we look at changes in the banks’ systematic risk as measured by bank equity’s betas.
Beta, computed as the ratio of the comovement of a bank’s returns with the market returns over the volatility of the market returns, measures the sensitivity of a bank’s returns to the market. For a change in beta to occur around the stress testing event, it must be that a bank’s returns are more sensitive to movements in the market than they were previously, suggesting that the stress testing event had an impact on the market’s perception of the systematic risk for the bank in question.
The betas are measured in the two months before and after the event. Our choice of a two-month window reflects our effort not only to preserve statistical precision by including enough observations, but also to avoid any contamination by other effects which had nothing to do with the stress testing event if we include too many observations. Nevertheless, we experimented with different windows and found qualitatively similar results.
Our main results are twofold.
First, we find that the market reacts to the publication of scenarios, suggesting that they do contain useful information. In fact, we find that the market reacts more strongly to the publication of scenarios than to the publication of results. While market valuation of banks increases following the 2014 and 2015 scenario releases, we find no clearly discernible reaction to the 2014 results. While some domestically-focused banks register negative abnormal returns following the release of scenarios and results, these returns are not significant.
Second, the change in the market perception of individual banks’ systematic risk is also stronger after the publication of the scenario than after that of results. Importantly, this impact differs across individual banks: banks whose business model seems more sensitive to the scenario register a higher increase in systematic risk. This can be seen in Charts 1-3 below, where the red bars correspond to the pre-event betas and blue ones correspond to the post-event betas. Following the release of the 2014 UK housing-bust scenario, domestically-focused banks (such as Barclays, RBS and LBG) witness a higher rise in systematic risk. In contrast, Asia-focused banks (such as HSBC and Standard Chartered (SCB)) drive the increase in systematic risk following the disclosure of the China’s hard-landing scenario in 2015. One interpretation is that the market considers that banks which are more affected by a particular scenario may be more dependent on real economic activity, because for instance they may have to raise equity on the back of the test.
Interpretations and lessons
This study is clearly based on a sample too small to draw firm lessons for policy-making. However, it suggests a number of conjectures and future research questions around stress test communication.
So, why do markets react more to the publication of scenarios than to results? Our initial conjecture is that markets can anticipate results based on the scenario and a high-level knowledge of banks’ business models. A related conjecture is that test results are in line with markets’ understanding of banks’ vulnerabilities, implying that the results are not surprising. Yet another possibility is that markets have not reacted to the publication of results simply because strongly negative results have been absent so far.
Either way, a cynical and hasty reader may be tempted to conclude that the Bank of England should spare itself the burden of publishing stress test results altogether.
This would be clearly misguided. Markets’ receptions of stress test results may change over time as familiarity with the framework increases. More importantly, informing markets is only one of the many potential benefits of being transparent about stress tests. Another key goal is to make regulators accountable to governments and the public. This requires not only informing about the degree of stress which regulators expect banks to withstand (the scenario), but also about how they scored banks against this criterion (the result).
Holding regulators to account on both these dimensions can help to maintain the value of stress tests, both to markets and the broader public. Measuring the broader benefits of stress test transparency beyond the contribution of new information to market participants is an empirically much more daunting task. However, this may be a necessary step for a truly comprehensive assessment of the benefits of transparency relating to all facets of stress-testing – from the question to the answer.
Matthieu Chavaz and Jeremy Chiu work in the Bank’s Stress Testing Strategy Division and Evarist Stoja works at the University of Bristol and is a Bank of England consultant.
If you want to get in touch, please email us at bankunderground@bankofengland.co.uk
Bank Underground is a blog for Bank of England staff to share views that challenge – or support – prevailing policy orthodoxies. The views expressed here are those of the authors, and are not necessarily those of the Bank of England, or its policy committees.
