Tell me why! Looking under the bonnet of machine learning models

Carsten Jung and Philippe Bracke

Whether in case of a breakup (Backstreet Boys), wondering why a relationship isn’t working (Mary J. Blige) or bad weather (Travis) – humans really care about explanations. The same holds in the world of finance, where firms increasingly deploy artificial intelligence (AI) software. But AI is often so complex that it becomes hard to explain why exactly it made a decision in a certain way. This issue isn’t purely hypothetical. Our recent survey found that AI already impacts customers – whether it’s calculating the price of an insurance policy or assessing a borrower’s credit-worthiness. In our new paper, we argue that so-called ‘explainability methods’ can help address this problem. But we also caution that, perhaps as with humans, gaining a deeper understanding of such models remains very hard.

Continue reading “Tell me why! Looking under the bonnet of machine learning models”

Build-your-own fancharts in R

Andrew Blake

Central banks the world over calculate and plot forecast fancharts as a way of illustrating uncertainty. Explaining the details of how this is done in a single blog post is a big ask, but leveraging free software tools means showing how to go about it isn’t. Each necessary step (getting data, building a model, forecasting with it, creating a fanchart) is shown as R code. In this post, a simple data-coherent model (a vector auto-regression or VAR) is used to forecast US GDP growth and inflation and the resulting fanchart plotted, all in a few self-contained chunks of code.

Continue reading “Build-your-own fancharts in R”

Handel and the Bank of England

Ellen T. Harris

This guest post is the third of an occasional series of guest posts by external researchers who have used the Bank of England’s archives for their work on subjects outside traditional central banking topics.

George Frideric Handel was a master musician — an internationally renowned composer, virtuoso performer, and music director of London’s Royal Academy of Music, one of Europe’s most prestigious opera houses. For musicologists, studying his life and works typically means engaging with his compositional manuscripts at The British Library, as well as the documents, letters, and newspapers that describe his interaction with royalty, relationships to others, and contemporary reaction to his music. But when I began to explore Handel’s personal accounts at the Bank of England twenty years ago, I was often asked why. For me the answer was always ‘follow the money’. Handel’s financial records provide a unique window on his career, musical environments, income, and even his health.

Continue reading “Handel and the Bank of England”

Bitesize: What can the MPC minutes tell us about the policymaker’s uncertainty?

Michal Stelmach

Policymaking is invariably uncertain. I created a new index of ‘policymaker’s uncertainty’ based on a textual search of the minutes of the MPC meetings since 1997. The index is constructed by simply calculating the number of references to the word ‘uncertainty’ (and its derivatives, including ‘not certain’ and ‘far from certain’) as a share of the total word count. To avoid double-counting, it also excludes the Monetary Policy Summary that was introduced in 2015. One caveat of this approach is that it doesn’t distinguish instances of low or falling uncertainty from those where uncertainty was high. That aside, this measure can offer a new insight into uncertainty compared to indicators based on media references or business surveys.

Continue reading “Bitesize: What can the MPC minutes tell us about the policymaker’s uncertainty?”

Bitesize: Fixing ideas – The Slowing of Interest-rate Pass-through to Mortgagors

Fergus Cumming

When choosing a mortgage, a key question is whether to choose a fixed or variable-rate contract. By choosing the former, households are unaffected by official interest-rate decisions for the length of the fixation period. We can use transaction data on residential mortgages to get a sense of how long it takes interest-rate decisions to filter through to people’s finances.

Continue reading “Bitesize: Fixing ideas – The Slowing of Interest-rate Pass-through to Mortgagors”

Opening the machine learning black box

Andreas Joseph

Machine learning models are at the forefront of current advances in artificial intelligence (AI) and automation. However, they are routinely, and rightly, criticised for being black boxes. In this post, I present a novel approach to evaluate machine learning models similar to a linear regression – one of the most transparent and widely used modelling techniques. The framework rests on an analogy between game theory and statistical models. A machine learning model is rewritten as a regression model using its Shapley values, a payoff concept for cooperative games. The model output can then be conveniently communicated, eg using a standard regression table. This strengthens the case for the use of machine learning to inform decisions where accuracy and transparency are crucial.

Continue reading “Opening the machine learning black box”

What’s in the News? Text-Based Confidence Indices and Growth Forecasts

Arthur Turrell, Nikoleta Anesti and Silvia Miranda-Agrippino.

As the American playwright Arthur Miller wrote, “A good newspaper, I suppose, is a nation talking to itself.” Using text analysis and machine learning, we decided to put this to test – to find out whether newspaper copy could tell us about the national economy, and in particular, whether it can help us predict GDP growth.

Continue reading “What’s in the News? Text-Based Confidence Indices and Growth Forecasts”

How do bonus structures affect risk and effort choices? Insight from a lab experiment

Qun Harris, Analise Mercieca, Emma Soane and Misa Tanaka.

The bonus regulations were introduced based on the consensus amongst financial regulators that compensation practices were a contributing factor to the 2008-9 financial crisis. But little is known about how they affect behaviour in practice. So we conducted a lab experiment to examine how different bonus structures affect individuals’ risk and effort choices. We find that restrictions on bonuses, such as a bonus cap, can incentivise people to take less risk. But their risk-mitigating effects weaken or disappear once bonus payment is made conditional on hitting a high performance target. We also find some evidence that bonus cap discourages effort to search for better projects.

Continue reading “How do bonus structures affect risk and effort choices? Insight from a lab experiment”

A question of interest: Is UK household debt unsustainable?

Lewis Kirkham and Stephen Burgess.

UK household debt is high relative to income. But is it “unsustainable”? Some commentators say “it is”; others say “there is no reason to worry”. To investigate, we build a simple model of the economic relationships between household debt, house prices and real interest rates which we believe must hold in the long run. In our model there is no single threshold beyond which debt suddenly becomes unsustainable, but we argue that household debt should be broadly sustainable under any rise in real interest rates of up to about 2 percentage points (pp) from current levels. We also show that falling real interest rates may have contributed around 20-25pp to the rise in the household debt-to-GDP ratio since the 1980s.

Continue reading “A question of interest: Is UK household debt unsustainable?”