Explainability in machine learning: do popular methods deliver on their promises?

Ivona Cickovic and Andrea Serafino

Machine learning models are increasingly used in organisational decision-making, yet their inner workings often remain opaque. When these systems influence real world outcomes, knowing what they predict is not enough – we also need to understand why. Explainability methods aim to illuminate this ‘black box,’ and feature attribution tools that link predictions to individual inputs are especially popular. They feel intuitive but rely on strict data assumptions that rarely hold, making their outputs unreliable. The 2019 Apple Card case illustrates why this matters: despite gender not being an explicit input, women appeared to receive lower credit limits than men with similar profiles – an outcome attribution methods struggle to explain. This post examines a key assumption underpinning these tools and how it distorts explanations.

Continue reading “Explainability in machine learning: do popular methods deliver on their promises?”

Using causal inference for explainability enhancement in the financial sector

Rhea Mirchandani and Steve Blaxland

Supervisors are responsible for ensuring the safety and soundness of firms and avoiding their disorderly failure which has systemic consequences, while managing increasingly voluminous data submitted by them. To achieve this, they analyse metrics including capital, liquidity, and other risk exposures for these organisations. Sudden peaks or troughs in these metrics may indicate underlying issues or reflect erroneous reporting. Supervisors investigate these anomalies to ascertain their root causes and determine an appropriate course of action. The advent of artificial intelligence techniques, including causal inference, could serve as an evolved approach to enhancing explainability and conducting root cause analyses. In this article, we explore a graphical approach to causal inference for enhancing the explainability of key measures in the financial sector.

Continue reading “Using causal inference for explainability enhancement in the financial sector”