It’s Still Not Magic: Framing the Risks Facing Financial Services in the Gen AI Era

This report introduces a new framework for making sense of AI-related risks in financial services, in a landscape transformed by the widespread adoption of generative AI. It builds on a framework first developed in a 2019 CSFI report.

Opportunity and risk often go together. But as AI reaches across every part of the financial services ecosystem – and indeed, every part of society – they are especially hard to separate. This report introduces a new framework for understanding AI risks; one which, at its core, is about the trade-offs that give rise to "uncomfortable tensions". 

In finance, "mostly right" may be good enough for some tasks, and dangerous for others. The framework in this report identifies nine risks which are grouped into three broad categories: "outcomes", "operating environment" and "system". This classification helps trace how AI risk moves through the financial services ecosystem: from the outcomes experienced by customers and society, to the environment in which firms deploy AI, to the system-level dynamics through which risks can scale and spread.

It also draws out the uncomfortable tensions; for example, that the same machinery can widen inclusion and sharpen exclusion; that "human in the loop" is not the same as human control; and that concentration is baked into how AI systems are built. Many of the thorniest risks in this report are ecosystem risks. These are not only those explicitly labelled "system" level, but any risk that can scale, spread, or reinforce other risks across the financial system. They often involve choices that look sensible for individual firms, but can create shared dependencies and fragility across the system.