This is my official (Substack wise at least) expression of interest for a paper on the regulatory gaps in SS1/23, more precisely what the framework assumes about human oversight during a crisis, and what fixing that assumption would require at the architectural level.
SS1/23 was written for a world where models fail and humans intervene. That assumption is not only load-bearing throughout the framework, but is — unfortunately — largely outdated. Principle 5 — Model Risk Mitigants — is built almost entirely around post-model adjustments, expert judgement and temporary overrides: all mechanisms that require a competent human to be present, paying attention, and capable of acting in time. In something like a batch-run model serving a weekly risk report (i.e. a static model), that is a reasonable assumption. However in an autonomous system making sequential decisions at machine speed during a market stress event, it’s certainly not.
What the framework does not account for is what I’ve been calling: the drift in the operating conditions of oversight itself. The human-in-the-loop is not a fixed resource. During a crisis, every operator in the firm is simultaneously overwhelmed. Attention fractures. Decision latency spikes. The exact moment when autonomous systems most need human oversight is the exact moment when human oversight is least available. SS1/23 treats this as someone else’s problem. It’s not. It is a structural gap between firm-level model risk management and macroprudential stability, and it is sitting in the framework unaddressed.
The Bank of England’s own AI Consortium has been circling this. At the inaugural AIC session in May 2025, members raised the risk that widespread use of similar models could amplify systemic vulnerabilities under stress, with no consensus on how existing frameworks would contain it. By October 2025, Workshop 4 had sharpened the concern considerably: correlated errors propagating across institutions, agentic workflows spreading flawed updates at speeds that almost completely outrun monitoring. The AIC named the problem clearly. Neither session proposed what to do about it at the level of architecture.
That’s what the paper does. The fix is not more human oversight, you cannot mandate attention that isn’t there. The fix is pre-committed system behaviour. Autonomous competence boundary enforcement: an architectural requirement that the system itself detects when it is operating outside its validated distribution and degrades its own autonomy transparently, shifting to human-readable outputs rather than continuing to execute decisions a human can no longer monitor. Not a new reporting obligation. Not another escalation procedure. A structural constraint, built into the system before deployment, that activates when the human layer drops out.
SS1/23 needs a new category of requirement. Autonomous systems must be mandated to degrade honestly when oversight conditions collapse — not because a human told them to, but because the architecture requires it. I am positioned to define this framework because SCARLET, a working proof of concept I built in Python, already does it: forecasting and execution are structurally decoupled, distribution gaps are flagged explicitly, and the system shifts to plain-language briefs rather than autonomous action when it reaches the edge of what it can reliably know. The paper generalises that principle into a formal regulatory proposal.
The paper is in progress. If this is your area, I’d like to hear from you — dataops_ml@proton.me
Top comments (0)