Note: This article is a developer-focused adaptation of my original blog post in Italian. You can read the full, in-depth personal reflection here:...
For further actions, you may consider blocking this person and/or reporting abuse
So many quotes to pull from this fantastic article but this one stood out to me "No risk committee will ever authorize replacing deterministic systems with probabilistic ones that have even a tiny margin of unassisted error. " Working in any regulatory sector requires the safety of deterministic systems as sometimes people's lives literally depend on it. There's a saying, "They don't make them like they used to." And to me, this applies in this context. In some use cases, not all, boring determinism is the most hardened and safest strategy that maybe wasn't intended to be built to last necessarily but here we are!
Thank you so much! It really means a lot to hear that—I’m glad I managed to translate my internal chaotic thinking into something clear and relatable!
Your point on 'boring determinism' (and why it will likely never die in certain domains) hits the nail on the head. It’s a debate we’ve been having as engineers long before the current AI wave—we saw the exact same tension back in the '90s and during previous automation shifts. In mission-critical systems, predictability isn't a limitation; it's the whole point.
Definitely! Message received. :-)
I'm with you on predictability. In come cases, AI has helped us shift away from precision. Not always or necessarily a bad thing but there's a certain satisfaction from something being certain.
Really appreciate this post. An analogy that comes to mind when I think about AI is plastic -- useful and life-saving, but way overused to the point of extreme harm, and no accountability. I keep thinking of the pool scene in The Graduate and you can replace it with AI and it's eerily relevant.
Thanks! That plastic analogy is spot-on. Replacing 'Plastic' with 'AI' in The Graduate captures the exact absurdity of today's hype cycle.
It’s fascinating how fast definitions bend for quarterly goals. Hours after my post, Nvidia's CEO claimed 'AGI has arrived'—simply by shifting the definition from human-level reasoning to completing specific tasks.
Between chipmakers declaring AGI is here to sell 400k GPUs, and ex-Anthropic insiders sounding apocalyptic doom alarms, it's all media theater. Both narratives keep the valuation bubble inflated while dodging real-world engineering accountability.
This is so so so true. And it takes the fun out of engineering. It's like everyone's chasing these trends like they're desperate teenagers, and we all get to feel the whiplash.
sounds terrible!
Fair enough! English is not my native language, so sometimes the phrasing gets a bit weird. I was thinking of 'debugging' in terms of looking under the hood line-by-line to understand the chaos. But I agree, using a developer dialect, 'tracing the hype' sounds way better. Consider it noted for my next article!
I meant not an English. But the idea that hype may be traceable or debugged. :) How's that?
Ah, I completely misread your first comment, my bad! 😄
You raise a great philosophical point: can we truly 'debug' something as irrational and emotional as market hype?
From a psychological perspective, probably not—human nature and Silicon Valley FOMO definitely don't follow deterministic logic. But philosophically, we can still analyze the cycles, observe the outputs over time, and see what happens under the hood.
It's a lot like black-box testing: when you don't have access to the source code, you focus on the inputs and outputs to figure out what's really going on underneath. A sort of reverse engineering of the hype, if you will.
We might not fix the bug in human enthusiasm, but we can certainly inspect the stack trace!
I'd underline the liability section. I work in regulated environments, and the question in the room is never whether the model can do the task. The question is who signs the form when it gets it wrong, and a probabilistic system has nobody to put in that box.
Your democratisation-of-error point has a more recent verified example than the spreadsheet boom. In late September 2020, Public Health England lost 15,841 positive COVID test results. Lab results were collected in the old .XLS format, which caps at 65,536 rows, so once the file filled up, new records were dropped. No error, no warning. Roughly 48,000 contacts were never traced, and a peer-reviewed study later estimated the delay was associated with more than 125,000 additional infections and over 1,500 additional deaths.
Nobody prompted an LLM there. That was a spreadsheet doing exactly what it was designed to do, inside a national health system, during a pandemic.
So the problem was never that the tool was bad at its job. The problem was that a failure travelled through a chain where nobody was positioned to notice it. AI does not introduce that failure mode. It widens it.
And the clean-up is harder this time, because AI-generated code looks correct. A bad spreadsheet formula at least looked like a mess. A confidently wrong function with clean naming and a tidy docstring gets through review far more easily.
Spot on, especially regarding the camouflage of AI-generated code: an old spreadsheet error at least looked like a mess, whereas an elegant algorithmic bug hides in plain sight during review. And on the "who signs the form" question—legal liability has no API to delegate to. Thanks for sharing such a fitting example!