Series: AI, Ego & Regret — Bonus Chapter
Editor's Note: While compiling the old series for the book, I found this draft in the archives. It d...
For further actions, you may consider blocking this person and/or reporting abuse
Ngl, I feel that. My current employer (day-job) asked me a year ago when I was about 2 months there, to upgrade their dashboard, make it drag and drop... It took me 2 months, because of all the mistakes in their calculations, flaws in their interface, slow/bloated endpoints, etc. Fast forward those 2 months, they had a clean drag and drop dashboard, which was FAST, the data was accurate and it wasnt just drag and drop, it had the capability to swap out blazor cards for js cards (their request), it could resize cards and dynamically scale to window size, in other words it was exactly what I should have built, with all the bells and whistles, even dynamic drilldowns that let a customer get deeper insight, in-depth upgrades to their existing shared infrastructure that allowed them to support simple excel exports instead of complete rewrites each case... They disagreed, saying the values are different (yeah, cuz their values were calculated wrong), but that's fine... So I got put into different tasks and the dashboard I built was benched, because it needed 'review', fast forward a year, it's still under review...
Sometimes, you do your best, put your all into it, just for them to look at it and say 'nah, I want it different' and different is almost always worse, unless you seriously know what you're doing... trusting a LLM as a rule-engine is a systematic flaw in thinking. A LLM can write rules, but it cant effectively enforce them, it'll always write self-gratifying rules that just clear 99% of the time, unless it's catastrophically wrong. For systems that require efficiency, redundancy and absolute certainty, you use a language like F# or Rust, you write strict rules, not leave it up to LLM interpretation... A detailed report from a LLM would require fine-tuning beyond what's possible, you'd need telemetry data from the day the machine rolled off the factory floor, detailed schematics, detailed reports on common failures and signals, etc. Essentially, you'd need to build the Tom system, just to validate the LLM's outputs, at which point why even have the LLM?
That dashboard story. God. "Under review" for a year — we all know what that means.
And you nailed it: LLM as rule engine is a contradiction. If you need another system to validate the first system's output, you've added a layer, not solved a problem.
Yip... Functionally scrapped. Though I did see that it's active on 3 of the companies' sites, so atleast someone gets a better dashboard?
Look fundamentally, having a LLM document isnt a bad idea, or find edge-cases beyond standard rules (ONLY FOR TAKING NOTES)... But to just swap a production system with a brand new 1, without side-by-side running them for atleast 6 months... That's just insanity. Eg. Doccit, my autonomous accounting suite, was originally built on a custom OCR pipeline, massive piece of work, using spline primitives instead of pixel marching, incredibly fast, it hit 700k chars/s with cuda, unfortunately, my ssd died... And I hadnt backed it up and I used a locally hosted git server... (never making that mistake again) So I had to re-engineer it now that the exclusivity license is over for it, so I practically had to build it from scratch again (500k LOC in 4 weeks, I think not too bad for doing it solo, admittedly, V.A.L.I.D. generated 80% of that and made it way more secure and stable), but in doing so, I switched to cloud LLMs to try out how that would work... It's decent for 90% of the cases, but it's expensive af for the last 10% and still makes mistakes, so I built a hybrid system, a custom rust OCR system + LLM for document understanding (not regex) and for spot-checking low confidence areas. I wont ever fully trust the LLM to do the job, but having it as a Tier 2 is a good idea, because in the event the OCR failed, the rule-validator failed, atleast it has some extra level of security to fix it. That's fine, because it's the backup, NOT the one and only system in use...
That's the right setup. LLM as tier-2 backup, not the whole pipeline. Hybrid beats pure every time — rules catch the hallucinations, LLM catches what the rules didn't think of.
I had my own automated testing platform too. Put everything into it. Same story — dormant on a local drive now. Know exactly how that feels.
Also — 500k LOC in 4 weeks is ridiculous. Even at 80% generated, it's 100k lines you owned. Mad respect.
Exactly, except that might actually change... If you get a chance, have a look at the 12 part series I posted today, V.E.L.O.C.I.T.Y. OS's development (sum total of 5 days) has made some incredible breakthroughs... 4x KV-Cache compression, Deterministic language running at Ring-0 and a LLM that's quantized to 2 bit, yet operates deterministically and keeps everything it's ever done in context all at once... A system like that might actually stand a chance at being a tier-1 safety system, because it'll just learn continuously and notice anomalies no matter how small. I'd still trust rules over it, but when you have a LLM that literally cant hallucinate, cant forget, runs hardware-sensors directly... It actually has a decent chance at being good enough to trust on it's own...
That's a big claim — "literally can't hallucinate." If you've cracked that, you've solved the fundamental problem. I'd be curious to see how the deterministic + 2-bit quantization works in practice — those two goals usually pull in opposite directions.
I'll take a look at the series when I get a chance.
Hey, I love to read your posts. Can you write something about ManifestGo (manifestgo.app), I Can give you 100 credits for free for testing it out.
Arbab — checked out your profile. 8 posts in a month at 16 is legit, most people never hit publish at all.
Appreciate the offer about ManifestGo but I don't really do sponsored posts or tool reviews. That said, free advice since you asked:
The "5 Things I Learned" post is your strongest one. The others read more like product announcements than stuff people actually want to discuss. You've got 0 comments across 8 posts — not because your product is bad, but because the posts sell instead of teach.
Next one: pick a specific problem → how you solved it → what broke → what you learned. That's what gets replies.
Building and writing at the same time at 16 — that's rare. Keep going.
This is easily the best advice I’ve received since I started publishing. You're totally right—I got caught up trying to 'sell' the product rather than sharing the actual developer journey.
Hearing that from someone else makes it click perfectly. I'm definitely going to ditch the announcement style and start writing about the real, messy parts of building.
Thanks for the massive perspective shift and the push to keep going. I won't drop the ball!"
Glad it landed. People remember the messy stuff — in code and in stories. Drop me the link when you post the next one 👊
They’ve published eight articles, while I’m so lazy I can’t even manage to do a single one properly.
Your comments are worth more than most people's articles. Quality over quantity — and you've got the quality part covered 😄
One thing AI can't automate is good leadership. Models can generate code, but they can't replace technical judgment, trust built with clients, or years of production experience. Those remain a company's real competitive advantage.
And yet companies keep trying to automate exactly that. The irony is the people who make those decisions are the ones who lack those things themselves. 🤷
This felt like much more than a story about nepotism. What stood out to me was how you showed that the real damage wasn't caused by AI itself, but by replacing years of production knowledge with confidence and titles.
The detail that really stayed with me was the contrast between the clean benchmark dataset and the messy reality of production. That's something many people underestimate. A model can look incredible in demos, but if it hasn't earned trust in real-world conditions, those impressive numbers don't mean much.
I also liked how Diana's decision wasn't driven by emotions or loyalty—it was driven by repeated operational failures. That made the ending feel believable instead of just satisfying.
The line about a system belonging to the person who truly built and understood it was a powerful way to end the story. Whether fictional or inspired by real experiences, it highlights an important engineering lesson: domain knowledge, careful validation, and humility are far harder to replace than organizations often realize.
Really enjoyed this bonus chapter. Looking forward to seeing how the next Stratagem connects to these themes.
You nailed the benchmark vs production gap — that's the part that scares me the most in real engineering. A clean test set never lies, but it never tells the full truth either. Kevin's 98.7% wasn't fake data, it was just measuring the wrong thing.
Stratagem 2 is coming — and it's about someone who thinks they've learned from the past. Appreciate you reading, glad the bonus chapter hit 👊
For me most times reading on dev.to, may not be the writers intent but a single sentence, phrase or a comment in the post that unlocks or clicks a different realization. 🤔
That's the best kind of comment to get. A story isn't a lecture — it's just a lens. What you see through it is yours. Do you remember which sentence it was?
Your post reminded me of the distinction between hypothesis, theory, and law. We often treat engineering philosophies as elegant theories. But when code hits bare metal, survives production, and endures messy real-world variance, that's when those ideas earn their credibility. The gap between a polished demo and production chaos is where engineering principles are truly tested.
Hypothesis → theory → law. Yeah I'm stealing that 😄 What gets me is how long something can sit in "hypothesis" while everyone talks like it's already gospel. What's your best "looked great on paper, died in prod" moment?
I tried building a mini Apache-like server inside my browser OS. On paper the philosophy made sense: abstract the filesystem, route requests, and let apps behave as if they were running on a local server.
In practice, it fell apart. Looking back, I don't think the philosophy failed—I think my understanding of the underlying syntax and implementation wasn't mature enough yet. It was one of those "looked great on paper, died in prod" moments that taught me the difference between a sound idea and a working implementation.
Still learned a lot from it though.
That last line hits different. "Not the philosophy — my understanding." Most people stop at the first part. So did you rebuild it with what you know now, or start over?
Tom's mistake wasn't getting laid off it was building something so deep into one company DNA that leaving it felt like amputation. The real horror isn't Kevin. It's that Kevin was allowed to happen because nobody above Tom understood what he'd actually built. The 99.97% recall rate wasn't a number, it was six years of edge cases nobody else knew existed. Diana knew. Kevin never did. That last line hit hard — a system is yours forever because you built it.
"Amputation" — that's the word I kept circling and never landed on. You're right.
Six years of edge cases that nobody wrote down because nobody thought they'd need to. Diana knew the number was real because her crew had dealt with the 0.03% false positives. Kevin walked in, read an API doc for three days, and decided he'd seen enough.
Appreciate you reading it that close, Basit 👊
'VP of Engineering' three months after a college project is the most honest summary of how some orgs actually allocate power — titles follow relationships, not track record. The scary part is how long the dashboard can stay green before reality catches up.
That last sentence is the one — "how long the dashboard can stay green before reality catches up." That gap is where all the damage happens. The longer the green, the harder the crash.
Title != competence is a lesson every org learns eventually. Some learn it after a month, some after a client walks.
Is this the kind of story where any similarities to real world entities are pure coincidence? xD
It is 😄 Just that coincidences keep piling up, you know?
Yeah this one is definitely too good not to publish - another thriller!
Means a lot 🙌 This one wrote itself — some stories don't need AI to go wrong, just bad decisions.
It was surely an entertaining read again ...
Stratagems #2 is already in the works. Buckle up 👀