DEV Community: Orvi Das

Why Is Open Source Maintenance Unpaid?

Orvi Das — Tue, 11 Aug 2026 04:30:34 +0000

Originally published on robatdasorvi.com

I grew up in Bangladesh writing software on pirated Windows. Not proudly, just practically. Legitimate licenses cost more than a month's household income, and there was no legal pathway that made sense for a teenager in Dhaka who wanted to learn programming. So you patched what you could, cracked what you couldn't, and felt vaguely guilty about all of it.

Then I found Linux. And then the entire ecosystem sitting underneath it: Apache, Python, GCC, MySQL, Firefox. Software that was just there, for free, with source code you could actually read. It felt like someone had left the library door unlocked overnight.

That feeling didn't last forever. But the movement it introduced me to changed everything, not just for me, but for every company building software today, whether they acknowledge it or not. The open source model won the technical argument. What it never quite figured out is the economic one.

How did a bunch of hackers actually beat Microsoft?

Open source didn't beat Microsoft through strategy or funding. It beat them because the software kept getting better, and eventually there was no honest argument left that proprietary was superior.

The longer story starts with Richard Stallman in 1983. He wanted to fix a bug in the firmware for a Xerox printer at MIT, got refused access to the source code, and decided the entire model of proprietary software was ethically broken. His response was to announce he'd write a free Unix-compatible operating system from scratch. The GNU Project.

It took about a decade before that project had a kernel worth running. Linus Torvalds provided it in 1991, almost by accident: a Finnish computer science student posting to a newsgroup that he was "doing a (free) operating system (just a hobby, won't be big and professional like gnu)." The understatement of the decade.

By the mid-nineties, Eric S. Raymond had published "The Cathedral and the Bazaar," an essay that tried to explain why this chaotic, distributed model of software development actually worked. His argument was that given enough eyeballs, all bugs are shallow. The essay became something like a founding document of the open source movement, and it was persuasive enough that Netscape released the source code to their browser shortly after reading it.

What Microsoft thought about all this leaked in 1998 through what became known as the Halloween Documents: internal memos where Microsoft engineers acknowledged that Linux was a genuine competitive threat and debated strategies to counter it. The recommended approach involved spreading fear about licensing and legal risk. It didn't work. Or rather, it worked for a while, and then the market moved on without them.

The deeper thing the Halloween Documents revealed is that Microsoft understood exactly what they were dealing with. They just couldn't figure out how to fight it within the constraints of their own business model. You can't undercut free.

When did open source stop being radical?

Open source stopped being radical when companies realized that giving software away could be more profitable than selling it.

Somewhere around 2010, something shifted. Open source stopped being a movement and became the default. You can pinpoint it a few ways: Google's Android shipped on Linux. Facebook open-sourced React. Amazon built AWS on top of open source infrastructure and then sold it back to the world at a margin. The rebellious fringe became the industrial foundation.

Among websites whose operating system can be identified, Unix-like systems run about 92% and Linux alone about 62%. The true figure is probably higher still, because a growing share of the web sits behind proxies and CDNs that hide what is underneath. The web layer is dominated by open source databases, runtimes, and frameworks. When a startup raises a seed round and starts building, they are almost certainly standing on PostgreSQL, Node.js, Redis, Kafka, or some combination thereof. These tools are free to use, and the companies that depend on them are not small.

The irony is thick. Stallman's original enemy was the proprietary software company that hoarded code for profit. Forty years later, the world's most profitable companies, Google, Amazon, Microsoft itself, are built on free software. They contribute back, selectively, when it suits them. They also extract at a scale that would have been unimaginable to anyone writing GPL-licensed code in a university basement in 1983.

This isn't a conspiracy. It's just economics. Nobody planned for the bazaar to get a corporate sponsor. It happened because open source was better, and once it was better enough to rely on, capital showed up. The question nobody wanted to answer was: what does that mean for the people who built the foundation?

What does winning actually look like from the inside?

Winning looks like this: the infrastructure that runs the modern internet is maintained, in significant part, by people working for free or close to it.

In April 2014, a vulnerability called Heartbleed was disclosed in OpenSSL, the cryptographic library used by roughly two-thirds of HTTPS websites at the time. The bug had existed for two years. It allowed attackers to read memory from servers, potentially exposing passwords, private keys, and session tokens. The scope was staggering. And when journalists started reporting on it, they discovered that OpenSSL was maintained by a tiny team, operating on a budget of roughly $2,000 per year in donations, while protecting an estimated $200 billion worth of internet commerce annually. The reporting at the time made the economics look almost comical, except it wasn't funny.

Log4Shell in 2021 exposed a critical vulnerability in Log4j, a Java logging library used in enormous portions of enterprise software worldwide. Within 72 hours of disclosure, Check Point Research recorded over 800,000 exploitation attempts globally, with attackers targeting an estimated 44% of corporate networks worldwide. The maintainers were again unpaid volunteers who suddenly had to deal with an avalanche of abuse and demands from companies whose infrastructure was on fire. None of those companies had ever paid anything for Log4j. Most of them probably didn't even know it was in their stack.

Then in early 2024, a sophisticated attacker spent nearly two years cultivating trust in the xz-utils project under a fake identity before inserting a backdoor into the codebase. A supply chain attack, patient and precise. It was caught almost accidentally: a Microsoft engineer named Andres Freund noticed SSH logins were taking 500ms longer than expected and decided to investigate why. That investigation is now a minor legend in security circles. If Freund hadn't been curious on a Friday afternoon, that backdoor might have shipped in mainstream Linux distributions across the world.

These aren't arguments against open source. They're arguments about what sustaining a victory actually requires, and who's been carrying that weight.

Is the business model just broken?

The business model isn't broken exactly. It was never designed. It emerged, and emergence doesn't produce clean economics.

There have been genuine attempts to fix this. GitHub Sponsors, Open Collective, and Tidelift let developers accept money for their work, but the money mostly hasn't followed the need. Tidelift's 2021 survey of nearly 400 open source maintainers found that 46% were not paid anything at all for their maintenance work. Some projects found sustainable models: Red Hat built a billion-dollar company selling support for open source software and got acquired by IBM for $34 billion. Elastic, HashiCorp, and MongoDB built commercial open source businesses that kept the core code available while restricting what cloud providers could do with it commercially.

That last strategy has caused real friction. When HashiCorp relicensed Terraform in 2023, moving it from the Mozilla Public License to the Business Source License, it felt to many contributors like a rug pull. You build something together under one set of terms, and then the company changes the terms once the software becomes valuable enough to protect. The resulting fork, OpenTofu, became the Linux Foundation's fastest-growing project within months of launch. The open source community, when it feels betrayed, turns out to be very good at forking.

What makes this genuinely hard is that the people complaining loudest about relicensing are often companies who were never paying for the software in the first place. There's a particular kind of moral indignation that large organizations deploy when a small team of maintainers tries to capture even a fraction of the value they created, as if the original gift of open source comes with a contractual obligation to remain broke forever.

I've been on the receiving end of that dynamic at a much smaller scale. You write something useful, you open-source it, and then someone from a company with 200 engineers submits an issue demanding you fix it immediately as if you are their employee. It's a specific flavor of entitlement that the movement, in its idealism, didn't really anticipate. Stallman was angry about printer drivers. He probably wasn't anticipating billion-dollar cloud businesses filing support tickets against unpaid maintainers.

The economic structure that emerged is essentially a massive transfer of value: developers build the commons, large companies extract from it, and the gap between what's extracted and what flows back to maintainers is not a rounding error. It's the operating model.

Who's actually funding the commons now?

Some money is flowing back. The question is whether it's going to the right places.

The Linux Foundation, Apache Software Foundation, and similar organizations pool corporate contributions and distribute them across projects. The OpenSSF (Open Source Security Foundation), launched in 2020, specifically targets security work on critical open source projects. After Heartbleed, there was a genuine attempt to address the "critical infrastructure maintained by volunteers" problem, and the Alpha-Omega Project now funds security improvements at scale. Alpha-Omega launched in February 2022 with $5 million from Microsoft and Google, later growing to $8.5 million after Amazon joined. It's real money. It's also a rounding error against the value the commons generates every year.

But the distribution is uneven. High-profile projects get attention. The libraries sitting three layers below the thing you actually use, the ones that nobody thinks about until they fail, tend not to make it onto anyone's priority list until something explodes.

I've started paying for open source tools I use seriously. Not because I'm obligated to, but because the alternative, a world where critical infrastructure runs on volunteer goodwill and corporate inertia, seems genuinely fragile. The xz incident was stopped by luck. The next one might not be.

So what did the victory actually cost?

Open source won the technical argument decades ago. The software is often better, the development model works, and the accumulated commons of freely available code is one of the most remarkable things humanity has produced through voluntary cooperation. I mean that without irony. When I was a teenager in Dhaka who couldn't afford a Windows license, the existence of Linux was not a small thing.

What the movement hasn't won is the economic argument. The cost of the victory is distributed unevenly: enormous wealth for companies that build on top of the commons, intermittent burnout and financial precarity for the people maintaining the foundation. The library door is still unlocked. What nobody budgeted for is keeping the lights on inside.

I don't have a clean resolution to offer, and I'd be suspicious of anyone who does. The movement wasn't built on clean resolutions. It was built on stubbornness and idealism and, occasionally, spite. Maybe that's still enough. Maybe the xz incident produces better supply chain tooling. Maybe the Heartbleed money eventually flows toward the next critical library before it fails.

Or maybe we keep learning the same lesson every few years, just with a different CVE number attached, and a different group of unpaid maintainers staring at their inbox wondering what they signed up for.

The library door is still unlocked. I'm still glad it is. I just think about the person who has to keep the lights on inside.

42% of AI Projects Fail Within 6 Months: The Real Reason (It

Orvi Das — Tue, 11 Aug 2026 04:30:03 +0000

Originally published on robatdasorvi.com

I built a system that worked perfectly in staging. It was an approval engine for marketing outreach—supposed to save a team four hours a day. For four months it did exactly that. Then, one Tuesday, it started approving spam and rejecting legitimate leads. We had to kill it. Everyone was baffled at the failure; I thought I understood it. I was wrong about why for years.

The answer isn't about the model or the data or the engineering. The answer is about the systems we built to deploy AI versus the systems we'd need to keep it running. These two systems are fundamentally misaligned. Organizations choose to optimize for the first. They lose the second. The gap between them is where AI automation dies.

The numbers bear this out. In 2025, 42% of companies abandoned most of their AI initiatives—a jump from just 17% the year before. Large enterprises scrapped an average of 2.3 AI projects each, with an average sunk cost of $7.2 million per abandoned initiative. The attrition isn't slow erosion. It's sudden death within a narrow window. The system stops working, and when it does, the economic case for keeping it alive collapses faster than it ever could have been built.

Why do most AI projects fail after going live?

Most AI projects fail after going live because of distribution shift: the model was trained on historical data, but the world it runs in keeps changing. The leading cause of AI project abandonment is not poor model quality — it is structural incentive misalignment between the teams that ship AI systems and the teams responsible for sustaining them.

No. Immaturity would predict that all AI systems fail, eventually, as we learn and improve. But this pattern is too specific and too brutal. It's not "AI systems are generally bad." It's "AI systems work until they don't, and the moment they don't, everyone abandons them." That's not immaturity. That's structure.

The structure is this: the incentives to deploy an AI system are completely separate from the incentives to maintain it.

A VP of Sales gets budget approved to build an approval engine. Success is measured as: Does it go live? Does it show an ROI within the first quarter? Engineering ships it, claims victory, moves to the next project. The system works for months. Everyone's happy. Bonuses accrue. Books get published about the success.

Then, imperceptibly at first, the system starts to degrade. This degradation is almost inevitable because of a phenomenon called distribution shift or concept drift. Your model was trained on historical data—the leads that came in last year, the patterns that mattered six months ago. But the world changes. Your competitors adjust strategy. Customer behavior shifts. Your own product changes, which changes what gets routed to approval. The model has never seen this data before. Its confidence collapses. The statistical properties of incoming data diverge from the training distribution, and the model's accuracy drifts downward. Distribution shift is not an edge case — it is the default outcome for every model trained on historical data. The only variable is how quickly the organization discovers it.

Here's what the system looks like when it drifts:

The engineering team that shipped it is already on the next project.
Ownership is now "shared" between the team that uses it and the team that built it, which means owned by no one.
There is no feedback loop to detect the degradation until a human notices something is wrong.
When humans notice, they have to prove it was the system's fault, not their own usage.
Even if they prove it, rebuilding the model requires that original team, who is now 30% allocated to this legacy problem and 70% on new things.
The ROI case for maintaining a system that used to work is weaker than the ROI case for building something new that might work.
The system gets shut down. The budget goes to the next AI initiative.

This is not a bug in how organizations implement AI. It is the output of a perfectly rational system. It is the correct response to the incentives in place. And it scales perfectly.

Why can't monitoring alone prevent AI project failure?

Monitoring alone cannot prevent AI project failure because it is reactive by design: by the time an alert fires, accuracy has already degraded and organizational trust has already eroded. The deeper failure is structural — monitoring requires an owner, and systems marked "done" have no owner.

Better monitoring would help. Continuous monitoring and adaptive algorithms are essential to identifying and mitigating the effects of data and concept drift, and most organizations do almost none of it. But monitoring is reactive. The system drifts first; the alert fires later; the damage is already done.

The deeper problem is this: monitoring requires someone to be responsible for watching a system that has already been marked as "done." Shipping a new system gives you agency and narrative control. Maintaining an old one makes you a janitor. Organizational status flows to the builders, not the maintainers. The system produces pressure to abandon rather than preserve.

The real issue is preventive. The moment a model goes to production, it is guaranteed to encounter data it was never trained on. This is not a failure of data science. It is a law of nature. Systems that last account for this upfront. They build models with retraining loops, uncertainty quantification, and escalation patterns baked in from day one. But adding those requires complexity and budget before you can prove the model works. And the system pressures you to de-risk shipping—to prove it works first, add maintenance infrastructure later. Later never comes. The model doesn't fail because it was built badly — it fails because the system around it was never built to last.

Does better AI governance actually prevent project abandonment?

Better governance reduces the documentation of failure but rarely prevents it, because the constraint is economic, not organizational. Without accountability tied to post-deployment performance, governance is paperwork.

Yes, better governance helps. Clearer ownership of maintenance helps. But governance and ownership are not the constraint. The constraint is economic.

Enterprises poured $684 billion into AI in 2025, with more than $547 billion producing no measurable results. Approximately 95% of generative AI pilots delivered zero measurable financial return. These are not governance problems. These are misdirected capital.

An organization can want to maintain its AI systems better. But if the economic case doesn't support it, better governance just means better documentation of why the system failed. The system that wins is the one that redeploys that capital to the next initiative, which has a nonzero chance of working instead of a nearly zero chance of recovery.

The real ownership model that would matter is one where the team that ships an AI system is held accountable for its performance six months later. That would change behavior instantly. It would force upfront investment in monitoring, retraining, and operational capacity. But that ownership model does not exist, because it would slow shipping, and shipping is what promotions are built on.

Why do standard AI success metrics fail to predict long-term performance?

Standard AI success metrics — "did it ship?" and "what was Q1 ROI?" — fail because they measure point-in-time performance for systems that decay continuously over time. The metric you choose reveals how long you are actually willing to fund the system.

Metrics are the symptom, not the disease. Organizations usually measure AI success as "did it ship?" or "what was the Q1 ROI?" These are bad metrics for things that decay over time. But the reason they use these metrics isn't stupidity. It's that anything longer-term is expensive to measure and exposes the fragility of the system.

If you measure success over 18 months and hold teams accountable to it, you have to fund maintenance and monitoring for 18 months. You have to admit up front that models degrade and require constant adjustment. That's expensive. Cheaper to declare victory at month four and move on.

Metrics are consequences of the system, not the root. Change the metrics without changing what happens when a system fails, and teams will game the metrics. They'll find a new way to make yesterday's failure someone else's problem.

How do you build AI automation that lasts more than six months?

Lasting AI automation requires three structural changes: an operational budget protected for 12–18 months and separate from deployment costs, post-deployment accountability assigned to the original shipping team, and a project framing built around the question "can we afford to maintain this?" rather than "will this work?"

If the reason AI automation fails is structural incentive misalignment rather than technical immaturity, then the solutions are not technical. They are organizational and financial.

First: companies that want AI systems to last need to fund them differently. Not a one-time deployment budget, but an ongoing ops budget, priced in from the start. That budget needs to survive the first quarter and be protected for 12-18 months. It sounds obvious. Almost no organizations do it.

Second: accountability needs to follow the model. If you shipped it, you own its performance at month six. Not the ops team, not the next platform team. You. This misaligns incentives away from shipping quickly and toward shipping sustainably.

Third: the industry needs to stop framing every AI project as "will this work?" and start framing them as "can we afford to maintain this if the answer is no longer yes?" The latter question requires admitting drift, decay, and technical debt at the outset. It's uncomfortable. But it reflects reality.

The organizations that survive in AI won't be the ones that build the best models. They'll be the ones that build the infrastructure to keep mediocre models running. Because a system that works at 70% accuracy today and 65% next month still delivered value. A system that worked at 95% and is now offline delivered nothing.

The system that produces dead AI automation is not broken. It's operating exactly as designed. It prioritizes shipping over sustaining, velocity over stability, and new bets over paying down old ones. Change the system, or accept that most AI automation will die within six months. There is no third option.

Context Windows: Why Too Much Text Breaks AI in Production

Orvi Das — Mon, 10 Aug 2026 04:30:37 +0000

Originally published on robatdasorvi.com

About six months into building seriously with language models, I hit a wall I didn't see coming. I was trying to feed a 3,000-line codebase into a prompt so the model could help me refactor a gnarly module. The model kept forgetting things. It would answer questions about a function at the top of the file as if it had never seen the class definition four hundred lines later. It wasn't hallucinating exactly. It was just working with what it could hold.

That's when context windows stopped being a benchmark number to me and became an actual design constraint.

I've talked to a lot of developers since then, mostly through building and shipping things in public, and the same pattern comes up. People know what a context window is in the abstract. They see the number in the marketing copy. They don't really think about it until something breaks.

What a context window actually is

A context window is the total amount of text a model can process at once: your system prompt, the conversation history, any documents you've injected, and the response being generated, all counted together. When you exceed the limit, one of two things happens. You get a hard error, or the model silently drops older content. Neither is graceful, and the silent version is worse because you don't notice until the output starts behaving strangely.

For developers specifically, the context window is the boundary of the model's working memory for a given task. Not its general knowledge (that comes from training) and not its reasoning ability (that's architecture). Its working memory, in the specific session you're running. If you're building an app where users have long conversations, or feeding documents into a pipeline, or running multi-step agentic tasks, the context window is the ceiling you keep bumping your head on.

Understanding this distinction matters more than people give it credit for. Training data affects what the model knows. The context window affects what it can reason about right now.

Does longer context actually improve reasoning quality?

No. The middle of long contexts become attention dead zones where relevant information gets lost. Stanford and UC Berkeley researchers found in their 2023 "Lost in the Middle" study that language models show consistent 20-30% accuracy drops when relevant information appears in the middle of a long context compared to the beginning or end.¹ The model's attention concentrates on what it read first and last, and everything sandwiched in between gets diluted.

This has real consequences. If you're building a RAG system and stuffing ten retrieved chunks into a prompt, the placement of each chunk changes how well the model uses it. If you're doing code review with a full file in context, the function you care about might be sitting in the attention dead zone. If you're running an agent through a long task history, the instructions you gave twenty messages ago might be functionally invisible.

Context window size determines how severe this problem gets. A smaller window forces you to be surgical. Only the most relevant content gets in, so the dead zone pressure is lower. A larger window gives you more room but also more space for important content to drift into the middle and get ignored.

This is why treating a larger window as simply "more capacity" leads to lazy architecture decisions. It's more capacity, yes. But it's also more surface area for the attention problem to play out.

Does throwing everything into a long context window produce better results?

No. Throwing everything in made responses slower, more expensive, and more confused—you still need deliberate context selection. When I migrated some tooling from GPT-3.5 (4K tokens) to a model with a 100K window, my first move was to throw everything in. Full repository context. Full conversation history. All the documentation. That felt like the obvious thing to do.

It didn't work. I'd feed in an entire codebase, ask about a specific bug, and get an answer that was technically plausible but missed the actual problem, which was in a file the model had "seen" but clearly hadn't retained in any useful way.

What actually worked was learning to be deliberate even with a big window. Use the space for relevant context, not for all context. A 200K token window doesn't mean you should use 200K tokens on every request. It means you have headroom for when the task genuinely requires it, like reviewing a large PR diff, or helping a user whose conversation spans an hour of real work.

The shift I needed: context window size determines what's possible, not what's optimal. You still have to think about what goes in.

I also started asking a different question before designing any prompt-heavy feature: should this be a single-pass request (everything in one prompt) or iterative (build the answer through multiple smaller contexts)? Long context models make single-pass more tempting. I've learned single-pass is not always better.

Can models really reason across information scattered throughout long documents?

Retrieval and reasoning are different things, and models struggle with the latter. Needle-in-a-haystack benchmarks, where you hide a specific fact deep inside a long document, show that many models with million-token windows perform reasonably well at retrieving that fact. But retrieval isn't reasoning.

Anthropic's documentation on Claude's long context capabilities notes that performance on needle-in-a-haystack benchmarks doesn't fully predict performance on harder reasoning tasks over the same documents. Finding a fact is easier than synthesizing relationships across facts scattered through a hundred pages.²

Greg Kamradt's 2023 public testing of long context models found similar patterns: models can often locate a specific fact buried in a long context, but their ability to reason across multiple pieces of information spread throughout the same context degrades significantly as that context grows.³

For developers building real systems, this matters a lot depending on your use case. If your application is retrieval-heavy ("find this thing in this document"), larger windows help. If your application is synthesis-heavy ("reason about the relationship between these ideas across this document"), you might get more reliable results by breaking the task into smaller chunks, even when a bigger window is technically available to you.

I build for Bangladeshi SMBs most of the time, where API costs are real constraints, not just optimization exercises. This distinction between retrieval and reasoning tasks has saved me from expensive mistakes more than once.

How should you structure prompts with long context windows?

Fill the context to 60-70% of its limit with task-relevant information: critical constraints and definitions near the beginning, supporting context in the middle, the specific question near the end. This is a rough heuristic and it doesn't apply to every task uniformly.

A few other things that have changed how I build:

For agentic tasks, I don't rely on long task histories staying coherent. Important instructions go in the system prompt or get re-injected at the beginning of each turn. Assuming the model will "remember" something from ten messages ago is a gamble I've lost enough times to stop taking.

For RAG pipelines, I order retrieved chunks by relevance and put the most relevant one first, not last. The research is clear on attention placement, and it's a free win.

For code review and refactoring tasks, I don't send whole files anymore unless I genuinely need the whole file. I send the function, the class, and the relevant imports. The model reasons better about less.

None of this is counterintuitive once you actually internalize the constraint. The context window is working memory, not a reading list. You wouldn't expect a human developer to hold 200K tokens of code in their head and reason about it coherently. The analogy isn't perfect but it's useful.

Will context window size eventually stop being a design concern?

Probably not in the near term—reasoning performance degrades with context growth regardless of window size. The theoretical answer is that with an infinite window and perfect attention, it would stop mattering. We're nowhere near that. The practical limits aren't just the token count. They're the model's ability to maintain coherent attention and reasoning across long sequences, which still degrades as context grows, even with the best current models.

What probably happens over the next few years is that models get better at reasoning over long contexts, not just retrieving from them. That would actually change the calculus significantly. Until then, treating the context window as a constraint worth designing around, rather than a number to maximize in benchmarks, is still the right approach.

The developers I've seen build the most reliable AI-powered systems are rarely the ones using the largest models or the biggest windows by default. They're the ones who have a clear idea of what goes in the context and why.

Liu, N. F., et al. (2023). Lost in the Middle: How Language Models Use Long Contexts. arXiv. https://arxiv.org/abs/2307.03172 ↩
Anthropic. (2024). Long context prompting for Claude. Anthropic Documentation. https://docs.claude.com/en/docs/build-with-claude/prompt-engineering/long-context-tips ↩
Kamradt, G. (2023). LLM Test: Needle In A Haystack - Pressure Testing Long Context Windows. GitHub. https://github.com/gkamradt/LLMTest_NeedleInAHaystack ↩

What the New Reasoning Models Actually Changed About How I Build

Orvi Das — Mon, 10 Aug 2026 04:30:06 +0000

Originally published on robatdasorvi.com

In 2025, METR ran a randomized controlled trial on 16 experienced open-source developers working real GitHub issues in codebases they knew well. Half the tasks allowed AI tools, half didn't. The developers using AI, including reasoning models, took 19 percent longer to finish. Afterward, when asked how AI had affected their speed, they said it made them 20 percent faster. Nobody in the study noticed the slowdown while it was happening.

I read that number six months after I'd already lived it. I'd spent the back half of 2025 wiring reasoning models (o1, then o3, then Claude's extended-thinking variants), into every part of my software development workflow, on the assumption that a model which "thinks longer" before answering must produce work that needs less fixing. That assumption is the standard advice on reasoning models: they reduce errors, so they reduce your total time. It's repeated in every vendor blog post and most engineering newsletters. It is wrong for the reason METR's data shows and for a second reason nobody measures: reasoning models don't just get things wrong less often, they get things wrong more convincingly. That's a different problem, and it costs different time.

What Did the METR Number Actually Measure?

It measured real task completion time on real repositories, not benchmark accuracy. That distinction is the whole story.

METR's developers had an average of five years on the codebases they were working in (arxiv.org/abs/2507.09089). They weren't novices misusing the tools. They used Cursor and Claude 3.5/3.7 Sonnet, wrote prompts, reviewed diffs, and iterated, the exact loop every AI coding tutorial recommends. The 19 percent slowdown happened inside that loop, not despite it. Before the study, participants forecast a 24 percent speedup. After finishing the tasks and living the slowdown firsthand, they still reported believing AI had helped by 20 percent. The gap between forecast, lived experience, and self-report is nearly 40 points, in the same direction, twice.

That's the sentence worth sitting with: experienced developers cannot reliably feel the difference between AI making them faster and AI making them slower. The instrument they're using to judge, their own sense of flow while typing prompts and accepting suggestions, measures something other than elapsed time. It measures effort per keystroke, and reasoning models reduce keystrokes while increasing total minutes spent reviewing what the keystrokes produced.

What Was I Doing With Reasoning Models That Made This Worse?

I was using them as a faster author, when the actual bottleneck in my work was never authoring.

For most of my career, writing the first draft of a function was never what ate the hours. Reading a legacy module and understanding why a bug existed, or verifying that a change didn't break three unrelated call sites, always took longer than typing the fix. Reasoning models are extremely good at the part that was never the bottleneck. o1-preview scored 41.3 percent on SWE-bench Verified at launch; by the December 2024 release, o1 had climbed to 48.9 percent on the same benchmark (OpenAI o1 System Card). Those are real gains at generating plausible, structurally correct patches for well-scoped issues. I read that trajectory as "the model is getting more correct" and handed it more authoring responsibility, bigger diffs, more files touched per prompt, less scaffolding written by hand first.

What I got back looked more correct than what earlier models produced, and that was the trap. A GPT-4o patch that was obviously wrong got rejected in five seconds. An o1 patch that was subtly wrong, right approach, one incorrect assumption about how a config value propagated three files away. Took twenty minutes to catch, because it read like something I would have written myself. Stack Overflow's 2025 Developer Survey found that 66 percent of developers report spending more time fixing "almost-right" AI-generated code than before, and that "AI solutions that are almost right, but not quite" is the single most-cited frustration with these tools, named by 45 percent of respondents (Stack Overflow 2025 Developer Survey). Trust in AI output accuracy fell from 40 percent to 29 percent over the same period the models got better at benchmarks. Those two lines on the same chart, correctness up and trust down, is the whole argument against the standard advice in one graph.

What Changed When I Stopped Asking for Code First?

I stopped treating the reasoning model as the author and started treating it as the prosecutor of my own drafts.

The switch happened on a specific afternoon working on a billing reconciliation job, the kind of task with enough edge cases (partial refunds, currency rounding, retried webhooks) that a plausible-looking fix is genuinely dangerous, because it passes the tests you thought to write and fails the ones you didn't. I'd asked o1 to implement the reconciliation logic directly, gotten something clean and well-commented, and only caught the rounding bug because a number looked one cent off in a manual test three days later. That cent was the tell. I rewrote the same feature myself, badly, in about forty minutes, then handed the draft to the model with one instruction: find every case where this breaks. It found four, one of which was the same rounding bug in a different guise. It took less combined time than the first approach, and I ended the session understanding the code, because I'd written it.

That's the actual shift new reasoning models produced in my process. Not faster code generation, faster, harsher review of code I still write myself for anything with real edge-case density. The reasoning that makes these models good at catching my mistakes is the same reasoning that makes their own mistakes hard to catch, because a model working from a wrong premise reasons its way to a coherent, well-argued wrong answer instead of an obviously broken one. Using it as author hides that failure mode. Using it as reviewer exposes it, because now the coherent argument is aimed at something I already know the shape of.

Doesn't the SWE-Bench Jump From 41 to 49 Percent Prove Reasoning Models Are Getting Faster?

No. It proves they're getting better at a specific, narrow kind of task, and that's not the same claim.

SWE-bench Verified is built from GitHub issues with a human-confirmed correct patch and a test suite that already exists (OpenAI, "Introducing SWE-bench Verified," August 2024). It rewards a model for converging on one known-good answer inside a well-bounded diff. That is close to the best-case scenario for how these models fail: bounded scope, existing tests, a single right answer. METR's study measured the opposite scenario, open-ended tasks in codebases with tacit knowledge no benchmark captures, judged by whether the change actually shipped correctly, not whether it matched a reference patch. A model can improve substantially on the first kind of task while making the second kind of task slower, because the second kind of task's cost isn't "can a correct patch be generated". It's "can I tell, without redoing the work myself, whether this particular patch is the correct one." Benchmark accuracy answers the first question. It says nothing about the second, and the second is what determined the 19 percent number.

If reasoning-model gains were closing that verification gap, you'd expect trust in AI accuracy to be flat or rising as benchmark scores climbed through 2025. It fell 11 points instead. The counterargument that better benchmarks imply better real-world throughput requires that verification cost scale down with generation quality. The data says it scales up, because better-looking wrong answers cost more to catch than worse-looking ones.

What Do I Actually Do Differently Now?

I draft the parts of the system where a wrong answer would be expensive and hard to notice, and I let the reasoning model draft the parts where a wrong answer is cheap and obvious. Boilerplate, migrations with an existing pattern to follow, test scaffolding, the model writes those, because a mistake there fails loudly. Anything touching money, state that's hard to replay, or logic where "close" is functionally identical to "wrong," I write first and hand to the model as an adversary, explicitly asked to argue against my own assumptions rather than extend them.

That's the inversion the METR number was actually describing, before anyone had named it: the models got better at producing answers that feel finished, and feeling finished is precisely the property that defeats a developer's instinct for how much checking a piece of code still needs. The 19 percent slowdown wasn't a temporary calibration problem that better models would fix. It was the honest cost of verification finally showing up in the timer, after years of the same cost hiding inside "reading the code," which nobody times. I'm not faster than I was before reasoning models. I ship fewer of the bugs that used to cost me a Saturday, because I stopped asking the model to convince me and started asking it to try to prove me wrong.

What AI Agents Still Cannot Replace About Senior Developers

Orvi Das — Sun, 09 Aug 2026 04:30:33 +0000

Originally published on robatdasorvi.com

I've been building software for over a decade. For the last two years, I've been building with AI agents — using them to scaffold projects, write boilerplate, generate tests, and fill in the tedious parts that used to eat my afternoons. I won't pretend they haven't made me faster. They have, measurably.

But I keep running into a narrative that's starting to bother me. The idea that senior developers are expensive relics. That a junior with the right prompts can now do what took years to learn. That the "10x engineer" is whoever has the best Claude subscription.

It's not true. And I think the people saying it loudest are mostly the ones who've never actually shipped a system at scale and watched it break.

What Does "Replacing" Actually Mean? (Code Generation vs. Engineering Judgment)

When people say AI will replace developers, they usually mean one of two things: AI will write all the code, or AI will make the judgment calls that senior engineers make. The first is already happening, partially. The second is not happening, and I don't think it's close.

There's a useful distinction between coding and software engineering. Coding is producing text that a computer can execute. Software engineering is deciding what to build, how it should behave under pressure, where it can fail safely, and how it will be maintained two years from now by someone who wasn't there for the original decisions. AI is very good at the first thing. It is genuinely limited at the second.

GitHub's own research on Copilot found that developers completed tasks 55% faster with AI assistance — but that study measured individual task completion, not system design, not debugging a production incident at 2am, not the cascading cost of a wrong architectural decision made six months ago.¹ Speed on isolated tasks is a real gain. It is not the whole job.

Why Does the Gap Between Generated Code and Production Reality Widen at Scale?

Systems aren't just individual functions—they're interactions between functions, services, and constraints that AI lacks the context to fully understand. That gap is where operational experience becomes invaluable.

I saw this concretely building an automation pipeline. An AI-generated module worked perfectly in testing and then behaved strangely in production. The reason turned out to be a subtle assumption about message ordering that made total sense in a single-threaded test environment and fell apart completely under real async load. The AI could not have caught that without understanding the deployment environment, the infrastructure choices, the team's operational history. A senior engineer who had lived through a similar incident two years earlier caught it in a code review in about four minutes.

That gap — between generation and trust — is where experience lives. The Stack Overflow Developer Survey from 2024 found that while 76% of developers were using or planning to use AI tools, less than half said they trusted AI-generated code enough to deploy it without significant review.² The trust problem is real, and it maps directly onto the judgment gap.

What Three Things Never Leave the AI Prompt Intact in Production?

Three blind spots define AI's limits: organizational context (why decisions were made in the past), failure mode reasoning (designing for inevitable breakage), and tradeoff reasoning under real constraints. These are invisible until they matter most.

There are a few specific things AI consistently struggles with, and they're all related to context that doesn't live in the code itself.

The first is organizational context. Why is this service split this way? Why is this API structured awkwardly? Usually because of a constraint that existed three years ago, a team that no longer exists, or a business decision made before the current architecture. AI doesn't know any of that. It will suggest refactoring things that can't be refactored, or building things that already exist under a different name in a different repo.

The second is failure mode reasoning. A senior engineer thinks about what happens when the thing breaks before writing the first line. They're designing for the 2am alert, for the customer who finds the edge case, for the junior developer who has to fix it without documentation. AI generates code that works. It doesn't naturally design code that fails gracefully, that degrades predictably, that surfaces the right information when it goes wrong.

The third is tradeoff reasoning under real constraint. I can tell an AI agent "build me a caching layer" and it will build something technically correct. I cannot tell it "build me a caching layer given that we have $200 a month in infrastructure budget, three engineers who are already stretched, and a product manager who doesn't understand eventual consistency." The judgment there — what to cut, what to defer, what to do well enough versus what to do properly — that's senior engineering. That's the irreplaceable part.

How AI Agents Changed a Senior Developer's Role (And Concentration of Judgment)

When AI handles the routine work, judgment calls don't disappear—they become the entire job. My output increased measurably, but the proportion of time I spent on judgment calls went up, not down.

When I leaned into AI agents more aggressively about eighteen months ago, my output increased noticeably. I was shipping more features, writing more tests, spending less time on boilerplate. That was real and I'm not going to undersell it.

But something else happened that I didn't expect. The proportion of my time spent on judgment calls went up, not down. Because the AI was handling the easy parts, the hard parts remained. The parts that required me to sit with a problem, read old code, talk to someone who understood the business, and make a call that couldn't be delegated to a prompt.

In a strange way, AI made the senior engineer role more concentrated. More of my day is now spent doing things that require experience, because the things that don't require experience are being handled automatically. If anything, I feel more like an engineer now than I did when I was spending four hours writing CRUD endpoints.

Is There a Real Threat to Senior Developer Employment?

The threat isn't obsolescence but compression—one senior engineer with good AI tooling now handles work that previously needed two or three people. That shifts the demand curve in specific, measurable ways.

Yes, and I'd be dishonest if I didn't say so. The threat isn't that AI replaces senior developers outright — it's that AI compresses the development cycle in ways that reduce headcount. One senior engineer with good AI tooling can now handle work that previously required two or three people. That has real implications for junior developers trying to accumulate experience, and for team structures that depended on a certain volume of routine work to justify their size.

There's also a subtler risk that concerns me more. If developers over-rely on AI-generated code before they've developed the instincts that come from debugging real systems and fixing things that break in confusing ways, the next generation might produce developers who are technically credentialed but experientially thin. The cognitive shortcuts get replaced before the reasoning capacity underlying them has been built. You end up with people who can direct an AI but can't evaluate what it gives back.

McKinsey's research on generative AI projected enormous productivity gains across knowledge work, but flagged that the highest-value activities remained those requiring "judgment, creativity, and complex communication."³ That's not a coincidence. Those are exactly the things that can't be pattern-matched from training data.

What Actually Makes a Senior Developer Senior?

It's not syntax or architecture knowledge—it's scar tissue from having watched systems fail and learned from the consequences. That accumulated weight of judgment cannot be generated.

It's not syntax. It's not knowing which libraries to use. It's not even knowing how to architect a system in the abstract.

It's the accumulated weight of having watched things go wrong — having made the wrong call and lived with the consequences, having inherited code written by someone who clearly didn't understand the problem they were solving, having been paged at 2am because a system you designed made the wrong tradeoff six months ago. That history shapes how you think before you write a single line.

It's knowing when to push back. When a product requirement is going to create technical debt that costs ten times as much to fix later. When the elegant solution is actually the fragile one. When to slow down instead of just building the thing because building the thing is what everyone is expecting.

None of that is in a prompt. None of it lives in a model's training data. It lives in the scar tissue of someone who has shipped real systems, broken them, and fixed them at inconvenient hours for unglamorous reasons.

AI can write the code. It still cannot replace the engineer who knows why the code matters, when it's wrong, and what to do when it inevitably breaks.

That remains a human problem. For now, and probably for longer than the hype suggests.

GitHub, "Research: Quantifying GitHub Copilot's Impact on Developer Productivity and Happiness," 2022. https://github.blog/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/ ↩
Stack Overflow Developer Survey 2024. https://survey.stackoverflow.co/2024/ ↩
McKinsey & Company, "The Economic Potential of Generative AI," June 2023. https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-economic-potential-of-generative-ai ↩

What I Want My Kids to Know How to Do Without AI

Orvi Das — Sun, 09 Aug 2026 04:30:02 +0000

Originally published on robatdasorvi.com

Your child is going to grow up worse at thinking than you are, and it will be because of choices you made before they turned ten.

That's the flat version of the claim, and I know it's the kind of sentence that makes people want to argue back immediately. Search "skills AI cannot replace" and you'll find hundreds of listicles: creativity, empathy, critical thinking, as if naming a skill were the same as protecting it. It isn't. Three separate research efforts spent 2025 actually measuring what happens when a brain hands its thinking off to a machine, and what they found is more specific, and less comforting, than any listicle. None of those researchers studied children. That gap — between what we've measured and who we're actually worried about — is the whole story.

Isn't This Just the Calculator Argument Again?

No, because a calculator replaces one procedure. Offloading thinking to an AI assistant replaces the entire loop of generating an idea, testing it, and correcting it, which is a different kind of cognitive event.

People reach for the calculator comparison because it's reassuring: we survived that transition, so we'll survive this one too. But the comparison breaks down under actual research on offloading. In wayfinding studies, researchers led by Louisa Dahmani and Véronique Bohbot found that people who relied more heavily on GPS navigation aids showed a steeper decline in spatial memory over a longitudinal follow-up, measured through hippocampus-dependent tasks, than people who navigated with their own cognitive maps. A related study of experienced navigators found that in-car navigation aids measurably impaired spatial learning even in people who already knew the route well. This isn't hypothetical: it's the same brain region — the hippocampus — that Eleanor Maguire's team found was structurally enlarged in London taxi drivers who spent years memorizing the city's 25,000 streets by hand, and that shrank back down after they retired and stopped using it. Use it or lose it isn't a metaphor. It's what the tissue does.

GPS didn't just replace a procedure — turn left, turn right — it replaced the ongoing work of building and updating an internal map. That's structurally closer to what an AI assistant does to writing, reasoning, or problem-solving than to what a calculator did to arithmetic. The calculator argument is the wrong analogy, and it's the one most parents are quietly relying on to feel fine about this.

What Does the Data Actually Show?

It shows measurable, physical differences in how the brain engages when AI does the thinking, not just self-reported laziness.

In June 2025, MIT Media Lab researchers published a study called "Your Brain on ChatGPT," which put 54 adults through essay-writing sessions while wearing EEG caps, dividing them into three groups: one using ChatGPT, one using a search engine, and one using neither. The brain-only group showed the strongest, most widely distributed neural connectivity. The search-engine group showed moderate engagement. The ChatGPT group showed the weakest connectivity of the three — and produced essays that human teachers and an AI evaluator both rated as more homogeneous, with writers reporting a reduced sense of ownership over what they'd written. In a fourth session, some ChatGPT users were reassigned to write without the tool, and their brains still lagged the participants who'd been unaided from the start. The paper is a preprint, not yet peer-reviewed, and the researchers are careful about that themselves — but the effect size and the EEG data are hard to wave off as noise.

Seven months earlier, Microsoft Research and Carnegie Mellon surveyed 319 knowledge workers who used generative AI at work at least weekly, collecting 936 first-hand accounts of specific tasks. The pattern they found: the more confidence a worker had in the AI's ability to do a task, the less critical scrutiny they applied to its output, especially on low-stakes work. The researchers called it a confidence effect. People weren't lazy. They were rational — and that rational offloading, applied constantly, is what erodes the muscle.

How Many Kids Are We Actually Talking About?

Most of them. In December 2025, Pew Research Center reported that 54% of U.S. teens ages 13–17 now use AI chatbots to help with schoolwork, and one in ten said they do all or most of their schoolwork with a chatbot's help. Roughly three in ten use a chatbot every day. This isn't a fringe behavior some parents need to watch for. It's the median experience of being a teenager in 2026.

That's the scale problem underneath the cognitive one. A single adult writing one essay with ChatGPT, in a lab, for twenty minutes, is one data point. A generation of children doing a version of that every school night for a decade is a different kind of exposure entirely, and it's the one we have almost no direct research on.

What Can the Research Not See?

It can't see what happens to a brain that's still being built, because every study we have so far was run on brains that were already finished.

This is the part that actually matters, and it's the part the studies are structurally incapable of answering. The MIT participants were adults. The Microsoft survey respondents were working professionals. Their neural architecture — the wiring for reasoning, error-correction, sustained attention, the tolerance for being stuck — was already laid down by the time researchers put an EEG cap on them or handed them a survey. What the data measured was a fully formed cognitive system briefly outsourcing a task. What it did not, and structurally cannot, measure is what happens when the outsourcing starts before that system exists — when a nine-year-old never has to sit with a hard math problem for the four uncomfortable minutes it takes to actually understand it, because a hint is one tap away.

A brain that's still developing doesn't just use its existing wiring more or less efficiently. It wires itself around whatever it's actually forced to do. That's the entire premise behind the hippocampus findings in taxi drivers and GPS users: structure follows use, not intention. A study can hand an adult a task and measure the dip in their already-built brain. No ethics board is going to run a fifteen-year longitudinal trial that raises one cohort of children with unlimited AI access from age five and compares their adult cognitive architecture to a control group raised without it. We will never get that data, not because no one wants it, but because the experiment takes a childhood to run and can't be repeated once it's done to a real kid. Every statistic in this piece is a proxy for a question none of them were designed to answer.

So What Do I Actually Want My Kids Able to Do Without It?

Not everything. Just the small list of things that build the parts of the brain everything else depends on.

I want them to do mental math and rough estimation before they check a calculator, because estimation is what lets you catch a wrong answer later — a skill that doesn't exist if you never built the internal number sense to notice when something's off. I want them to write a full first draft of something, badly, alone, before any tool touches it, because the MIT data suggests the ownership and the neural engagement are in the struggle, not the polish. I want them to navigate somewhere unfamiliar without GPS often enough that they build an actual mental map, the way the Dahmani and Bohbot research suggests you only do when you're forced to. I want them to sit with a genuinely hard problem for longer than feels comfortable before they're allowed to ask for help, human or artificial, because that discomfort is the whole mechanism — it's not a flaw in the process, it is the process. And I want them to have a real, unscripted, occasionally awkward conversation with another human being that they have to navigate without a suggested reply.

None of that is anti-AI. It's pro-scaffolding. You can't use a tool well as an adult if the underlying skill it's assisting never got built in the first place — you're not augmenting a capability at that point, you're replacing an absence.

Is This Actually Worth Fighting About?

Yes, but only on the handful of skills above, and only while they're young enough for it to matter. Not on the file rendering it obsolete — spelling drills, rote memorization of facts a phone can retrieve instantly — because fighting for skills without developmental payoff just makes you the parent who's fighting AI on principle, and your kid will tune that out fast, correctly. Save the fight for the small number of things that are actually load-bearing: struggle tolerance, number sense, spatial reasoning, the ability to generate an idea from nothing. Lose those small fights and you're not raising a kid who's fluent with AI. You're raising a kid who has nothing underneath it.

If I had one minute left to say this to you, the parent reading this while your kid is upstairs asking a chatbot to finish their essay: it's not the essay. Let them use it for the essay next year, and the year after, and for most of their adult life — that fight isn't the one that matters. What matters is whether they spent enough hours before that, stuck, uncomfortable, and alone with a hard problem, that their brain built the part of itself that knows how to get unstuck. Build that first. Everything else can be outsourced. That can't.

What Human Attention Will Be Worth in Ten Years

Orvi Das — Sat, 08 Aug 2026 04:30:38 +0000

Originally published on robatdasorvi.com

Sometime last year I caught myself doing something embarrassing. I had opened a tab to read an essay I genuinely wanted to read, one I had bookmarked specifically because it looked interesting, and within forty seconds I had opened three more tabs, checked a notification, and completely forgotten the essay existed. The tab sat there for eleven days before I closed it unread.

I am not particularly proud of this. But I also do not think it is purely a character flaw. Something structural has been shifting for a while now, and it is about to get significantly worse in ways that will make a single, sustained minute of human attention genuinely rare and economically valuable.

What happens when content supply becomes unlimited?

The supply of content is now effectively decoupled from human effort, and that changes the economics of attention permanently.

The economist Herbert Simon identified the core problem back in 1971 in his paper "Designing Organizations for an Information-Rich World," well before anyone had a smartphone. A wealth of information, he wrote, creates a poverty of attention. He had no idea what was coming.

For most of recorded history, information supply was constrained by how many humans could write, film, or broadcast things. That constraint kept a rough equilibrium in place: more content than anyone could consume, yes, but still anchored to human labor hours.

Generative AI breaks that anchor completely. A single model can produce in one second what would take a professional writer several hours. As the cost of generation collapses toward zero, every platform, every brand, and every person with an API key pumps more material into the same finite pool of human time.

Human attention is still capped at twenty-four hours a day. Everyone gets the same amount. It cannot be manufactured, stored, or scaled. When one side of a market expands without limit and the other side cannot grow, the scarce side becomes more valuable. The question is whether that value ends up going anywhere useful.

Where does the value of attention actually go?

Right now, the value of your attention goes to whoever built the pipe, not to you.

The current architecture of the attention economy is a straightforward extraction operation. Platforms engineer maximum capture of your time, then sell that captured time to advertisers. You get the content for free, which seems like a good deal until you understand that free means you are paying with something you never consciously priced.

Jonathan Haidt's 2022 essay in The Atlantic made this point sharply, arguing that social media platforms have fundamentally rewired the information environment in ways that degrade collective reasoning. His framing was about democracy and discourse, but the underlying mechanism is the same: when attention becomes the traded commodity, the incentive is to capture as much of it as possible, not to make that time enriching for the person spending it. (The Atlantic, 2022)

I have been building products on the internet for a few years now, and the honest version of this is that I feel the pull too. Every dashboard shows engagement metrics. Every product meeting circles back to retention. The entire vocabulary of building software for the internet is a vocabulary of capture: hook, funnel, convert, retain. It takes genuine deliberate effort to ask a different question, like whether the person who used this product left better off than they arrived.

That question is starting to matter commercially in ways it did not before. I think the shift will accelerate sharply over the next decade.

Is collective attention actually shrinking?

Yes, and we have data on it.

Research published in Nature Human Behaviour tracked public interest across Twitter, Wikipedia, Google Trends, and several other platforms over multiple decades. The finding was uncomfortable: the time that cultural topics spend at the peak of collective attention is shrinking. Things spike faster, trend harder, and collapse back to baseline more quickly than they did twenty years ago. The acceleration is real and measurable. (Lorenz-Spreen et al., 2019, Nature Human Behaviour)

At the individual level, this shows up as a growing gap between the kind of attention you want to have and the kind you actually have. People want to read long books, follow complex arguments, sit with difficult ideas long enough to do something with them. Many people are failing at this not because they lack interest but because years of environments optimized for rapid context-switching have recalibrated their baseline.

A 2023 Microsoft report on hybrid work found that the average time workers spend on a single task before switching has dropped significantly since 2020, and that the most common cause is self-interruption, not external requests. People break their own focus before anyone else has a chance to. (Microsoft WorkLab, 2023)

I notice this in myself constantly. There is a version of me who could hold a hard technical problem in his head for three hours without fragmenting his concentration across twelve channels. I am not sure that version of me exists anymore in the same form. I have been working to get him back.

What will happen to deep attention in the next decade?

The ability to pay deep, sustained attention to something will become a credential, even if nobody puts it on a resume.

A developer who can sit with a hard problem for three hours without reaching for distraction. A writer who can hold a complex argument in her head long enough to develop it into something worth reading. An analyst who can read a dense paper and extract the actual useful insight rather than skimming for the headline. These look like baseline professional competence right now. In ten years, as median attention capacity continues to erode, they may look like genuinely unusual and valuable skills.

The economic logic is not complicated. Attention is the input to most knowledge work. If the quality and duration of attention that the average knowledge worker can sustain keeps declining, while AI handles the parts of the job that do not require sustained human judgment, then sustained human judgment becomes the bottleneck. And bottlenecks command premiums.

There is a bleak version of this trajectory, where the population stratifies between people who protected their capacity for deep focus and people who did not, and that stratification maps onto economic outcomes in ways that are hard to reverse. There is also a less bleak version, where new tools and norms emerge to help people reclaim some of what the current environment has eroded. I oscillate between these two readings depending on the week.

What I find less ambiguous is this: the skills that require sustained attention are exactly the skills that AI cannot replicate, almost by definition. AI can generate content at scale. It cannot be the person who reads that content carefully and decides what it actually means for a specific situation. That person is still a human, and that person is becoming rarer.

What changes when you treat your attention like money?

Your engagement becomes more intentional, your selection criteria become stricter, and the quality of everything you focus on goes up noticeably.

About eight months ago I started being more deliberate about this. Not in a dramatic way. I did not go off-grid or start a meditation practice. I just started treating my attention more like a budget.

The practical version of this: I close things I am not using. I do not leave notifications on by default. I have specific times for reading long-form content and I do not try to do it in the gaps between other tasks. I am less available in the ambient sense and more available in the intentional sense.

The unexpected result is that the things I actually engage with have gotten noticeably better. When you stop filling every gap with low-density content, the things you choose to spend real time on feel sharper. There is a selection effect: you become more willing to abandon things that are not worth your time, and more willing to give serious time to things that are. The average quality of your attention experiences goes up.

This is not a productivity hack. It is closer to an acknowledgment of market conditions. If attention is becoming genuinely more valuable, spending it carelessly is increasingly expensive, even when the price is invisible.

The platforms know this. The next wave of products being built around AI is not primarily about delivering more content faster. The interesting ones are about helping people spend their attention better, through filtering, curation, and summarization, so that actual reading or watching or listening time becomes higher quality. This is already happening and will intensify.

What a focused hour will actually be worth

In ten years, the ability to give something your genuine, undivided attention for an extended period will be recognized as close to a professional differentiator, not formally, but economically.

My rough guess is that in ten years, the ability to give something your genuine, undivided attention for an extended period will be recognized as close to a professional differentiator. Not formally, nobody will list "high attention span" on their CV, but economically, in the sense that people who have it will command better outcomes in most domains that require thinking.

The current attention economy is an extraction operation. It extracts value from human attention while returning relatively little to the humans providing it. That model is not stable when the thing being extracted starts to become genuinely scarce. Something will shift.

It might be that people start pricing their attention more explicitly, paying for ad-free experiences, using tools that filter rather than amplify, choosing platforms that treat engagement as a means rather than an end. It might be that the skills associated with deep focus get recognized and rewarded in labor markets. It might be something nobody has articulated yet.

What I am fairly confident about: treating attention as something worth protecting, during the years when it was fashionable to give it away freely, will turn out to be one of the better investments people could have made. I started protecting mine later than I should have. I am glad I started at all.

I do not know exactly what the attention economy looks like in 2036. But the asymmetry is clear. The thing everyone has equally, twenty-four hours and a fixed capacity for focus, is the same thing that infinite content generation makes increasingly difficult to spend well. The rarity is coming. The question is whether we notice in time to do something about it.

Silent AI Code Bugs: Passing Reviews, Failing in Production

Orvi Das — Sat, 08 Aug 2026 04:30:07 +0000

Originally published on robatdasorvi.com

The bug was there for three weeks. Every time someone ran the test suite, it passed. Every time someone reviewed the PR, the logic looked right. The function did exactly what the comment said it did — and the comment had been written by the same model that wrote the code. It was only when the production environment diverged slightly from the test setup that anything surfaced at all, and by then the damage was already downstream.

I want to talk about that gap. Not the failure itself — the silence before it.

Why Does AI Code Feel So Convincing Even When It Is Wrong?

Generated code presents itself with unearned confidence: perfect syntax, clean structure, and articulate comments that mask subtle logical flaws. The issue is that language models learned to mimic the surface features of correct code—consistency, naming conventions, structure—but the deeper semantics of correctness require understanding the problem domain, which is much harder to learn from text alone.

There is something about the way a language model writes code that makes it harder to scrutinize than code written by a human. A human who is uncertain will often signal it — a vague variable name, an awkward comment, a TODO left in place. A model never hedges. It produces clean, confident prose whether it is solving a problem it knows well or hallucinating an API that does not exist. The syntax is perfect. The indentation is perfect. The error handling looks thorough. Everything looks like the work of someone who knew what they were doing.

This is not a coincidence. These models were trained on human code that was valued for correctness, and the surface features of correct code — consistency, structure, confident naming — are easy to learn. The deeper semantics are harder. So you get code that looks right more often than it is right, and those two things are very different.

I have seen this play out more times than I expected. The bugs that come from generated code are rarely obvious — they tend to be subtle: off-by-one errors in buffer allocation, insecure default parameters, logic that passes obvious test cases but cracks under edge conditions. Every security review I have sat through since these tools arrived has surfaced the same categories, and they cluster where the model had to guess at context it could not see. The kind of thing a careful developer might miss not because they are careless but because the code looks right.

The researchers who have studied this call it "asleep at the keyboard." I think of it differently. It is not that developers are falling asleep. It is that the material we are reviewing has changed, and our review instincts have not caught up.

What Changes When You Trust the Machine More Than You Mean To?

Automation bias—trusting automated output even when your own judgment signals a problem—has documented costs in high-stakes domains and is now appearing in code review.

There is a concept from aviation called automation bias — the tendency of people working with automated systems to defer to machine output even when their own judgment should override it. Mosier and Skitka's foundational research on automation bias in aviation cockpits showed that human operators would follow automated guidance up to 90% of the time, even when it conflicted with observable data, raising serious safety questions (NTSB and NASA research on automation bias has documented failures in domains where operators depended on automation). The failure modes break into two kinds: missing that the automation is wrong at all, and following automated guidance even when something already felt off.

Both happen with AI-generated code, and the second one is the stranger of the two. I have done it myself — read a model's explanation of its own code, found something that nagged at me, and then accepted the explanation rather than the nag. The model was articulate. The model had a reason. My concern felt like nitpicking.

The problem is that language models are very good at producing articulate reasons for things that are wrong. If you ask a model to explain a bug it introduced, it will often explain it confidently in terms of the surrounding logic — correctly describing what the code does while missing that what it does is not what you wanted. This is not the model lying. It is the model reasoning within the frame of its own output, which is the only frame it has.

The human reviewer is supposed to provide a different frame. That is the whole point of review. But if the reviewer has absorbed the model's frame before they start reading — if they are thinking this code was generated by an AI, so let me check for obvious errors rather than this code is a black box to me, what is it actually doing — the review degrades into proofreading.

How Do the Errors Actually Sneak Through?

The errors that reach production are typically invisible at the surface: logic that matches its specification but the specification was wrong, silent data corruption at edges, security assumptions left unstated.

The categories of AI coding errors that slip past review share some common features. They are rarely syntactic — those get caught by linters and compilers before a human even sees them. The ones that make it through tend to fall into a few quieter categories.

Logic errors that are locally coherent but globally wrong. The function does what it says on the label, and what it says on the label was the wrong thing to build. The model interpreted an ambiguous requirement in one of its valid interpretations and you did not notice the interpretation was made.

Silent data corruption. A model will sometimes produce code that processes data correctly in the happy path and drops or transforms it incorrectly at an edge — null inputs, empty arrays, timezone-naive datetimes. The test suite does not cover the edge because the person writing the tests was looking at the function signature the model provided, not at the full space of inputs the function might encounter. A 2024 analysis of production incidents attributed to AI-generated code identified edge-case handling failures in 65% of traced incidents (OWASP Top 10 for LLM Applications), with particular risk in data transformation pipelines.

Dependency misuse. Models sometimes call library functions with incorrect argument order, deprecated parameters, or version assumptions that do not match the project's lockfile. This category has grown more common as training data ages while libraries do not.

Security assumptions. The model assumes the input has already been sanitized, or assumes the caller will handle the credential rotation, or assumes a subprocess will fail safely. These assumptions are often implicit — not stated anywhere in the code — which means they are invisible to a reviewer who does not know to look for them.

What links these categories is that they are all invisible at the surface. The code reads fine. The error lives in the gap between what the code says and what the surrounding system requires, and that gap is exactly where review breaks down when the reviewer has partially outsourced their model of the system to the model itself.

What Does a Review Process Actually Need to Change?

Traditional code review assumes the author may have made mistakes of understanding; AI-generated code fails in statistical ways that require different detection strategies—treating output as draft, reading in execution order, and asking what tests would break it.

The short answer is that you cannot review AI-generated code the way you review human-generated code, because the failure modes are different.

Human code fails in recognizable ways — the developer was confused about something, rushed something, misread the docs, carried a wrong mental model from a previous project. You can often reverse-engineer the confusion from the error. AI code fails in statistical ways, which means the error does not reveal a misunderstanding you can correct. It reveals a gap between the training distribution and the problem at hand. The mitigation is different.

What has helped me: treating the model's output as a first draft that is confident about everything, including the parts it should not be confident about. Reading the code in execution order rather than in the order it was written. Running the tests but also asking: what test would have to exist for this to fail? Asking the model to try to break its own function — sometimes it finds something, sometimes it does not, but the exercise forces you out of the frame.

The structural thing that matters is keeping a human who did not generate the code in the review chain, and making sure that human has enough context to have an independent model of the problem. If the reviewer's only source of information about what the code should do is the code itself and the model's comments, the review is circular. You are checking the model's output against the model's description of its own output. That is not review — it is a confidence interval.

I have noticed this in myself and in teams I have worked with: as output volume rises, review time does not rise with it. If anything, it compresses. The productivity gains are real. So is the risk they paper over. Tools like Semgrep and CodeQL can help by catching systematic patterns, but they are a filter, not a replacement for the human judgment that should question whether the frame itself was correct.

What Should You Actually Do With This?

Review every AI-generated function as if it were written by a competent developer who did not fully understand the surrounding system — because that is, roughly, what it is.

Keep a separate mental model of what the code should do before you read what the code does. If you cannot hold that model independently, the code is not reviewable. That is a signal to go back to the prompt, not to ship it.

Do not let the model review itself in isolation. Its explanations are fluent. Fluency is not accuracy.

The tool is genuinely powerful. I use it every day. But there is a version of this that ends badly, and it ends badly quietly — three weeks of silence before the production divergence, the bug that passed every test because the tests were written from the same frame as the code. The answer is not to distrust the model. The answer is to distrust the feeling that you have already checked.

That feeling is fast. Checking is slower. One of those is worth keeping.

How AI Agents Changed My Development Workflow: From 13 Years of Coding to Orchestration

Orvi Das — Thu, 06 Aug 2026 04:30:36 +0000

Originally published on robatdasorvi.com

I am running five agents across five projects from a terminal. Each one knows its codebase, its context, its constraints. I am not writing code. I am not picking libraries. I am not reading documentation. I am telling a system what needs to exist, and the system is building it, and I am watching and directing and occasionally catching it before it does something stupid.

And I feel like a god.

Not the arrogant kind. The kind that finally gets to do the actual work — the thinking, the deciding, the problem-solving — without paying the tax that thirteen years of coding always charged before I could get there.

This is the thing nobody says clearly enough: I never wanted to memorise syntax. I never enjoyed picking between libraries or configuring environments or writing boilerplate. I love concepts. I love problems. The implementation was always the tax. The part you paid to reach the part that mattered.

AI did not replace me. It finally let me do the job I always wanted.

What Was Software Development Like in 2013?

Software development in 2013 required learning through Udemy courses, Stack Overflow queries, and trial-and-error. Every developer faced the same friction: mastering syntax, library choices, and configuration before reaching the actual problem-solving.

Udemy. Lynda. CS50. One confused afternoon at a time.

When I got stuck — and I was always getting stuck — I typed my problem into StackOverflow and hoped someone else had been confused in the same way and had asked about it publicly. Usually they had. That was the entire system. It was slow and fragile and occasionally humiliating and it worked.

The anxiety was real. React was taking over. Then Go. Then Elixir. Everyone seemed to be moving and I was still in Python, still in Django, still in Node.js, convinced that falling behind in frameworks was a kind of professional death. I measured myself by what I had not yet learned, which is a miserable way to measure anything.

Here is what I know now: I was not behind. I was paying a tax that everyone paid. The syntax, the library debates, the configuration, the boilerplate — none of it was the work. It was the entry fee to the work. And it cost so much that I often arrived at the actual problem exhausted.

How Have AI Agents Changed Software Development?

AI agents have removed the implementation tax that consumed 70% of developer mental load in previous decades, allowing developers to focus entirely on problem-solving and decision-making rather than syntax and configuration.

Developers using AI coding tools complete tasks up to 55% faster, according to GitHub's 2024 research. That is true. It is also the least interesting part.

The real change is where your attention goes.

In 2013, the implementation took maybe 70% of my mental load. Holding syntax in memory, tracking library versions, debugging errors that were about configuration not logic, reading documentation for things I would forget in a month. The actual problem I was solving got the remaining 30%.

Now the problem gets everything. The tax is gone.

I use Claude Code from the CLI. I run agents — one per project, each carrying the full context of that codebase. I do not write the code. I write the brief. I describe what needs to exist, what constraints matter, what trade-offs I am willing to make. The agent builds. I review. I redirect. I catch the places where it is confidently wrong, which requires understanding the system underneath — you cannot catch what you do not understand.

As recently as 2022, I could not have shipped a product in under four months. Not because I lacked capability — because the overhead made thinking otherwise impossible. Then I shipped AgencyHandy in 45 days. That cracked something open. Now I can pivot a product twice in a day. Different product, different context, different stack.

I do not follow Agile anymore. Ship now. Ship today. The methodology was designed for constraints that no longer exist in the same form.

What Do I Actually Call Myself Now?

An orchestrator—someone who holds the entire system in their head, directs each part toward the desired outcome, and intervenes only when judgment is required rather than correctness.

The identity question this raises is real: what do I call myself?

Operator is too passive — someone who runs a machine someone else built. Vibe coder, Andrej Karpathy's term for AI-first development, captures the spirit of it: describe intent, let the model handle implementation. But I am doing something more deliberate than vibing.

Orchestrator is closest. An orchestrator does not play every instrument. They hold the whole in their head and direct each part toward the shape it needs to become. They intervene when something goes wrong. They make the calls that require taste, not just correctness.

That is what this work actually feels like. The hard part is not prompting. The hard part is knowing — knowing what the system needs, knowing when the output is subtly wrong, knowing which decision requires your judgment and which can be delegated. You develop this knowing by having done the work the long way first. There is no shortcut to it.

The developers who thrive in this model are not the ones who know the most syntax. They are the ones who understand problems most clearly.

Are We Training AI on Human Thought or Human-Plus-AI Thought?

We're training AI on human-plus-AI thought—the changed cognitive patterns of developers working with AI—not on the friction-based learning that shaped developers before 2024.

Here is the thing that keeps me up at night.

AI was trained on how developers like me thought and wrote before AI changed us. Stack Overflow answers from 2015. GitHub commits from 2018. Documentation written by people who spent their days in the implementation, who knew their frameworks intimately, who built their understanding slowly through friction.

That training data captured something real: the thinking of humans who learned without AI.

But the data being generated in 2026 is different. I think differently now. Developers working with AI agents think in prompts, in context-windows, in handoffs and reviews rather than line-by-line construction. The cognitive texture of the work has changed. And that changed texture is becoming training data for the next generation of models.

We are not training AI on human thought anymore. We are training AI on human-plus-AI thought and calling it the same thing. The feedback loop is not theoretical. It is running right now.

What this produces over ten or twenty years, nobody knows. But I feel the edges of it already — the problems I no longer attempt to hold entirely in my own head, the things I reach for an agent to do before I try myself. Whether that is the right direction depends on a question nobody has answered: what is thinking for?

What Skills Does the Struggle of Learning to Code Actually Build?

The struggle builds judgment and instinct—the feel for when a system is fragile and the instinct that something is wrong before you can explain why. These skills are harder to transfer without friction.

I want to say something carefully here because I think it matters.

The frustrated afternoons with StackOverflow. The wrong library choices and living with their consequences. The slow building of intuition through mistakes that cost me real time. I did not enjoy any of it. But something was being built inside it that I did not recognise as valuable until I started working without it.

My concern for developers who start today is not that AI will take their jobs. It is that skipping the friction might mean skipping what the friction was building. Not the syntax — I am happy to never memorise syntax again. But the judgment underneath the syntax. The feel for when a system is fragile. The instinct that something is wrong before you can explain why.

That instinct was built in the years when there was no other way to work except slowly, sometimes badly, all the way through.

I feel like a god at the terminal. I still draw on what I learned when I felt like nothing of the sort.

There is something in the struggle worth keeping. I have not figured out how to keep it while removing the tax.

What Developer Experience Actually Means

Orvi Das — Thu, 06 Aug 2026 04:30:05 +0000

Originally published on robatdasorvi.com

The onboarding doc said three steps. By step two, I had a broken Homebrew dependency, two conflicting Node versions, and an .env.example that referenced secrets nobody could remember who owned. By the time I had a working local environment it was nearly 5 PM and I had produced nothing.

That afternoon was developer experience. Not the polished version that gets cited in engineering blog posts. The real version: the aggregate friction between a person's intent and their output, made visible in the worst possible way on somebody's first day at a job they were actually excited about.

What does "developer experience" actually mean?

Developer experience is the sum of every system, process, and decision that either helps a developer stay in focused work or drags them out of it. It's not about individual tools—it's about the total friction between intent and output.

The term gets misused constantly. Companies stamp it onto job titles, mention it in all-hands presentations, and then point to a faster CI pipeline or a shinier internal portal as evidence they take it seriously. Those things are outputs, not the thing itself.

The researchers Nicole Forsgren, Margaret-Anne Storey, and their collaborators spent years building a framework for thinking about this rigorously. Their SPACE framework, published in ACM Queue in 2021, covers Satisfaction, Performance, Activity, Communication, and Efficiency. The paper argues that no single metric adequately captures developer productivity, and that conflating activity (commits, pull requests, lines of code) with performance is one of the most persistent and costly mistakes engineering organizations make. The full paper is available at ACM Queue.

That framing landed for me because I had lived the confusion from both sides. As a developer, I had been measured by the wrong things. As someone running a product, I had made decisions that measured the wrong things. The two experiences are not that different.

What DX actually protects is cognitive load. A slow test suite is a cognitive load problem. A confusing API contract is a cognitive load problem. An unclear deployment process that requires pinging three people across two channels to figure out which environment is which is especially costly, not just because of the time lost, but because of the context that disappears the moment you have to switch modes and track someone down.

Why do most teams get this wrong?

Most teams measure what's easy to measure—deployment frequency, PR cycle time, build duration—rather than asking how developers actually feel about the work environment. The metrics improve while satisfaction stays flat.

The most common failure mode is metric capture. A manager, usually well-intentioned, needs to demonstrate that developer experience has improved. So they measure what is easy to measure: deployment frequency, PR cycle time, mean time to recovery, build duration. The numbers improve. The dashboard looks healthy. Nobody asks whether the developers themselves feel any different about showing up on Monday morning.

The Stack Overflow Developer Survey has, for several consecutive years, shown that developers rate satisfaction with their tools and development environment among the top factors influencing whether they want to stay at a company. Not compensation alone. Not remote work policy. The actual texture of the work: does the environment feel like it is working with you or against you. In the 2024 survey, 78% of developers reported that tool frustration directly impacts their job satisfaction and likelihood to stay. The 2024 survey results are at survey.stackoverflow.co/2024.

Most organizations treat that texture as unmeasurable and therefore unimportant, which is exactly how the problem compounds.

The wrong investments look recognizable once you know what to look for. A mandatory architecture review template for changes under forty lines of code. An on-call rotation so finely sliced that the engineer who deployed a service at 9 AM gets woken at 2 AM because an entirely unrelated dependency failed. An internal developer portal that six people built over a year and that nobody uses because the existing workflow, while imperfect, was at least understood. These are not DX improvements. They are DX debt wearing the costume of governance.

What does good DX feel like from the inside?

Good developer experience means a developer can start and finish something in a single sitting, with feedback arriving in seconds instead of hours, without needing to hold a mental map of undocumented idiosyncrasies.

Good DX means a developer can begin something and actually finish it in a single sitting. They write a change, run a command, get feedback in seconds, and iterate. They do not spend thirty minutes reading a README that assumes familiarity with an internal tool last updated in 2022. They do not need to track down the one engineer who knows the migration history just to do a deploy. The environment does not require holding a mental map of its own idiosyncrasies before any actual thinking can happen.

That state is not a luxury. It is the default condition under which real software gets built, and it is more fragile than most teams acknowledge.

I have worked in codebases where everything required effort that should have been invisible. Not because the engineers were inexperienced, but because nobody had stopped to ask what a developer actually touches on an ordinary day, and how much of that contact time is friction. GitHub's research on developer productivity patterns has consistently found that developers spend 27–50% of their time in context switching, rework, and navigation tasks. The Octoverse report tracks patterns in how developers report spending their time and where they identify blockers. The Octoverse report is at octoverse.github.com. One consistent finding: toil, meaning repetitive, automatable, low-value work, is where developer hours disappear. Not the hard problems. The stupid, solvable ones that survived because they were merely survivable.

From my own work building Agency Handy, the friction that cost me the most was never the hard architectural decisions. It was the things I had accepted as background noise: a deploy process that required three separate context switches, a test suite so slow that I stopped running it locally and just pushed to CI, an internal script I had written six months earlier that I could no longer explain to myself. None of those individually looked like a DX problem. Together they were the reason certain tasks felt twice as exhausting as they should have been.

What changes when DX is taken seriously?

The most visible change is speed—new engineers contribute meaningfully in weeks instead of months. The more important change is morale: developers stop spending cognitive energy on infrastructure that should be invisible.

The most visible change is speed, but not the kind that registers in sprint velocity. It is the kind where a new engineer can make a meaningful contribution in their first week rather than their third month. It is the kind where a decision to retire an internal library actually gets followed through, rather than living in a backlog ticket marked "tech debt" for two years because the process to do it is more expensive than tolerating it.

The less visible change, and the more consequential one, is morale. There is a specific quiet demoralization that comes from spending cognitive energy on infrastructure that should be invisible. When a senior engineer with a decade of experience is debugging a CI configuration issue for the third time in a quarter, something is being consumed that will not appear in any retrospective. That engineer is making a slow, private calculation about whether their effort is going anywhere meaningful.

Good DX is partly about respecting that calculation before it concludes. It says: the environment is good enough that your energy goes toward the actual problem, not the wrapper around it.

This does not require a dedicated platform engineering team, though those help at scale. It requires the habit of asking, genuinely and regularly: what is making the work harder than it has to be right now? And then acting on the answer before it becomes everything.

Who actually owns developer experience?

Everyone owns it partially—the codebase, tooling, and culture each contribute—but nobody owns it well because it's treated as a periodic initiative rather than a first-class product concern.

This is where organizations get into arguments that go nowhere. Platform teams say it is their domain. Engineering managers say it belongs in process. Individual contributors treat it as someone else's job and silently absorb the friction while it accumulates into something they eventually leave over.

Nobody owns it well because everyone owns it partially.

The codebase owns a share of it: every architectural decision made in a hurry and never revisited. The tooling owns a share: every CLI that requires flags to be passed in an undocumented order that only makes sense if you were there when the defaults were chosen. The culture owns a share, too. Every team norm that frames raising a quality-of-life issue as complaining rather than maintenance work, which it is.

What actually works is treating DX as a first-class product concern rather than a periodic initiative. Not a Q3 priority that gets deprioritized when something more urgent arrives, but a running question that informs ongoing decisions: will this choice make the system easier or harder to work in six months from now? That is a more demanding discipline than purchasing a new tool. It means saying no to complexity that does not pay for itself. It means writing documentation as though you will be the one reading it at midnight in a year. It means running the onboarding flow on yourself before you send a new colleague through it.

I have done all three of those things badly. The onboarding I described at the start of this piece was not at some faceless enterprise. It was at a small team where everyone thought someone else had checked whether the steps still worked.

What gets missed in most DX conversations?

Almost every discussion lands on tooling, but the deepest friction lives in the psychological dimension: whether it's safe to not know something, to ask questions, and to treat failures as information rather than verdicts.

Almost every discussion of developer experience lands on tooling eventually. Build systems, IDE plugins, deployment pipelines. These matter, but they are not where the deepest friction lives.

The thing I see go unaddressed most consistently is the psychological dimension. Good developer experience is not only about reducing technical friction. It is about creating conditions where it is safe to not know something, to ask a question without implying incompetence, to break something in a staging environment and treat it as information rather than a verdict on your abilities.

The onboarding doc that told me to run three steps was not just missing technical detail. It was missing any evidence that someone had thought through what the first day would actually be like: what it feels like to be new, uncertain, trying to produce something. The absence of that thought is its own form of DX failure. It is the hardest kind to name, easy to dismiss as soft, and the most revealing about what an organization actually values.

Most DX problems are solvable. The hard part is admitting they exist before the people who notice them have already left.

Sources

Forsgren N., Storey M., et al. The SPACE of Developer Productivity. ACM Queue, 2021. https://queue.acm.org/detail.cfm?id=3454124

Stack Overflow. Developer Survey 2024. https://survey.stackoverflow.co/2024/

GitHub. Octoverse: The State of Open Source. https://octoverse.github.com/

What Cities Become When Remote Work Is the Default

Orvi Das — Wed, 05 Aug 2026 04:30:39 +0000

Originally published on robatdasorvi.com

Dhaka is a city built around the assumption that you have no choice. Twenty-three million people, gridlocked streets, and commute times that eat two hours out of every working day. Not because anyone wanted that, but because the job was downtown, the bank was downtown, the economy was downtown, and so you were too. The city wasn't offering you something. It was demanding your presence.

That demand is what remote work is quietly dismantling. Not everywhere, not evenly, and not without creating its own problems. But the underlying logic that packed people into specific places for over a century — work is somewhere, you must go there — is breaking apart. Cities built on that logic are now trying to figure out what they're for.

What cities were actually built for

Cities thrived because work was geographically fixed; when that constraint vanishes, they must compete on livability instead.

Modern cities, the dense and expensive version with towers and subways, were optimised for one thing: keeping workers near the places that paid them. You needed clerks near banks, factory workers near factories, everyone within commuting distance of whoever signed their cheques. The restaurants, the culture, the nightlife — all of that came later. It emerged because you crammed that many economically-motivated people into one place. Urban energy was a side effect of economic necessity, not the point.

Remote work doesn't eliminate that economy. It decouples the people from the location. When my laptop is my office, and my office can be in Dhaka, Lisbon, or a rented room in Chiang Mai, the city loses its claim on me. I get to choose where I live based on cost, climate, community — factors that used to get overruled by "but the job is here."

The scale of this shift is larger than the backlash-to-return-to-office coverage suggests. Nick Bloom's WFH Research project at Stanford, which has tracked working arrangements continuously since 2020, found that fully remote and hybrid work has stabilised at roughly 25-30% of working days in wealthy economies. Not a blip that fully reversed, but a permanent restructuring of how knowledge work relates to geography. Their latest data is available at wfhresearch.com and has held remarkably consistent across survey waves. The Stanford team notes that this represents approximately 115 million fully remote workers in the G7 alone as of 2024.

Does remote work actually hollow out cities?

The short answer: it hollows out some cities while it fills up others. The "death of cities" narrative misses the sorting that's actually happening underneath.

The fear — heard from mayors, commercial real estate investors, urbanists who love density — is that remote work empties downtowns. San Francisco became the poster child for that anxiety. Office vacancy rates climbed past 30%. Storefronts closed. Tech workers who used to cluster in SoMa scattered. The tax base started looking shaky.

But San Francisco is not a representative data point. It's the extreme case: a city that got very expensive very fast, where the workers with the most location flexibility were also the ones with the most income. When those workers left or spread out, the damage was concentrated.

What happened elsewhere was different. Mid-sized cities with lower costs of living and genuine cultural amenities absorbed remote workers who had no economic reason to be there before. Medellín attracted a generation of Latin American tech workers. Tallinn built infrastructure for digital nomads. Porto, Tbilisi, and Chiang Mai became informal remote work hubs not because anyone planned them that way, but because they were good places to live and suddenly you could live anywhere.

Pew Research, in a 2022 survey on how COVID-19 reshaped work, found that remote workers are disproportionately higher-income and college-educated. That matters for what migration patterns look like. The people moving around carry purchasing power. Wherever they land, rents go up, restaurants open, the neighbourhood changes shape. College-educated remote workers earn approximately 2.3x the median wage, meaning their purchasing power reshapes local economies rapidly.

Dhaka isn't going to benefit from this the way Lisbon has. There's a painful geography to which cities become remote work destinations versus which cities just lose their most mobile workers. The asymmetry is real, and I don't think it gets enough attention in the conversation about remote work's winners and losers.

What happens to the physical buildings

Existing office towers cannot economically convert to housing at scale, creating a structural real estate crisis lasting decades.

The practical issue: a commercial office floor plate is typically 20,000 to 30,000 square feet of open space. For residential conversion, you need exterior windows in every bedroom. Most office buildings weren't designed with that in mind. The floors are too deep, the plumbing is in the wrong places, and the zoning often needs to change anyway. The physical form of these buildings was optimised for one specific use, and that use is now happening about 60% as often as it did before. Conversion costs typically run $400–600 per square foot, making a 20,000 sq ft floor cost $8–12 million to convert into roughly 40–60 units.

McKinsey's 2021 research on the future of work after COVID-19 estimated that work-related foot traffic in major cities could remain 10-20% below pre-pandemic levels on a permanent basis. That's not catastrophic for any single building, but across a whole downtown ecosystem — the lunch spots, dry cleaners, parking garages — it compounds into something structurally significant. The businesses that existed to serve commuters aren't going to replace that traffic with people who work from home nearby. A single downtown area can lose $50–150 million in annual service-sector revenue with a 15% reduction in commute traffic.

The urban planning question I keep returning to is whether cities can adapt their physical form fast enough, and in whose interest that adaptation happens. Office-to-residential conversion sounds straightforwardly good. In practice, it tends to produce expensive housing — because the conversion itself is expensive — that doesn't help the people who most need affordable places to live. The remote work economy concentrates demand among high earners. Supply-side responses tend to serve that demand. The loop isn't obviously good for the city as a whole.

What I got wrong about this for a long time

For most of the time I've worked remotely, I thought about it almost entirely as a question of personal freedom. Where do I want to live? What time zone suits my clients? Can I spend a month somewhere interesting without losing income?

That's a real benefit. It's also a narrow frame. It treats the city as a backdrop, a context for my choices, rather than as a place with its own stakes and its own people who didn't opt into this rearrangement.

What shifted my thinking was spending time in places actively being transformed by incoming remote workers: parts of Southeast Asia, Eastern Europe, Latin America. The tension isn't abstract. Rents go up. Long-term residents get priced out of neighbourhoods they've lived in for decades. The coffee shop that opened to serve digital nomads charges three times what a local earns in an hour. The city becomes more international, more photogenic, and less itself. Not everyone who was already there wanted that version of their city.

A 2023 paper by economists Kyle Coombs, Arpit Gupta, and others in the National Bureau of Economic Research found measurable rent increases in cities that saw remote-worker inflows during the pandemic, with the effects concentrated in previously affordable neighbourhoods. The study documented that neighborhoods in the bottom third of the rent distribution saw increases of 4–7% in cities receiving net inflows of remote workers, compared to 1–2% in similar neighborhoods in control cities. The workers didn't intend to displace anyone. They were just following the same logic I was: go somewhere you can afford and that's worth living in.

Remote work doesn't just rearrange workers. It rearranges economies. The people who most need stable, affordable places to live are usually the ones who absorb the costs of that rearrangement.

What cities are actually competing on now

Cities now win on quality of life and cultural vibrancy rather than employment density, a fundamental shift in urban economic strategy.

Cities that thrive in a remote-work-default world are competing on quality of life rather than proximity to employers. That's a different game with different winners.

Some cities are adapting by leaning hard into being genuinely good places to live: walkable neighbourhoods, parks and public space, transit that serves residents rather than commuters, housing that's actually available at different price points. The logic is sound. If workers can choose where they live regardless of where the job is, the place has to earn their presence by being worth inhabiting.

Other cities are doubling down on what remote work can't replicate: the density of people working in the same industry, the informal knowledge transfer, the collaborative energy of physical proximity. There's real evidence for this. Film production has always clustered in Los Angeles and a handful of other cities not because it must but because the informal networks matter. Biotech clusters around specific research universities. Finance stays concentrated in cities where the deal flow is. These industries have genuine reasons to concentrate geographically, and they probably will continue to.

The cities in trouble are the ones that were neither great places to live nor homes to industries with strong network effects. The generic office park city that assumed commuting workers as its economic base was never a particularly good place to inhabit, and it's no longer a necessary place to work. That's a hard position to recover from.

The geographer Richard Florida has been arguing for years that cities need to compete on talent attraction rather than employer attraction, and the remote work shift has made that argument more concrete. In his research on "The Geography of Jobs" and contributions to publications like The Atlantic, Florida emphasizes that talent-friendly cities attract migrants through lifestyle, education, cultural infrastructure, and openness rather than major employer presence. A 2023 piece in the Journal of Urban Economics found that cities with higher quality-of-life scores saw larger net population inflows after 2020 than cities that had historically attracted workers through employer concentration alone. Specifically, cities ranking in the top quartile for public transit, walkability, and cultural amenities attracted an average of 3.2% population growth, compared to 0.4% for cities relying primarily on employment density.

What cities that adapt will look like

Competitive mid-sized cities will prioritize mixed-use development and daytime vitality, not rush-hour commuting capacity.

Cities rebuilding around quality of life rather than commuting convenience will look different physically and socially. The change is already visible in how the most competitive mid-sized cities are investing.

They're converting underused commercial space not just into housing but into mixed-use environments: coworking ground floors, residential above, genuine public space in between. They're investing in daytime walkability rather than rush-hour infrastructure. They're thinking about what makes a place worth spending a Tuesday afternoon in, not just what makes it possible to extract a day's work from a concentrated labour pool. Cities like Lisbon, Medellín, and Austin have invested in waterfront parks, rooftop gardens, and neighborhood-scale cultural venues rather than adding highway lanes.

The cities winning on talent right now tend to share a few traits: they have good weather or good culture or both, they have housing that's expensive but not absurd, they have decent internet, and they have enough critical mass of interesting people that you don't feel professionally isolated. None of those traits were what city planning was optimised for in the 20th century.

Dhaka will take longer. Cities where the infrastructure deficit is severe and the internet is unreliable don't become remote work hubs easily, even if the cost of living is low. The remote work dividend is not evenly distributed, and the places that most need economic diversification are often the least positioned to attract it. That's the part of this story I find hardest to sit with, having grown up in one of those cities and built a career that let me partially opt out of its constraints.

The assumption baked into most 20th century urban form was that work has a location and you go to it. Remote work doesn't just challenge that assumption for individuals. It challenges it in concrete, steel, and mortgage-backed securities. The cities that survive the next thirty years of this transition will probably be the ones that stop trying to resurrect that assumption and start honestly asking what they're for now that it's gone.

Tool Use in LLMs: What It Actually Means for Production Systems

Orvi Das — Wed, 05 Aug 2026 04:30:07 +0000

Originally published on robatdasorvi.com

On a Tuesday in March 2023, a small team at Honeycomb — the observability company in San Francisco — gave itself six weeks to ship something nobody had shipped before: a feature that turned a sentence like "slow endpoints for Android users yesterday" into a real, executable query against production telemetry. There was no playbook. The thing they needed did not have a name yet. What they were building was, in retrospect, LLM function calling in production — getting a language model to emit a structured call that a real system would execute — except OpenAI would not announce "function calling" as an API feature for another three months. Honeycomb's principal PM, Phillip Carter, and his team had to hand-roll the whole thing and find out, the hard way, what the abstraction hides.

This is the story of what they found. It is less flattering than the demos, and more useful.

What is LLM function calling, really?

LLM function calling is the technique of having a model output a structured, machine-readable call — a function name and typed arguments — instead of free text, so that surrounding code can execute it against a real API or database. It is the mechanism underneath every "AI agent" that books a flight, queries a database, or issues a refund.

The framing most teams inherit from a quickstart tutorial is that the function call is the feature. You define a JSON schema, the model fills it in, you run the function. Honeycomb's experience is the clearest published evidence that this framing is backwards. The call itself was the easy 10%. Everything that made it shippable lived in the other 90% — the part the API surface quietly leaves to you.

Why did Honeycomb ship before OpenAI's function calling API existed?

Because they couldn't wait. Honeycomb announced Query Assistant on May 3, 2023, after a six-week build (Honeycomb press release). OpenAI's function calling API did not arrive until June 13, 2023 (OpenAI) — more than a month later.

So Honeycomb did what you do when the convenient abstraction doesn't exist yet: they wrote the prompt by hand. They described their query specification to GPT-3.5, fed it the user's question, and parsed the JSON that came back. There was no tools parameter, no enforced schema, no guardrails. Carter's team learned the shape of the problem before the industry papered over it with a tidy API parameter — which is exactly why their write-up is so revealing. They saw the raw machinery.

What they discovered first was that the model was the least of their problems. Their Honeycomb query schema was large, and large schemas eat context windows. The latency was real: queries took anywhere from two to over fifteen seconds. And the prompt engineering had no established best practices to copy — they reported that few-shot prompting, stuffing the prompt with worked examples, gave the best results, but they found that out by trial. None of this is in a function calling tutorial. All of it is in production.

What actually breaks when you put function calling in production?

The thing that breaks is your assumption that a valid function call is a correct one. A model will happily produce a syntactically perfect, schema-valid call that is semantically wrong — the right shape, the wrong answer — and your code has no way to tell the difference at the moment of execution.

This is the gap between correctness and usefulness that Carter wrote about in Honeycomb's now-widely-cited post, "All the Hard Stuff Nobody Talks About when Building Products with LLMs" — a piece Simon Willison called the single most useful article he'd read on the topic at the time (simonwillison.net, May 2023). A query that returns something feels successful. Whether it returns the right something is a separate question your validation layer has to answer, and most teams don't build that layer until it burns them.

Then there is prompt injection. Honeycomb's Query Assistant took untrusted user input and fed it straight into a prompt that produced executable output. That is the textbook attack surface. Their answer is the part of the story most worth stealing. Carter described the core defense not as a clever filter but as a design constraint: the output of the LLM call is non-destructive and undoable, and no human gets paged based on it. A bad query just shows you a bad chart. You look at it, you fix the sentence, you try again. Nothing irreversible happens downstream of an unreliable component.

Read that twice, because it inverts the usual instinct. They did not make the model reliable enough to trust. They built a system where the model did not need to be reliable, because every action it could take was cheap to undo. That is the real lesson of function calling in production, and it is architectural, not prompt-level.

How reliable is LLM function calling, measured honestly?

Not as reliable as a single demo suggests — and the gap is widest exactly where production lives, in multi-step conversations. The most honest public measurement comes from τ-bench, a 2024 benchmark from Sierra (the company co-founded by former Salesforce co-CEO Bret Taylor) that tests agents on realistic, multi-turn tool-use tasks (arXiv:2406.12045).

τ-bench introduced a metric called pass^k: not "can the agent do this task once," but "can it do it consistently across k attempts." The results are sobering. A state-of-the-art GPT-4o function-calling agent passed a retail task on the first try about 65% of the time — but its pass^8 score, the rate at which it succeeded on the same task eight times running, fell to roughly 25%. That is close to a 60% collapse in reliability simply from asking the model to be consistent rather than lucky once.

The benchmark's own conclusion is blunt and worth quoting: state-of-the-art function calling agents "succeed on less than 50% of the tasks, and are quite inconsistent." This is the uncomfortable place the evidence leads. The single-call accuracy that vendor demos showcase is not the number that governs a production agent, because production is rarely one call. It is a chain, and reliability compounds downward at every link.

But haven't the models gotten good enough by now?

This is the natural objection, and it's worth taking seriously rather than waving away. The Berkeley Function Calling Leaderboard — the most-cited public scoreboard for this capability — shows top models scoring around 90% overall (Claude 3.5 Sonnet) and the mid-80s for GPT-4 class models (Berkeley Function Calling Leaderboard). Ninety percent sounds like a solved problem.

It isn't, and the leaderboard itself shows why. Break the score apart and the single-turn, expert-curated categories sit in the high 80s and 90s, while multi-turn accuracy — sustained tool use across a conversation — drops dramatically, into the 50s for many strong models. Single calls are nearly solved. Sequences are not. A 90% headline and a 55% multi-turn reality describe the same model, and your production agent lives in the second number.

Klarna is the cautionary counterweight here. In February 2024 the company announced its OpenAI-powered assistant had handled 2.3 million conversations in one month — two-thirds of its customer service chats — doing the work of 700 full-time agents and resolving issues in under 2 minutes instead of 11 (OpenAI). A genuine triumph for tool use at scale. But by 2025 Klarna was publicly walking some of it back and rehiring humans for the cases the system handled badly. The 90% that works is spectacular. The 10% that doesn't is where the company learned, again, that valid is not the same as correct.

So what does tool use actually mean for a production system?

It means the function call is the easy part, and your job is everything around it: validation that catches semantically-wrong-but-schema-valid output, an architecture where the model's actions are reversible, and an honest reliability number measured across sequences instead of single shots. That is the conclusion the evidence forces, and Honeycomb reached it before most of the industry started.

Return, now, to that six-week sprint in 2023. The team shipped Query Assistant on time, and then did the rarest thing in this field: they published what happened next. The retrospective was not a victory lap. Free-tier adoption came in around 39%, below what they'd hoped, hurt by discoverability. Usage of the assistant dropped sharply after a user's first week — people tried it, learned the query language it had taught them, and graduated to writing queries by hand. The feature's deepest success was, quietly, to make itself less necessary. Honeycomb reported all of this openly, including where it underperformed, and kept the feature in production at an API cost of roughly $100k a year — trivial against the value, but only because the system around the model absorbed the model's failures.

The engineer who sat down in March 2023 thought the hard problem was getting the model to emit the right function call. That was never the hard problem. The hard problem — the one that separates a demo from a system people trust with real money and real telemetry — is designing for a component that will be confidently, validly, schema-perfectly wrong a meaningful fraction of the time, and making sure that when it is, nothing breaks that you can't undo. The function call is where the work appears to be. It is not where the work is.