DEV Community: Chetan Munigangappa

State Machines & Durable Execution with just Postgres

Chetan Munigangappa — Sun, 28 Jun 2026 16:40:13 +0000

I spent a weekend learning every component I'd need to build a durable state machine from scratch. Queues, checkpoints, recovery logic, lease management, idempotency keys, backoff strategies, the lot. I was sketching out the architecture for a walkthrough post, convinced that durable execution meant either assembling it yourself or buying into someone else's distributed system. I had pages of notes. Diagrams. A half-written Python prototype that handled polling and crashes, complete with a janky requeue mechanism for orphaned tasks that I already knew would race under load. Then I found Absurd, and realised someone had already built exactly what I was mapping out — except they had done it inside PostgreSQL, using stored procedures and native tables, and the result was so obvious in retrospect that I felt slightly foolish for not seeing it sooner.

The database you already run is the only orchestrator you need for durable execution, and the specialised platforms exist to solve a scale problem most teams do not have.

The Lie of Stateless Logic

We design backend systems as if execution is atomic and infrastructure is polite. The gap between that assumption and production reality is where user data disappears.

I learnt this the hard way on a sign-up flow. A container restart mid-deployment killed the process halfway through creating an account. The user record had been written to the users table. The payment intent was stuck in "processing" because the Stripe API call had fired but the callback handler never ran. The welcome email never sent. When I traced it the next morning, I found three half-created accounts, two duplicate emails from users who'd hit "sign up" again after hearing nothing, and a trail of application logs that simply stopped mid-line. The database had partial state. The email provider had no record. The users had no confirmation. And the application itself had no memory of any of it because the execution state had lived in a process that no longer existed.

The fix my team reached for was familiar. Add retries. Make the steps idempotent. Wrap the whole thing in a transaction and hope the database commit finishes before the pod dies. But transactions do not span external API calls. Idempotency keys expire. Retries without state are just gambling with different odds. We treated the symptoms because treating the cause would mean admitting that our architecture was built on a fiction: the fiction that a function call is a single uninterrupted breath. In production it is interrupted constantly. Deployments roll. Nodes evict. Networks partition. The state you need to survive these interruptions is not in your application code. It is in the execution state itself — the exact line you reached, the result you obtained, the decision you made — and if you haven't made that explicit and durable, you do not have it at all. You have hope dressed up as architecture.

The worst part is that this pattern is not rare. It is the default. Every framework that promises "scale out of the box" teaches you to keep your functions stateless and your data in the database. But it never teaches you to keep your execution state there too. So you end up with a database full of business data and a fleet of processes full of implicit, ephemeral, unrecoverable execution state. When the process dies, the business data is safe but the execution is nonsense. The database is durable. The application is not. And the gap between them is where your users suffer.

The Orchestration Tax

The standard response to this fragility is to outsource the problem. Buy a platform. Let AWS Step Functions, Temporal, or Airflow handle the state machine for you. The promise is durability without the thinking. The reality is that you have traded one failure mode for another, and most teams pay the tax before they need the benefit.

My engineering lead said something like: "Step Functions is the safe choice. AWS manages it." So I opened the pricing calculator. A three-step sign-up flow — validate, process, confirm — costs per state transition. Every retry, every pause, every wake-up is a billable event. The console shows you a pretty graph of your execution, but when something sticks at 02:00, that graph is the last thing you need. You need the raw state. You need to know which checkpoint failed and why. And the managed abstraction has deliberately made that opaque because its business model is based on you not needing to look inside. You are paying for a black box that bills you by the glance. The JSON DSL compounds it — you are asked to express business logic in a format designed for machine consumption, then charged for the privilege of executing it. A simple loop becomes a nested state machine. Error handling is a separate branch.

Temporal is more honest — it gives you the code — but it also gives you a cluster to run, a database to back it, and a set of failure modes that now belong to you. Airflow brings its own scheduler, its own workers, its own dependency hell. You are now running a distributed system to manage a state machine, when the database you already run is itself a state machine.

The pattern is not new. Telephone exchanges in the 1920s used discrete states and explicit transitions. TCP handles packet loss by recording progress and resuming from the last acknowledged byte. The idea of durable execution is ancient. What is modern is the insistence that implementing it requires infrastructure you do not already have.

Yes, these platforms handle scale and cross-team coordination that a single Postgres instance cannot. But most workflows are not at that scale. The tax is paid from day one: the DSL to learn, the cluster to operate, the pricing model that penalises you for every transition in a long-running flow. The smaller your team, the larger the burden of the orchestrator relative to your actual work. A team of five should not be running a Temporal cluster. Step Functions is a plane. Temporal is a plane. Absurd is a bicycle. Most of your journeys are short enough that the bicycle is faster, cheaper, and more fun.

The Postgres State Machine

Absurd proves that the only part of the stack that needs to be durable is the database, and it already is.

The "Just Use Postgres" meme has circulated for years. Absurd makes it concrete. It pushes workflow state, queues, and task coordination directly into native PostgreSQL tables using stored procedures. Each queue provisions its own set of tables — t_ for tasks, r_ for runs, c_ for checkpoints — all native rows you can query with psql when you need to. Workers are thin stateless processes. They poll for tasks via SKIP LOCKED, acquire a time-limited database lease, execute business logic, and write checkpoints back. The lease is just a timestamp on the row. If a worker crashes before writing the next checkpoint, the timestamp expires, the row unlocks, and another worker picks it up on the next poll. No coordinator. No push pipeline. Just Postgres and a polling loop.

Before any of that runs, you provision the queue. One command, or a Liquibase changeset if you prefer migrations tracked in source control:

absurdctl create-queue signup --storage-mode partitioned

That creates the t_signup, r_signup, c_signup tables and registers the queue's retention policy. Everything from here lives in the database.

The worker is a Python process, thin enough to read in one screen:

import logging
from datetime import datetime, timezone
from absurd_sdk import Absurd

logger = logging.getLogger("absurd-worker")
app = Absurd(queue_name="signup")

@app.register_task("provision-user", default_max_attempts=5)
def provision_user(params, ctx):

    # Step 1: Create user record
    def create_user_record():
        return {
            "user_id": params["user_id"],
            "email": params["email"],
            "created_at": datetime.now(timezone.utc).isoformat(),
        }

    user = ctx.step("create-user-record", create_user_record)

    # Step 2: Simulate a transient failure so the retry behavior is visible
    outage = ctx.begin_step("demo-transient-outage")
    if not outage.done:
        ctx.complete_step(outage, {"simulated": True})
        raise RuntimeError("temporary email provider outage")

    # Step 3: Send activation email
    def send_activation_email():
        return {"sent": True, "provider": "demo-mail", "to": user["email"]}

    delivery = ctx.step("send-activation-email", send_activation_email)

    # Step 4: Suspend until the activation event arrives
    activation = ctx.await_event(
        f"user-activated:{user['user_id']}",
        timeout=3600,
    )

    return {"user_id": user["user_id"], "status": "active", "activated_at": activation["activated_at"]}

if __name__ == "__main__":
    app.start_worker(worker_id="worker-1", concurrency=4)

Each ctx.step() call is a checkpoint. The function inside only runs if that checkpoint does not already exist in c_signup. The deliberate failure in step 2 is the thing worth watching closely. When the worker crashes after completing step 1, the next run reads the cached result for create-user-record, skips the side effect entirely, and continues from send-activation-email. You can see this in the logs directly:

# Attempt 1 — creates the user record, then fails
[019ee555] creating user record for alice
[019ee555] simulating a temporary email provider outage

# Attempt 2 — replays the checkpoint, skips step 1, continues from step 3
[019ee555] sending activation email to alice@example.com
[019ee555] waiting for user-activated:alice

Step 1 did not run twice. No duplicate record. No duplicate email. The database remembered what the application had forgotten.

The await_event call suspends the task — releasing the worker's lease — until the client emits user-activated:<id>. At that point, a new run is scheduled and the task completes. The Bun client that triggers all of this is equally thin:

import { Absurd } from "absurd-sdk";

const app = new Absurd({ queueName: "signup" });

Bun.serve({
  port: 3000,
  routes: {
    "/users": {
      POST: async (req) => {
        const body = await req.json();
        const spawned = await app.spawn("provision-user", body, {
          queue: "signup",
        });
        return Response.json(spawned, { status: 202 });
      },
    },
    "/users/:userId/activate": {
      POST: async (req) => {
        const { userId } = req.params;
        await app.emitEvent(`user-activated:${userId}`, {
          activated_at: new Date().toISOString(),
        });
        return Response.json({ emitted: `user-activated:${userId}` });
      },
    },
  },
});

Here is the full sequence of what happens between them:

sequenceDiagram
    autonumber
    participant Client as App / Scheduler
    participant DB as PostgreSQL (Absurd Engine)
    participant Worker as Pull Worker

    Client->>DB: Spawn Task (Payload + Idempotency Key)
    Note over DB: Validates key, writes row to t_<queue>

    Worker->>DB: Poll Task via SKIP LOCKED
    DB-->>Worker: Lock acquired, lease assigned
    Note over DB: Writes attempt tracking to r_<queue>

    Worker->>DB: Write Step Checkpoint
    Note over DB: Persists value to c_<queue>, extends lease

    Note over Worker: Crash / Network Fault (Lease Timeout)

    Worker->>DB: Re-Poll Task via SKIP LOCKED
    Note over DB: Increments attempt in r_<queue>
    Worker->>DB: Read Checkpoints
    DB-->>Worker: Cached Step Results
    Note over Worker: Skips executed side-effects

    Worker->>DB: Complete Task (Final Result)
    Note over DB: Updates state in t_<queue>

When I simulated a container restart with this setup, the task resumed from the last completed checkpoint. No half-created accounts. No stuck payment intents. The difference is not incremental — it is architectural. Before, the durability of the flow was an emergent property of multiple systems behaving correctly together: the application server, the message queue, the database, the orchestrator. Now the durability is intrinsic to the storage. The queue is a table. The checkpoint is a row. The lease is a timestamp. There is no emergent behaviour to debug when things go wrong. There is only PostgreSQL, and you already know how to debug that.

The Visibility Problem

Trading away the orchestrator is only half the battle. You cannot trade away the need to see what is happening.

The primary risk of database-native workflows is the black box. When your state machine lives in PostgreSQL tables, you need a way to inspect it without writing raw SQL at 02:00 while a production task is stuck. Habitat is the UI Absurd provides for this: a read-only web dashboard that connects directly to Postgres and shows queues with task counts, per-task run history with each attempt's status and error, checkpoint JSON for every completed step, and event payloads with timestamps. You can see exactly which step a stuck task reached and what it returned, without reconstructing it from CLI output. It is not a luxury. It is the difference between a database trick and a system you can actually operate.

Without it, you are left with psql and a growing set of internal queries that someone wrote once and no one dares change. Habitat removes that friction. It runs as a single Go binary with an embedded frontend, pointed at the same Postgres instance:

./habitat run -db-name mydb -db-user postgres -db-password postgres
# navigate to http://localhost:7890

Or drop it into Docker Compose alongside the worker and client, and it just works.

But the limitations are real and worth naming. This is single-instance PostgreSQL. It does not have built-in cross-region replication. The operational model shifts from "managed service I pay for" to "system I own and understand." You need to know how PostgreSQL locking works. You need to monitor table bloat and configure pg_cron for cleanup. This is not a managed service. It is a tool that respects your intelligence and demands your attention. That is a trade-off, not a defect.

The honest boundary is this: if you need cross-team coordination at scale, or you are orchestrating workflows across a hundred microservices, Step Functions or Temporal earn their keep honestly. They are not bad tools. They are solving a real problem — just not the one most teams have. For the majority of workflows that live inside a single service boundary, you are paying for complexity you do not use.

The most sophisticated architecture is often the one that removes the most components. The platforms selling you durability are not wrong. They are just solving a problem most teams do not have, at a price most teams should not pay.

The full working example — queue setup, Python worker, Bun client, Docker Compose, and Liquibase migrations — is in the Absurd repository. The documentation covers partitioning, cron patterns, rolling deployments, and agent tooling.

The Quiet Restructuring: AI inside Corporations

Chetan Munigangappa — Sun, 08 Mar 2026 19:44:19 +0000

I spent the first two weeks of this year reading every AI prediction I could find, then stopped. Not because the writing was bad. Some of it was sharp. But the forecasts were expiring faster than I could finish them. A new model would drop, a vendor would announce something, a benchmark would get shattered, and whatever someone had written in January felt like archaeology by February. The half-life of a one-year AI prediction is shorter than a sprint cycle.

The deeper problem was what they were measuring. They counted automatable tasks, ran benchmarks, estimated job exposure. None of them asked how organisations actually decide to restructure, or what happens when the cost curves shift but the org chart doesn't. They treated software engineering as a set of tasks to optimise rather than a function embedded in institutions with their own logic and friction.

So instead of betting on what a model release does to your Q3 velocity, this piece looks at what five years of compounding AI adoption does to the organisations building software. To answer that properly, you have to start somewhere most predictions skip: how companies actually work.

How Software Organisations Work

The org chart on the company website is not wrong, exactly. It's just incomplete in the ways that matter.

Organisations don't function as pyramids. They function as translation machines, and the translation happens in layers, each with a different time horizon and a different kind of work.

The C-suite operates on the longest horizon. They own the three-to-five year bets: market position, regulatory programmes, platform migrations that span multiple budget cycles. Crucially, they often don't know every project currently running in the organisation. They don't need to. Their job is to set direction and constraints, not track the work. A CEO who knows the details of every active sprint has bigger problems than an inefficient org chart.

Below them sits the first middle layer. Directors, senior managers, heads-of. These are the translators. They take a strategic initiative like "reduce infrastructure costs by 30% over three years" and decompose it into programmes that can actually be staffed, scoped, and delivered. This is not mechanical work. It requires understanding both the strategic intent and the operational reality well enough to know when the two are in tension. A director who can't push back on an initiative that's been scoped unrealistically isn't doing the job. McKinsey's research found that just over half of companies regularly translate strategic goals into three-to-seven year financial plans, meaning the long-horizon bet always exists in tension with quarterly cost pressure. The first middle layer lives in that tension every day.

Below them sits the second middle layer: managers, project managers, delivery leads, scrum masters. These are the executors. They take programmes and make them happen, coordinating across teams, tracking dependencies, translating requirements into tickets, escalating blockers, reporting status upward. The work is real and the pressure is constant. But the nature of the work is fundamentally different from the layer above it. The first layer exercises judgment about what to build. The second layer exercises coordination to make sure it gets built.

That distinction — judgment vs coordination is the one the org chart doesn't show you. And it's the one that matters for everything that follows.

One important caveat before we go further: none of this describes small businesses or lower-medium sized organisations. In a 20-person company, the founder is the C-suite, the first middle layer, and often the second. Decision-making is fast, hierarchy is flat, and the layers described above either collapse into one person or don't exist at all. The dynamics here apply to organisations large enough to have grown the full stack, typically from around 200 people upward, where the coordination burden has grown large enough to justify dedicated roles for it.

What AI Changes

The second middle layer doesn't grow because companies are wasteful. It grows because visibility has a cost, and that cost scales with complexity.

When a director hands a programme to an engineering team, they need to know if it's on track. Not in real time, but reliably enough to escalate before something becomes a crisis. Status reports flow upward. Risk registers get updated. Sprint ceremonies create checkpoints. The system works because human communication between layers requires human intermediaries to manage it.

As organisations grow, the surface area requiring visibility expands faster than the programmes themselves. Each new team is another node. Each new dependency is another handoff that needs tracking. The answer companies have consistently reached for is more coordination: more people whose job is to make sure other people know what is happening. This is how you end up with what Zuckerberg described when restructuring Meta: managers managing managers managing managers managing the people actually doing the work. His diagnosis was right even if the fix was blunt. You cannot remove a coordination layer without replacing the coordination function. The question is whether that function still requires humans to perform it.

For most of the current second middle layer, it doesn't. Not entirely, and not immediately, but directionally.

The project manager's work splits into two functions that rarely get separated. The first is synthesis: reading across Jira, GitHub, budget trackers, and vendor systems to assemble a coherent picture of programme health for the director. The second is planning: working with directors on next quarter's capacity, budget allocation, and programme scope. Today both require a human because the underlying systems don't talk to each other in any meaningful way. Someone has to read each one, reconcile the signals, and produce the output.

Agentic AI systems are beginning to make the synthesis function redundant. The early harnesses like OpenClaw that genuinely execute across tools rather than simply respond to queries point at what this looks like in practice: an agent that reads your Jira board, watches your GitHub PRs, tracks budget burn, and surfaces a coherent programme picture without a human assembling it. When that synthesis becomes reliable, the project manager's time reclaims itself. What remains is the judgment work: scope tradeoffs, stakeholder management, pushing back on unrealistic timelines, working with directors on planning cycles that currently consume weeks of manual data assembly. That work was always the more valuable half of the role. It just got buried under the synthesis overhead.

For engineering managers the change cuts deeper, and in a different direction. The constraint on how many direct reports a manager can effectively handle has never been the number of humans they could track. It has been the quality of attention they could give each one. Career conversations, technical mentorship, performance calibration, helping someone work through a problem they are stuck on. These are not coordination functions and they cannot be delegated to a dashboard. What AI changes is the overhead around them: the status chasing, ticket grooming, and ceremony facilitation that consumes the hours that should be spent developing people. In the 1980s the average managerial span was 1-to-4 direct reports. Information technology moved that closer to 1-to-10. AI moves it further still.

But span widening is the smaller part of the story. The bigger part is what engineers are now being asked to become.

For most of the past decade, engineers were reduced to implementers. Handed tickets, kept away from stakeholders, insulated from the business context that would have made their work meaningful. AI is reversing that. The engineer who only closes tickets is being displaced by tooling. What survives is the engineer who can engage with the problem domain, work directly with stakeholders, own outcomes rather than tasks. That is a significantly harder job to grow someone into than the one the industry settled for. Career conversations get more complex. Mentorship requires more than code review. Performance calibration becomes about judgment and domain understanding, not velocity metrics. The engineering manager who was already stretched thin on four direct reports, spending most of their time in ceremonies and status updates, now has more reports, deeper development conversations, and engineers whose scope of responsibility has expanded substantially. The coordination overhead coming down is what makes that possible. It is not optional relief. It is the condition that makes the expanded role survivable.

For directors the change is about bandwidth. Their job is translating strategy into programmes and making judgment calls about priority and scope. AI doesn't do that. What changes is the fidelity and speed of the information they are working from. A director who currently waits for a weekly status report will instead have a live synthesis across the programme portfolio. The planning cycle that currently takes weeks of manual data assembly collapses to days. The director constrained by information velocity becomes constrained by their own judgment speed, which is as it should be.

For the C-suite the change is about calibration. They don't need to know every project, but they do need to sense when strategic intent and execution are drifting apart. AI-assisted synthesis makes that drift visible earlier. The quarterly business review becomes less about assembling the picture and more about interrogating it. The leaders who thrive will be those who use the freed bandwidth to engage more deeply with the long-horizon bets that actually determine their company's position, not those who use it to meddle in execution they never needed to own.

The through-line across all layers is the same. Coordination work that was performed by humans because the systems didn't talk to each other becomes automated. Judgment work that was always the point becomes the primary occupation. The hierarchy doesn't disappear. It shrinks. The same organisational function gets performed by fewer people, each with a wider span and a sharper focus on the work that actually matters.

What shrinks is not value. What shrinks is the overhead that was obscuring it.

Opportunities, Competition and New Industries

The scripted work goes away first. Password resets, order status, appointment scheduling, tier-one support. It should. That work was never the interesting part of any job. What remains is what was always hardest to scale: the customer disputing a charge they don't understand, the stakeholder who needs someone to work through a problem that fits no category in any decision tree. Those interactions need someone who can listen past the surface to the actual problem, make a judgment call, and leave the other person feeling heard rather than processed. The role doesn't disappear. It sheds what it was never good at.

The WEF's 2025 Future of Jobs report puts numbers on the shape of this: 92 million jobs displaced by 2030, 170 million new ones created. Over 85% of employment growth since 1940 came from technology-driven job creation and the pattern has been consistent across every wave. This cycle is no different in that respect. What changes is where the new jobs concentrate: toward complexity, toward domain knowledge, toward the work that requires someone who understands the problem well enough to exercise genuine judgment about it.

The mechanism driving the expansion is inference cost. GPT-3.5-level performance dropped from $20 per million tokens in November 2022 to $0.07 by October 2024, a 280-fold reduction in eighteen months. Projects that failed an ROI calculation in 2021 need recalculating at 2025 prices. The long tail of businesses that enterprise software never properly served — too small for SAP, too complex for off-the-shelf tools is now economically addressable. A 15-person logistics company can build custom route optimisation. A regional accountancy firm can offer AI-powered client tools that would have required a dedicated engineering team two years ago. The assumption that custom software was for enterprises is becoming false faster than most small business owners realise. The constraint has shifted from cost to capability: not "we cannot afford this" but "we need someone who understands our domain well enough to build this right."

That space also includes a significant amount of bad code. Humans have been shipping debt-laden software since long before AI arrived. AI didn't invent the problem, it inherited the tendency and gave it a faster engine. GitClear tracked an eightfold increase in duplicated code blocks during 2024, with 46% of code changes consisting entirely of new lines while refactored and moved code dropped sharply. MIT professor Armando Solar-Lezama called it a brand new credit card for accumulating technical debt in ways we were never able to before. The expansion creates its own counter-demand: the backlog of systems needing someone who can read them, diagnose them, and make principled decisions about what to fix grows alongside the new projects. Who fixes the slop matters more than who created it.

Healthcare is the clearest domain that has crossed a viability threshold. Buying cycles have compressed from 12-18 months down to under six, yet 80% of the market remains untapped. Prior authorisation systems that trap clinical staff in paperwork producing no patient value. Voice interfaces for patient engagement a two-doctor practice could never previously afford. Diagnostic support tools surfacing patterns across patient records at a scale no clinician could maintain unaided. These are not incremental improvements. They are categories of software that didn't exist as commercially viable products three years ago. The engineering required to build them correctly — integrating with legacy clinical systems, navigating data governance constraints, designing for safety-critical failure modes — is hard in ways no amount of AI assistance substitutes for. That hardness is the opportunity.

Legal and compliance follows the same pattern with added regulatory tailwind. The EU AI Act, the EU Cyber Resilience Act, evolving data sovereignty requirements across jurisdictions: each a new surface area of compliance work organisations need software to manage. Contract review, regulatory change monitoring, audit trail generation. Work that previously required expensive specialist time, or simply wasn't done rigorously, is now viable at a cost that makes sense for organisations of all sizes. Regulation is a forcing function for software investment, and the current environment is generating more of them than the industry has seen in a decade.

Manufacturing and industrial software is crossing a different threshold entirely. Predictive maintenance systems previously requiring expensive specialist integration can now run on existing sensor infrastructure at a fraction of the cost. Digital twins for factory floors, simulations that let operators model consequences before making changes, are moving from enterprise-only to mid-size manufacturers. The engineering problems are genuinely difficult: real-time control loops, safety-critical systems, integration with legacy industrial hardware designed before the internet existed. That difficulty is not a barrier. It is a moat for engineers who can navigate it.

Security sits in a category of its own because AI is simultaneously creating the problem and generating the demand for people to solve it. The AI cybersecurity market is projected to reach $86 billion by 2030, driven by accelerating attack surface expansion and a talent shortage that already stood at 4.8 million unfilled positions before agentic AI began proliferating across enterprise stacks. Attackers use the same foundation models, the same code generation tools, the same agentic frameworks that defenders do. The threat surface expands every time a new agent ships, every time a developer uses AI to generate integration code without understanding the security model of the library they're calling. What the next five years demand is the engineer who reasons adversarially: who identifies how a system might be exploited before it is, who treats security as a design constraint from the first conversation rather than a compliance checkbox before launch. Every IT position is becoming a cybersecurity position.

The competitive consequence of all this is the part most organisations haven't fully priced in. The inference cost collapse doesn't just create new markets. It compresses the time advantage incumbents used to enjoy. A category of software that took three years and a dedicated team to build in 2021 now takes months for a well-scoped team with domain knowledge and the right tools. The window between a new entrant identifying an opportunity and being able to ship something real into it has shortened dramatically. Organisations that have restructured around speed — flatter hierarchies, engineers with broader scope, faster decision cycles — will move through that window. Organisations still running the full coordination stack will still be assembling the business case.

Evolution, Not Revolution

The next five years of AI in software organisations will disappoint everyone waiting for a dramatic moment.

There will be no single announcement, no model release, no product launch that draws a clean line between before and after. The AI-augmented software of 2025 — features bolted on, LLM wrappers shipped as products, "powered by AI" in every marketing deck will quietly give way to something less visible and more consequential. AI-native organisations: built around what the technology actually does well, with engineers who understand the domain before they touch the model, and hierarchies structured around judgment rather than coordination. The transition will feel unremarkable as it happens and significant in retrospect.

The hype will persist throughout. Every new model release will generate another wave of disruption predictions, another round of job displacement headlines, another set of quarterly forecasts that expire before anyone finishes reading them. The noise is structural. The signal is slower and less exciting. DORA's finding that a 25% increase in AI adoption produces a 2.1% productivity lift is not a headline. It is the honest starting point for a compounding argument that plays out over years, not quarters.

What compounds is not the technology. It is the organisational adaptation. The companies figuring out how to work AI natively rather than additively. The engineers expanding their scope rather than defending their tickets. The managers using freed coordination overhead to actually develop their people. The directors making faster, better-informed decisions because they are no longer waiting for a human to assemble the picture. Those advantages are invisible quarter by quarter. Over five years, they are structural.

The organisations that get this right will not look like they did something dramatic. They will look like they quietly got better at the things that always mattered: judgment, speed, and people who understand the problem deeply enough to solve it. The ones that get it wrong will still be running the same coordination overhead with a layer of AI tools bolted on top, wondering why the productivity gains never arrived.

Not disruption. Evolution & Accumulation.

AI in SDLC: A 2025 Retrospective

Chetan Munigangappa — Sat, 28 Feb 2026 14:01:32 +0000

It's February 2026 and I've got a fractured leg, which turns out to be the right conditions for looking back at a year that moved too fast to examine properly at the time. Laptop on the sofa, nowhere to be, no standups to attend, no Slack pings filling the gaps between thoughts. Just the kind of enforced pause that doesn't happen in a normal engineering career.

And what keeps coming back is 2025. Specifically, how different the work felt, and why.

David Farley defines software engineering as "the application of an empirical, scientific approach to finding efficient, economic solutions to practical problems in software. It requires practitioners to become experts at both learning effectively and managing complexity sustainably."

Sitting here with time to actually think, I keep returning to that definition. Not as a textbook quote — as a recognition. It describes exactly what 2025 forced a return towards.

We forgot this. Or rather, we let others forget it for us. And 2025 was the year the consequences became impossible to ignore.

How Software Engineers Became the Least Important People in Software Engineering

Somewhere in the 2010s, we collectively decided that software engineering was a coding problem. The bootcamp explosion promised that anyone could learn to code in twelve weeks and immediately qualify for a six-figure job. General Assembly, Lambda School, Flatiron School — they all taught variations of the same curriculum: React, Rails, JavaScript fundamentals, maybe some basic database work. The implicit promise was that coding was the skill that mattered. Learn the syntax, learn the frameworks, and you were an engineer.

This was never true, but it was convenient. Convenient for the industry, which needed implementers faster than universities could produce them. Convenient for business people, who wanted to believe that the "vision" and the "strategy" were the hard parts, while the coding was just execution. Convenient for the bootcamps themselves, which could sell a transformational experience in a timeframe that fit between unemployment benefits and desperation.

What we actually produced was a generation of developers who knew how to build components but not how to decide what components to build. Developers who could write React but couldn't sit with a stakeholder and understand why a feature mattered. Developers who knew the technical implementation of a user story but had no context for the business problem that story was supposed to solve.

The "full-stack" developer became the industry's darling — not because full-stack represented deep competence, but because it represented flexibility. One person who could do everything, which really meant one person who could be assigned to any ticket without complaining. The stack didn't matter; what mattered was that we'd found a way to make engineers interchangeable.

Meanwhile, the complexity didn't go away. It just got managed by people who weren't trained to manage it. Business founders and product managers took ownership of "the vision" — often without understanding what was technically possible. Designers took ownership of "the experience" — frequently without understanding the data models that would have to support their interfaces. Architects emerged to "guide" technical decisions, often from positions where they no longer wrote production code. Engineering managers optimised for velocity metrics that measured activity rather than outcomes.

Every new role that appeared was built on the same assumption: engineers couldn't be trusted with the full picture. They needed translation, guidance, oversight. The person who actually understood the system — who knew where the complexity lived, which assumptions were fragile, what would break under load — had the least authority to influence decisions.

Technical debt became a "developer problem" rather than a business reality. Refactoring became something you did "when you had time" between feature deliveries. The complexity engineers managed was invisible to business stakeholders, which meant it was unvalued. When engineers tried to explain why a simple-sounding feature would take weeks, they were seen as making excuses rather than describing constraints.

The bootcamp model reinforced this dynamic by design. They taught React not because React was the right tool for every problem, but because React was what employers wanted. They taught CRUD applications because CRUD applications were easy to teach and easy to evaluate. They produced developers who could follow tutorials, copy patterns, and implement specifications — but not developers who could define problems, evaluate tradeoffs, or own outcomes.

This wasn't the fault of the bootcamp graduates. They were doing exactly what the system asked of them. The fault was with an industry that had convinced itself that coding was the valuable part, and that everything else — understanding context, designing approaches, managing complexity sustainably — was someone else's job.

You could say I'm being elitist about bootcamps — that they gave access to people who couldn't afford CS degrees. But this isn't about credentials. It's about what we taught. Bootcamps taught coding as a commodity skill because that's what the industry demanded. The critique is of an industry that wanted implementers, not engineers. Access matters. What we give people access to matters more.

By the early 2020s, software engineers had become the least important people in software engineering. We were the ones who actually built the systems, who understood how they worked, who managed the complexity that everyone else ignored. But we weren't the ones who decided what to build, or why, or for whom. We'd been reduced to expensive typists, implementing decisions made by people who didn't understand their implications.

We called it "collaboration." It was mostly translation — endless meetings where engineers tried to explain technical constraints to business people who didn't want to hear them, and business people tried to explain user needs to engineers who weren't allowed to talk to users directly. The boundary between roles wasn't about efficiency; it was about control. And engineers had lost it.

Then 2025 happened.

When AI Exposed the Gap

The first time I used Copilot to generate a React component in early 2025, I felt a strange mix of exhilaration and dread. The exhilaration was obvious — I'd just written a complex form handler in seconds instead of minutes. The dread took longer to identify. It wasn't that the AI was going to replace me. It was that the AI was making visible something I'd been trying not to see: the coding was never the hard part.

I'd spent the previous decade optimising for coding speed. Learning new frameworks, mastering type systems, keeping up with the JavaScript ecosystem's relentless churn. All of that became nearly worthless overnight — not because the AI could do it better, but because the AI could do it fast enough that the difference between "good at coding" and "competent at coding" stopped mattering.

What the AI couldn't do was understand why we were building something. It couldn't sit with risk engineers and learn how they actually processed reports. It couldn't evaluate whether a technical approach would scale with the business, or whether we were solving the right problem, or what would happen when the edge cases we hadn't considered inevitably appeared.

Those weren't coding problems. They were engineering problems. And they'd been my problems all along — I just hadn't been allowed to own them.

In January 2025, I started building a risk assessment platform for insurance underwriters. LightRAG would process their reports and generate standardised grading according to company guidelines. In 2024, this would have triggered the full organisational machinery: product manager for discovery, designer for workflows, architect for technical approach, probably three engineers for six months of implementation.

Instead, I started with the SDK. Not because someone prioritised it in a roadmap, but because my data scientist partner needed something to test their prompt engineering against. They needed real inputs and real outputs, a way to iterate on scoring guidelines without waiting for a full platform. So I built a simple Python library — ingest reports, run them through LightRAG, return structured grading. A few days' work.

While they tested prompts, I built the server backend. No handoff documents. No estimation rituals. Just parallel work coordinated through conversation. The speed was disorienting. I'd spent years waiting — for requirements, for designs, for approvals — and now there was nothing to wait for. The work was just... done. Then I could do more work.

But the real shift came when I did something I'd never done before. I sat down with the risk engineers themselves. Not through a product manager who would translate. Not by reviewing personas someone else created. I sat in their workspace, watched them process reports, understood the actual pain of their current workflow. Then I sketched UI flows on a whiteboard whilst they told me what would work.

This would have been impossible in 2024. Not because I lacked the skills — I could always sketch, always ask questions. But because the structure prevented it. The structure said that was product's job, or design's job. The structure said engineers implement, they don't discover. The structure said you wait for specifications, you don't create them.

We refined those mockups over a week. I'd sketch something, they'd try it in their actual workflow, we'd identify what didn't work, I'd iterate. When we landed on something that felt right, I wrote the requirements documentation myself — translating their domain knowledge into technical specs whilst I could still ask clarifying questions.

The platform grew organically. SDK became foundation. Backend took shape. UI emerged from those collaborative sessions. When the first business line was stable, I started conversations with the second business line directly — understanding their variations, adapting what we'd built. In 2024 this would have required a product manager, a designer, multiple engineers, six months to get started. I delivered the complete platform — SDK, backend, UI, two business lines, end-to-end stakeholder management within a year, alone.

What made this possible wasn't "10x coding." The AI helped with boilerplate, sure. But what actually made it possible was exercising the full engineering competence that Farley defined: understanding the problem deeply enough to design an efficient, economic solution. The coding was trivial. The engineering was not.

When coding becomes fast, engineering judgement becomes visible. When implementation is cheap, understanding the problem becomes valuable. The things we'd offloaded to PMs and designers — understanding stakeholders, designing workflows, making tradeoffs — turned out to be engineering work after all. We'd been solving problems all along. We just weren't allowed to own the solutions.

What the Acceleration Broke

The transition wasn't clean. Three things broke, and they all point to the same root cause: treating engineering as coding for so long meant that when the full competence was suddenly required, neither organisations nor people were ready for it.

My engineering manager said something like: "Now that AI handles the coding, you've got bandwidth for more." The scope expanded. The timeline didn't. Headcount stayed static. The assumption was that coding had become solved, freeing up capacity for "higher value work."

But what he called "higher value work" was actually the engineering I'd been doing all along — understanding context, designing approaches, owning outcomes. The cognitive load increased whilst the recognition of that load didn't. The organisation saw efficiency gains and demanded more capability from the same people, without acknowledging that "doing it all" requires different skills, different energy, different support than doing one well-defined piece.

In mid-2025, an intern joined our team. Bright, eager, armed with the same AI tools I used daily. I gave them a CI pipeline to migrate — a straightforward work based on a template and documented decisions. They stared at it, then reached for the AI. The AI completed the job, but not the whole job, there were environments and team standards and internals the AI didn't understand. The deployment failed, the tool suggested a fix and they applied it without understanding it. As expected this led to a successful deployment with the failed application. In this particular case, the error was trivial enough to be missed by us even in PR review.

This isn't their fault. They were doing exactly what the system taught them: code fast, use tools, deliver features. The system just never taught them that understanding matters more than output. AI accelerates experienced engineers because we already have patterns in our heads. For juniors, the same tools prevent those patterns from forming. We're creating a generation who can generate but can't understand.

Then there was the production incident. The platform shipped fast because implementation had "no delay." There was no pause to learn Datadog properly, to understand observability best practices, to set up meaningful dashboards and alerts. I knew how to build the thing. I hadn't given myself time to learn how to operate it.

When something broke at 2am, I was debugging blind. The dashboards were there — I'd set them up quickly, checking boxes without understanding what I was looking at. The metrics didn't tell me what I needed to know because I hadn't learned which metrics mattered. I fixed the immediate issue, but I didn't understand why it had happened, and that meant I couldn't be sure it wouldn't happen again.

All three incidents stem from the same source. When coding is all you value, everything else becomes invisible. The organisation saw speed and demanded more of it. The junior saw tools and skipped the work. The senior saw a delivery target and missed the operational depth. Same mistake, three levels.

The Return to Form

The concrete problem isn't philosophical. The tools that accelerate experienced engineers actively damage junior formation. AI removes the struggle that builds intuition. Bootcamps already taught implementation over understanding. Combine them and you get developers who can produce but can't evaluate — who can generate code but can't hold a mental model of what that code actually does.

The industry will need to deliberately rebuild how engineers are trained and mentored. Not by going back to the old gatekeeping — access matters — but by designing structures that force understanding. Assign problems where AI can't be the first answer. Require explanation, not just generation. Build in the pause that acceleration removes. Create the friction that learning requires.

This is structural, not personal. New graduates aren't doomed. The structure that trained them needs to change — from teaching coding as a commodity skill to teaching engineering as problem-solving. From producing implementers who follow specifications to producing engineers who can sit with stakeholders, understand constraints, and own outcomes.

"Finding efficient, economic solutions to practical problems." That's the job. It always was.

The rest — the tickets, the ceremonies, the handoffs, the theatre of process that let everyone feel important whilst obscuring who was actually responsible — that was the deviation. We built an industry around the assumption that engineers couldn't handle complexity, then wondered why the complexity kept overwhelming us. We optimised for coding speed and forgot that understanding matters more than typing.

AI didn't create a new kind of engineer. It revealed that the old definition was always the right one. We let business people convince us that coding was the valuable part. In doing so, we let ourselves become the least important people in the room. 2025 was the year that stopped working.

The engineers who thrive in 2026 won't be the ones who prompt best. They'll be the ones who can understand a problem, design a solution, implement it reliably, and learn from it properly. The ones who never forgot — or who are now remembering — what software engineering actually means.