DEV Community

Cover image for Why Software Deadlines Always Slip (And Why Everyone Seems Fine With It)
Arthur
Arthur

Posted on • Originally published at pickles.news

Why Software Deadlines Always Slip (And Why Everyone Seems Fine With It)

Every senior delivery manager I have ever known has the same career anecdote. They have never, across some round number of years, watched a major release ship on the date set at kickoff. Not once. They will say this with a kind of dry resignation, in the same tone people use to describe the weather. It is the universal experience of running software projects, and it is the universal experience of running them in companies that produce software for a living.

I want to take that observation seriously rather than file it under "project management is hard." If something happens consistently across forty years, across sectors, across team sizes, across methodologies that openly disagree about almost everything else, that is not a series of unfortunate local failures. It is a structural feature of how the industry runs. The interesting question, the one this piece is about, is not why deadlines slip. The data on that is settled. The interesting question is why the deadline ritual continues to be performed exactly the same way despite the data being settled.

The data, briefly

The numbers are easy to find and have not improved in any direction that matters in twenty years.

BCG's 2024 study, "Most Large-Scale Tech Programs Fail: How to Succeed," reports that more than two-thirds of large-scale technology programs fail to deliver on time, on budget, or to scope, and that the share of programs failing on all three has been roughly constant for a decade. The Standish Group's CHAOS Report, in its 2020 edition, put the breakdown at 31% successful, 50% challenged, and 19% outright failed — and Standish then announced it would stop publishing the report on the grounds that nothing was changing and the act of publishing it had stopped being useful. The miss rate has been measured, restated, benchmarked across tens of thousands of projects, and the consistent finding is that the deadline-promising ritual produces predictable misses on a predictable schedule.

This is not a team-specific failure. It is the industry's running average. Any given project is, by historical baseline, more likely to slip than to ship.

The four mechanisms behind the miss

Practitioners who have run a few hundred projects describe four mechanisms that, between them, account for almost every miss they have personally watched.

The first is requirements churn. Pretty much no project survives the journey from kickoff to release without absorbing changes the team did not anticipate. A stakeholder sees a draft mockup and realises they meant something else. A regulator publishes a new rule mid-build. A competitor ships a feature that becomes a must-match. The pattern in mid-sized projects is something like three to five serious scope shifts during a six-month build. Estimates given at kickoff are not estimates of the project that will eventually ship. They are estimates of a project that no longer exists by week three.

The second is hidden complexity. The first integration with a third-party API turns out to be against documentation that was written eighteen months ago and is no longer accurate. A legacy table in the data warehouse has six edge cases nobody flagged at design time. A "trivial" auth flow has a quirk that surfaces on the seventh week of the build. There is a documented pattern, repeated across delivery managers, of integration estimates that come in two to four times larger than the kickoff number — not because the team underestimated the integration but because the integration's actual surface area was not visible from the outside until the team was already in it.

The third is the planning fallacy. The cognitive bias has a name and a citation. Daniel Kahneman and Amos Tversky introduced it in 1979 in "Intuitive prediction: Biases and corrective procedures," describing the universal human tendency to underestimate the time, cost, and risk of future tasks even when the same person, in the same domain, has documented evidence that prior tasks took longer than predicted. The bias is robust across cultures and disciplines. Experienced engineers are no less susceptible than first-year ones; the experiment was repeated through the 1990s and 2000s with the same result. The mind, when planning, takes an inside view of the task — the specifics of this case, this customer, this codebase — and systematically discounts the outside view that asks how similar tasks have actually played out historically.

The fourth is the keyboard-time-versus-cycle-time confusion. When an engineer says "this will take three days," they almost always mean three days of focused implementation. The release calendar pays for the entire task lifecycle: design review, code review, revisions, test runs, integration tests, environment provisioning, deployment, stakeholder demo, customer feedback, fixes, redeploy. A commonly-cited rule of thumb among delivery managers is that pure coding time is roughly 30 to 40 percent of the total task duration. The team is not wrong about the coding. The team is being asked to predict the lifecycle and pricing only the part of it they directly control.

Hofstadter's Lawit always takes longer than you expect, even when you take into account Hofstadter's Law — was published in 1979, the same year as the planning-fallacy paper, in Gödel, Escher, Bach. Software's main contribution to the literature in the half-century since has been to confirm both findings.

The honest practice already exists

What is striking about all of this, on close inspection, is that the working version of estimation is not unknown. Senior delivery managers, when they have the latitude, do exactly the same handful of things.

They decompose work into chunks of one or two days, because the planning-fallacy literature is clear that estimates on small chunks aggregate more accurately than estimates on the same body of work made in one large breath. They quote ranges instead of points: not "three weeks" but "two to four weeks, with a P50 of 2.5 and a P90 of 4." They itemise risks as separate line items rather than burying them inside a padded estimate, on the principle that a risk that is named can be retired and a risk that is hidden cannot. They keep a rolling forecast, updated weekly or per sprint, on the basis that a week of actual data is worth more than any number of pre-kickoff predictions. Mature teams keep an internal log of past estimates against past actuals and derive a multiplier for their own work; the multiplier is reportedly stable around 1.7 to 2.0 in delivery groups that track it consistently.

None of this is new. None of this requires new tooling. The Project Management Institute has been describing variants of this practice in its body of knowledge since the 1990s. The agile literature describes essentially the same approach in different vocabulary. There is no estimation-method gap.

So why the single date

The interesting question is what the single date is doing in the room at all, given that everyone competent who has worked on the problem agrees the single date is wrong.

The single date is doing a social job, not a forecasting one. A range with risk lines makes the team's uncertainty visible to the customer, the executive, the board. It says, in writing: we do not know yet. Most organisations, by their cultural physics, find that statement intolerable. The customer asks "when will it be ready?" and the political reflex is to give them a number. The executive asks the project manager to commit. The project manager commits to a date that nobody in the room actually believes. The build runs, the date misses, the team takes the heat, and the next project starts the same conversation in the same language.

This is not the engineers' fault, and it is not the project managers' fault, and it is not even the executives' fault in any meaningful sense. It is the standing trade-off the field has chosen: a comforting fiction at kickoff, paid for at the end in slippage costs, blame, and burned-out teams. The data on slippage rates is forty years old, and it has stayed flat through every methodology that promised to fix it, because the methodology has never been the problem. The approaches that try — the ones that ask the customer to absorb uncertainty in writing, up front — get adopted in pockets, and almost never spread outside organisations whose senior leadership has personally chosen to treat estimation as a probability problem.

The 2026 AI footnote

By 2026 most major issue trackers ship some form of AI-assisted estimation. They look at the team's historical data, recognise patterns, flag anomalies, and produce probability distributions instead of single numbers. Industry coverage of McKinsey-cited research reports that ML models trained on a team's own delivery history produce estimation accuracy improvements of roughly 20 to 30 percent over expert-only estimates, which is real but should not be oversold. The AI does the part of the job the literature already knew how to do. It does not change the social transaction. The customer still asks for a single date; the system records the distribution's mean; the dashboard shows the comforting number; the political incentive is unchanged. The tools improve the input. The conversation that consumes the input is the same conversation it has been since 1979.

Why nobody seems to mind

The honest answer is that the deadline ritual is, for most stakeholders, not actually about deadlines.

It is about commitment theatre. The customer wants to feel that the supplier is taking the work seriously. A confident date communicates seriousness in a way that a probability distribution does not. The supplier wants to feel that the customer is committed enough to the work to be told a date in the first place. The PMO wants a clean Gantt chart for the board deck. The board wants the Gantt chart to be green. The team wants the executive to stop asking. Everyone in the system has a small, local, rational reason to prefer the comforting number, and the system as a whole pays for those small rational preferences with two-thirds of large programs missing on time, budget, or scope.

The miss is not a bug. The miss is the price the system has decided to pay for the comfort of the kickoff conversation. Forty years of data suggest the price is stable.

What an honest practice would look like

A team that wanted to break the pattern, on a single project, could do exactly what the senior delivery managers already know how to do. Decompose to small chunks. Quote a range, not a point. Itemise risks. Update the forecast every week. Keep a journal of estimates versus actuals and let it train the next round. Tell the customer the truth: that ranges land more often than dates do. The data on the practice is good. The practice is not the obstacle.

The obstacle is what happens in the meeting where the customer says "yes, but when?" — and somebody in the room has to decide whether to say a number that the team does not believe, or to insist on a probability distribution and absorb the social cost of doing so. That moment is the entire problem, and it is reproduced, in roughly the same form, in every kickoff meeting in the industry, every week.

The ritual

The deadline ritual will keep producing the deadline outcome for as long as the social cost of "I do not know yet" exceeds the political cost of being wrong six months later. That is not a project-management problem. That is the field's standing trade-off, and the field has been making the same trade for forty years.

The data is not the problem. The methods are not the problem. The estimation literature is not the problem. The problem is that the room does not, when push comes to shove, want to hear a probability distribution. The room wants a date, accepts the date, and treats the slippage as a discrete misfortune rather than the rate the system was always going to produce. As long as that remains the standing arrangement — and it has remained the standing arrangement through every wave of "this methodology will fix it" — every project will keep shipping late, every senior delivery manager will keep collecting the same anecdote, and the next kickoff will keep asking the same question and getting the same answer, in the same tone, from the same people who already know better.

Top comments (0)