DEV Community: Doogal Simpson

Software Estimation: Why It Fails and How to Fix It

Doogal Simpson — Fri, 14 Aug 2026 15:51:03 +0000

TL;DR: Software estimation fails because uncertainty increases with unfamiliar tasks. While frameworks like story points, t-shirt sizing, and hours exist to abstract this complexity, accurate estimation ultimately relies on comparative sizing, tracking historical velocity, and maintaining strict team discipline rather than finding a magical unit of measurement.

If I ask you how long your morning routine takes, you can probably pin it down to a 15-minute window. If I ask you how long it takes to drive 300 miles, you'll give me a solid estimate, though a bad traffic jam might add an hour or two. But if I ask you to paint a bedroom, and you've never held a roller, your estimate goes out the window. You'll forget to buy primer, underestimate drying times, or realize halfway through that you don't actually know how to prep drywall.

Software development is exactly like painting that unfamiliar bedroom. The less familiar the codebase, API, or framework, the more spectacularly our estimates fall apart. We are constantly asked, "How long will this take?" and yet, our answers are notoriously wrong. Why is this, and how can we do it better?

Why is estimating software delivery time so hard?

Estimating software is difficult because we are rarely building the exact same thing twice, meaning every project introduces novel, unseen technical challenges. As unfamiliarity with a codebase or technology increases, our ability to predict obstacles drops exponentially. We are not just performing repetitive labor; we are solving unique logic puzzles where the scope of work often changes as we uncover new information.

Think of software estimation as navigating a maze where the walls shift. When you estimate a task, you're accounting for the code you know you have to write. But you aren't accounting for:

The legacy module that breaks when you update a dependency.
The undocumented API behavior that requires a complete rewrite of your integration layer.
The local environment configuration issues that eat up an entire afternoon.

Because our work is inherently non-repetitive, traditional time-based estimation models fail us.

What are the most common software estimation methodologies?

Teams generally use three main methodologies to estimate work: hourly estimates, t-shirt sizing (S, M, L, XL), and Fibonacci-based story points. Hourly estimates focus on literal duration, while t-shirt sizes and story points attempt to abstract time by focusing on relative effort and complexity. Each approach trades precision for speed of planning.

Here is how these three methodologies stack up against each other:

Estimation Method	Primary Metric	Best For	Main Drawback
Hourly Estimates	Real-world calendar time	High-discipline, well-defined legacy tasks	Prone to optimism bias; ignores context-switching
T-Shirt Sizing	Broad buckets of effort	High-level roadmap planning and discovery	Too vague for sprint-level execution planning
Story Points	Relative complexity & risk	Agile sprint planning with historical velocity	Teams often secretly convert them back to hours

While many Agile purists champion story points (using Fibonacci numbers like 1, 2, 3, 5, 8 to compare tasks rather than assign hours), the truth is that these numbers are just comparators. They don't represent days; they represent how one task relates to another.

How can development teams make software estimates more accurate?

To improve accuracy, teams must shift from trying to guess absolute time to measuring relative complexity against historical reference tasks. By comparing new tasks to completed ones and tracking actual velocity over time, you can project delivery dates based on real data rather than developer optimism. Additionally, building in a buffer for "unknown unknowns" is essential for handling unexpected technical debt.

Interestingly, the only time I have ever seen estimation work flawlessly—where burndown charts actually burned down to zero on schedule—was with a team that used raw hours. But it wasn't the unit of measurement that saved them; it was their sheer discipline.

They tracked every hour meticulously, adjusted their remaining hours daily, and aggressively descoped tasks the second they slipped. If your team lacks that level of extreme discipline, trying to estimate in strict hours will lead to broken promises. Instead, use relative estimation (like story points), look at your team's historical delivery rate, and let the data calculate the timeline for you.

FAQ

Should we use story points or hours for Agile estimation?

Use story points if your team struggles with optimism bias or works in a highly variable environment, as they focus on complexity rather than time. Only use hours if your team has the extreme discipline required to track and update remaining hours daily, and if the work consists of highly predictable, repeatable tasks.

What is the "Cone of Uncertainty" in software engineering?

The Cone of Uncertainty is a concept showing that project estimation accuracy is lowest at the beginning of a project when the least is known. As development progresses, requirements become clearer, and technical paths are validated, the range of estimation error narrows significantly.

How do you handle estimates when a task has high technical uncertainty?

Do not estimate the entire task upfront. Instead, create a time-boxed "spike"—a short research task designed to write throwaway code, test integrations, or explore a library. Once the spike is complete and the uncertainty is resolved, you can estimate the actual implementation task with much higher accuracy.

Graph Databases for Real-Time Fraud Detection

Doogal Simpson — Wed, 12 Aug 2026 15:35:50 +0000

Quick Answer: Graph analytics detects fraud by mapping connections between seemingly unrelated entities—like people, addresses, and transactions—into a unified network. Unlike traditional relational databases, graph databases use relationship-focused algorithms to uncover hidden fraud rings, sometimes revealing connections that are a little too close to home.

I want to tell you a quick story about bankers, software, and corruption.

A while back, I was involved with a really cool piece of technology. The premise was simple: you take a massive pile of unstructured documents from a company, parse them, and extract key entities—people, addresses, credit cards, and cars. By linking these entities together, you build a massive "social network" of data.

Once you have this network, you run graph algorithms over it to find fraud. And let me tell you, fraud is incredibly easy to spot when you visualize it as a network.

We sent some consultants to demo this software at a tier-one bank in a country that historically struggled with systemic corruption. The consultants spent two weeks on-site, processing the bank's data and mapping out these networks.

At the end of the two weeks, they gathered a room full of senior executives for the final sales pitch. They pulled up the software, showed how the network visualization worked, and began presenting real examples of detected fraud rings.

Unfortunately for the consultants, the software worked a little too well.

As they walked through one of the largest fraudulent networks on the screen, they realized they had accidentally identified some of the executives sitting right there in the room as active participants in the fraud ring.

The consultants were politely—and then very rapidly, not-so-politely—escorted out of the building, told to get on a plane, and leave the country immediately.

Interestingly, the bank did eventually buy the software. It worked, after all. But that demo proved a fundamental truth: graph-based relationship analysis is one of the most powerful tools in a developer's arsenal for uncovering hidden patterns.

How does graph-based entity resolution find hidden fraud?

Graph-based entity resolution identifies fraud by combining disparate data points into a unified network of nodes and edges, exposing indirect relationships that traditional analysis tools miss. By linking identifiers like shared phone numbers, physical addresses, or device IDs, it exposes coordinated networks rather than isolated bad actors.

When you look at fraud through a standard transactional lens, everything can look perfectly normal. A transaction occurs, a form is filled out, and an account is created.

But the magic happens when you shift focus from the individual data points to the connections between them. Think about how we might analyze credit card applications: individually, five different applications look completely clean. But if your graph database reveals that all five registered with the same physical address and two share a phone number, a coordinated network suddenly jumps out at you.

Why do traditional relational databases fail at network analysis?

Relational databases fail at network analysis because they rely on rigid table structures and resource-intensive JOIN operations to connect data points. As the depth of relationships increases past two or three degrees of separation, the computational cost of running these JOINs makes real-time analysis impossible.

The architectural divide between how relational databases and graph databases manage highly connected data is night and day:

Feature	Relational Databases (RDBMS)	Graph Databases (NoSQL/Graph)
Data Model	Tables, rows, and foreign keys	Nodes (entities) and Edges (relationships)
Query Mechanism	SQL JOIN operations	Graph traversal (index-free adjacency)
Deep Link Performance	Degrades exponentially with each JOIN	Constant lookup times regardless of depth
Schema Flexibility	Rigid; altering schemas requires migrations	Flexible; properties can be added dynamically
Primary Use Case	Transactional logging and structured reporting	Relationship mapping and network analysis

In a graph database, relationships are stored as first-class citizens directly on disk. Finding a connection doesn't require scanning an index table; the database simply hops from one node to the next. This makes traversing deep networks lightning-fast.

Which graph algorithms are most effective for fraud detection?

The most effective graph algorithms for fraud detection are community detection, shortest path, and node centrality. These algorithms allow systems to automatically group suspicious accounts, track the flow of illicit funds, and identify the primary orchestrators of fraud rings.

Once your data is structured as a graph, you don't have to manually hunt for bad actors. You can deploy specialized algorithms to surface these anomalies out of the box:

Community Detection (e.g., Louvain Modularity): This algorithm partitions the graph into tightly knit clusters. If a cluster has an unusually high density of connections and shares a few key resources, it often indicates a coordinated fraud ring.
Pathfinding (e.g., Shortest Path): This determines the fewest steps between two nodes. It is incredibly useful for anti-money laundering (AML) because it can quickly show if clean account A is connected to blacklisted account B through a chain of intermediary accounts.
Degree Centrality: This measures how many connections a single node has. A node with an unusually high degree of centrality—such as one phone number linked to dozens of bank accounts—is a red flag for synthetic identity theft.

Frequently Asked Questions

How does graph database performance scale with massive financial datasets?

Graph databases scale efficiently because they use index-free adjacency, meaning query times are determined by the size of the subgraph being traversed, not the overall size of the database. However, memory management is critical, as large-scale graph traversals require significant RAM to keep active nodes cached.

What are the challenges of extracting graph nodes from unstructured document data?

The main challenge is entity resolution—ensuring that "John Doe," "J. Doe," and "John A. Doe" extracted from different PDFs are correctly mapped to a single node. This requires integrating Natural Language Processing (NLP) pipelines and fuzzy matching algorithms before writing the clean data to the graph.

Can graph databases run fraud detection in real-time?

Yes. While deep community detection algorithms are typically run as batch jobs, simple pathfinding and neighborhood traversal queries can run in milliseconds. This allows financial institutions to evaluate the risk profile of an entity during the transaction authorization window.

How Defensive UI Design Prevents Production Outages

Doogal Simpson — Tue, 11 Aug 2026 15:08:55 +0000

When a system failure occurs, firing the operator who triggered it does absolutely nothing to prevent the next incident. Real reliability comes from building fault-tolerant user interfaces and processes. By focusing on fixing the UI rather than blaming the operator, we keep our systems online and our teams productive.

I still remember reading about the absolute panic in Hawaii back in 2018. For thirty terrifying minutes, citizens thought a ballistic missile was heading their way because of a text alert that ended with: "This is not a drill."

After the dust settled, the agency pointed fingers at an operator—let's call him Greg—and fired him because he got confused during a routine test. To me, firing Greg wasn't just unfair; it was bad engineering. If a single user can accidentally trigger a state-wide panic because of a bad interface, the system is what's broken, not the user.

Why do human errors happen in critical systems?

I’ve always believed that human error is a symptom of a broken system, not the root cause. If a user can bring down a critical service with a single mistyped command or a bad click, that's on the engineers who built the application, not the person who clicked it. We have to design our software to assume the operator is tired, distracted, and running on too little sleep.

I look at the Hawaii setup and see a classic UI trap. During the drill, the audio recording Greg listened to actually contained the phrase "this is not a drill" to simulate a real-world scenario. On top of that, the dropdown menu in the alert software placed the option for a "live alert" directly next to "test drill."

If I'm reviewing a pull request for a dashboard like that, I'm flagging it immediately. Relying on an operator to have 100% perfect focus every second of their shift is a recipe for a production outage. Human brains aren't built to be flawless compilers.

How does pointing fingers affect our team's code quality?

When we blame an individual for an outage, we leave the actual bug completely untouched and guarantee it will happen again. It also destroys trust, turning a healthy team into a group of terrified developers who hide their mistakes instead of fixing them. If you want a resilient codebase, you need a blameless culture where people feel safe calling out bad UI and sketchy workflows.

Firing Greg didn't rewrite the confusing broadcast script, it didn't fix the crowded dropdown menu, and it didn't add a cancel button. The exact same trap lay in wait for the next person hired to fill Greg's shoes.

In my experience, when you start pointing fingers after an outage, developers stop taking risks. They stop writing code quickly, they avoid touching legacy systems, and they sweep close calls under the rug.

Action	Immediate Impact	Long-Term Risk of Recurrence	Impact on Engineering Culture
Fire the Operator	Fast and cheap; makes management look like they are taking "action."	Extremely High. The bad UI and confusing processes are still live in production.	Toxic. Developers hide bugs and avoid deploying out of fear of being fired.
Redesign UI & Guardrails	Requires dev cycles, UX testing, and process updates.	Near Zero. The UI trap is removed, making the mistake physically impossible to repeat.	Healthy. Promotes blameless post-mortems and active risk mitigation.

What practical UI patterns stop us from breaking production?

Preventing critical errors requires implementing defensive design patterns like clear visual distinction, double-confirmation inputs, and instant "undo" actions. These safety nets ensure that a slip of the mouse or a momentary lapse in focus doesn't escalate into a massive outage. If a mistake is easy to make, the system is what's broken, not the user.

I like to build things defensively. If you are designing any dashboard that has the power to destroy data or send public alerts, you should build around three simple guardrails:

Inject Intentional Friction: Never use a simple button click for destructive actions. Force the user to explicitly type a confirmation phrase (like typing the name of the database or environment) before running the action.
Visual Isolation: Give staging, testing, and production entirely different color schemes. If the admin panel is bright red when you are in production, your brain instantly registers the danger.
Build an Escape Hatch: The Hawaii alert system had no "undo" or recall mechanism. Every high-impact command should have a built-in cancellation window or a quick rollback pathway.

FAQ

How do you run a blameless post-mortem?

Focus on the chronological sequence of events and the tools involved, not the people. Frame the discussion around how the system allowed the mistake to happen and what guardrails can be added to prevent it from happening again.

What is the "Swiss Cheese" model of system failure?

It is the idea that a disaster only happens when multiple independent safety gaps line up perfectly. To prevent outages, we don't try to make people perfect; we add different types of defensive layers so a mistake gets caught before it hits production.

Should an engineer ever be let go for making a mistake?

Only if there is documented, intentional sabotage or a repeated refusal to follow established safety protocols. If an engineer makes a mistake because of a confusing interface or a lack of documentation, firing them just wastes institutional knowledge without fixing the actual problem.

Why Exponential Math Breaks Software Scaling

Doogal Simpson — Tue, 11 Aug 2026 15:04:23 +0000

TL;DR

Humans struggle to intuitively grasp exponential growth and probability, like the 1-in-33-million odds of flipping 25 consecutive heads. While the gambling industry exploits this cognitive gap for profit, software engineers must master these mathematical realities to avoid catastrophic errors in system scaling, hashing, and availability.

If I gave you two hours and twenty quid, do you think you could flip a coin and get 25 heads in a row?

Most engineers I talk to hesitate. It feels doable. 25 is a small number. You can count to 25 on your fingers and toes. It feels like a minor challenge you could knock out before lunch.

But you won't. If you sat there flipping a coin every few seconds for the next decade, you'd still likely fail.

Our brains think linearly. The systems we build run on exponents. That gap between intuition and cold math is exactly what gets exploited.

What is the probability of flipping 25 heads in a row?

The probability of flipping 25 consecutive heads is exactly 1 in 33,554,432. Because each coin flip is an independent binary event, the odds scale exponentially as a power of two: 2^25.

A 1-in-33-million chance means you are far more likely to be struck by lightning this year than to hit this coin-flipping streak.

We can calculate these odds programmatically to see how quickly the scale explodes:

const consecutiveFlips = 25;
const odds = Math.pow(2, consecutiveFlips);

console.log(`Odds: 1 in ${odds.toLocaleString()}`);
// Output: Odds: 1 in 33,554,432

Because 25 is a small, digestible number, we mistake the scale of the operation. We focus on the linear count of flips rather than the exponential depth of the exponent.

Why do we miscalculate exponential growth and odds?

We miscalculate exponential growth because our brains evolved to navigate linear environments, mapping progress in additive steps rather than multiplicative ones. This cognitive blindspot is the exact business model for lotteries and casinos.

These companies hire extremely smart mathematicians who design systems to feed your linear illusions. When you play the lottery and match three numbers, your brain flags it as progress: "I was so close! I'll get it next time."

Matching three numbers doesn't bring you any closer to the jackpot on your next ticket. The odds remain identical because every draw is a fresh, independent event.

Scenario	Human Intuition (Linear Expectation)	Mathematical Reality (Exponential Truth)
25 Coin Flips	"It's a quick task, I can get a streak going in an hour."	A 1 in 33,554,432 probability of occurring.
The Lottery "Near Miss"	"I got three numbers right; I am close to winning."	Past results have exactly 0% influence on future draws.
99.999% SLA (Five Nines)	"A tiny bit of downtime is fine, it's basically 100%."	Allows for only 5.26 minutes of total downtime per year.

How does probability misperception break software systems?

In software engineering, misjudging exponential scaling leads to database bottlenecks, security vulnerabilities in cryptography, and unrealistic service level agreements (SLAs). Engineers relying on gut feeling underestimate the likelihood of rare system failures, hash collisions, or data loss.

Imagine you are designing a microservice that generates short, unique promotional codes. If you rely on a 6-character alphanumeric string, it feels like plenty of combinations. But as your user base grows linearly, the probability of a collision scales exponentially due to the Birthday Paradox.

Without running the actual math, you risk deploying a system that crashes under load because "it felt like we had enough combinations." We have to write code based on the math as it actually is, not how we feel it should be.

FAQ: Understanding Probability and Exponents

How do you calculate the probability of consecutive independent events?

Multiply the probability of a single event by itself for each occurrence. The formula for N consecutive coin flips is (1/2)^N.

Why do we fall for the gambler's fallacy?

We expect short-term sequences of random events to represent long-term statistical averages. If a coin lands on tails five times in a row, our brains falsely believe a head is "due," even though each flip has a static 50% chance.

How does this probability gap affect system scaling?

When scaling APIs or distributed networks, we often underestimate collision rates in hashing or concurrent requests. Using UUIDs or secure hashes feels safe, but without calculating the exponential bounds, a linear increase in traffic can trigger collision-based failures.

AI Agent Architecture: Skills, Evals & Dark Factories

Doogal Simpson — Mon, 10 Aug 2026 13:51:46 +0000

TL;DR: AI agents can write incredible code, but they can also write absolute garbage. Because there is no industry standard yet for managing agent skills or running evaluations (evals), developers must rely on collective, peer-to-peer learning at events like AI Native DevCon to establish best practices.

Remember the early days of Docker? Before Kubernetes won the container wars, we were all just trial-and-erroring our way through configuration files, desperately trying to figure out how to orchestrate microservices without blowing up our staging environments.

That is exactly where we are with AI agents today.

Our entire profession is being flipped on its head. In the last year, AI agents have become incredibly capable of writing software. But whether they write clean, maintainable systems or compile a mountain of unmitigated technical debt depends entirely on how we use them. The reality? Nobody has all the answers yet. We are collectively designing the playbook in real time.

How do you make AI agents write good code instead of garbage?

To make AI agents write high-quality code, you must move away from generic prompting and instead build constrained environments with modular skills and robust evaluation frameworks. Without these boundaries, an agent acts like a junior developer with infinite speed and zero oversight, quickly producing chaotic boilerplate.

Think of an AI agent as a highly capable but radically unpredictable contractor. If you give them a vague ticket and unrestricted access to your codebase, they will make a mess. Instead, we have to transition to an architectural mindset where we design:

Strict Context Windows: Giving the agent only the exact files and APIs it needs.
Sandboxed Runtimes: Allowing the agent to run, test, and break code in an isolated environment before it ever hits a pull request.
Deterministic Guardrails: Forcing the agent's output to conform to strict type definitions and linting rules.

What is the difference between AI agent skills and evals?

Agent skills are the specific, sandboxed tools and APIs an agent can execute to perform tasks, while evals (evaluations) are the automated test suites used to measure the accuracy and safety of the agent's output. Think of skills as what the agent is capable of doing, and evals as the grading rubric that proves they did it safely and correctly.

When we build agentic workflows, we generally break the architecture down into three core concepts:

Concept	Definition	Production Challenge
Agent Skills	Modular, pre-defined functions an agent can call (e.g., "read_file", "execute_test", "search_db").	Restricting scope so agents do not execute destructive or runaway operations.
Evals (Evaluations)	Automated testing frameworks designed to score LLM outputs for correctness, security, and style.	Creating reliable, deterministic assertions for non-deterministic model outputs.
Dark Factories	Fully autonomous, end-to-end software delivery pipelines where a ticket goes in and a verified PR comes out.	Eliminating the need for human code review while maintaining 100% system trust.

Why is peer-to-peer learning essential for AI native development?

Because AI native engineering is shifting daily, no single documentation page, tutorial, or textbook has the "correct" answers yet. Sharing hard-won production secrets—like how to handle agent drift or structure evals—with other developers is the only way to establish reliable industry standards.

I recently attended the London AI Native DevCon and spent time working the "Agent Bar," chatting with engineers and troubleshooting their setups. The disparity in where teams are at is wild. Some developers are brand new to the concept of agentic skills. On the exact same day, I talked to an engineer who had already implemented a fully automated "dark factory" where a ticket comes in, the agent writes the feature, runs the tests, and deploys it end-to-end.

That developer didn't want to talk about basic prompting; they wanted to discuss advanced eval strategies. This is why getting together in the same room—or the same virtual chat—is so valuable. We have to learn from each other's experiments.

If you want to get involved, the next AI Native DevCon is happening in New York this November. You can attend in person or stream the entire event virtually for free.

If you want to head to New York, use my discount code DOOGA30 to get 30% off. If you are heading over with your team, drop me a DM and I can grab you 50% off.

FAQ: Navigating the AI Agent Landscape

What is an AI "dark factory" in software engineering?

An AI dark factory is a fully automated software development pipeline where human intervention is minimized or entirely removed. A user story or bug ticket is fed into the system, and a network of coordinated AI agents autonomously writes the code, runs the test suite, fixes its own errors, and submits a verified deployment.

How do you write reliable evals for non-deterministic AI code?

Writing evals requires shifting from exact-match assertions to semantic and functional testing. Instead of checking if the code looks exactly like a reference solution, your eval framework should execute the generated code in a secure sandbox, verify that it passes unit tests, analyze it with static analysis tools (like AST parsers), and run LLM-as-a-judge prompts to grade code quality.

What are the best resources for learning AI agent architecture?

Because the field is changing rapidly, the best resources are active developer communities, open-source agent frameworks (like LangChain, AutoGen, or CrewAI), and peer-to-peer technical conferences like AI Native DevCon where engineers share real-world production post-mortems.

Premature Optimization: What Donald Knuth Actually Meant

Doogal Simpson — Sat, 08 Aug 2026 16:07:24 +0000

Quick Answer: The famous quote "premature optimization is the root of all evil" is often misunderstood. Donald Knuth actually wrote that we shouldn't pass up opportunities in a "critical 3%" of cases. Identifying this 3% means distinguishing between wasting time on micro-efficiencies and making foundational architectural choices that prevent future system collapse.

We all know that person who packs absolutely everything for a three-day weekend trip. They pack sandals, heavy snow boots, a raincoat, a heavy wool scarf, and three bottles of sunscreen. They spend hours packing for every hypothetical climate event, only to spend the entire trip wearing a t-shirt and jeans.

In software engineering, we recognize this behavior instantly. We call it premature optimization.

But lately, I have noticed a massive swing in the opposite direction. Engineers have started weaponizing Knuth's famous adage to shut down valid, necessary architectural discussions. By screaming "premature optimization!" at the first sign of a design debate, we end up traveling to a rainy climate without even checking the weather forecast—and we freeze because we refused to pack a jacket.

Why do developers misunderstand Donald Knuth's quote?

Most engineers only remember half of Donald Knuth’s famous adage. The full quote states that while we should forget about small efficiencies 97% of the time, we must not pass up opportunities in that critical 3%.

The full quote from Knuth's 1974 paper, Structured Programming with go to Statements, reads:

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."

By ignoring the second half of this quote, developers fall into the trap of architectural laziness. They treat all optimization as a sin. There is a massive difference between wasting three days trying to shave four microseconds off a utility function and spending three hours designing a system boundary so it doesn't fall over when traffic spikes. The 97% refers to micro-optimizations; the 3% refers to foundational decisions.

How do you distinguish between the 97% and the critical 3%?

You identify the 3% by focusing on decisions that are extremely difficult or expensive to change later, rather than micro-tuning code execution. This means prioritizing architectural boundaries, data models, and hard scaling bottlenecks over micro-optimizations like loop structures or variable caching.

Imagine your team is building a new payment processing pipeline. The choice of database schema, the decision to use an asynchronous message queue instead of synchronous HTTP calls, and the strategy for handling distributed transactions are part of the critical 3%. If you get these wrong, you cannot easily refactor your way out of them later.

Conversely, worrying about whether to use a for loop or a .map() array helper is firmly in the 97% bucket. If that loop ever becomes a bottleneck, you can swap it out in thirty seconds.

Scenario Component	The 97% (Defer Optimization)	The Critical 3% (Optimize Early)
Data Modeling	Optimizing specific index sizes before you have data.	Choosing a relational model when you have deeply nested, dynamic graph data.
Network Operations	Trimming JSON payload keys to save minor bytes.	Choosing synchronous cascading API calls over asynchronous event-driven queues.
Code Execution	Refactoring loops into low-level assembly-style syntax.	Choosing a poorly suited algorithm (e.g., O(N^2)) for a dataset that scales exponentially.

What are the risks of using 'premature optimization' as a shield?

When team members use this quote to shut down healthy design discussions, it breeds architectural neglect. It turns a blind eye to fundamental flaws, resulting in massive, painful refactoring projects when the system inevitably fails under load.

I often see developers shut down discussions about scale by saying, "Let's just build it the simple way first; we can optimize it when we need to." While this sounds pragmatic, it often leads to what I call "accidental complexity."

If you build a service that performs synchronous HTTP requests to three other microservices inside a database transaction, you haven't just built a "simple" version. You have built a fragile system prone to cascading failures. Fixing this later requires rewriting the entire service, changing client contracts, and migrating live data. That is not an optimization issue; it is a structural defect.

Learn to spot the difference. If a design decision impacts your system's core boundaries, data gravity, or operational model, it belongs in that critical 3%. Spend the time to get it right from day one.

Frequently Asked Questions

Is choosing a specific database paradigm considered premature optimization?

No. Choosing between a relational database (SQL) and a non-relational database (NoSQL) is an architectural foundation. Because data migration and paradigm shifting are incredibly costly after production launch, this decision is firmly in the critical 3%.

How can we quantitatively identify the critical 3% in production?

Use profiling tools and APM (Application Performance Monitoring) solutions. These tools show you exactly where CPU cycles, memory, and network latency are concentrated, allowing you to target the 3% of your codebase that causes 97% of your performance bottlenecks.

Does writing clean code conflict with the critical 3% optimization?

Almost never. Clean, modular code is actually easier to optimize because the system boundaries are clear. When you do find a bottleneck in your critical 3%, a clean codebase allows you to isolate and optimize that specific component without rewriting the entire application.

Why Software Engineers Prefer YYYY-MM-DD (ISO 8601)

Doogal Simpson — Sat, 08 Aug 2026 16:02:39 +0000

Quick Answer: The YYYY-MM-DD date format (codified as ISO 8601) is the global standard in software because it enables natural chronological sorting as a plain string. Unlike regional formats (MM-DD-YYYY or DD-MM-YYYY), descending from the largest to the smallest unit allows databases and file systems to sort dates alphabetically without complex parsing logic.

If there is one thing that consistently makes my eye twitch during a late-night debugging session, it is opening a log folder and seeing dates formatted in three different regional styles. To me, date formatting should not be an art form—it should be a solved, predictable science. Yet, we still find ourselves battling regional madness daily.

While human languages have spent centuries complicating how we record calendar days, software engineering solved this problem decades ago. The solution is elegant, simple, and silently powers almost every app I write.

Why do programmers prefer the YYYY-MM-DD date format?

Programmers prefer the YYYY-MM-DD format because it sorts chronologically using standard lexicographical (character-by-character) sorting algorithms. By structuring dates from the largest unit (year) to the smallest (day), computers can order time-series data without needing to parse strings into date objects first.

To understand why I find this format so beautiful, consider how a computer sorts strings. It compares characters from left to right. If you use the British format (DD-MM-YYYY), a computer will clump all your files by the day of the month. You’ll get every "1st of the month" grouped together, regardless of the month or year. If you use the American format (MM-DD-YYYY), you get everything clumped by month.

But year-month-day? Large, medium, small. Because the year comes first, the alphabetical sort order perfectly mirrors the chronological sort order. It just makes computers happy.

// Lexicographical sorting just works with ISO 8601 strings
const dates = ["2023-11-05", "2021-04-12", "2024-02-01"];

dates.sort();
console.log(dates);
// Output: ["2021-04-12", "2023-11-05", "2024-02-01"]

What is the history behind regional date formats like MM-DD-YYYY and DD-MM-YYYY?

The regional differences in date formatting stem from historical colonial patterns and post-colonial divergent standards. The US inherited the MM-DD-YYYY format from Britain in the 18th century, while the UK later shifted to DD-MM-YYYY to align with European standards, leaving the US with the older system.

I always find the history here fascinating. The Americans went with month, day, year—which to me has always felt like sorting things medium, small, large. It feels completely chaotic. The British went with day, month, year—which is small, medium, large. That at least has some logical progression.

So why did the Americans stick with the "medium, small, large" madness? It turns out they just kept the standard that was active in the UK when the colonists first moved over 300 years ago. After the Americans left, the British decided to change their own standard to match Europe, leaving the US frozen in time with the old British way. Cheers for that.

How does ISO 8601 compare to other date standards?

ISO 8601 (YYYY-MM-DD) is an international standard designed to eliminate ambiguity in global data exchange. While regional formats prioritize colloquial speech patterns, ISO 8601 prioritizes data integrity, programmatic readability, and computational efficiency.

When I am designing an API, I have a golden rule: never trust regional date string formats. Imagine you are building a global checkout system. A timestamp of 04-03-2026 is a ticking time bomb. To a user in London, it is March 4th. To a user in New York, it is April 3rd. ISO 8601 removes this ambiguity entirely.

Format	Structure (Unit Size)	Chronological String Sortable?	Ambiguity Risk	Common Use Case
ISO 8601 (YYYY-MM-DD)	Large -> Medium -> Small	Yes	None	Databases, APIs, Log files
British (DD-MM-YYYY)	Small -> Medium -> Large	No (Groups by day)	High (Confused with US)	Everyday use (UK/EU/AU)
American (MM-DD-YYYY)	Medium -> Small -> Large	No (Groups by month)	High (Confused with UK)	Everyday use (US)

FAQ

Why does sorting by DD-MM-YYYY or MM-DD-YYYY fail in code?

Because string sorting is lexicographical (left-to-right). If you sort a list of DD-MM-YYYY strings, "01-12-2021" (December) will sort before "02-01-2020" (January) because the computer evaluates the "01" first, completely ignoring the year and month.

Is ISO 8601 limited only to dates, or does it include times?

No, ISO 8601 fully supports times by appending a "T" separator and the time in descending order (e.g., YYYY-MM-DDTHH:mm:ssZ). This preserves the same large-to-small hierarchy, meaning highly precise timestamps remain perfectly sortable as raw strings.

How do databases handle these different date formats?

Internally, databases do not store dates as strings; they store them as binary timestamps (like Unix epoch time). However, during data ingestion, serialization, and API communication, I always rely on ISO 8601 as the standard interchange format to prevent parsing errors and timezone mismatches.

Is 100% Test Coverage Worth It? Realistic Testing Targets

Doogal Simpson — Thu, 06 Aug 2026 17:48:19 +0000

Many developers treat test coverage as a game where the only acceptable high score is 100%. But in the real world of software delivery, chasing absolute perfection often introduces diminishing returns.

While 100% test coverage sounds ideal, achieving it often introduces diminishing returns. For most applications, aiming for 70% to 80% coverage secures the happy paths and critical business logic without draining development time. Reserve 100% coverage strictly for high-stakes, mission-critical systems where software failure is not an option.

Imagine sitting down with a blank piece of paper and being asked to write a short story set in the Game of Thrones universe. Here is the catch: you cannot make a single grammatical error, break character continuity, or mismatch the lore. Not one slip-up.

Writing flawless prose is incredibly hard—yet as software engineers, we are expected to do exactly this with thousands of lines of code every single day. How do we keep our "story" from falling apart when someone else edits a chapter? We write tests. But the real debate isn't whether we should test; it is how much we should test.

Why is 100% test coverage usually a trap?

Striving for 100% test coverage often forces you to write low-value tests for trivial code, such as basic getters, setters, or third-party integrations. This consumes valuable engineering time that could be spent shipping features, with little to no reduction in actual production bugs.

To understand why, we have to look at the cost curve. Testing the "happy paths"—the primary routes your users take—is straightforward and high-yield. However, mapping out every microscopic edge case, transient network failure, and esoteric state variation takes an exponential amount of effort. If you spend 80% of your time writing tests for 20% of your codebase's most obscure corners, you are dealing with severely diminishing returns. Time spent writing those low-value tests is time stolen from building features that actually help your users.

What is a realistic test coverage goal for most applications?

For the vast majority of software projects, a target of 70% to 80% test coverage strikes the perfect balance between reliability and development speed. This range ensures that your core business logic, integration points, and high-risk pathways are fully verified without bogging down your team in test maintenance.

Let's look at this pragmatically: if you are launching a new SaaS product or building a standard web API, shipping fast is your primary competitive advantage. The code is going to change next month anyway. Maintaining a strict 100% coverage requirement means that every minor refactor requires rewriting dozens of brittle tests.

Here is how I bucket coverage targets when I'm evaluating a project:

Coverage Tier	Best Suited For	Key Focus Area	Trade-off
95% - 100%	Medical devices, aerospace, fintech core transaction engines	Edge cases, error handling, safety-critical loops	Slow development velocity, high maintenance overhead
70% - 80%	Standard SaaS apps, internal business tools, e-commerce sites	Happy paths, business logic, primary integrations	Good safety net, fast velocity, manageable maintenance
Under 50%	Prototypes, early-stage MVPs, marketing landing pages	Basic smoke tests on critical user flows	High risk of regression, rapid initial delivery

When should you actually aim for 100% test coverage?

You should only mandate 100% test coverage for mission-critical systems where software failure results in severe physical, financial, or reputational harm. If you are writing software for a pacemaker, an autonomous vehicle, or a satellite orbiting Earth, every single line of code must be meticulously verified.

Imagine a microservice that coordinates satellite trajectories. A single unhandled exception could mean losing a hundred-million-dollar piece of hardware forever. In these environments, development velocity is secondary to absolute correctness. But unless your code is quite literally rocket science, treating your standard CRUD application like a space mission is a recipe for stalled roadmaps.

FAQ: Common questions about testing strategies

Is high test coverage a guarantee of bug-free software?

No, it is not. Test coverage only measures which lines of code were executed during a test run, not whether those tests actually asserted the correct behavior or handled unexpected user inputs.

How do you prioritize what to test first?

Focus first on your core business value drivers—such as the checkout process, authentication flow, and data persistence layers—before worrying about presentational UI states or simple utility helpers.

Should we use automated tools to block pull requests with low coverage?

Yes, but set the thresholds pragmatically. Setting a gate at 75% acts as a helpful safety guardrail without becoming a frustrating barrier that encourages developers to write bad, meaningless tests just to pass the build.

Why JavaScript Has Quirks (And Why TC39 Can't Fix Them)

Doogal Simpson — Tue, 04 Aug 2026 10:53:29 +0000

TL;DR: JavaScript has notorious quirks because it was designed in just ten days as a prototype to help Netscape survive the early browser wars. Because the internet relies on absolute backwards compatibility, these rushed design flaws became permanent APIs that we must support forever to avoid breaking the web.

I've lost count of the times I've stared at a console, questioning my career choices because of some bizarre JavaScript coercion behavior. Maybe it is the fact that typeof null returns "object", or that sorting an array of numbers alphabetically arranges them as [1, 10, 2, 20].

Whenever I am digging through these quirks, I try to remind myself of how we got here. I find it absolutely mad that the language powering almost every modern user interface on earth was built as a rushed, two-week prototype. It is the ultimate real-world case of "protoduction"—a temporary hack thrown directly into production under existential company pressure.

Why does JavaScript have so many design flaws?

JavaScript has so many design flaws because it was designed and implemented in just ten days by Brendan Eich at Netscape in 1995. Rushed to market to win the early browser wars against Microsoft, the language was never intended to be the final product, meaning early prototype compromises became permanent.

To understand why these quirks exist, I always look back to the mid-1990s browser wars. Imagine your team is building a web browser, and your company's survival depends entirely on shipping a scripting language before your competitors do. If you do not get there first, your product is dead.

That was the pressure cook Brendan Eich was under at Netscape. I think any engineer can sympathize with that level of stress. He had to compromise on language design, cobbling together elements of Scheme, Self, and Java in less than two weeks. It worked brilliantly for the business, but it left us with shortcuts that were never meant to survive the weekend, let alone three decades.

// I love this classic symptom of a 10-day deadline:
console.log(typeof null); // "object"
console.log([] + {});     // "[object Object]"

Why can't TC39 simply remove JavaScript's legacy quirks?

TC39 cannot remove legacy quirks because doing so would break millions of existing websites that rely on those exact behaviors. The web operates on a strict rule of backwards compatibility, meaning we can only add new features to the language, never delete old ones.

I often hear developers ask why we don't just run a massive cleanup on the specification. The reality is that the web's core philosophy is strict backwards compatibility.

I like to use the analogy of a historical city. Imagine you have a cardboard bridge built decades ago. It was only meant to be temporary, but over the years, thousands of delivery trucks started crossing it daily. You cannot tear it down because those trucks—representing millions of legacy websites—would crash into the river. All you can do is build a modern steel bridge right next to it, point at the cardboard one, and warn people not to use it.

Quirk	Original Root Cause	Modern Mitigation
`typeof null === 'object'`	A bug in the original 1995 type tag implementation.	I use strict null checks (`=== null`).
Global variable leakage	Rushed scope resolution that defaults to the global object.	I enforce strict mode (`"use strict"`) and block-scoped `let`/`const`.
Automatic Semicolon Insertion	An attempt to make the language forgiving for non-programmers.	I rely on linters (like ESLint) to enforce consistent styling.

How does the modern web evolve around these issues?

Modern JavaScript evolves by layering clean, opt-in APIs and syntax over the legacy core rather than modifying the underlying behaviors. Standards committees use tools like strict mode, compiler flags, and transpilars to allow developers to write clean code while preserving the old runtime environment.

I've watched the ecosystem shift toward build-time tooling to solve this. Instead of trying to fix the runtime engine, we use tools like Babel and TypeScript to write clean, modern, type-safe code that compiles down to the highly compatible (and highly quirky) JavaScript that browsers actually run.

We are essentially papering over the cracks. When I look at modern ES6+ features, I see a beautiful, highly capable language, but it is built entirely on top of a 10-day prototype. It is a remarkable engineering feat that we have managed to scale this temporary bridge to support the entire digital world.

FAQ

Why was JavaScript created in only 10 days?

Netscape was in an intense race against Microsoft to establish the dominant web platform. They needed a lightweight scripting language to pair with their browser immediately, forcing developer Brendan Eich to deliver a working prototype under an impossibly tight deadline.

What does "Don't Break the Web" mean for developers?

"Don't break the web" is the core philosophy of web standards bodies. It means that any code written in 1995 must run in a modern browser today without modification, preventing developers from removing old, buggy features from the JavaScript specification.

Is it possible to completely replace JavaScript in the browser?

While WebAssembly (Wasm) allows languages like Rust, C++, and Go to run in the browser at near-native speeds, it is designed to complement JavaScript rather than replace it. JavaScript remains the only native language for direct DOM manipulation and web scripting.

How to Prevent SQL Injection with Parameterized Queries

Doogal Simpson — Sun, 02 Aug 2026 10:10:10 +0000

Quick Answer: SQL injection attacks exploit vulnerabilities in how web applications construct database queries. By sending malicious SQL code through user input fields (like search bars), attackers can manipulate or steal data. The primary defense is using parameterized queries, which treat user input as data, not executable code, thereby preventing unauthorized database operations. This is a fundamental security practice that significantly reduces your attack surface.

I love the Hollywood trope of hackers in hoodies, furiously typing in dark rooms, with streams of green text on black screens. It's dramatic, but in the real world, security exploits often come from much simpler, dumber mistakes. One of the classic and still relevant ones? SQL injection attacks.

What is an SQL Injection Attack?

At its core, an SQL injection attack is when a malicious actor inserts dangerous SQL code into your application's input fields, tricking the database into executing unintended commands. If your application blindly trusts user input and directly embeds it into database queries, you're leaving the door wide open for attackers to steal, modify, or delete your data.

Think about a search bar on your website. Normally, a user types in "widgets," and your backend constructs a query like SELECT * FROM products WHERE name = 'widgets';. This is fine. But what if an attacker inputs something like ' OR '1'='1? If you're not careful and just concatenate that string, your query could become SELECT * FROM products WHERE name = '' OR '1'='1';. Suddenly, you're not just searching for "widgets"; you're telling the database to return everything because '1'='1' is always true. It’s a simple example, but it illustrates the danger: user input is being treated as executable SQL, not just data.

How Do Attackers Exploit Input Fields?

Attackers look for any place your application accepts user input and sends it to the database without proper sanitization or parameterization. This commonly includes:

Search Bars: As discussed, users can inject SQL through search queries.
Login Forms: Malicious input in username or password fields can bypass authentication.
URL Parameters: Sometimes, data passed in the URL can be manipulated.
Form Submissions: Any data submitted via HTML forms is a potential vector.

When your backend code takes this input and simply sticks it into an SQL string, you're essentially giving the attacker a direct line to command your database. They could ask for sensitive user data, delete records, or even drop entire tables if you're not careful. As the transcript puts it, "you get what you deserve kind of" if you blindly execute raw input.

How Can I Protect My Application?

The absolute best defense against SQL injection is using parameterized queries (also known as prepared statements). This is the industry standard and what I rely on.

Here's a breakdown of the difference:

Feature	String Concatenation (Vulnerable)	Parameterized Queries (Secure)
How it works	User input is directly embedded into the SQL string.	SQL query structure and user data are sent separately.
Input Treatment	Treated as executable SQL code.	Treated strictly as literal data values.
Security Risk	High; vulnerable to code injection.	Minimal; prevents code injection by design.
Example	`query = "SELECT * FROM users WHERE id = " + user_id`	`query = "SELECT * FROM users WHERE id = ?"` `execute(query, (user_id,))`

When you use parameterized queries, you define your SQL command with placeholders (like ? or named parameters). Then, you provide the user's input as separate parameters. The database driver ensures that this input is treated only as data for those placeholders, never as executable SQL commands. This fundamentally prevents injection attacks.

When Should I Use Parameterized Queries?

From my perspective, you should use parameterized queries every single time your application interacts with a database using user-supplied input. This isn't a niche security measure; it's a fundamental best practice. Most modern programming language libraries and ORMs (Object-Relational Mappers) provide straightforward ways to implement parameterized queries. For instance, in Python with psycopg2 for PostgreSQL, it might look like this:

# Imagine we're getting a user ID from a web request
user_id_from_request = request.args.get('user_id')

# This is the secure way:
sql = "SELECT * FROM users WHERE id = %s"
cursor.execute(sql, (user_id_from_request,))

The key is that the cursor.execute method, when given a tuple or list as the second argument, handles the parameterization. It escapes special characters and ensures the user_id_from_request is treated purely as a value, not as SQL code. This simple change dramatically hardens your application against SQL injection.

Frequently Asked Questions

Can modern frameworks completely prevent SQL injection?

Modern frameworks are built with security in mind and often default to using parameterized queries, which is excellent. However, it's still possible to introduce vulnerabilities yourself if you manually construct SQL queries by concatenating strings instead of using the framework's built-in, safe methods for database interaction.

Is input validation still important if I use parameterized queries?

Absolutely. Input validation is a critical defense-in-depth strategy. While parameterized queries prevent SQL injection, validating input ensures that the data your application receives is in the expected format and range. For example, validating that a user ID is actually an integer prevents other types of errors or unexpected behavior, even if it wouldn't lead to an SQL injection.

What happens if I don't use parameterized queries?

If you don't use parameterized queries and instead rely on string concatenation for dynamic SQL, your application is vulnerable to SQL injection. Attackers can craft input that manipulates your queries, potentially leading to unauthorized access, data breaches, data corruption, or denial of service. It's one of the most common and dangerous web security flaws.

How to Prompt LLMs for Clean Code Using Trigger Words

Doogal Simpson — Fri, 31 Jul 2026 11:26:54 +0000

Stop wasting your context window explaining industry standards to LLMs. Large Language Models already have best practices baked into their training data; you just need to activate those specific regions of their latent space. By using targeted "trigger" phrases instead of manual rules, you get cleaner, production-ready code with minimal prompting.

Think of an AI’s brain as a massive, dark 2D grid. When you initialize a prompt, you get dropped right into the middle of this dormant space. The model doesn't automatically fire up its entire repository of knowledge. Instead, your prompt acts as a localized light switch.

To get the clean, secure code you want, you do not need to manually reconstruct the rules of modern software engineering. You just need to navigate that grid and tickle the specific pathways that represent established industry standards until they light up.

How does AI handle coding best practices without explicit instructions?

AI models store industry-standard patterns, security protocols, and architectural best practices directly within their latent training data. You do not need to teach the model how to write secure code; you simply need to use targeted prompts that activate that pre-existing knowledge.

Imagine you are building a standard backend service. If you ask an LLM for a generic handler, it gives you a bare-bones implementation. The knowledge of how to write a secure, scalable version of that handler isn't missing—it is simply dormant. Because the training data is saturated with high-quality open-source repositories, the model already knows what "good" looks like. It defaults to lazy outputs unless you explicitly signal that you expect production-grade architecture.

Why do detailed prompts yield the same results as simple triggers?

Spelling out every architectural rule manually is redundant because the LLM already maps those concepts together in its high-dimensional space. Triggering a concept like "do auth right" lights up the exact same neural pathways as pasting a multi-paragraph checklist of security standards.

I recently ran an experiment building Express web apps to test this exact behavior. I generated three different services:

The Control: I asked for a book service with no mention of authentication.
The Manual Prompt: I spelled out detailed, explicit best practices for Express authentication (password hashing, JWTs, secure cookies).
The Trigger Prompt: I simply asked the AI to "write a web app and do auth right."

The results were eye-opening. The control app had no authentication, which was expected. However, the code generated by the manual prompt and the simple "do auth right" prompt was virtually identical.

Prompt Strategy	Developer Effort	Code Quality	Context Token Overhead
Implicit (No mention of auth)	None	Unsecured	Zero
Manual Specification (Spoon-feeding rules)	High	Production-grade	Very High
Trigger-Based ("Do auth right")	Low	Production-grade	Low

By trying to write the rules yourself, you are wasting valuable context tokens. The AI already knows what "right" means for your stack.

What is the most efficient way to prompt LLMs for clean code?

The most efficient strategy is to use concise, high-leverage trigger words that point to established paradigms instead of writing long, rule-based checklists. This saves context tokens and reduces the chance of conflicting instructions confusing the model.

Instead of listing out libraries, try triggering the ecosystem standard. For instance, if you want a secure Express route, don't write a 10-line prompt detailing token validation. Just trigger the paradigm:

// Prompt: "Set up a protected Express route, do auth right"
const express = require('express');
const helmet = require('helmet');
const { requireAuth } = require('./middleware/auth');

const app = express();
app.use(helmet()); // Triggered automatically by "doing it right"

app.get('/api/books', requireAuth, (req, res) => {
  res.json({ books: [] });
});

By using the phrase "do auth right," the model immediately pulled in security middleware like helmet and structured the route protection cleanly, without needing a step-by-step lecture.

FAQ

Does this mean I should never write detailed system prompts?

Not quite. Detailed system prompts are necessary for custom business logic, unique domain models, or proprietary APIs. However, for industry-standard tasks like authentication, database connection pooling, or error handling, simple trigger phrases are far more efficient.

How do I know if an LLM actually knows the best practice for a niche library?

If a library or tool is relatively obscure or has undergone major API changes since the model's knowledge cutoff, trigger words might fail. In those cases, you should provide a brief code block of the target API structure alongside your trigger phrase to anchor the model.

Will relying on trigger words cause hidden security gaps?

Always review AI-generated code. While "do auth right" triggers standard security implementations, the AI cannot verify your actual infrastructure configuration. Use trigger words to generate the boilerplate and structure, but perform a manual security audit on the output before deploying it to production.

Why the Netflix Keeper Test Hurts Engineering Teams

Doogal Simpson — Wed, 29 Jul 2026 10:08:17 +0000

The Netflix Keeper Test asks managers: "If this engineer resigned today, would you fight to keep them?" While it ensures high talent density, I believe it often destroys psychological safety. For most software engineering teams, a collaborative culture built on trust outperforms a high-stress, sports-team model driven by the fear of sudden severance.

Imagine walking into your next 1-on-1, and instead of discussing career growth or system architecture, your manager is running a silent background process: If you walked out the door right now, would I actually care? Personally, I think this is a terrifying way to run an engineering organization. The "Keeper Test" has always fascinated me because, in my view, applying this raw, survival-of-the-fittest model to software development comes with massive, hidden architectural costs.

What is the Netflix Keeper Test and how does it work?

The Keeper Test is a management framework where leaders regularly evaluate their team members by asking if they would fight to retain them if they resigned. If the manager wouldn’t fight to keep them, the employee is promptly let go with a generous severance package. From what I've observed of the Netflix culture, they frame this by saying they aren’t a family—they are a high-performing professional sports team.

I’ve definitely been in situations where I’ve looked at certain colleagues and thought: If I were starting a company tomorrow, I would fight tooth and nail to hire them. I’ve also looked at others and thought: They’re fine, but I wouldn’t exactly go to bat to keep them. It’s a natural human distinction. However, Netflix codifies this. In their model, "adequate performance gets a generous severance." They prioritize talent density above almost everything else, replacing anyone who isn't an absolute superstar.

Why does the Keeper Test harm psychological safety in software engineering?

In my view, software engineering is an inherently creative and experimental discipline, meaning it requires psychological safety to thrive. I believe the Keeper Test directly undermines this safety by fostering a continuous fear of termination, which forces developers to avoid risky but potentially breakthrough ideas. When every mistake or temporary dip in velocity could mean failing the "test," I've noticed that engineers naturally default to safe, predictable choices.

Consider what happens when you build systems under threat. You don't refactor that fragile legacy module because if you break production, you might be out of a job. You don't propose a bold architectural shift because it’s safer to ship incremental, boring features. When we treat engineers like professional athletes who are only as good as their last game, we ignore how software is actually built. To me, writing code isn't a sprint down a track; it's a highly collaborative, iterative process of debugging and learning.

Team Metric	The Keeper Test Model (Sports Team)	The Psychological Safety Model (Collaborative)
Primary Driver	Performance & Talent Density	Trust, Growth & Innovation
Risk Tolerance	Low (Mistakes can lead to exit)	High (Failure is treated as a learning tool)
Collaboration	Competitive (Individual survival)	Cooperative (Collective ownership)
Long-term Retention	Low (High churn by design)	High (Preserves institutional domain knowledge)

How should engineering leaders balance high performance and trust?

In my experience, leaders can balance high performance and trust by establishing clear, objective expectations and treating performance issues as collaborative debugging sessions rather than immediate failures. Instead of a binary "keep or fire" test, I recommend implementing transparent growth paths that give engineers a fair runway to improve. This maintains high technical standards without triggering a constant state of survival mode.

I like to think of your team's talent like a complex microservice architecture. If a service starts throwing 500 errors, you don't instantly delete it from the cluster and deploy a brand-new, untested service. You look at the logs, find the root cause, and refactor. I believe people deserve the same systematic approach. High standards and psychological safety aren't mutually exclusive; in fact, I don't think you can have sustainable high standards without trust.

Frequently Asked Questions

Does the Keeper Test increase developer turnover?

Yes. By design, the Keeper Test increases both voluntary and involuntary turnover to maintain what Netflix calls "talent density." However, in my observation, this often results in a massive loss of institutional domain knowledge, making systems harder to maintain over time.

What is the alternative to the Keeper Test for managing underperformance?

The best alternative is a transparent, continuous feedback loop coupled with a structured growth plan. This gives underperforming developers clear, measurable milestones and the support needed to hit them, treating termination as a last resort.

Can psychological safety and high standards coexist?

Absolutely. High psychological safety combined with high standards creates an environment of accountability and continuous learning. When developers know they won't be fired for an honest mistake, they are far more willing to take the calculated risks necessary to build outstanding products.