DEV Community: Steve McDougall

The Polling API Is the Most Underrated RFC PHP Has Shipped in Years

Steve McDougall — Tue, 23 Jun 2026 21:09:47 +0000

While most of the community spent the spring arguing about generics, a different RFC slipped into PHP 8.6 with almost none of the attention it deserved. No long Twitter threads. No blog posts dissecting the implications. Just a quiet vote that closed on the third of June with 33 in favour, one against, and four abstaining, from a list of names that includes the Composer author, the FrankenPHP creator, and a healthy chunk of the people who actually maintain the async libraries you depend on.

That RFC is the Polling API, authored by Jakub Zelenka as part of his ongoing stream evolution work. I want to make the case that it is the most consequential thing to land in PHP in years, and that the reason nobody is talking about it is exactly the reason it matters.

The problem you have been quietly working around

If you have ever written anything that needs to watch more than one socket at a time, you have met stream_select(). It is the only I/O multiplexing primitive PHP has shipped for most of its life, and it is built on the select() system call from 1983. It works. It also carries a set of limitations that every async library author has had to engineer around.

The first is the file descriptor ceiling. The current implementation caps out at around 1024 descriptors on most systems, which is fine until the moment it very much is not. The second is the complexity. select() is O(n): it scans every descriptor you hand it on every single call, so performance degrades as you add connections. The third is that it gives you no access to the mechanisms the rest of the world has been using for two decades. There is no epoll on Linux, no kqueue on BSD or macOS, no event ports on Solaris. There is no edge-triggered mode and no one-shot mode.

This is why every serious async runtime reaches past select(). Node, Nginx, Go's net package, Rust's Tokio: they all sit on epoll or kqueue, because that is the foundation high-concurrency networking is built on. PHP was the last major runtime without native access to it, which is why libraries like AMPHP and ReactPHP have shipped multiple driver backends for years. You write your StreamSelect driver for the common case, then a separate libuv driver for the people who installed ext-uv, because a single native high-performance option simply did not exist. That "install ext-uv for performance" line in the README is a workaround for a hole in the language itself.

The Polling API fills the hole.

What it actually is, and what it is not

The proposal adds a small set of classes under a new Io\Poll namespace. It automatically selects the best polling backend for the platform you are running on, and exposes a consistent interface on top of it. On Linux you get epoll, on macOS and BSD you get kqueue, on Solaris and illumos you get event ports, on Windows you get WSAPoll, and everything else falls back to poll(). You do not have to think about any of that. You create a context and it picks the right backend.

Here is the whole thing in one breath. You create a Context, you wrap a stream in a StreamPollHandle, you add it to the context with the events you care about, and you call wait():

<?php

declare(strict_types=1);

use Io\Poll\{Context, Event};

$poll = new Context();

$server = stream_socket_server('tcp://0.0.0.0:8080', $errno, $errstr);
stream_set_blocking($server, false);

$poll->add(new StreamPollHandle($server), [Event::Read], ['type' => 'server']);

while (true) {
    foreach ($poll->wait(timeoutSeconds: 1) as $watcher) {
        // a watcher only comes back if it has events ready
        $handle = $watcher->getHandle();

        if ($watcher->hasTriggered(Event::Read)) {
            // accept, read, write, whatever this descriptor needs
        }
    }
}

That is the shape of it. wait() blocks until something is ready or the timeout expires, then hands you back only the watchers that actually triggered. No scanning the full set yourself, no guessing which descriptor woke you up.

The important thing to understand is what the RFC deliberately leaves out. This is not an event loop. There are no timers in the first cut, no signal handling, no child process management. It is one primitive: a way to watch file descriptors efficiently using whatever the operating system does best. If you want a full event loop with all the trimmings, you still reach for AMPHP or ReactPHP. The difference is that those libraries can now sit on a single native backend instead of maintaining their own.

The part nobody is talking about

Here is where I think the coverage has missed the real story. Every article I have seen frames this as a userspace feature, a faster stream_select() for people building WebSocket servers. That is a genuine benefit, and it is also explicitly the secondary one.

Read the RFC carefully and the primary motivation is the internal API. Jakub is building a unified polling interface that PHP core and extensions can share, and the userspace classes are a gift that falls out of having done that work properly. The internal php_poll.h header is the actual point.

Why does that matter to you, someone writing application code who will probably never type new Context()? Because of what it unlocks underneath. Safe, efficient signal handling in ZTS, which is exactly what FrankenPHP needs for its goroutine-based threading mode. More flexible event handling in PHP-FPM workers, replacing the ad-hoc implementations that exist today. A cross-platform timer foundation, which has been a genuine pain on macOS. A standard interface that the sockets and curl extensions can build on rather than each rolling their own.

The future scope section reads like a roadmap for the next few years of PHP's networking internals: SocketPollHandle, CurlPollHandle, TimerHandle, SignalHandle, FPM migrating onto the internal loop, and the consolidation of all the scattered polling code across the codebase into one place. None of that is glamorous. All of it is the kind of foundational plumbing that quietly raises the ceiling for everything built on top.

The best infrastructure changes are invisible. You do not notice epoll. You notice that Nginx handles ten thousand connections without sweating, and you never think about why.

The design choices worth admiring

A couple of decisions in the API are worth slowing down on, because they tell you something about how carefully this was put together.

The first is the Handle interface. It is a marker interface with no methods, and you cannot implement it from userland. Try, and you get a fatal error at class declaration time:

Fatal error: Io\Poll\Handle cannot be implemented by user classes

That looks restrictive until you understand why. Every concrete handle type has to register a C-level operations table (php_poll_handle_ops) that tells the backend how to extract the underlying file descriptor and check whether the handle is still valid. A userland class cannot provide that, so allowing it to implement the interface would only produce handles that do not work. By locking the interface to internal classes, the contract stays enforceable: any Handle that reaches the polling backend is guaranteed to have a working ops table. If you need to poll a custom resource, you wrap a stream in StreamPollHandle and let the existing machinery do its job. Clean userspace surface, all the messy file descriptor logic kept on the C side where it belongs.

The second is the Watcher. You never construct one directly; Context::add() returns it to you, and it carries everything about that registration. You can ask it which events it is watching, which ones just triggered, and the arbitrary user data you attached. You can modify it in place or remove it:

<?php

declare(strict_types=1);

use Io\Poll\{Context, Event};

$poll = new Context();
$stream = fopen('php://temp', 'r+');

$watcher = $poll->add(new StreamPollHandle($stream), [Event::Read], 'some data');

// switch what you are watching for without rebuilding the registration
$watcher->modifyEvents([Event::Write]);

// or change both events and the attached data at once
$watcher->modify([Event::Read, Event::Write], 'updated data');

if ($watcher->isActive()) {
    $watcher->remove();
}

The Event enum is where the modern mechanisms surface. Alongside the obvious Read and Write, you get Event::OneShot to have a watcher remove itself automatically after it fires once, and Event::EdgeTriggered for the high-performance mode that only reports state changes rather than state. Edge triggering is the technique that lets epoll and kqueue scale to tens of thousands of connections, and it is available here through a single enum case. You can even ask the backend whether it supports it before you rely on it, via $poll->getBackend()->supportsEdgeTriggering().

The narrative this quietly kills

There is a line you have heard a thousand times, usually from someone who last touched PHP in 2014. "PHP can't scale." For a long time there was a kernel of technical truth buried under the reputation, because the language genuinely did not give you native access to the polling primitives that high-concurrency servers are built on. You could get there with userland libraries and an optional extension, but the foundation was not in the language.

With 8.6 it is. AMPHP, ReactPHP, and Revolt can move their event loop backends onto it. The ext-uv footnote can start disappearing from async READMEs. Benchmarks handling the C10K problem on a stock PHP build will follow, because epoll and kqueue hold steady at ten thousand connections and beyond where select() falls apart after a few hundred. The last technical leg under "PHP can't scale" gets kicked away, and what remains is a perception problem rather than a language one.

I want to be honest about the limits of the excitement, because the accurate version is more persuasive than the hype. This passed with one dissenting vote and four abstentions, not the unanimous landslide some of the early write-ups claimed. Most application developers will genuinely never write new Io\Poll\Context() in anger. You will feel this through your framework, through a faster FPM, through an async library that finally ships one backend instead of three, through FrankenPHP handling signals correctly under threads. The value is real and it is almost entirely indirect.

That is precisely why it is underrated. A loud feature you use every day gets attention by default. A quiet foundation that raises the floor for the entire ecosystem has to be pointed at, because the people it helps most are the ones who will never see it directly. Jakub did the unglamorous work of building the primitive that was missing for twenty years, and the people who understood what it does voted it through without much noise.

Sometimes the most important thing in a release is the line nobody is arguing about.

Next time you read a README that tells you to install ext-uv for performance, remember that the footnote has an expiry date now, and go and read what Io\Poll is doing underneath.

A Proper Look at Tabstack

Steve McDougall — Tue, 23 Jun 2026 20:07:27 +0000

I have spent a fair bit of time recently with Tabstack, including building a CLI around its API, and I think it is worth writing down what it actually is, what you would reach for it to do, and whether it earns its place in your stack. There is a lot of noise around anything with "AI" in the description at the moment, so I want to cut through that and talk about the thing on its own terms.

Let me start with the problem it exists to solve, because the product makes far more sense once you have felt the pain it removes.

The problem nobody wants to own

If you have ever tried to give a program reliable access to the web, you know how quickly it turns into a swamp. You begin with a simple fetch. Then a site renders its content client-side, so a raw fetch returns an empty shell and you reach for a headless browser. Then you are managing proxies because you keep getting rate limited. Then the markup changes and your carefully written selectors snap. Then you are writing bespoke parsing logic for every single site you touch, and every one of those is a small liability waiting to break at three in the morning.

None of that is the interesting part of your product. It is plumbing. You are maintaining a rendering engine, a proxy layer, and a pile of brittle extraction code purely so your actual application can read a page. That is the work Tabstack is offering to take off your hands.

What Tabstack actually is

Tabstack is a web execution and data transformation API built at Mozilla. The pitch is straightforward: you hand it a URL, a schema, a question, or a task, and the browser, the model, and the orchestration all run on their side. You get back clean data, cited answers, or completed browser tasks. There is no headless Chrome for you to provision, no LLM for you to wire in, and no parsing layer for you to babysit.

It is a developer product, not a consumer one. There is no dashboard where you sit and browse the web by hand. It is a REST API with TypeScript and Python SDKs, an MCP server, and a CLI, and it is aimed squarely at people building agents, research tools, data pipelines, and automated workflows.

At the time of writing it is in public early access, and every account starts with ten thousand free credits and no card required, which is generous enough to actually evaluate it properly rather than just kicking the tyres.

The four things it does

The whole surface comes down to four capabilities, and once you see them laid out, the mental model clicks into place.

Extract turns a page into something a model can actually use. The cheapest form is markdown. You give it a URL and you get clean markdown back, frontmatter and all, which is exactly what you want when you are feeding a context window and do not want to burn tokens on rendered HTML noise.

curl -X POST "https://api.tabstack.ai/v1/extract/markdown" \
  -H "Authorization: Bearer $TABSTACK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

The more interesting form of extract is JSON. You describe the shape you want with a schema, you pass a URL, and you get back data that matches that schema. No selectors, no parsing, no handling of unexpected shapes downstream.

import Tabstack from '@tabstack/sdk';

const client = new Tabstack();

const { jobs } = await client.extract.json({
 url: 'https://www.google.com/about/careers/applications/jobs/results',
 json_schema: {
  jobs: [{ title: 'string', location: 'string' }],
 },
});

That schema enforcement is the bit that matters. The contract is yours, not the website's. When the site reshuffles its layout next week, your pipeline does not care, because you are asking for a shape rather than scraping a structure.

Generate is the sibling to extract, and the distinction is worth getting right. Extract pulls data that already exists on the page. Generate uses a model to create something new from that content: a summary, a categorisation, a rewritten output, a tailored message. If you want the price that is printed on a product page, that is extract. If you want a one-line sentiment-tagged summary of a review, that is generate.

Automate is the ambitious one. You give it a task in plain language and it drives a real browser to completion: clicking, scrolling, filling forms, submitting, working through multi-step flows on sites you do not control.

import os
from tabstack import Tabstack

tabs = Tabstack(api_key=os.getenv('TABSTACK_API_KEY'))

result = await tabs.agent.automate(
    url="https://news.ycombinator.com",
    task="Go through 5 pages of the top posts. For each post, determine the "
         "website it is from and group all posts by website. Return the 10 "
         "websites with the most posts.",
)

This is the capability that replaces the headless-browser-and-glue-code stack entirely. It also has guardrails you can set, a max-iterations cap, and an interactive mode where your own orchestrator can stay in the loop, which I will come back to.

Research runs an autonomous multi-source pass and hands you back a synthesised answer with every claim cited to its source. The selection of sources, the reading, the synthesis, and the citation all happen inside one call.

const stream = await client.agent.research({
 query: 'Key risks in CRE lending right now?',
 mode: 'fast',
});

for await (const event of stream) {
 if (event.event === 'complete') {
  console.log(event.data.report);
  console.log(event.data.metadata.citedPages);
 }
}

The citations are the selling point here. If you are shipping a research feature to real users, an answer they can trust because every claim points at a source is worth a great deal more than a confident paragraph with no provenance.

What you would actually build with it

The four capabilities are abstract until you map them onto real work, so here is where they land.

Price and competitor monitoring is the obvious one. A scheduled extract call against a set of product pages, returning schema-matched JSON, keeps a dashboard current without anyone copy-pasting from a browser. Lead enrichment is another clean fit: take a raw domain, extract headcount, tech stack, and funding signals, and push it into your pipeline instead of stitching together a row of data vendors. Job aggregation, market research, and competitive intelligence all sit in the same bucket, where the win is structured output from messy pages.

Research mode opens up in-product features: a research assistant inside your application that answers from the live web with citations, rather than from a stale index. And automate covers the operations work that used to need a fragile RPA script, where an agent has to log into a dashboard, work through a flow, and pull something back out.

The common thread is that all of these need live web context, and none of the teams building them wants to own a scraping stack to get it.

Why you would reach for it over rolling your own

The honest answer is the abstraction. You are buying back the time you would otherwise spend on rendering, proxies, parsing, and the maintenance tail that follows all three. For most teams that maintenance tail is the real cost, because it never shows up in the estimate and never stops.

There is a second reason that is specific to Tabstack, and it is the Mozilla backing. The privacy posture is not an afterthought bolted on for the marketing page. Requests carry a dedicated Tabstack user-agent so site owners can identify the traffic, robots.txt opt-outs aimed at that agent are honoured, and retrieved content is treated as ephemeral and is never used to train models. If you care about being a responsible actor on the web, and increasingly you have to, that matters. It is web access that is built to be observable and well-behaved rather than a mass harvester wearing a disguise.

The third reason is just developer experience. Scoped API keys, a clean Bearer-token auth flow, two first-party SDKs, an MCP server, a CLI, and per-action cost shown in the console before you run anything. The "open in Claude, ChatGPT, or Cursor" buttons on every docs page tell you who they are building for. It feels like a product made by people who have actually had to integrate other people's APIs.

The pricing, plainly

Tabstack is credit-based, and I appreciate that one currency covers the whole platform rather than a separate line item for extraction versus research. Every call spends credits, and the cost depends on the endpoint and how hard you make it work.

The per-action rates are easy to hold in your head. Markdown extraction is ten credits, JSON extraction is fifty, generate is a hundred, automate is a hundred per action, fast research is two hundred and fifty, and balanced research is three hundred and fifty. The important wrinkle is that extract and generate are always a single action per call, so the cost is fixed and predictable, while automate and research chain as many actions as the task needs and bill for each one. A research call that touches more sources costs more than one that touches fewer. You pay for work done, not per request, which is fairer but does mean automate and research costs are variable by nature.

The plans are Individual at zero a month on pay-as-you-go, Team at ninety-nine a month with five hundred thousand credits, and Pro at four hundred and ninety-nine a month with three million. Overages are on by default for Team and Pro and are cheaper per credit than pay-as-you-go, and you can set a spend threshold in the console to either notify or stop. The free tier of ten thousand credits is enough to build something real before you commit, which is exactly how a trial should work.

Where it is rough, and who it is not for

I would be doing you a disservice if I only listed the good parts.

It is not interactive-fast. Extraction sits in the low single-digit seconds and automate or research can run considerably longer, because they are doing real multi-step work. That is fine for background automation and scheduled jobs. It is the wrong tool if you need sub-second responses inside a live user-facing interaction.

It is not built for true web-scale crawling. The credit model is comfortable for thousands to tens of thousands of calls. If your plan is to pull millions of pages a day, you are into custom infrastructure or a dedicated crawler, and the economics will tell you so quickly.

It is not a no-code tool, and it is not a consumer assistant. There is no visual workflow builder and no chat window you talk to. You write API calls. If you want point-and-click automation, this is not it, and if you want a research assistant to use yourself rather than to embed, you want a different category of product entirely.

The variable cost on automate and research is worth a flag too. Single-action endpoints are easy to budget. The agentic ones are not, by design, so you will want guardrails, sensible max-iteration caps, and an eye on the console while you learn the shape of your own workloads.

Where I land

Tabstack is doing something I genuinely respect, which is treating browsing as a distinct infrastructure problem and solving it properly rather than bolting a scraper onto an LLM and calling it a day. The schema-first extraction is the standout feature, the citation-backed research is the one I would build a product feature on, and the Mozilla privacy stance is the kind of thing that should be table stakes and somehow rarely is.

If you are building agents or AI features that need live web data and you do not want to own a scraping stack, it is straightforwardly worth your time. Start narrow. Pick one workflow, point it at the sites you actually care about, and see how the output holds up against your real targets rather than against a demo. The free credits make that an easy experiment to run, and the cost of finding out is close to nothing.

I will likely follow this with a walkthrough of building something small and real on top of the automate endpoint, because that is where the interesting edges live.

The Tips Behind API Artisan: Building Laravel APIs Developers Actually Want to Use

Steve McDougall — Tue, 16 Jun 2026 21:38:58 +0000

I have just finished writing API Artisan: A Guide to Building APIs with Laravel, and I am giving it away for free. Before you commit to 300-odd pages, let me give you the short version: the tips, patterns, and small decisions that separate an API that technically works from one that developers are genuinely happy to depend on.

None of this needs more hardware, a different framework, or a bigger team. It needs you to point your attention at the right things. These are the ones I keep coming back to.

Start by measuring the right thing

Ask most teams how they know their API is good and you get a single question back: does it work? Can I hit this endpoint and get a response? That question is necessary, and it is nowhere near enough.

The question I want you to ask instead is whether your API is liveable with. Can a developer read your docs, understand your auth model, make a successful request, and handle an error without contacting support, trawling a forum, or guessing what a status code is trying to tell them? The gap between "works" and "liveable with" never shows up in a sprint retro, but it shows up everywhere else: in support volume, in integration timelines that overrun, and in the quiet moment a developer decides to build around your API rather than with it.

Everything else in the book hangs off one mindset shift: an API is a product. It has users. Treat it as an implementation detail and it will behave like one. It will change without warning when your internals change, and it will be inconsistent because different people wrote different parts on different days with different conventions.

Write the contract before the code

The natural way to build an endpoint is to write the handler, return some data, and document it afterwards if there is time. It feels efficient, and in the short term it is. The problem is what it produces: a contract that was never designed, only discovered.

Let me show you the trap, because I have watched it catch good developers. You have a keys table with a revoked_at column, the Eloquent model is right there, so you reach for $key->toArray(). The response ships with revoked_at as a field name, because that is what the column is called. A few weeks later you realise the name is ambiguous and rename it. To every integration built against you, that rename is a breaking change, and nothing warned anyone.

Designing the response shape before you write the query closes that gap at the source. You cannot leak your schema if you worked out what the caller needs before you touched the database. The question changes from "how do I expose this data?" to "what does the developer calling this actually need?", and the answers genuinely differ.

Here is the asymmetry that makes the discipline worth it. Your implementation can be refactored whenever you like. A published field name lives in codebases you cannot see, running in production systems that will not be updated the day you push a change. So front-load the contract decisions. The cost of getting them wrong is real, and it is paid by other people, later, at the worst possible time.

Know what a breaking change actually is

Most of us can name the obvious ones: remove an endpoint, rename a field. The real list is longer and far more surprising, and every item on it breaks an integration even when the change looks like an improvement.

Changing a field's type, say expires_at from a date string to a Unix timestamp. Changing a field from nullable to non-nullable. Adding a required field to a request. Changing the meaning of a field without touching its name, which is the one that keeps me up at night. Changing error codes or error shapes, because clients build handling logic around them. Switching offset pagination for cursor pagination. Wrapping a collection in a meta object that clients were destructuring directly.

The pattern is always the same. A change that looks internal turns out to be a commitment you made to everyone who integrated with the old behaviour. The answer is not to stop changing things. It is to know what you are committing to before you commit, and to have a mechanism for changing it safely.

Version from day one

Here is a fact that surprises people: adding a /v1 prefix to an API that is already in production is itself a breaking change. Every consumer hitting /keys now has to hit /v1/keys. That is exactly why versioning from day one is not premature optimisation. It is the cheapest moment you will ever get to add it, because no integrations exist yet. The cost is one URL segment, and the payoff is a stable, versioned contract from the first endpoint you ship.

Make the version explicit at every level, not just in the URL. Namespace your controllers under App\Http\Controllers\Keys\V1. When v2 arrives it lives in Keys\V2, the v1 controllers stay exactly as they are, and you physically cannot put v2 logic in a v1 controller because the boundary is visible in the code.

When you do retire a version, reach for the Sunset pattern from RFC 8594. Send a Sunset header carrying the retirement date and a Deprecation header carrying the date it was deprecated. The part almost everyone skips is the Link header pointing at the migration guide, and it is the part that earns its keep. A consumer whose monitoring picks up the Sunset header can follow that link straight to the docs. The header becomes actionable instead of just informational. My rule, and the one I would encourage you to adopt, is that no version retires without at least six months of notice.

Treat REST as conventions, not religion

Let me save you some pull request arguments. Almost nobody implements true hypermedia controls, and almost every "REST API" you have ever used is really HTTP with JSON and some conventions about URLs. That is fine. The conventions are useful. The pretence that we are all implementing a rigorous architectural style is not.

So skip HATEOAS, unless you happen to be building a generic hypermedia browser, which you are almost certainly not. Stop treating the Richardson Maturity Model as a quality score. A level 3 API with inconsistent errors and no docs is a worse API than a level 1 one with a thoughtful, stable contract.

Spend that energy on your status codes instead, because they are part of your contract and clients build retry logic directly on them.

201 not 200 for a creation, so clients can read the Location header
422 not 400 for validation, because 400 means malformed and 422 means well-formed but invalid
403 not 401 when the caller is authenticated but not authorised

Confuse 401 and 403 and you create genuine bugs. A client that gets a 401 may try to re-authenticate. If it should have been a 403, re-authentication will not help, and now your client is sitting in a retry loop wondering what it did wrong.

For the operations that refuse to map to CRUD, like revoking or rotating a key, use sub-path actions: POST /v1/keys/{id}/revoke. Is it pure REST? No. Is it immediately understandable to anyone who reads it? Yes, and that is the trade I will take every time. On nesting, the rule I use is simple: nest one level when it clarifies ownership, and flatten when the nested resource has its own identity.

Make the controller boring

Every controller in the book is a single-action invokable class. One class, one __invoke method, one responsibility. IssueController tells you exactly what it does. KeyController tells you almost nothing.

A handful of small decisions compound here. Reference your controllers as a class, never a closure, because closures cannot be route-cached and a class reference is testable and navigable in your editor. Put declare(strict_types=1) at the top of every file. Mark controllers final. Keep validation in Form Requests and out of the handler, so the framework's automatic failure handling applies cleanly and you never end up with validation logic smeared between rules() and the top of a controller method.

The single best trick in the Laravel section is the payload() pattern. A Form Request's job is not only to validate input, it is to turn that validated input into a typed, immutable value object the rest of your application can trust.

public function payload(): IssueKeyPayload
{
    return new IssueKeyPayload(
        name:      $this->string('name')->toString(),
        scopes:    $this->collect('scopes')->map(KeyScope::from(...)),
        expiresAt: filled($this->input('expires_at'))
            ? CarbonImmutable::parse($this->string('expires_at')->toString())
            : null,
    );
}

Because payload() runs only after validation has passed, it can cast with confidence. KeyScope::from(...) will not throw, because the rules already confirmed every value is a valid enum case. Your controller receives a typed payload, hands it to an action, and returns a resource. The action works in domain types and never touches the request layer, which means it behaves identically whether you call it from a controller, a console command, a queue job, or a test. The payload is the handshake between HTTP and your domain, and it seals a boundary that otherwise quietly leaks.

Give every error the same shape

Out of the box, Laravel hands you a different response shape for a validation error, an unauthenticated request, and a missing model. Four situations, four shapes, and a client that wants consistent error handling has to special-case every one of them.

Pick RFC 9457 Problem+JSON and use it for every error on every endpoint, with no exceptions. One shape, carrying type, title, status, and detail, plus an errors extension when you need per-field validation. Two details are worth stealing outright. Set the Content-Type to application/problem+json so clients can detect it programmatically. And in your fallback Throwable handler, return null outside production, so Laravel keeps showing you full stack traces in development while production returns a clean, generic 500 that never leaks your internals.

Make invalid states impossible

When a resource has a lifecycle, model it as an enum-backed state machine with an explicit transition map. Invalid transitions throw a domain exception. The calling layer does not check the state before it acts; the entity enforces its own rules. That is the whole difference between a system that guarantees its own invariants and one that relies on developers remembering to check a field first.

Treat your domain exceptions, things like InvalidKeyTransitionException, as first-class citizens. They are not unexpected failures, they are known and named outcomes, and they deserve an informative 422 rather than a 500. Register them in your exception handler right alongside the framework ones.

Respect transaction boundaries

A database transaction protects you when a group of operations must all succeed or all fail together. It cannot protect you from everything, and two rules will save you real production pain.

Never make external HTTP calls inside a transaction. If the call fires and the transaction then rolls back, you have caused a side effect the database has no memory of. Push your jobs and webhooks with DB::afterCommit() so they only fire once the data is genuinely committed.

Transactions also cannot help when the network drops after you return a 201 but before the client receives it. The client retries, and you happily process the same operation twice. That is what idempotency keys are for. Accept a client-generated Idempotency-Key header on state-changing endpoints, store the original response, and replay it on any retry that carries the same key.

Build the audit log first, literally

An audit log is not a log file. It is an append-only database table of meaningful domain events: a key was issued, a key was revoked, a member was invited. Append-only is not a polite convention either, it is enforced by the schema. No updated_at, no soft deletes, no update operations anywhere in the code that touches it. A record you can update is not an audit record, it is a mutable row with a timestamp, and those are very different things.

Two design notes I would not skip. Store the event-specific context as JSONB rather than forcing every event type into a fixed set of nullable columns, because the relevant metadata is genuinely different for each one. And capture actor_type as well as actor_id, because an action taken by a human, by an API key, and by the system itself are three different stories in a security review.

The sequence inside every action that records an event is worth memorising: begin the transaction, perform the operation, write the audit record, commit, then dispatch side effects via afterCommit. The audit record is written before anything that could fail independently, which is what makes it the most durable thing in the whole system.

Make the system answer questions under pressure

Attach a request ID to every log line with a tiny piece of middleware.

$requestId = $request->header('X-Request-ID') ?? Str::uuid()->toString();
Log::withContext(['request_id' => $requestId]);
$response = $next($request);
$response->headers->set('X-Request-ID', $requestId);

Now every log entry within a request shares one ID, tracing a request becomes a single filter, and because you echo the ID back in the response header, a consumer reporting a problem can hand you the exact value to search for. One field, every relevant entry.

The rest of operability runs on the same instinct. Go spec-first with OpenAPI so your documentation cannot drift away from your implementation. Run contract tests in CI so an accidental breaking change fails the build instead of a consumer. Treat onboarding as part of the API, with runnable collections rather than a wall of prose. Give the API a clear owner, a deprecation cadence you actually review twice a year, and post-mortems that produce a test or a doc rather than blame.

The thread that ties it together

The patterns are not really the point. The reasoning is. Consumer empathy as an engineering discipline, stability as a feature you ship, observability as an architectural guarantee. Frameworks and conventions will change underneath all of us. That reasoning is durable, and it is what lets you make these same calls on your next project because the system needed them, not because a book told you to.

Every tip here is drawn from a single project I build end to end across the book: Portkey, a production-quality API key management platform with versioning, separate JWT and HMAC layers, an enforced key lifecycle, async rotation, signed webhooks, idempotency, rate limiting at three levels, and a Pest suite with contract tests in CI. Every pattern shows up because the project genuinely needed it, never to show off a technique.

If these notes were useful to you, the full book takes each one apart properly, with the code, the trade-offs, and the reasoning behind every decision. It is free. Download API Artisan and go build the API your consumers deserve.

Giving your agents a terminal: a first look at the tabstack CLI

Steve McDougall — Tue, 16 Jun 2026 21:38:57 +0000

Every project I touch lately ends up needing the same awkward thing: a reliable way to pull the web into a script or an agent. Not a brittle scrape held together with CSS selectors and hope, but something that takes a URL and hands back clean, structured text I can actually pipe into the next step. I have built that wrapper more than once, and it is never as small as you think it will be. So when Mozilla dropped the tabstack CLI, a single Go binary that wraps the Tabstack AI API, I wanted to spend a proper afternoon with it.

The pitch on the README is direct: every web interaction your agent or stack needs, from the terminal or a script. It turns any URL into clean Markdown or schema-shaped JSON, runs natural-language browser automation, and answers research questions with cited sources. The part that made me sit up is that the output is pretty in a terminal and pipeable into jq without a flag. That is a small detail, and it tells you the people who built it actually live on the command line.

Let me walk you through it the way I poked at it myself.

Getting it installed

There is no runtime to install and nothing to bootstrap, because it ships as a single static binary built for macOS, Linux, and Windows. The quickest route is the install script:

curl -fsSL https://tabstack.ai/install.sh | sh

That fetches the right binary for your platform and puts it on your PATH, and you are ready to go. If you would rather not pipe a script into your shell, there are a couple of alternatives. With Go on your machine you can use:

go install github.com/Mozilla-Ocho/tabstack-cli/cmd/tabstack@latest

That drops the binary in $GOPATH/bin, which is usually ~/go/bin. If your shell cannot find tabstack afterwards, you almost certainly have not got that directory on your PATH:

export PATH="$HOME/go/bin:$PATH"

And if you want to avoid Go entirely, there are pre-built binaries on the Releases page, or you can clone the repo and run make install-local, which builds it and copies it to /usr/local/bin so it works in any terminal straight away. Several routes, all of them boring in the best possible way.

Authentication done sensibly

This is the bit I want to praise first, because credential handling is where a lot of CLIs get careless. You log in once:

tabstack auth login            # prompts for the key, input hidden, saved to the config file
tabstack auth status           # shows how your key is being resolved, never prints it

A key can come from three places, and the precedence is exactly the order you would hope for:

the --api-key flag
the TABSTACK_API_KEY environment variable
the config file at $XDG_CONFIG_HOME/tabstack/config.toml, defaulting to ~/.config/tabstack/config.toml

The config file is written 0600, so it is not world-readable, and auth status will tell you which source won without ever leaking the value. This is the contract I want from any tool that holds a secret: a flag for one-off overrides, an environment variable for CI, and a locked-down file for everyday local use. If no key is found, the API commands exit with code 2 and tell you how to set one, rather than failing with a stack trace. We will come back to those exit codes, because they matter.

The core: turning a page into something useful

Every command follows the same shape: tabstack <group> <action> <target>. Once that clicks, the whole surface area feels predictable.

The first thing I reached for was extract. At its simplest it converts a page to clean Markdown:

tabstack extract markdown https://example.com --metadata

The --metadata flag pulls in the title, author, and similar bits alongside the content. Useful, but the version I keep coming back to is structured extraction, where you hand it a JSON schema and it shapes the output to match:

tabstack extract json https://example.com --schema @schema.json
tabstack extract json https://example.com --schema '{"type":"object","properties":{"title":{"type":"string"}}}'

Notice the @schema.json syntax. That @file convention reads the schema from a file, and it works the same way for the other input flags too. You can pass a literal string, point at a file with @, or read from stdin with -, which is the same ergonomics as curl -d. So this is perfectly valid:

echo '{"type":"object"}' | tabstack extract json https://example.com --schema -

If you have ever written the glue that decides "is this argument a path or a literal", you will appreciate that they solved it once and applied it everywhere.

Generate: extraction with an opinion

Where extract pulls what is on the page, generate fetches a page and transforms it with AI into the shape you describe. You give it instructions as well as a schema:

tabstack generate json https://example.com \
  --instructions "Summarise the article and list the key points." \
  --schema @schema.json

The mental model I landed on is this: extract json is for getting the data that is genuinely there, and generate json is for asking a model to produce something new from the page, summaries, classifications, restructured points, while still constraining the output to a schema you control. Keeping those two as separate verbs is a good call, because it stops you reaching for the heavier, slower path when all you wanted was the title.

Agents: automation and research

This is where it stops being a fetch tool and starts being something more interesting. The agent group runs server-side and streams progress back as it works.

Browser automation takes a natural-language task:

tabstack agent automate "Find the pricing for the Pro plan" --url https://example.com

Research searches the web, synthesises an answer, and prints a report with numbered, cited sources:

tabstack agent research "What are the latest developments in quantum computing?" --mode balanced

The feature I did not expect, and rather liked, is interactive automation. You can start a run that is allowed to pause and ask you for input partway through:

tabstack agent automate "Log in and download the latest invoice" --url https://example.com --interactive

When it pauses, it gives you a request ID, and you answer it with a separate command:

tabstack agent input <request-id> --data '{"fields":[{"ref":"field1","value":"yes"}]}'

Or you decline:

tabstack agent input <request-id> --data '{"cancelled":true}'

That agent input command only applies to runs started with --interactive. Without the flag, an automation never stops to ask. It is a clean way to handle the reality that some tasks genuinely need a human in the loop, without baking that assumption into every run.

The part that makes it scriptable

Here is the design decision I keep telling people about. The output is pretty and styled when you are sitting at a terminal, and it switches to JSON automatically when the output is piped. No flag required:

tabstack extract markdown https://example.com | jq .

You can force a mode with -o pretty or -o json if you want to be explicit, and you can kill colour with --no-color or the NO_COLOR environment variable. The streaming commands, automate and research, emit one NDJSON line per event when they are in JSON mode, so you can process events as they arrive rather than waiting for the whole thing to finish.

The other half of scriptability is the exit codes, and they are thought through:

Code	Meaning
`0`	success
`1`	runtime or network error
`2`	usage, invalid input, or missing config such as no API key
`3`	API error or in-band task failure

Distinct codes mean you can branch on the actual cause in a shell script, telling a bad argument apart from a network blip apart from an API rejection:

if ! tabstack extract markdown "$url" > out.md; then
  case $? in
    2) echo "check your arguments" ;;
    3) echo "the API rejected the request" ;;
    *) echo "network or runtime error" ;;
  esac
fi

I cannot tell you how often I have wanted exactly this from a tool and been handed a flat exit 1 for every conceivable failure instead.

A few common options round it out. --effort lets you pick the speed and capability tradeoff on extract and generate: min is fastest with no fallback, standard is the balanced default, and max does full browser rendering for JavaScript-heavy sites at the cost of taking longer. --geo GB routes the fetch through a given country using an ISO 3166-1 alpha-2 code, which is handy when a page behaves differently by region. And --nocache bypasses the cache when you need a fresh read.

Built to be driven by agents too

The detail that tells you where this is headed is the AGENTS.md file in the repo. The CLI is designed to be driven by LLM agents as well as humans, and that file documents every command, flag, and exit code in a form tuned for machine consumption. If you are wiring tabstack into Claude Code or your own harness, you point the agent at that file and let it learn the surface area itself.

This is the right shape for the moment we are in. A well-behaved CLI with predictable verbs, machine-readable output, and meaningful exit codes is exactly the kind of tool an agent can use safely, because every outcome is legible. The same properties that make it pleasant for me at the terminal make it tractable for a model in a loop.

Would I reach for it?

Yes, and for a specific reason. The web-access problem keeps showing up in my work, and I have written enough of these wrappers to know the unglamorous parts: credential precedence, the file-or-literal-or-stdin question, knowing whether a failure was mine or theirs, behaving differently when piped. The tabstack CLI has answered all of those the way I would want them answered, and it has done it in a single binary with no runtime to manage. It is v1.0.0, MIT licensed, and it came out of Mozilla, so it is not a weekend experiment you are betting your pipeline on.

If your stack or your agents need to read the web, give it an afternoon. Start with tabstack auth login and tabstack extract markdown on a page you know, and build out from there. The shape of the thing rewards exploration.

I have a feeling the interesting work is not in the extraction at all, but in what you wire on the other end of that pipe. That is the article I want to write next.

The Art Of Keeping Business Logic Honest

Steve McDougall — Wed, 20 May 2026 18:55:24 +0000

There is a moment in most long-lived applications where you open a controller and find a block of conditionals that nobody quite understands anymore. Something like if ($entity->status === 'pending' && $this->someFlag) buried inside a service class, half-guarding a transition that was probably fine when someone wrote it but now nobody wants to touch. The business logic has drifted from the code, and the code is quietly lying about what the system actually does.

I have been building a platform recently where I could see this problem coming from a long way off. The domain has entities that go through well-defined lifecycles - things move through stages, transitions have rules, and when a transition happens, a bunch of other things need to follow. The instinct is to reach for a big use case class, or a service that handles everything in sequence. But that approach tends to collapse under its own weight as requirements change.

Instead, I ended up with a two-layer pattern: a strict state machine for each entity that owns nothing but transition rules, and a separate workflow engine that responds to those transitions and orchestrates the follow-on work. This article is about how that pattern holds together and why I think it is worth the setup cost.

The Problem With Ad-Hoc Status Management

Before getting into the pattern, it is worth being honest about what you are replacing. Most applications manage entity status as a string column and a handful of conditionals scattered across services. When you need to add a new status, you add it to the column's allowed values and start writing if checks wherever it matters. This works fine for simple lifecycles. It starts to hurt when:

You have 5+ statuses and a non-trivial transition graph
Different transitions need different side effects
Some transitions involve async operations that can fail halfway through
You need an audit trail of who triggered what and when

The state machine pattern addresses the first two. The workflow engine addresses the last two. Together they handle the full picture.

Layer One: The State Machine

A state machine in this pattern is deliberately narrow. Its only job is to know which transitions are legal and to produce a domain event when one is performed. It does not send emails. It does not touch the database. It does not call other services.

Here is what a simple order lifecycle might look like:

OrderStatus:
  draft -> submitted -> approved -> fulfilled
  submitted -> rejected
  approved -> cancelled

The entity enforces this. If you try to transition from draft directly to fulfilled, you get a domain exception - not a silent data integrity problem discovered six months later.

// src/Orders/Domain/Order.php

final class Order
{
    private OrderStatus $status;

    public function submit(): OrderSubmitted
    {
        if ($this->status !== OrderStatus::Draft) {
            throw new InvalidTransitionException(
                "Cannot submit an order in status {$this->status->value}."
            );
        }

        $this->status = OrderStatus::Submitted;

        return new OrderSubmitted(
            orderId: $this->id,
            submittedAt: new \DateTimeImmutable(),
        );
    }

    public function approve(string $approvedBy): OrderApproved
    {
        if ($this->status !== OrderStatus::Submitted) {
            throw new InvalidTransitionException(
                "Cannot approve an order in status {$this->status->value}."
            );
        }

        $this->status = OrderStatus::Approved;

        return new OrderApproved(
            orderId: $this->id,
            approvedBy: $approvedBy,
            approvedAt: new \DateTimeImmutable(),
        );
    }
}

Notice that each method returns a domain event rather than dispatching it. The calling layer - a use case - is responsible for taking that event, persisting it to an event log, and firing it into the Laravel event system. The domain entity stays clean.

// src/Orders/Application/ApproveOrder.php

final class ApproveOrder
{
    public function __construct(
        private readonly OrderRepositoryContract $repository,
    ) {}

    public function execute(string $orderId, string $approvedBy): array
    {
        $order = $this->repository->findById($orderId);
        $event = $order->approve($approvedBy);

        $this->repository->save($order);

        return [$event];
    }
}

// app/Http/Controllers/Orders/V1/ApproveController.php

public function __invoke(ApproveOrderRequest $request, string $orderId): JsonResponse
{
    $events = $this->approveOrder->execute($orderId, $request->user()->id);

    foreach ($events as $event) {
        $this->eventLog->record($event, $orderId, 'order', $request->user()->id);
        Event::dispatch($event);
    }

    return new JsonDataResponse(data: ['approved' => true]);
}

This separation matters more than it might look. The use case is testable without Laravel. The domain entity has no infrastructure dependencies. The event log gets written before any side effects fire, so you always have a record of what happened even if something downstream blows up.

Layer Two: The Workflow Engine

The state machine tells you that a transition happened. The workflow engine decides what to do about it.

A workflow is a named, ordered list of steps triggered by a domain event. Steps are discrete PHP classes. The engine runs them in order, persists progress after each one, and can pause mid-workflow waiting for an external signal before continuing.

A workflow instance is the running execution of a workflow definition for a specific entity. The definition - the list of steps - lives entirely in code. The instance - which step we are on, what has happened so far - lives in the database.

This distinction is important. Changing a workflow is a code change with a readable diff. You do not need a migration to add a step. The engine always uses the current definition code when it resumes a paused instance.

Defining a Workflow

// app/Workflows/OrderApproval/OrderApprovalWorkflow.php

final class OrderApprovalWorkflow implements WorkflowDefinitionContract
{
    public static function name(): string
    {
        return 'order_approval';
    }

    public function steps(): array
    {
        return [
            ValidateOrderItems::class,
            ReserveInventory::class,
            CreateInvoice::class,
            AwaitPayment::class,
            NotifyFulfillmentTeam::class,
        ];
    }
}

Every step implements the same contract:

interface WorkflowStepContract
{
    public function execute(WorkflowContext $context): StepResult;
    public function timeoutSeconds(): ?int;
    public function maxAttempts(): int;
}

A step returns one of three outcomes: complete, await, or fail.

// src/Workflow/Domain/StepResult.php

final class StepResult
{
    private function __construct(
        public readonly string $outcome,
        public readonly ?string $awaitSignal,
        public readonly ?string $failReason,
        public readonly array $contextUpdates,
    ) {}

    public static function complete(array $contextUpdates = []): self
    {
        return new self('complete', null, null, $contextUpdates);
    }

    public static function await(string $signal, array $contextUpdates = []): self
    {
        return new self('await', $signal, null, $contextUpdates);
    }

    public static function fail(string $reason): self
    {
        return new self('fail', null, $reason, []);
    }
}

The AwaitPayment step, for example, does not poll an external payment service. It parks the workflow:

// app/Workflows/OrderApproval/Steps/AwaitPayment.php

final class AwaitPayment implements WorkflowStepContract
{
    public function execute(WorkflowContext $context): StepResult
    {
        return StepResult::await('payment_succeeded');
    }

    public function timeoutSeconds(): ?int
    {
        return 604800; // 7 days
    }

    public function maxAttempts(): int
    {
        return 1;
    }
}

The engine sets the instance status to awaiting, stores the signal name it is waiting for, and stops. When the payment provider's webhook arrives, it delivers the signal:

// app/Jobs/ProcessPaymentWebhook.php

$instances = $this->workflowRepository->findAwaitingSignalForAggregate(
    aggregateId: $orderId,
    signal: 'payment_succeeded',
);

foreach ($instances as $instance) {
    $this->engine->signal(
        instanceId: $instance->id,
        signal: 'payment_succeeded',
        signalData: [
            'payment_id' => $payment->id,
            'paid_at'    => now()->toIso8601String(),
        ],
    );
}

The engine resumes from the step after AwaitPayment and carries the signal data forward in the context. The NotifyFulfillmentTeam step can read payment_id from context without knowing anything about how payment was confirmed.

WorkflowContext: Shared State Without Shared Mutable State

Context is an immutable bag of data that flows through all steps. Steps cannot write directly to it - they return updates via StepResult::complete(['key' => 'value']) and the engine merges those in before passing context to the next step.

// src/Workflow/Domain/WorkflowContext.php

final class WorkflowContext
{
    private function __construct(
        public readonly string $workflowInstanceId,
        public readonly string $aggregateId,
        public readonly string $aggregateType,
        private readonly array $data,
    ) {}

    public function get(string $key, mixed $default = null): mixed
    {
        return $this->data[$key] ?? $default;
    }

    public function with(array $additions): self
    {
        return new self(
            $this->workflowInstanceId,
            $this->aggregateId,
            $this->aggregateType,
            array_merge($this->data, $additions),
        );
    }
}

This keeps steps honest. A step cannot accidentally clobber data set by a previous step except through the explicit return value. It also makes testing straightforward:

it('awaits payment signal when order is approved', function () {
    $context = WorkflowContext::make(
        workflowInstanceId: 'wf_001',
        aggregateId: 'ord_001',
        aggregateType: 'order',
        initialData: [],
    );

    $step = new AwaitPayment();
    $result = $step->execute($context);

    expect($result->outcome)->toBe('await');
    expect($result->awaitSignal)->toBe('payment_succeeded');
});

Each step is independently testable. You construct a context, run the step, assert the result. No Laravel bootstrapping required.

Branching Without a Tree Structure

Not all workflows are linear. Some paths depend on conditions that are only known at runtime, and this is where a lot of workflow implementations go wrong. The temptation is to model branching as a tree: if condition A, follow path X; if condition B, follow path Y. That works fine until you have three conditions and four paths, at which point you have a directed graph masquerading as code, and nobody can read it without drawing a diagram first.

My preference is context flags over nested step trees. The idea is simple: a routing step runs early in the workflow, evaluates the conditions, and writes its findings into context. Later steps read those findings and decide whether to skip themselves or do their work. The step list stays flat. You can read it from top to bottom and understand every possible path the workflow might take.

Here is a concrete example. An order workflow might need manual review for high-value orders, a compliance check for orders from certain regions, and expedited processing for customers on a priority tier. Rather than splitting into separate workflow definitions (which duplicates a lot of shared steps) or nesting conditional blocks inside the engine, a single routing step evaluates all of this upfront:

// app/Workflows/OrderApproval/Steps/RouteApproval.php

final class RouteApproval implements WorkflowStepContract
{
    public function __construct(
        private readonly CustomerRepositoryContract $customers,
        private readonly ComplianceService $compliance,
    ) {}

    public function execute(WorkflowContext $context): StepResult
    {
        $customer = $this->customers->findById($context->get('customer_id'));

        return StepResult::complete([
            'requires_manual_review'  => $context->get('order_value') > 10000,
            'requires_compliance_hold' => $this->compliance->requiresHold($customer->region),
            'is_priority_customer'     => $customer->isPriority(),
        ]);
    }
}

Later steps consume these flags independently. Each one is responsible for its own skip logic:

// app/Workflows/OrderApproval/Steps/AwaitManualApproval.php

final class AwaitManualApproval implements WorkflowStepContract
{
    public function execute(WorkflowContext $context): StepResult
    {
        if (! $context->get('requires_manual_review', false)) {
            return StepResult::complete();
        }

        return StepResult::await('manager_decision');
    }
}

// app/Workflows/OrderApproval/Steps/AwaitComplianceClearance.php

final class AwaitComplianceClearance implements WorkflowStepContract
{
    public function execute(WorkflowContext $context): StepResult
    {
        if (! $context->get('requires_compliance_hold', false)) {
            return StepResult::complete();
        }

        return StepResult::await('compliance_cleared');
    }
}

// app/Workflows/OrderApproval/Steps/NotifyFulfillmentTeam.php

final class NotifyFulfillmentTeam implements WorkflowStepContract
{
    public function __construct(
        private readonly NotificationService $notifications,
    ) {}

    public function execute(WorkflowContext $context): StepResult
    {
        $priority = $context->get('is_priority_customer', false);

        $this->notifications->notifyFulfillment(
            orderId: $context->get('order_id'),
            queue: $priority ? 'priority' : 'standard',
        );

        return StepResult::complete();
    }
}

And the workflow definition itself reads clearly, top to bottom:

public function steps(): array
{
    return [
        ValidateOrderItems::class,
        RouteApproval::class,          // sets context flags
        ReserveInventory::class,
        AwaitComplianceClearance::class, // skips if not required
        CreateInvoice::class,
        AwaitPayment::class,
        AwaitManualApproval::class,    // skips if not required
        NotifyFulfillmentTeam::class,  // uses priority flag
    ];
}

Reading that list, you can already build a mental model of what happens. A standard order hits RouteApproval, skips both await steps, and lands at NotifyFulfillmentTeam on the standard queue. A high-value order from a restricted region pauses twice before it gets there. A priority customer skips the holds but gets the priority queue at the end.

When the Context Flag Approach Breaks Down

It is worth being honest about the limits of this pattern. Context flags work well when branches share a significant chunk of their steps. If two paths genuinely have nothing in common beyond the trigger event, separate workflow definitions are probably the right call. Trying to force them into one definition just to keep things tidy results in a step list full of steps that almost always skip themselves, which is its own form of confusion.

The other case where context flags get awkward is when a branch decision depends on the outcome of an earlier awaiting step. Suppose a manual approval step can produce one of three decisions: approved, approved with modifications, or sent back for renegotiation. The step waiting for that signal needs to write the decision into context so the next step can act on it:

// app/Workflows/OrderApproval/Steps/AwaitManualApproval.php

final class AwaitManualApproval implements WorkflowStepContract
{
    public function execute(WorkflowContext $context): StepResult
    {
        if (! $context->get('requires_manual_review', false)) {
            return StepResult::complete(['approval_decision' => 'auto_approved']);
        }

        // Check if the signal has already been delivered (i.e. we are resuming)
        $decision = $context->get('approval_decision');

        if ($decision !== null) {
            return StepResult::complete();
        }

        return StepResult::await('manager_decision');
    }
}

When the signal arrives, the engine calls execute again with the signal data merged into context. The step finds approval_decision already set, returns complete, and the next step can read the decision and act accordingly. This is slightly counterintuitive at first - the step runs twice - but it keeps signal handling inside the step that owns that state, rather than leaking it into the engine or a separate handler class.

Reading Signal Data in the Next Step

Once the signal data is in context, consuming it downstream is straightforward:

// app/Workflows/OrderApproval/Steps/ProcessApprovalDecision.php

final class ProcessApprovalDecision implements WorkflowStepContract
{
    public function execute(WorkflowContext $context): StepResult
    {
        $decision = $context->get('approval_decision');

        return match ($decision) {
            'approved'              => StepResult::complete(['proceed' => true]),
            'approved_with_changes' => StepResult::complete(['proceed' => true, 'notify_changes' => true]),
            'sent_for_renegotiation' => StepResult::complete(['proceed' => false]),
            default                 => StepResult::fail("Unknown approval decision: {$decision}"),
        };
    }
}

Steps further down the chain check proceed and notify_changes as needed. The branching logic is distributed across the steps that care about it, rather than centralised in a router that has to know about everything.

Timeouts: When Waiting Is Not Infinite

Every awaiting step raises a question the engine cannot answer on its own: what happens if the signal never arrives? A payment that is never completed. A manager who goes on holiday without approving the order. A compliance team that sits on a request for three weeks. Real workflows have to handle these cases, and "wait forever" is rarely the right answer.

The timeout mechanism is built directly into the step contract. Every step declares how long it is willing to wait before something has to happen:

interface WorkflowStepContract
{
    public function execute(WorkflowContext $context): StepResult;
    public function timeoutSeconds(): ?int;  // null means no timeout
    public function maxAttempts(): int;
}

When the engine parks a workflow at an awaiting step that declares a timeout, it dispatches a delayed job alongside the parking:

// Inside WorkflowEngine::handleAwait()

if ($timeout = $step->timeoutSeconds()) {
    TimeoutWorkflowStep::dispatch(
        instanceId: $instance->id,
        expectedSignal: $result->awaitSignal,
    )->delay(now()->addSeconds($timeout));
}

When the job fires, it checks whether the workflow is still waiting on the same signal. If the signal already arrived and the workflow advanced, the job finds a different state and does nothing. If the workflow is still parked, it delivers a synthetic timeout signal:

// app/Jobs/TimeoutWorkflowStep.php

final class TimeoutWorkflowStep implements ShouldQueue
{
    public function __construct(
        private readonly string $instanceId,
        private readonly string $expectedSignal,
    ) {}

    public function handle(WorkflowEngine $engine, WorkflowRepositoryContract $repository): void
    {
        $instance = $repository->findById($this->instanceId);

        if (! $instance->canReceiveSignal($this->expectedSignal)) {
            return;
        }

        $engine->signal(
            instanceId: $this->instanceId,
            signal: 'timeout',
            signalData: ['timed_out_signal' => $this->expectedSignal],
        );
    }
}

The step itself then decides what a timeout means for its context. This is the key design decision: the step owns the timeout behaviour, not the engine. The engine just delivers a signal. The step interprets it.

Timeout Behaviour Is Step-Specific

Different awaiting steps need very different responses to a timeout. A payment step timing out probably means the order should be cancelled. A manual approval step timing out might mean the workflow should auto-approve, or escalate, or just fail loudly and wait for human intervention. Making the step responsible for this means each one can handle it appropriately.

Here is a payment step that cancels the order on timeout:

// app/Workflows/OrderApproval/Steps/AwaitPayment.php

final class AwaitPayment implements WorkflowStepContract
{
    public function execute(WorkflowContext $context): StepResult
    {
        // Already resolved - either paid or timed out
        if ($context->get('payment_status') !== null) {
            $status = $context->get('payment_status');

            if ($status === 'timed_out') {
                return StepResult::fail('Payment window expired. Order will be cancelled.');
            }

            return StepResult::complete();
        }

        // Signal just arrived
        $signal = $context->get('_signal');

        if ($signal === 'timeout') {
            return StepResult::complete(['payment_status' => 'timed_out']);
        }

        if ($signal === 'payment_succeeded') {
            return StepResult::complete([
                'payment_status' => 'paid',
                'payment_id'     => $context->get('_signal_data.payment_id'),
            ]);
        }

        // First time through - park and wait
        return StepResult::await('payment_succeeded');
    }

    public function timeoutSeconds(): ?int
    {
        return 604800; // 7 days
    }

    public function maxAttempts(): int
    {
        return 1;
    }
}

Compare that to a manual approval step that auto-approves rather than failing when the window expires. The business rule here is that if a manager does not review within 48 hours, the workflow proceeds as though it were approved:

// app/Workflows/OrderApproval/Steps/AwaitManualApproval.php

final class AwaitManualApproval implements WorkflowStepContract
{
    public function execute(WorkflowContext $context): StepResult
    {
        if (! $context->get('requires_manual_review', false)) {
            return StepResult::complete(['approval_decision' => 'auto_approved']);
        }

        $signal = $context->get('_signal');

        if ($signal === 'timeout') {
            // Business rule: no response in 48h = auto-approved
            return StepResult::complete([
                'approval_decision' => 'auto_approved',
                'auto_approved_reason' => 'timeout',
            ]);
        }

        if ($signal === 'manager_decision') {
            return StepResult::complete([
                'approval_decision' => $context->get('_signal_data.decision'),
            ]);
        }

        return StepResult::await('manager_decision');
    }

    public function timeoutSeconds(): ?int
    {
        return 172800; // 48 hours
    }

    public function maxAttempts(): int
    {
        return 1;
    }
}

Two awaiting steps, two completely different timeout behaviours, zero timeout logic in the engine. The engine is just a postman. It delivers signals and persists state. It does not have opinions about what those signals mean.

Runtime-Configurable Timeouts

Hard-coding timeout values as constants in step classes is fine during early development but tends to become a problem in production. Business rules about payment windows and approval deadlines have a habit of changing, and changing them should not require a deployment.

The step contract is a PHP class, which means timeoutSeconds() can read from wherever it likes:

// app/Workflows/OrderApproval/Steps/AwaitPayment.php

final class AwaitPayment implements WorkflowStepContract
{
    public function __construct(
        private readonly SystemSettingsRepositoryContract $settings,
    ) {}

    public function timeoutSeconds(): ?int
    {
        return $this->settings->get('payments.window_days') * 86400;
    }
}

The service container resolves the step, injects the repository, and the timeout is read from the database at the moment the step executes. Change the setting, and the next workflow instance that reaches this step picks up the new value. Existing parked instances are not affected - their timeout job was already dispatched when they parked - but that is usually the right behaviour. You do not want a settings change to retroactively alter the expectations for workflows already in flight.

What About Timeouts on Timeouts?

One edge case worth thinking through: the TimeoutWorkflowStep job is itself a queued job, which means it can fail. If your queue worker crashes repeatedly and the job exhausts its retry attempts without ever firing, the workflow stays parked indefinitely. For most applications this is an acceptable risk - queue failures are rare and observable - but if you need a harder guarantee, a scheduled job that sweeps for instances whose timeout_at has passed and have not yet received a signal is a reasonable backstop. It trades immediacy for reliability: the sweep might fire 5 minutes late, but it will fire.

// app/Console/Commands/ExpireStaleWorkflowSteps.php

public function handle(WorkflowRepositoryContract $repository, WorkflowEngine $engine): void
{
    $stale = $repository->findTimedOutInstances(now());

    foreach ($stale as $instance) {
        $engine->signal(
            instanceId: $instance->id,
            signal: 'timeout',
            signalData: ['source' => 'sweep', 'timed_out_signal' => $instance->getAwaitingSignal()],
        );
    }
}

Running this as a scheduled command every few minutes means even a prolonged queue outage does not leave workflows stranded forever. The signal data includes a source flag so you can distinguish between a timeout that fired on time via the job and one that was caught by the sweep - useful for monitoring and alerting.

Connecting the Two Layers

The state machine and the workflow engine are connected by a Laravel event listener. The state machine fires a domain event. The listener starts a workflow.

// app/Providers/OrdersServiceProvider.php

public function boot(): void
{
    Event::listen(
        OrderApproved::class,
        StartOrderApprovalWorkflow::class,
    );
}

// app/Listeners/StartOrderApprovalWorkflow.php

final class StartOrderApprovalWorkflow
{
    public function __construct(
        private readonly WorkflowEngine $engine,
    ) {}

    public function handle(OrderApproved $event): void
    {
        $instance = $this->engine->start(
            workflowName: OrderApprovalWorkflow::name(),
            aggregateId: $event->orderId,
            aggregateType: 'order',
            initialData: [
                'order_value' => $event->orderValue,
            ],
        );

        AdvanceWorkflow::dispatch($instance->id);
    }
}

The engine and the state machine have no direct dependency on each other. The listener is the only thing that knows about both.

What the Database Stores

Two tables do the heavy lifting.

workflow_instances tracks a running execution. The key columns are status (running, awaiting, completed, failed, cancelled), current_step_index, awaiting_signal, and context as a JSON blob. When the engine persists between steps, it writes the new index and any context updates.

workflow_signals is an append-only log of every signal delivered to every instance. This gives you a complete record of what happened and when, including which user or system actor delivered each signal.

Because the engine persists before moving to the next step, a failed job always resumes from a known good state rather than re-running work that already succeeded. The AdvanceWorkflow job is safe to retry because the engine checks the current status before doing anything.

What This Pattern Buys You

After living with this for a while, a few things stand out.

The business logic becomes auditable. The state machine defines every valid transition explicitly. The workflow definition defines every step in a process explicitly. Anyone can read both and understand what the system does, without chasing conditionals through layers of service classes.

Async work is a first-class concept. The await mechanism makes it natural to model processes that span hours or days. Waiting for a payment, waiting for a human decision, waiting for an external API callback - these are all just signals. The workflow does not care whether they arrive in 50 milliseconds or 5 days.

Each piece is independently testable. Domain entities with no infrastructure dependencies. Steps that take a context and return a result. The engine with an in-memory repository. You can test the full logic of a workflow without touching the database or the queue.

Adding steps does not require touching existing ones. Inserting a new step into a workflow is a one-class change. The engine picks it up from the definition array. Existing steps are unaffected.

The setup cost is real. You are building an engine, not just writing business logic. But for domains with complex entity lifecycles and multi-step processes that need to survive failures, the investment pays back quickly. The alternative - a growing tangle of service classes, status checks, and jobs that only make sense if you know the history - tends to get more expensive with every feature added.

If you are working on something where entity status matters, where some transitions need to trigger chains of work, and where some of that work is async, this pattern is worth considering. The state machine gives you confidence that your data is always in a valid state. The workflow engine gives you confidence that the work that follows a transition always happens in the right order, even when things go wrong in the middle.

The Mid-Level Mindset

Steve McDougall — Tue, 19 May 2026 12:01:05 +0000

We started this series with a single paragraph from a fictional client. A vague brief about a tool where clients can submit requests and developers can track them. Nine articles later, we have a fully interrogated set of requirements, a collection of properly shaped features, a trusted data model, a system architecture diagram, a layered application design, and a sequenced build plan with individual tickets ready to pick up.

We have not written a single migration. We have not scaffolded a controller or configured a route. And yet the most important work is done.

That is the point this entire series has been building towards.

What Actually Changed

Think back to where we started. The instinct to open your editor the moment a brief lands. The pull towards action, towards visible progress, towards the feeling of productivity that comes from lines of code appearing on a screen.

That instinct is not wrong. It is just premature.

What changed over the course of this series is not the tools you have access to. Laravel was always capable of building Clarity. The database was always going to need an organisations table and a users table and a requests table with the right foreign keys. The layered architecture was always the right approach. None of that knowledge was new.

What changed is the order of operations. The willingness to invest thinking before typing. The discipline to ask "what do I not yet know?" before asking "how do I build this?". That shift in sequence is what defines the move from junior to mid-level, and it compounds over time in ways that are hard to see until you look back.

What Mid-Level Actually Means

The industry uses the term loosely, but when I think about what a mid-level developer actually is, I come back to a few specific qualities.

A mid-level developer can be handed a brief and produce something buildable from it without constant direction. Not because they know every answer upfront, but because they know which questions to ask and how to find the answers before they start building.

A mid-level developer understands that the most expensive code to write is code that solves the wrong problem. They protect against that by doing the thinking work first.

A mid-level developer can communicate the shape of a problem to other people. They can draw a diagram, write a pitch, or explain a data model in plain language. That communication ability is what makes them valuable beyond their own output.

A mid-level developer knows when to apply a pattern and when to leave it out. They have moved past the phase of applying everything they have learned uniformly, and into the phase of using judgment about what a given situation actually needs.

None of those things are about programming language knowledge or framework familiarity. They are about thinking habits. And thinking habits are built through deliberate practice, not through reading documentation.

The Process in Brief

If you take one thing from this series, let it be this sequence. Apply it to every project you work on, every feature you are handed, every ticket that lands in your queue.

First, interrogate the requirement. Surface every unstated assumption. Ask the business questions before the technical ones. Write down what you know and what you do not know yet.

Second, define the actors and their needs. Write user stories with acceptance criteria. Make sure every feature has a clear definition of done that is anchored to a real user getting real value.

Third, shape the work. Decide how much it is worth before you estimate how long it will take. Draw the edges of what is included and write down what is explicitly out of scope.

Fourth, model the data. Draw the ERD before you touch a migration. Check every relationship for cardinality, every foreign key for direction, every array in a column for the related table it should be.

Fifth, draw the system. Show the major components, how they connect, where external dependencies sit, and where data flows. Write the plain-language description. If you cannot describe it clearly in prose, the diagram needs more work.

Sixth, sequence the build. Map the dependencies. Identify what has to exist before each feature can be built. Break each feature into discrete tickets with clear definitions of done.

Then build.

Where to Take This

This process is a foundation, not a ceiling. As you apply it to more projects, you will start to develop instincts about where the complexity hides in a brief, which ERD patterns recur across different problem domains, and which architectural decisions tend to cause the most pain if you get them wrong.

Those instincts are what senior developers have. They have run this process enough times, on enough different problems, that parts of it become automatic. They spot the ambiguous noun in a brief before the client finishes reading it to them. They see the missing pivot table in a data model before anyone has drawn it. They know from experience which features sound small and turn out to be enormous.

You build those instincts by doing the work deliberately, even when it feels slower than just opening your editor. Especially then.

A few resources worth spending time with as you continue:

Ryan Singer's Shape Up is the deepest treatment of the shaping process I have found. It is free, it is short, and every developer who works on product software should read it.

Jeff Patton's User Story Mapping goes further on the user story side than we covered here, and is particularly useful when you are working on complex products with many different user types.

John Ousterhout's A Philosophy of Software Design is the best writing I know of on the question of where complexity comes from and how to fight it. It will change the way you think about every design decision you make.

And then there is the work itself. Take a project, real or fictional, and run the full process from brief to build plan before you write the first line of code. Do it again on the next one. The process becomes faster with practice, but it never stops being valuable.

A Final Word on Clarity

We built something real in this series. Clarity started as a vague paragraph and became a fully designed application with a clear data model, a considered architecture, and a sequenced plan ready to execute.

We never wrote a migration. We never wrote a controller. But anyone who has followed this series could pick up the build plan from article nine and start building Clarity today, with confidence that the foundations are solid and the direction is clear.

That confidence, grounded in thinking rather than assumption, is what the mid-level mindset feels like from the inside.

It is worth developing.

Thank you for following along with From Requirements to Reality. If this series helped you think differently about how you approach a problem before you start building, that is exactly what it was designed to do.

From Diagram To Implementation Plan

Steve McDougall — Tue, 19 May 2026 12:01:04 +0000

We have covered a lot of ground in this series. We started with a vague client brief, interrogated it until the real requirements surfaced, wrote user stories with acceptance criteria, shaped those stories into bounded features, drew an ERD, built a system diagram, and looked at how layers give every piece of code a clear home.

That is the full design process. Now we need to translate it into something a developer can actually execute.

This article is about sequencing. Taking everything we have designed and turning it into an ordered build plan that respects dependencies, minimises wasted work, and lets you deliver value as early as possible.

Why Sequence Matters

A common junior developer instinct is to start with the most interesting part. The feature that sounds fun to build, the component that uses a technology you want to learn, the part of the system that feels most novel. That is understandable, but it is usually the wrong starting point.

The right starting point is the foundation. The code that everything else depends on. If you build the request management system before you have authentication in place, you will either need to stub out user identity in ways that do not reflect reality, or you will need to go back and retrofit it later. Both options cost time you cannot get back.

Sequencing is the discipline of asking: what has to exist before this can be built? Work backwards from every feature until you hit something that has no dependencies, and that is where you start.

Identifying Dependencies

Let me map the dependencies for the Clarity features we shaped in article four.

Client authentication and onboarding has no dependencies on other features. It needs the database and the user model, but those are foundational. This goes first.

Team member management depends on authentication being in place (admins need to be logged in to invite people) and on the user model having role support. It is almost as foundational as authentication itself, and it needs to exist before any meaningful testing of the application can happen.

Request submission depends on authentication (to know who is submitting) and on organisations existing (to associate the request correctly). Team member management should also be done first so there is at least one admin user to test with.

Request status and assignment depends on requests existing. It also depends on team members existing, because you cannot assign a request to a developer who does not exist in the system. This comes after submission.

Comment thread depends on requests existing. It could technically be built in parallel with request status and assignment, but sharing the request detail page means they are better built in sequence to avoid integration friction.

Request list views depend on requests existing and on the basic request detail page being in place. This should be the last feature built, because it is a read view over data that all the other features create.

The Build Order

With dependencies mapped, the sequence becomes clear:

Foundations: database, models, migrations, and base application structure
Authentication: client login, team member login, invitation flow
Team member management: invite, role assignment, deactivation
Request submission: form, validation, attachment upload, status initialisation
Request status and assignment: status transitions, developer assignment, client visibility
Comment thread: posting, internal flag, client-facing view
Request list views: client list, team list, basic status filtering

Each step builds on the last. Each step produces something testable before the next step begins. And each step delivers a piece of working software rather than a collection of half-built components.

Writing the Tickets

A build order is not yet a set of tickets. You still need to break each step into discrete pieces of work that a developer can pick up and complete in a day or two.

Here is how I would break down step four, request submission, into individual tickets.

Ticket: Request model and migration
Create the requests table migration with all columns from the ERD. Create the Request Eloquent model with the HasUlids trait, fillable columns, and the submitter and assignee relationships. Write a model factory for use in tests.

Ticket: Attachment model and migration
Create the attachments table migration. Create the Attachment model with its relationship back to Request. Configure the file storage driver. Write a model factory.

Ticket: Submit request endpoint
Create the StoreRequestRequest form request with validation rules and a payload() method. Create the StoreRequestPayload DTO. Create the SubmitRequest action. Create the StoreController. Wire up the route. Write feature tests covering successful submission, validation failures, and attachment handling.

Ticket: Request detail page
Create the Livewire component for the request detail view. Display title, description, status, assignee, and attachments. Scope the view so clients only see their own requests. Write feature tests covering client access and unauthorised access attempts.

Four tickets for one feature, each independently completable and testable. A developer picking up any one of these knows exactly what done looks like, because the acceptance criteria from the user stories map directly onto the test cases.

The Relationship Between Stories and Tickets

User stories describe what the system should do from a user's perspective. Tickets describe the technical work required to make that happen. They are not the same thing, and conflating them is a common source of confusion.

A single user story often maps to multiple tickets. The "submit a request" story from article three maps to at least the four tickets above. That is fine. The story is the goal. The tickets are the path.

When you write a ticket, you should be able to point to the user story it is serving. If a ticket cannot be connected to a user story, ask whether it needs to exist. Infrastructure work (setting up CI, configuring deployment, writing base test helpers) is an exception, but application feature work should always be traceable back to a user need.

Checking the Plan Against the ERD

Before you start building, do one final check. Go back to your ERD and make sure every entity and every relationship has a corresponding ticket.

In the Clarity plan, let me verify:

Organisation: covered in the authentication step
User: covered in authentication and team member management
Request: covered in request submission
Comment: covered in the comment thread step
Attachment: covered in request submission
All relationships: each foreign key maps to a ticket that creates the parent entity before the child entity

If any entity in your ERD does not have a corresponding ticket, you have a gap. Either you have forgotten something, or that entity is not actually needed for the current scope. Either way, better to find that now than two weeks into the build.

The Plan as a Living Document

A build plan is not a contract. Requirements change, technical discoveries shift priorities, and features that seemed simple turn out to be more complex than they appeared. A good plan is one that absorbs those changes without falling apart.

The way you make a plan resilient is by keeping the dependencies clear. If you know that request status depends on request submission, and request submission depends on authentication, then when authentication takes longer than expected you can immediately see what is blocked and communicate that clearly. You are not surprised. You are just updating the schedule with an accurate picture of what is affected.

That kind of clarity is what teams rely on mid-level developers to provide. Not perfect predictions, but an accurate, up-to-date map of where the work stands and what it connects to.

Putting It Into Practice

Take the project you have been working with throughout this series. Map the dependencies between your shaped features. Write a build order. Then break the first two steps into individual tickets.

For each ticket, make sure you can answer these questions: what does done look like? What does it depend on? Which user story does it serve?

If any of those questions is hard to answer, the ticket needs more work before anyone picks it up.

In the final article, we are going to step back from Clarity and talk about what this entire process represents: the shift in thinking that defines a mid-level developer, and where to take these skills from here.

Thinking In Layers

Steve McDougall — Tue, 19 May 2026 12:00:32 +0000

Ask a junior developer where a piece of logic should live and you will usually get one of two answers: the controller, or the model. Ask a mid-level developer the same question and they will ask you a question back: what kind of logic is it?

That distinction is the heart of this article.

Separation of concerns is one of those principles you hear about early in your career and nod along to without fully internalising what it means in practice. It sounds obvious. Of course different concerns should be separated. But when you are staring at a controller that has grown to three hundred lines and you are not sure how it got there, the theory suddenly feels less clear than it did.

In this article we are going to look at what separation of concerns actually means inside a Laravel application, why fat controllers are a symptom of an architectural problem rather than a code quality problem, and how thinking in layers changes the way you make decisions about where code should live.

What a Layer Actually Is

A layer is a group of code that has a single, well-defined job. Code inside a layer should only do that job. When it needs to do something outside that job, it should hand off to a different layer rather than doing the work itself.

In a Laravel application, the layers I work with look like this.

The HTTP layer handles everything related to the incoming request and the outgoing response. Controllers, middleware, form requests, and responses live here. This layer's job is to receive input, validate it, hand it to the business layer, and return a response. It does not make business decisions. It does not query the database directly.

The business layer contains the logic that makes your application do what it is supposed to do. Actions, services, and domain objects live here. This layer does not know anything about HTTP. It receives data, applies rules, and returns results. It can talk to the data layer to read or write records.

The data layer handles persistence. Models, query builders, and repository classes live here. This layer knows how to find, create, update, and delete records. It does not apply business rules. It does not know about HTTP.

Three layers. Three jobs. Each one knowing about its own responsibilities and nothing else.

Why Fat Controllers Happen

A fat controller is not a discipline problem. It is an architectural vacuum.

When a team does not have a clear shared understanding of what belongs where, logic accumulates in the most obvious place. Controllers are obvious. They are where the request arrives, so they are where people start writing code. And once code is there, more code gets added next to it because it is easier to extend an existing file than to create a new one.

Before long you have a controller that validates input, queries the database, sends emails, dispatches jobs, formats responses, and handles error cases. It has absorbed everything because nothing had a clear home.

The fix is not to refactor the controller. It is to give every kind of logic a home so that developers always know where to put new code.

Applying Layers to Clarity

Let me walk through a concrete example using Clarity's request submission feature.

A client submits a new request. Here is what needs to happen:

Validate the input (title required, description required, attachments optional)
Create the request record associated with the client's organisation
Store any uploaded attachments
Set the initial status to "submitted"
Return a response confirming the submission

Without layers, all of that logic tends to end up in the controller. Here is what that looks like, and why it is a problem:

// The kind of controller you want to avoid
public function store(Request $request): JsonResponse
{
    $validated = $request->validate([
        'title' => ['required', 'string', 'max:255'],
        'description' => ['required', 'string'],
        'attachments.*' => ['nullable', 'file', 'max:10240'],
    ]);

    $projectRequest = ProjectRequest::create([
        'organisation_id' => auth()->user()->organisation_id,
        'submitted_by' => auth()->id(),
        'title' => $validated['title'],
        'description' => $validated['description'],
        'status' => 'submitted',
    ]);

    if ($request->hasFile('attachments')) {
        foreach ($request->file('attachments') as $file) {
            $path = $file->store('attachments');
            $projectRequest->attachments()->create([
                'user_id' => auth()->id(),
                'filename' => $file->getClientOriginalName(),
                'path' => $path,
                'mime_type' => $file->getMimeType(),
            ]);
        }
    }

    return response()->json(['data' => $projectRequest->load('attachments')], 201);
}

This is not terrible code. It works. But it is doing too many jobs at once. It is validating, creating records, handling file storage, and formatting a response, all in the same method. Testing it in isolation is difficult because it is deeply coupled to the HTTP request object. Reusing any of this logic (say, if requests could also be submitted via an import job) means duplicating it or extracting it under pressure.

Now here is the same feature with layers applied:

// The Form Request handles validation
class StoreRequestRequest extends FormRequest
{
    public function rules(): array
    {
        return [
            'title' => ['required', 'string', 'max:255'],
            'description' => ['required', 'string'],
            'attachments.*' => ['nullable', 'file', 'max:10240'],
        ];
    }

    public function payload(): StoreRequestPayload
    {
        return new StoreRequestPayload(
            organisationId: auth()->user()->organisation_id,
            submittedBy: auth()->id(),
            title: $this->input('title'),
            description: $this->input('description'),
            attachments: $this->file('attachments', []),
        );
    }
}

// The Action handles the business logic
final readonly class SubmitRequest
{
    public function execute(StoreRequestPayload $payload): ProjectRequest
    {
        $projectRequest = ProjectRequest::create([
            'organisation_id' => $payload->organisationId,
            'submitted_by' => $payload->submittedBy,
            'title' => $payload->title,
            'description' => $payload->description,
            'status' => 'submitted',
        ]);

        foreach ($payload->attachments as $file) {
            $path = $file->store('attachments');
            $projectRequest->attachments()->create([
                'user_id' => $payload->submittedBy,
                'filename' => $file->getClientOriginalName(),
                'path' => $path,
                'mime_type' => $file->getMimeType(),
            ]);
        }

        return $projectRequest;
    }
}

// The Controller handles HTTP in and HTTP out
final readonly class StoreController
{
    public function __construct(
        private SubmitRequest $submitRequest,
    ) {}

    public function __invoke(StoreRequestRequest $request): JsonResponse
    {
        $projectRequest = $this->submitRequest->execute(
            payload: $request->payload(),
        );

        return response()->json(
            ['data' => $projectRequest->load('attachments')],
            201,
        );
    }
}

The controller is now four lines of meaningful logic. It receives a validated payload, calls an action, and returns a response. It knows nothing about how the request is created or how attachments are stored. Those are the action's job.

The action knows nothing about HTTP. It receives a payload object and does the work. You could call it from a console command, a queued job, or a test, and it would behave identically each time.

Where the Model Fits In

You will notice the model does not appear much in the code above beyond ProjectRequest::create() and the attachments() relationship. That is intentional.

Models in a layered application are responsible for representing a database record and its relationships. They are not responsible for business rules, validation, or application logic. A model that has methods like submitAndNotifyClient() or assignToAvailableDeveloper() is doing the action's job, and it will become harder to test and maintain as that logic grows.

Keep models focused. They know about their table, their columns, their relationships, and their casts. Everything else belongs in a layer above them.

This Is Not Dogma

I want to be clear that layers are a tool, not a religion. A simple CRUD endpoint that reads a record and returns it does not need an action class. Adding indirection where none is needed creates noise without creating value.

The question to ask is: is this logic complex enough, or reusable enough, that it deserves its own home? If the answer is yes, extract it. If the answer is no, keep it in the controller and move on.

Mid-level developers learn to make that call. Juniors often apply patterns uniformly regardless of context, which leads to over-engineering simple things. The goal is judgment, not rule-following.

Putting It Into Practice

Take a controller from a project you have worked on and look at it honestly. How many jobs is it doing? Can you identify which lines belong in the HTTP layer, which belong in the business layer, and which belong in the data layer?

You do not need to refactor it right now. Just practice the act of categorising what you see. That categorisation instinct is what you are building, and it comes from practice more than from reading.

In the next article, we are going to bring everything together and look at how to turn your ERD and architecture diagram into a sequenced build plan that a developer can actually follow.

Introduction To Systems Architecture

Steve McDougall — Tue, 19 May 2026 12:00:30 +0000

We have a clear problem definition, a set of shaped features, and a data model we trust. At this point, a lot of developers would consider the design work done and start writing code. And honestly, for a project the size of Clarity, you probably could. The application is small enough that its structure is mostly implied by the framework.

But this series is about building the habits that carry you through larger, more complex projects. And on those projects, skipping the architecture diagram is where things start to quietly go wrong.

In this article I want to introduce you to what architecture actually means for application developers, and show you how to draw a simple system diagram for Clarity that captures how the pieces fit together.

What Architecture Means at This Level

When most developers hear "architecture", they think about large distributed systems, microservices, cloud infrastructure diagrams with dozens of boxes and arrows going everywhere. That is one kind of architecture, and it is not what we are talking about here.

For application developers, architecture is about a simpler set of questions. What are the major components of this system? How does data flow between them? What are the boundaries between different concerns? Where do external dependencies plug in?

You do not need to be designing a distributed system to benefit from answering those questions. Even a straightforward Laravel application has meaningful architecture: a web layer that handles HTTP, a business logic layer that does the actual work, a data layer that talks to the database, and often a set of external integrations sitting alongside all of that. Understanding where those layers are and how they connect is what lets you make good decisions about where code should live.

Components vs Layers

There are two ways to think about application structure, and both are useful in different contexts.

Layers describe a vertical slice through your application. Think of the classic presentation layer, business logic layer, data access layer split. Data comes in at the top, passes through the layers, and comes out as a response. Each layer only talks to the layer directly below it. This is a useful mental model for thinking about separation of concerns inside a single application.

Components describe horizontal groupings of related functionality. Authentication is a component. The request management system is a component. Notifications are a component. A component might span multiple layers internally, but it has a clear boundary and a clear responsibility.

In practice, most applications use both mental models at once. The layers tell you how code flows inside a component. The components tell you how the system is divided at a higher level.

Drawing a System Diagram

A system diagram does not need to be elaborate. Its job is to show the major components of your system, how they connect, and where external dependencies sit. You are not trying to document every class or every database query. You are drawing a map that helps you and your team understand the shape of what you are building.

Here is a Mermaid diagram for Clarity:

flowchart TD
    Browser["Browser / Client"]

    subgraph App ["Clarity Application"]
        Web["Web Layer\n(Routes, Controllers, Middleware)"]
        Auth["Auth Component\n(Login, Invitation, Sessions)"]
        Requests["Request Management\n(Submit, Status, Assignment)"]
        Comments["Comments\n(Post, Internal Flag)"]
        Attachments["Attachments\n(Upload, Storage)"]
        Notifications["Notifications\n(Status Changes, Comments)"]
    end

    subgraph Data ["Data Layer"]
        DB[("PostgreSQL")]
        Storage["File Storage"]
        Cache["Cache"]
    end

    Browser --> Web
    Web --> Auth
    Web --> Requests
    Web --> Comments
    Web --> Attachments
    Requests --> Notifications
    Comments --> Notifications
    Auth --> DB
    Requests --> DB
    Comments --> DB
    Attachments --> DB
    Attachments --> Storage
    Auth --> Cache

This is not a detailed technical specification. It is a conversation starter. It shows that Clarity has a web layer handling incoming requests, five functional components sitting behind it, a database and file storage for persistence, a cache layer, and a notifications pathway that gets triggered by changes in the request and comment components.

What the Diagram Reveals

Even a simple diagram like this surfaces useful questions.

The notifications component appears as a box but is currently undefined. What does it actually do? Does it send emails? Does it push in-app notifications? Does it use a queue? That is a decision we have deferred, and the diagram makes that deferral visible. For the initial version of Clarity, we scoped email notifications on comments out of the first cycle. The component box is still there, which means when we come back to build it, we have a clear place for it to sit.

The file storage box is separate from the database. That is intentional. Attachments are stored on disk or in an object store, not as blobs in the database. The diagram captures that decision explicitly. A developer joining the project later can see at a glance that attachments have their own storage concern, rather than discovering it buried in a model.

The cache layer connects only to the auth component in this version. That is a starting point, not a permanent constraint. As Clarity grows and request list queries become more expensive, caching will expand to cover other components. Having it on the diagram from the start means that growth is natural rather than bolted on.

Architecture as a Communication Tool

One of the things I value most about architecture diagrams is that they create a shared vocabulary for a team. When everyone can point at the same diagram and say "the request management component" or "the web layer", discussions become sharper. You spend less time talking past each other and more time talking about the actual problem.

That shared vocabulary is especially valuable when something goes wrong. If a bug is affecting comments but not requests, a team with a clear component diagram can focus their investigation quickly. If performance is degrading under load, having the data flow documented means you can reason about where the bottleneck is likely to be.

This is one of the reasons mid-level developers tend to be better debugging partners than juniors, even when the junior has been on the codebase longer. It is not that they know more code. It is that they think in components and flows rather than individual files and functions. The architecture diagram is how you start building that mental model.

Clarity's Architecture in Plain Language

Let me describe the Clarity architecture in prose, the way you might explain it to a new team member.

Clarity is a single Laravel application served over HTTP. All incoming requests pass through the web layer, which handles routing, authentication middleware, and request validation. Behind the web layer, the application is divided into four main functional areas: request management, comments, attachments, and authentication.

Request management is the core of the application. It covers the full lifecycle of a client request from submission through completion, including status transitions and developer assignment. Comments sit alongside request management and share the same request context, with the addition of the internal flag that controls client visibility. Attachments handle file uploads and connect to external file storage rather than the database.

Authentication covers client login via password, team member login, and the invitation flow. Session state is cached to reduce database load on authenticated requests.

Notifications are triggered by events in the request and comment components. In the first version of Clarity, notifications are out of scope. The component is acknowledged in the architecture but not built yet.

Data persistence uses PostgreSQL for relational data and a file storage service for attachments. The application does not use a search index or a message queue in the initial version.

Putting It Into Practice

Draw a system diagram for your own project using the same approach. Start with the major components you can identify, then add the data layer and any external dependencies. Draw arrows to show how data flows between them.

Then write a plain-language description of the architecture the way I did above. If you struggle to write it in a few clear paragraphs, the diagram probably needs more work. The prose and the diagram should tell the same story.

In the next article, we are going to look at separation of concerns in more detail, understand why the "fat controller" problem is really an architectural symptom, and start thinking about what belongs in each layer of a Laravel application.

Relationships, Cardinality, and the Questions Your Schema Is Asking

Steve McDougall — Tue, 19 May 2026 11:59:58 +0000

In the last article we drew the Clarity ERD and ended up with a clean diagram covering five entities and seven relationships. If you followed along and drew one for your own project, you probably found a few things that surprised you. That is normal, and it is exactly the point.

Now I want to go deeper. Drawing an ERD is one skill. Reading what it is telling you is another, and the second one is where the real value lives.

In this article we are going to look at the three relationship types in detail, walk through the most common mistakes I see developers make when modelling data, and show you how to catch them at the diagram stage before they become painful migrations.

The Three Relationships, Properly

We covered these briefly in the last article, but they deserve more attention.

One-to-One

A one-to-one relationship means one record in table A maps to exactly one record in table B, and that mapping is unique on both sides.

These are less common than people expect. When a developer reaches for a one-to-one, I always ask: why is this data in a separate table? Sometimes there is a good reason. You might have a users table and a user_profiles table where the profile data is large, infrequently accessed, and cleaner to separate. Or you might have a base users table that is extended by either a client_profiles table or a team_member_profiles table depending on role.

But a lot of the time, a one-to-one relationship is a sign that the data should just live in the same table. Before you commit to one, ask yourself whether there is a genuine reason for the separation or whether you are adding complexity without adding value.

In Clarity, we do not have any one-to-one relationships. The model is clean enough that everything fits in its own entity without needing auxiliary tables.

One-to-Many

This is the workhorse of relational data modelling. One record in table A is referenced by many records in table B. The foreign key lives on the "many" side.

In Clarity: one organisation has many users, so organisation_id lives on the users table. One request has many comments, so request_id lives on the comments table. The pattern is consistent.

The mistake I see most often with one-to-many is putting the foreign key on the wrong side. If you put user_ids (as some kind of serialised array) on the organisations table instead of organisation_id on the users table, you have inverted the relationship and created a mess. You cannot join efficiently, you cannot enforce referential integrity, and you will hit a wall the moment you need to query it properly.

The rule is simple: the foreign key always goes on the child. The "many" side. If you are unsure which side is the child, ask yourself which record is owned by the other. A user is owned by an organisation. A comment is owned by a request. The owned record holds the foreign key.

Many-to-Many

A many-to-many relationship means records on both sides can be connected to multiple records on the other side. You cannot model this directly with a foreign key on either table. You need a pivot table that sits between them and holds the connection.

Clarity does not have a many-to-many in the current design, so let me add one to illustrate the point. Suppose we wanted to add tagging to requests, where a request can have many tags and a tag can be applied to many requests. That is a classic many-to-many.

The entities involved:

Request {
    uuid id
    string title
    ...
}

Tag {
    uuid id
    string name
    string colour
}

RequestTag {
    uuid request_id
    uuid tag_id
}

The RequestTag pivot table is the relationship. It holds nothing but the two foreign keys (and sometimes additional data about the relationship itself, like a created_at timestamp or a user who applied the tag).

In Laravel, this maps to a belongsToMany relationship on both models:

// Request model
public function tags(): BelongsToMany
{
    return $this->belongsToMany(Tag::class, 'request_tags');
}

// Tag model
public function requests(): BelongsToMany
{
    return $this->belongsToMany(Request::class, 'request_tags');
}

The common mistake here is not recognising when you have a many-to-many. Developers often start by modelling it as a one-to-many and then discover mid-build that the constraint they assumed does not hold. When you spot a noun in your application that sounds like it could belong to many different records of another type, treat it as a signal to check whether you are looking at a pivot.

Common ERD Mistakes

Let me walk through the mistakes I see most often and how to spot them before they cause problems.

Storing arrays in a column

If you ever find yourself writing tags: string or assigned_to: json on an entity, stop. That is almost always a sign that the data belongs in a related table. Arrays in columns cannot be indexed properly, cannot be joined against, and are a maintenance headache the moment the data inside them grows or needs to be queried individually.

In the Clarity ERD, assigned_to is a single foreign key pointing at one user. That is correct because our business rule says a request has one assignee. But if the requirement had been "a request can be assigned to multiple developers", the right answer would be an assignments pivot table, not a JSON array of user IDs in a column.

The missing pivot

Related to the above: when you draw a many-to-many relationship between two entities without drawing the pivot table, you have an incomplete diagram. Always draw the pivot explicitly. It forces you to think about what data the relationship itself carries, and it makes the migration obvious when you get to that stage.

Nullable foreign keys as a shortcut

It is tempting to make a foreign key nullable when you are not sure whether a relationship will always exist. Sometimes that is correct. In Clarity, assigned_to on Request is nullable because a request might not have an assignee yet. That is a real business rule.

But sometimes a nullable foreign key is a sign that the relationship is modelled incorrectly. If you find yourself with a record that should always belong to a parent but the foreign key keeps being null in practice, the relationship direction might be wrong, or the data might belong in a different table entirely.

Conflating user roles with user types

This one is specific to applications like Clarity, but it comes up constantly. When your system has multiple types of users (clients and team members, customers and staff, students and teachers), there is a temptation to use a single role column and call it done.

Sometimes that is fine. But if the two user types have genuinely different data requirements, different relationships to other entities, or different constraints on what they can do, a single role column will not hold the weight. You end up with columns that are only relevant to one type of user, nullable columns everywhere, and logic scattered across your codebase to handle the differences.

In the Clarity design I chose a single users table with a role column and a nullable sub_role. That works for our use case because the data requirements for clients and team members are similar enough. But it is a decision worth examining on your own projects. If the profiles for your two user types are genuinely different shapes, a polymorphic users table or a base users table with separate profile tables might serve you better.

Revisiting the Clarity ERD

Now that we have covered these in detail, let me take one more look at the Clarity diagram and point out the decisions that are worth noting.

The Request entity has two foreign keys pointing at User: submitted_by and assigned_to. This is intentional, but it means the Eloquent relationships on Request need to be named explicitly to avoid ambiguity:

public function submitter(): BelongsTo
{
    return $this->belongsTo(User::class, 'submitted_by');
}

public function assignee(): BelongsTo
{
    return $this->belongsTo(User::class, 'assigned_to');
}

If you wrote $this->belongsTo(User::class) without the second argument, Laravel would look for a user_id column that does not exist. Naming the foreign key explicitly in the relationship definition is the correct approach here, and the ERD is what made this obvious before we wrote a single line of code.

The Comment entity has is_internal as a boolean. This is a simple field, but it carries real access control logic: when is_internal is true, the comment must be excluded from any query that is serving data to a client user. That is an application-layer concern rather than a schema concern, but spotting it on the diagram is a reminder to handle it consistently across every query that touches comments.

Putting It Into Practice

Go back to the ERD you drew in the last exercise. Now look at every relationship line and ask these questions for each one:

Is the foreign key on the correct side?
Is this truly a one-to-many, or could it become a many-to-many as the application grows?
Are there any arrays or JSON fields that should be a related table?
Are there any nullable foreign keys that feel like shortcuts rather than genuine business rules?

Fix what you find. A diagram is cheap to change. A migration is not.

In the next article, we are going to step back from the data model and look at the bigger picture: what application architecture actually means for backend developers, and how to draw a simple system diagram that shows how the pieces of Clarity fit together.

Your First ERD

Steve McDougall — Tue, 19 May 2026 11:59:56 +0000

Up until now, everything we have done has been about understanding the problem. We interrogated the brief, surfaced the assumptions, wrote user stories, and shaped the features into bounded pieces of work. That is a lot of thinking before touching anything technical, and it is exactly the right order to do things in.

Now we get to use that thinking. Because once you know what your application needs to do, the next question is: what does it need to remember?

That question is what an Entity Relationship Diagram is designed to answer.

What an ERD Actually Is

An ERD is a visual map of the data your application works with. It shows you the things your system needs to store (entities), the properties those things have (attributes), and the connections between them (relationships).

Before you write a single migration, before you think about table names or column types, drawing an ERD gives you a bird's-eye view of your entire data model. It lets you spot problems, ask questions, and make decisions on paper, where changing your mind costs nothing.

I have seen developers skip this step and pay for it later. A misunderstood relationship between two entities can mean a painful migration weeks into a build, or worse, a data model that quietly produces wrong results and nobody notices until a client reports it. Fifteen minutes with a pencil and a blank page is a much cheaper way to find those problems.

The Building Blocks

Every ERD is made of three things.

Entities are the nouns in your system. The things your application tracks and stores. In Clarity, the entities we can already see from our user stories are: organisations, users, requests, comments, and attachments.

Attributes are the properties of each entity. A user has a name, an email address, and a role. A request has a title, a description, and a status. Attributes map directly to the columns you will eventually write in your migrations.

Relationships are the connections between entities. An organisation has many users. A user can submit many requests. A request can have many comments. These connections are what give your data model its shape, and getting them right is the most important part of the exercise.

Cardinality

When you draw a relationship between two entities, you need to describe how many of one thing can be connected to how many of another. This is called cardinality, and there are three basic types.

One-to-one. One record in table A corresponds to exactly one record in table B. These are relatively rare. An example might be a user having one profile, where the profile data lives in a separate table.

One-to-many. One record in table A corresponds to many records in table B. This is the most common relationship in most applications. One organisation has many users. One user has many requests. One request has many comments.

Many-to-many. Many records in table A can be connected to many records in table B. This always requires a pivot table. An example in Clarity might be if we allowed a request to be tagged, where a request can have many tags and a tag can belong to many requests.

Getting cardinality wrong is the most expensive ERD mistake. If you model a one-to-many relationship as a one-to-one, you will hit a wall the moment a second record tries to attach itself somewhere it cannot go. Drawing it out first makes these mistakes obvious before they are baked into your schema.

Drawing the Clarity ERD

Let me walk through how I would build the Clarity ERD from our user stories and shaped features.

I start by listing every entity I can see:

Organisation
User
Request
Comment
Attachment

Then for each entity, I list its attributes. I am not worrying about data types yet, just the fields that need to exist.

Organisation: id, name, created_at

User: id, organisation_id, name, email, password, role (client or team_member), sub_role (developer or admin, nullable), invited_at, created_at

Request: id, organisation_id, submitted_by (user_id), assigned_to (user_id, nullable), title, description, status, created_at, updated_at

Comment: id, request_id, user_id, body, is_internal, created_at

Attachment: id, request_id, user_id, filename, path, mime_type, created_at

Now the relationships:

An Organisation has many Users
An Organisation has many Requests
A User (client) submits many Requests
A User (developer) is assigned to many Requests
A Request has many Comments
A Request has many Attachments
A User posts many Comments

In Mermaid syntax, which I would recommend learning because it lets you write diagrams as code and store them in version control alongside your project, this looks like:

erDiagram
    Organisation {
        uuid id
        string name
        timestamp created_at
    }

    User {
        uuid id
        uuid organisation_id
        string name
        string email
        string role
        string sub_role
        timestamp invited_at
        timestamp created_at
    }

    Request {
        uuid id
        uuid organisation_id
        uuid submitted_by
        uuid assigned_to
        string title
        text description
        string status
        timestamp created_at
        timestamp updated_at
    }

    Comment {
        uuid id
        uuid request_id
        uuid user_id
        text body
        boolean is_internal
        timestamp created_at
    }

    Attachment {
        uuid id
        uuid request_id
        uuid user_id
        string filename
        string path
        string mime_type
        timestamp created_at
    }

    Organisation ||--o{ User : "has many"
    Organisation ||--o{ Request : "has many"
    User ||--o{ Request : "submits"
    User ||--o{ Comment : "posts"
    Request ||--o{ Comment : "has many"
    Request ||--o{ Attachment : "has many"

Render that in any Mermaid viewer and you get a complete picture of the Clarity data model. Every entity, every attribute, every relationship, on a single page.

Reading the Diagram

The notation on the relationship lines is worth understanding. In Mermaid ERDs, each side of a relationship is described with two symbols: one for the minimum cardinality and one for the maximum.

|| means exactly one. o{ means zero or more. So ||--o{ reads as: exactly one on the left, zero or more on the right.

Put together, Organisation ||--o{ User reads as: one organisation has zero or more users. An organisation can exist with no users yet (useful when you first create it), but every user belongs to exactly one organisation.

What the ERD Is Telling You

The real value of drawing this out is what you notice when you step back and look at it.

I can already see a potential question in the Clarity diagram. The Request entity has two foreign keys pointing at User: submitted_by and assigned_to. That is fine and intentional, but it means when you build the Eloquent relationships on the Request model, you will need named relationships rather than a single belongsTo. Something like submitter() and assignee() rather than just user(). That is a small thing, but it is much better to notice it here than halfway through writing a controller.

I can also see that Comment and Attachment both have a user_id. That means both clients and team members can post comments and add attachments, which matches our user stories. If the stories said only team members could add attachments, that user_id column would need a constraint we would need to enforce at the application layer. The diagram surfaces that decision.

This is what ERDs are actually for. Not documentation. Not formality. They are a thinking tool that forces you to look at your data model as a whole before you start building pieces of it in isolation.

Putting It Into Practice

Take the project from the previous exercises and draw an ERD for it. Start with just the entities and relationships, then add attributes once the structure feels right.

Use Mermaid if you want to keep it in code, or draw.io if you prefer a visual tool. The format matters less than the act of drawing it out and looking at what it tells you.

Pay particular attention to your foreign keys. For every relationship line you draw, ask yourself: does this direction make sense? What happens if the parent record is deleted? Is there anything here that I assumed was a one-to-many that might actually be a many-to-many?

In the next article, we are going to go deeper on relationships and cardinality, look at some of the most common ERD mistakes, and talk about how to catch the wrong relationship on paper before it becomes the wrong migration in production.

Shaping Before You Build

Steve McDougall — Tue, 19 May 2026 11:59:24 +0000

We now have a solid set of user stories for Clarity. We know who the actors are, what they need to do, and what done looks like for each feature. That is real progress. But there is a gap between a well-written user story and something a developer can confidently pick up and build within a reasonable timeframe.

That gap is what shaping is designed to close.

Shaping is a concept from the Shape Up methodology, written by Ryan Singer at Basecamp. If you have not read it, I would strongly recommend it. It is free at basecamp.com/shapeup, and it is one of the most practical pieces of writing on software product development I have come across. This article is not a replacement for it, but it will introduce you to the core ideas and show you how to apply them to Clarity.

The Problem Shaping Solves

User stories are great for describing what a user needs. They are not great at describing how much work that need represents, or how to build it in a way that is actually feasible within a given timeframe.

Without shaping, a story like "as a client, I want to add a comment to my request" can balloon into a real-time threaded messaging system with emoji reactions and read receipts, or it can be a simple text box that posts to a list. Both satisfy the story. One takes two days to build, the other takes two months.

Shaping forces you to make that decision deliberately, before anyone writes a line of code. It gives features a specific form: a defined scope, a considered approach, and an explicit boundary around what is included and what is not.

The Three Elements of a Shaped Feature

Every shaped feature has three components.

The first is appetite. This is the amount of time you are willing to spend on the feature, decided upfront. Not an estimate of how long it will take, but a deliberate decision about how much it is worth. This is one of the most powerful ideas in Shape Up, and it inverts the normal way teams think about time. Instead of estimating a feature and then planning around that estimate, you decide how much time a feature deserves and then shape the work to fit inside that boundary.

The second is the solution. This is a rough sketch of how the feature will work, at a level of detail that is enough to build from without being so prescriptive that it removes all creative problem-solving from the developer. Shape Up calls these "fat marker sketches" because the point is to communicate structure and flow, not pixel-perfect detail.

The third is the boundaries. This is an explicit list of what is not included. Boundaries are as important as the solution itself, because they are what prevent scope from quietly expanding during the build. If you do not write down what is out of scope, someone will assume it is in scope.

Writing a Pitch

The output of shaping is called a pitch. A pitch is a short document that captures all three elements and makes the case for why the feature is worth building in this cycle.

A pitch is not a specification. It does not tell developers exactly how to implement something. It tells them what problem they are solving, roughly how it should work, how much time they have, and what the edges of the work are.

Let me write a pitch for one of the Clarity features.

Pitch: Request Comments

Appetite: Two days

Problem: Clients and team members currently have no way to communicate directly on a specific request within Clarity. Context gets lost in email threads, and there is no record of decisions or questions attached to the work itself.

Solution: A simple comment thread on each request detail page. Any user with access to the request can post a comment. Team members can mark a comment as internal, which hides it from clients. Comments appear in chronological order. No threading, no reactions, no editing after posting in the first version.

Rough sketch:

Request detail page
---------------------
[ Request title ]
[ Status badge ] [ Assigned to: Sarah ]
[ Description ]

Comments
---------
[ Client User ] 14 May
Can you confirm the deadline for this?

[ Developer ] 15 May  [internal]
Need to check with the PM before responding.

[ Text area: Add a comment... ]
[ Internal? checkbox ] [ Post comment ]

Out of scope:

Threaded replies
Editing or deleting comments
Emoji reactions or mentions
Email notifications on new comments (deferred to a later cycle)
File attachments on comments

Look at what that pitch gives you. A developer picking this up knows exactly what they are building, roughly how it should look and behave, how long they have to build it, and where the edges of the work are. They are not going to spend a day building a notification system because they assumed it was included. They are not going to build a threaded reply UI because no one said not to.

That clarity (again, no pun intended) is what makes teams fast. Not velocity metrics, not sprint ceremonies. Clear, bounded work.

Appetite as a Design Tool

I want to spend a moment on appetite specifically, because it is the idea most developers initially resist.

The instinct when you hear "two days for a comment system" is to think about all the things a comment system could be and conclude that two days is not enough. But that framing is backwards. The appetite is not a constraint on what you could build. It is a statement about what the problem is worth solving right now.

A comment thread that works and ships in two days is more valuable than a fully-featured messaging system that takes six weeks and delays everything else. You can always come back and add threading, notifications, and mentions in a later cycle if the basic version proves its value. What you cannot do is un-spend six weeks.

This is one of the most important mental shifts in moving from junior to mid-level. Juniors often think about features in terms of what they could be. Mid-level developers think about features in terms of what they need to be, right now, to solve the problem in front of them.

Shaping the Clarity Features

Let me apply appetites to the full set of Clarity user stories, so we have a shaped backlog to build from as the series continues.

Client authentication and onboarding
Appetite: three days. Clients are invited by an admin, set a password via an invitation link, and log in. No social auth, no self-registration.

Submitting a request
Appetite: two days. Title, description, and optional file attachments. Status set to "submitted" on creation.

Request status and assignment
Appetite: two days. Team members can update status and assign a developer. Status and assignee are visible to clients. No real-time updates in the first version; a page refresh is acceptable.

Comment thread
Appetite: two days. As pitched above.

Team member management
Appetite: one day. Admins can invite team members by email, set their role, and deactivate their account. Invitations expire after 48 hours.

Request list views
Appetite: one day. Clients see a list of their own requests. Team members see all requests with basic filtering by status.

That is the full Clarity application shaped into six discrete pieces of work, with a total appetite of twelve days. That is not an estimate of how long it will take. It is a decision about how much time the initial version of Clarity is worth. If any piece of work is running over its appetite, the right question is not "how do we go faster?" but "what can we remove from this piece to fit inside the time we agreed it was worth?"

Putting It Into Practice

Take one of the user stories you wrote in the last exercise and write a pitch for it. Define an appetite, sketch a rough solution, and write out the out of scope list.

The out of scope list is the most important part. Be honest about what you are tempted to include, and then deliberately leave it out.

In the next article, we are going to move from features into data, and look at how to draw your first Entity Relationship Diagram before you write a single migration.