DEV Community: Qasim Parray

How to Handle Scope Creep: The Quick One That Cost Me a Week

Qasim Parray — Sat, 12 Sep 2026 02:39:56 +0000

Confession: the worst scope creep I ever dealt with was my own fault, and it started with the words "sure, that's a quick one."

It was a Laravel admin panel for a logistics company. Fixed price, six weeks, clear spec. Around week three the client asked if the export button could also email the CSV. Quick one. Then could the email go to a list of addresses. Quick one. Then could the list be managed in the panel. Then could those users get their own login with read-only access. By week eight I had built a small user management system, a notification layer, and a permissions model, none of which were in the contract, and I was the one who'd said yes every time.

I've been freelancing and running small projects for long enough now that I've stopped thinking of scope creep as something clients do to you. It's something that happens in the gap between what was written down and what everyone assumed. This post is about how I handle that gap now: what I put in contracts, what I say in the moment, and the one habit that fixed most of it.

Scope creep is mostly a definition problem

The standard definition, the one you'll find in the Atlassian guide to scope creep and every project management textbook, is uncontrolled growth in a project's requirements after the project has started. That's accurate and almost useless, because it makes it sound like a thing with a clear start line.

In practice the growth is never uncontrolled from the client's point of view. Each request is small, reasonable, and obviously related to the thing you're already building. The export button emailing the CSV is a two-hour job. The client isn't trying to get free work. They're seeing the software for the first time and having ideas, which is what you'd want from an engaged client.

The problem is that "related to the thing you're building" and "in the thing you agreed to build" are different sets, and if the contract doesn't draw the line, the line gets drawn by whoever is more uncomfortable saying no. That's usually the developer.

So the first fix isn't a script for pushing back. It's writing the scope down in a way that makes the line visible to both of you before anyone has to be the bad guy.

What goes in my scope document now

Every fixed-price project I take gets a scope section that has two lists, and the second one matters more than the first.

The first list is what's included. Screens, features, integrations, whatever the unit of work is for that project. Pretty standard.

The second list is what's explicitly not included, and it's specific. Not "additional features are out of scope", which is legally fine and practically meaningless. It says things like: user roles beyond admin and viewer, email notifications of any kind, data import from the old system, mobile layouts for the admin panel. I write this list by imagining the project is done and asking what the client will ask for in the first week. I'm usually right about half of it.

The point of the second list isn't to refuse those things. It's to make the conversation, when it comes, sound like "that's on the not-included list, so let's price it" instead of "well, I didn't think that was included." The first sentence is a fact both parties signed. The second is an argument.

I also add one sentence that has done more work for me than any other: changes to scope are welcome and will be quoted separately before work begins. Welcome is the important word. It tells the client that asking is fine, and that asking has a process.

The five-minute rule

Here's the habit that fixed most of it. When a request comes in mid-project, I do not answer yes or no in the same message. I answer with an estimate.

"Emailing the CSV: that's about two hours, so roughly 250. Want me to add it to the change list?"

This does several things at once. It makes the cost visible, which is the entire problem with small requests: the client can't see the cost, so to them it's free. It gives them a real decision to make rather than a favour to accept. And it keeps my tone neutral, because I'm not saying no, I'm saying here's the price.

About a third of requests disappear at this point. The client sees 250 and decides emailing the CSV can wait. Another third get a yes and go on the change list. The remaining third turn into a conversation, which is where the actual project management happens.

What I don't do anymore is absorb the two hours to be nice. I did that on the logistics project maybe fifteen times, and fifteen two-hour favours is nearly a week of unpaid work. The client didn't know they'd received a week of free work, because I never told them. That's not generosity. That's just bad bookkeeping with extra resentment.

When the request is actually a spec bug

Sometimes the request isn't creep at all. Sometimes it's the client noticing that the spec was wrong, and if you'd noticed it during scoping you'd have included it.

The logistics panel had a reporting screen. The spec said "monthly report of deliveries by region". When I built it, the client said, reasonably, that they also needed it by driver, because that's how they actually ran their operation. Nothing in the spec said driver. But the report as specified was useless to them, and a useless report was never what either of us intended.

I treat these differently. If the spec was ambiguous and the client's reading is reasonable, I eat it. If the spec was clear and the client wants something different, it's a change. The test I use is: would a competent third party reading the spec have built what I built? If yes, the client is asking for a change. If they'd have asked a clarifying question I failed to ask, that's on me.

I still tell the client which bucket it landed in, either way. "This one's on me, the spec should have caught it" costs nothing and buys a lot of goodwill for the next time I quote a change. This is also where I've learned to be honest about my own mistakes rather than quietly fixing them; I wrote about a much more expensive version of that lesson in the post about the refund bot that cost me a client.

Retainers make this easier, projects make it harder

One structural note. Almost all of my scope creep pain has come from fixed-price projects. On retainers, where the client buys a block of hours per month, scope creep mostly stops being a concept. A new request is just a new item in the queue, and the question is what it displaces, not whether it's free.

That doesn't mean retainers are always right. Clients with a defined one-off build want a fixed price, and I'd want one too in their position. But when a project turns out to be the first of many, I now suggest moving to a retainer early, sometimes before the first project is finished. The client gets a predictable bill, I get a predictable month, and the "is this in scope" conversation is replaced with "where does this sit in the priority list", which is a much better conversation to be having.

If a client resists a retainer but keeps asking for changes, that's information. The Asana article on scope creep lists unclear priorities as a leading cause, and in my experience the client who wants both a fixed price and an open feature list hasn't decided what the project is yet. Better to find that out in week two than week eight.

What to change this week

If you've got a project in flight right now, do one thing: open a shared document called "Change list", put a table in it with columns for request, estimate, status, and date, and send the link to your client with a one-line note that says you'll log any new requests there so nothing gets lost. Then, the next time a request arrives, put it in the table with an estimate before you do anything else.

That table has turned more awkward conversations into boring ones than any contract clause I've written. The client can see what they've asked for, what it costs, and what they've approved. You can see how much unpaid work you were about to do. Nobody has to be the bad guy, because the table is the bad guy, and it's very polite about it.

If you're setting up your first few contracts and want to see how I structure the scope section on real projects, the work page on my site has a few case studies where I've been fairly open about what was in and out of scope, and what it cost when I got that wrong.

Originally published at abrarqasim.com. I write there about React, PHP, Rust, Go and the AI tooling around them.

ARC-AGI-3 and GPT-6 Astra: Two Scores, One Model, and the Harness Gap

Qasim Parray — Fri, 11 Sep 2026 22:39:54 +0000

Short version for the impatient: GPT-6 Astra scored 62.7% on ARC-AGI-3 and also 99.9% on ARC-AGI-3, on the same day, on the same test set, and both numbers are real. The difference is the harness. If you only read one headline this week, make it that one, because it's the most useful thing anyone has said about benchmarks in a long time.

I spent Thursday evening reading the ARC Prize write-up on Astra instead of the thing I was supposed to be doing, which was fixing a flaky test in a client's checkout flow. The test is still flaky. But I came away with a clearer picture of why the agent evals I run for my own projects keep giving me numbers I don't trust, and it's the same reason Astra has two scores.

This isn't a "is it AGI" post. ARC Prize themselves say it isn't, and I have no interest in relitigating that. It's a post about what the 37-point gap between those two scores means for anyone who ships agents, which at this point is most of us.

What ARC-AGI-3 actually tests

Quick background, because ARC-AGI-3 is only six months old and the earlier versions were different animals. ARC-AGI-1 and 2 were grid puzzles: here's a few input-output pairs, infer the rule, apply it. Static. One shot.

ARC-AGI-3 launched in March 2026 as an interactive benchmark. Each task is a small turn-based game with no instructions, no stated goal, and no rules explained. The agent has to poke at the environment, work out what the mechanics are, figure out what "winning" even means, and then do it, across levels that get harder. ARC Prize built hundreds of these and calibrated difficulty with about 500 members of the public, who solved all of them. At launch, every frontier model scored under 1%.

The thing I find interesting about the design is that it scores action efficiency on top of completion. Finishing a level in 400 moves when the median human took 40 counts against you. That's a much closer match to how I'd judge an agent doing real work. An agent that eventually fixes the bug after 300 tool calls is not the same as one that fixes it in twelve.

The two harnesses, in plain terms

Here's the part that matters. ARC Prize ran Astra two ways.

The Standard harness is their provider-neutral interface. Every model gets the same thing: the game state, the action space, and the ability to carry forward whatever notes it chooses to write for itself between turns. What it keeps is up to the model. What it doesn't write down, it loses.

The Provider Adapter harness lets the model use the context-management features its provider built for it. For Astra, that means the opaque reasoning state (the stuff OpenAI doesn't show you) persists between requests, and long conversations get compacted rather than truncated. The model gets to reuse its own prior thinking instead of reconstructing it from notes.

Standard: 62.7% at max reasoning effort, for about $26,000. Provider Adapter: 99.9% at high effort, for about $19,000. Faster too. ARC Prize reports the adapter runs were roughly 3.66 times quicker by elapsed time and used 49% fewer tokens across the 167 game-and-reasoning-level pairs both harnesses solved.

Read that again. Cheaper, faster, and 37 points better. Same model weights. The only thing that changed is whether the model was allowed to remember how it had been thinking.

Why the gap is the story, not the 99.9

I've seen a few takes this week along the lines of "the 99.9 is fake, the real score is 62.7". I don't think that's right, and ARC Prize don't frame it that way either. They're going to report both on the leaderboard, labelled, going forward. The Standard number answers "how do models compare on an identical minimal interface". The adapter number answers "how well does this model do when it's used the way its provider intends".

Both are legitimate questions. But if you're building something, the second one is the one you're living with. Nobody ships an agent on a deliberately handicapped interface to keep the comparison fair.

What the gap tells you is how much of an agent's performance lives outside the weights. In Astra's case, more than a third of the score. That lines up uncomfortably well with something I noticed on my own projects and wrote up in my post on agent memory being a dose, not a switch: the same model, given a slightly different memory policy, behaved like a different product. I'd assumed that was me being sloppy with prompts. Turns out the best-resourced eval team in the field sees the same thing at a scale of 37 points.

The Standard-harness results across reasoning levels are also worth a look. Astra at "none" (no reasoning) scored 35.2%. At "low" it scored 17.5%. Lower. Some reasoning did worse than no reasoning on this task under the Standard harness, and I don't have a tidy explanation for that. ARC Prize don't offer one either. Under the adapter harness, every level from none to max lands between 96.7% and 99.9%. The harness raised the ceiling and flattened the whole curve at the same time.

What Astra was actually doing in there

The bit of the write-up I keep coming back to isn't the numbers. It's the replays.

Under the Standard harness, Astra invented its own shorthand for tracking game state. Not English notes. Things like L8: hub q2 (8↓). Lengths: 14=1, 9=1 to record a level, a rotation index, and mechanism lengths, and extend8 to3; retract10 to2; shorten8 to1 as an ordered plan. ARC Prize call it an on-the-fly algebraic notation. It's a model compressing its own world model into the smallest thing that survives the note-passing boundary, because the harness forces it to.

That's the same thing a person does when the whiteboard is too small. It's also, I think, why the Standard score is as high as it is. The model built a workaround for the missing memory. The adapter harness just removed the need for the workaround.

In a third setup, the PRO-LONG harness where the model gets a code sandbox, Astra wrote itself small libraries per game: maze_solver.py, patrol_solver.py, a sync_state.py to check predictions against what it observed. ARC Prize are careful to say that's model-plus-tools and not comparable to the human baseline. Fair. But it's also exactly what you'd want an agent to do when handed a codebase it's never seen.

The part that should worry you about your own evals

Here's my uncomfortable conclusion. If a 37-point swing can come from the harness alone on a benchmark this carefully built, then the numbers coming out of my own agent evals, run on a harness I threw together in an afternoon, are measuring my harness at least as much as they're measuring the model.

I made this point from a different angle in the post about LLM evaluation tools missing the long game. Most eval tooling scores single turns. Agents don't live in single turns. The thing that made Astra go from 62.7 to 99.9 is a multi-turn concern: what persists, what gets compacted, what the model is allowed to reuse. If your eval harness resets context per task and your production agent doesn't, you're not testing what you ship.

The fix isn't complicated, it's just work. Run your evals through the exact context-management path your production agent uses. Same compaction, same memory, same tool wiring. If the provider offers a native session or reasoning-state feature, test with it on and off, and record both, the way ARC Prize now does. If the two numbers are close, great, your harness isn't the variable. If they're 30 points apart, you've just learned where your engineering time should go, and it isn't prompt tweaking.

Simon Willison's note on the Astra launch makes the related point that Astra's headline benchmark wins arrived alongside Artificial Analysis still ranking it below Fable 5.1 on their intelligence index. Different harnesses, different questions, different winners. That's not a contradiction. That's what measuring agents looks like now.

What I'm doing about it this week

Two things, and you can steal both.

First, I'm adding a "harness" column to my eval results table. Every run records which context policy it used. I've been treating that as an implementation detail. It's a variable, and it's a bigger one than the model version in some cases.

Second, I'm re-running my agent evals with provider-native session state turned on, for the two providers I use that support it, and comparing against my hand-rolled note-passing. I have a suspicion my notes approach is the Standard harness of my own little world and I've been paying a tax I didn't know about.

If you want to argue with any of this, the replays are public and you can play the games yourself. I lasted about four minutes on the first one before I understood the rules, which puts me somewhere behind Astra and, I suspect, most of the 500 test participants. This is the kind of evaluation work I do on the agent projects I take on; there's more about that on my site.

Originally published at abrarqasim.com. I write there about React, PHP, Rust, Go and the AI tooling around them.

GitHub Actions Reusable Workflows: The Bug I Fixed Eleven Times

Qasim Parray — Fri, 11 Sep 2026 18:39:51 +0000

Okay, this is going to sound dumb, but last month I fixed the same bug in eleven repositories. Not eleven bugs. One bug, eleven times. A Node version pin in a CI workflow that I'd copy-pasted across every client project I set up in 2024, and every one of them started failing the week a dependency dropped support for Node 18.

Eleven pull requests. Eleven "bump node to 22" commits. Eleven times waiting for the green tick. Somewhere around repo seven I stopped and asked myself why I was doing this by hand when GitHub Actions has had reusable workflows since 2021 and I had, apparently, never bothered to learn them properly.

So this is the post I should have read three years ago. It covers what reusable workflows are, how they differ from composite actions (the two get confused constantly, including by me), the before and after of my own setup, and the handful of gotchas that cost me an afternoon. If you maintain more than three repos with CI, you probably have this problem too, you just haven't hit the eleven-PR morning yet.

The copy-paste workflow, and why it rots

Here's roughly what lived in .github/workflows/ci.yml in every one of those repos. You've written this file. Everyone has.

# Before: the same file, pasted into every repo
name: CI
on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 18
          cache: npm
      - run: npm ci
      - run: npm run lint
      - run: npm test -- --coverage
      - uses: actions/upload-artifact@v4
        with:
          name: coverage
          path: coverage/

Nothing wrong with it on day one. The rot sets in slowly. One repo gets a fix for flaky caching. Another gets a matrix added because a client needed Node 20 support. A third gets a security tweak after I read something about pinning action versions to SHAs. None of those improvements travel. Six months later I have eleven workflows that are 80% identical and 20% mysteriously different, and I couldn't tell you which one is the "good" one.

The Node pin was just the symptom that finally hurt enough to notice.

Reusable workflows vs composite actions

This is where I got stuck the first time, so let me be blunt about the distinction before showing code.

A reusable workflow is an entire workflow, with its own jobs and runners, that another workflow can call as if it were a job. You trigger it with workflow_call, and the caller uses uses: at the job level. The GitHub docs on reusing workflows are decent here, but they bury the important part: the called workflow runs in the context of the caller's repository, on the caller's runners, with the caller's secrets if you pass them.

A composite action is a bundle of steps. It runs inside a job that already exists. You can't define a runner or a matrix in a composite action, because it doesn't own the job. It's closer to a function you call from within a step list. The composite action docs cover the mechanics.

The rule I use now: if the thing I want to share is "a job" (test this, build this, deploy this), it's a reusable workflow. If it's "a few steps that always go together" (set up Node with our cache config, or log in to our registry), it's a composite action. I ended up needing both, and they nest fine: a reusable workflow can call composite actions.

I got this wrong for two weeks by trying to make one composite action do everything, then wondering why I couldn't give it a matrix.

The after: one workflow, one line per repo

I made a repo called qasim/workflows (public, because reusable workflows in private repos need extra access config and I didn't want to think about it yet). The shared workflow lives at .github/workflows/node-ci.yml:

# .github/workflows/node-ci.yml in the shared repo
name: Node CI (reusable)
on:
  workflow_call:
    inputs:
      node-versions:
        description: JSON array of Node versions to test
        type: string
        default: '["22"]'
      run-lint:
        type: boolean
        default: true
      coverage:
        type: boolean
        default: false
    secrets:
      NPM_TOKEN:
        required: false

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        node: ${{ fromJSON(inputs.node-versions) }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node }}
          cache: npm
      - run: npm ci
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
      - if: ${{ inputs.run-lint }}
        run: npm run lint
      - run: npm test ${{ inputs.coverage && '-- --coverage' || '' }}
      - if: ${{ inputs.coverage }}
        uses: actions/upload-artifact@v4
        with:
          name: coverage-node${{ matrix.node }}
          path: coverage/

And in each of the eleven repos, ci.yml became this:

# After: the caller, in every project repo
name: CI
on:
  push:
    branches: [main]
  pull_request:

jobs:
  ci:
    uses: qasim/workflows/.github/workflows/node-ci.yml@v1
    with:
      node-versions: '["20", "22"]'
      coverage: true
    secrets:
      NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

Eight lines. The next time a Node version dies, I change one file, tag v1.1, and move the v1 tag. Every repo picks it up on its next run.

Notice the matrix lives in the shared workflow but the versions come from the caller. That's the bit composite actions can't do, and it's the reason I picked reusable workflows for this layer. The fromJSON trick is ugly but it's the documented way to pass an array through a string input; matrix inputs can't be typed as arrays directly. Yes, I find that annoying too.

Secrets, permissions, and the `inherit` shortcut

Two things bit me here.

First, secrets don't flow automatically. The called workflow only sees what you explicitly pass in the secrets: block, unless you write secrets: inherit, which hands over everything the caller has access to. I used inherit for a day, then changed my mind. It's convenient, but it means the shared workflow can read every secret in every repo that calls it, and if I ever add a step that echoes environment for debugging (I have, more than once), that's a bad afternoon. Passing secrets by name costs one extra line per secret and lets me see exactly what crosses the boundary.

Second, permissions. If your shared workflow needs to write to the repo (pushing a tag, commenting on a PR), the caller has to grant that. The called workflow can only narrow permissions, never widen them beyond what the caller allowed. GitHub's secure use guide for Actions covers the reasoning, and the short version is: put permissions: contents: read at the top of your caller and only open things up per job when you have to.

On the topic of security, pin the shared workflow reference to something you control. @v1 is a moving tag I own, so I'm fine with it. If you're calling someone else's reusable workflow, pin to a full commit SHA. Same rule as third-party actions.

Composite actions for the boring steps

Once the job-level workflow was shared, I noticed the same four setup steps kept appearing in my other workflows too: the deploy workflow, the release workflow, a nightly one. Node setup, cache, npm ci, registry auth. That's a composite action.

# action.yml in qasim/workflows/setup-node-project/
name: Setup Node project
inputs:
  node-version:
    default: '22'
  npm-token:
    default: ''
runs:
  using: composite
  steps:
    - uses: actions/setup-node@v4
      with:
        node-version: ${{ inputs.node-version }}
        cache: npm
    - run: npm ci
      shell: bash
      env:
        NODE_AUTH_TOKEN: ${{ inputs.npm-token }}

Called with uses: qasim/workflows/setup-node-project@v1. The shell: bash line is required on every run step in a composite action, and forgetting it produces an error message that does not mention the word "shell". You've been warned.

One genuine limitation: composite actions can't use secrets.* directly. You pass the secret in as an input from the caller, which is what the npm-token input is doing above. It feels slightly wrong to pass a secret as an input, but it's how the model works, and the value is still masked in logs.

Caching still needs care

I assumed moving to a shared workflow would make caching "just work" everywhere. It mostly did, with one wrinkle: cache keys are scoped to the repo that runs the workflow, not the repo that defines it. That's the right behaviour, and it means each of the eleven repos has its own node_modules cache, which is what you want. But the 10 GB per-repo cache limit that GitHub documents in its dependency caching reference applies per caller, so a shared workflow that caches aggressively can fill a small repo's quota faster than you'd expect. I trimmed the coverage artifacts to seven days of retention and stopped caching Playwright browsers in the shared job. Not a big deal, just not free.

If you're also shrinking Docker images in CI, the same "measure before you assume" attitude applies. I wrote about that in my post on multi-stage builds and the 1.2 GB image I stopped shipping, and about half of that post's advice is really about what your CI cache is silently doing.

What I'd tell someone starting today

Don't extract too early. I wrote the copy-paste version of that workflow in maybe fifteen minutes back in 2024 and it served me fine for a year. The shared version took most of a day, including the two weeks of low-level confusion about composite actions that I'm compressing here. Three repos with slightly different CI is fine. Eight is where it starts to hurt. Eleven is where you write a blog post about it.

Also, version the shared repo like a library. Moving v1 is fine for backward-compatible fixes. Anything that changes an input name gets v2, and the old tag stays around. I learned this the cheap way, by breaking one repo instead of all of them, because one repo happened to still pin @main. Don't pin @main.

I'm still not sure whether I should have just used a monorepo for the client projects and skipped all of this. Probably not; the clients own their repos and want them separate. But I wrote about that tradeoff at length in the Turborepo vs Nx post, and the honest answer is that shared CI workflows are the "we have multiple repos and that's not changing" solution.

Something to do this week

Open your most-copied workflow file. Grep your org for its job name. If it appears in more than three repos, make a workflows repo, move that job into a workflow_call file with two or three inputs, and replace the copies with an eight-line caller. Tag it v1. You'll spend an hour, and the next time a runtime version dies you'll fix it once.

If you want to see how I set up CI for the projects I run for clients, the work section of my site has a few examples where the pipeline setup was most of the job.

Originally published at abrarqasim.com. I write there about React, PHP, Rust, Go and the AI tooling around them.

When Not to Use AI Automation: The Refund Bot That Cost Me a Client

Qasim Parray — Fri, 11 Sep 2026 14:39:51 +0000

Confession: the first AI automation I sold to a client was a mistake, and I didn't know it for four months. It was a small e-commerce shop, three people, and they wanted the "AI reads the support inbox and replies" thing everyone was selling in 2024. I built it. It worked in the demo. Then it spent a quarter politely telling customers their refund was "being processed" when nobody had processed anything, because the model was excellent at sounding like a refund had been processed and nobody had wired it to the actual refund system. The owner found out from a one-star review.

I've been thinking about that project again this week, for an odd reason. OpenAI published a case study on Gilbert + Tobin, a large Australian law firm that has rolled ChatGPT Enterprise and Codex across most of its operations. It's a vendor story, so take the adoption numbers with the usual salt. But the interesting part isn't the numbers. It's what they explicitly did not automate, and how much governance a firm with real money and real lawyers decided it needed before letting AI touch anything. If a law firm with a CIO and a business transformation team draws the line that carefully, a three person shop with me as the entire IT department should probably draw it more carefully, not less.

So this is a post about when not to use AI automation in a small business, written by someone who makes part of his living building AI automation for small businesses. That should tell you I'm not against it. I'm against the version of it that ends in a one-star review.

What a law firm chose not to automate

Read the Gilbert + Tobin piece with an eye for the negatives and a pattern shows up. The firm introduced ChatGPT to operations teams first, then marketing, recruitment, finance and technology. The legal work itself, the advice clients pay for, stayed on separate approved platforms with their own controls. The CEO's line was "AI is not cheating", but the framing throughout is that people stay responsible for constraining the task, checking outputs and approving the final work product.

Their Codex examples are the same shape. Audit reports across 300 entities, prepared by Codex and reviewed by a person. Eleven hundred files checked and renamed for upload, reviewed by a person. A KYC and anti-money-laundering check workflow that does the research and processing steps and then produces a report for human sign-off. In every example the model does the long boring middle and a human owns the start and the end.

There's a detail I keep coming back to. They built a custom GPT trained on the CEO's writing and priorities so executives could pressure test ideas before taking up his time. And then the case study goes out of its way to say it does not make decisions on his behalf or speak for him. Someone thought hard about where that line was and wrote it down.

That's a firm with hundreds of staff, an information security function, and clients who will sue if data leaks. My clients are usually a founder, two employees and a Shopify store. They don't have a governance team. So the question for me is: which of those lines transfer down to a business that size, and which ones are big company overhead?

The line that transfers: automate the middle, not the ends

Almost every AI automation I've seen fail in a small business failed at one of the ends. Either the model decided what the task was, or the model decided the task was done. The middle, the part where it reads forty PDFs and pulls the invoice numbers into a spreadsheet, almost never fails in a way that matters. If it gets one wrong, a person catches it at the end, because a person is at the end.

My refund bot broke both ends at once. It decided a support email was a refund request (start), and it decided to tell the customer the refund was handled (end). The bit in the middle, drafting a polite reply, was flawless. That was the problem. A flawless middle with no one at the ends is the most dangerous configuration there is, because it looks like it's working right up until it very much isn't.

So the first rule I now apply, before a single line of code, is this. Draw a box around the automation. Write down what enters the box and who decided it should. Write down what leaves the box and who checks it before it has an effect in the real world. If either of those "who" answers is "the model", stop and ask whether the business can survive the model being confidently wrong there a hundred times before anyone notices. Usually it can't.

I wrote about the positive version of this, the jobs that actually are worth handing to a model, in my post on AI automation examples and which skills LLMs replace. This post is the mirror image.

When not to use AI automation at all

Some things aren't a middle. I've built up a short list of situations where I'll now tell a client no, or at least "not like that", and most of them come from getting it wrong once.

Anything that sends money or makes a commitment. Refunds, discounts, delivery promises, quotes. A model that promises a customer a discount has made a contract on your behalf, and "the AI said it" is not a defence anyone will accept. Gilbert + Tobin let Codex prepare audit reports, not sign them. The same applies to a Shopify store's refund policy.

Anything where being wrong is invisible. A summarizer that drops one line from a contract, a classifier that files a complaint under "general enquiry". Nobody sees the mistake at the moment it happens. The cost turns up weeks later as a churned customer or a missed deadline, and you can't trace it back. If you can't measure the error rate, you can't decide whether it's acceptable, and "we'll find out" is not a plan.

Anything you don't do often enough to review. This one surprised me. The Gilbert + Tobin examples are irregular jobs: an audit sweep, a bulk rename. Those work because a person is sitting right there when it runs. The danger zone is the low volume, unattended automation. It runs twice a week, no one is watching, and by the time someone looks at the output there are two months of confidently wrong records to unpick. High volume tasks get caught fast because someone is always looking. Low volume, unattended tasks rot.

Anything where the business hasn't written down the rule. If the founder can't tell me in plain English how they decide whether a lead is worth calling, I can't encode it, and the model will happily invent a rule that sounds plausible. Half the "AI automation" projects I've turned down were really "please make a decision for us that we haven't made ourselves" projects. That's a consulting job, and the model is not the consultant.

The permission problem nobody budgets for

The part of this that doesn't get talked about in small business AI automation pitches is what the automation is allowed to touch. Docker published a piece this week, Below the Harness, that opens with Norm Hardy's 1988 confused deputy problem: a program that takes an action using its own permissions rather than yours. Their argument is that every AI agent is that deputy now, because it inherits your credentials, your repository access, your ability to call whatever API you can call, and that organizations will need a runtime layer underneath all their agents where rules get enforced in one place.

That's aimed at enterprises running fleets of agents. But the confused deputy shows up in a three person business just as hard, and with fewer safeguards. The typical small business automation I'm asked to build runs on the owner's own accounts. The owner's Gmail. The owner's Stripe key. The owner's admin login to the store. Because setting up a scoped service account is friction and the demo is due Friday. So the automation can do anything the owner can do, which includes issuing refunds, deleting products and emailing the entire customer list. It probably won't. But "probably won't" is doing a lot of work when the thing deciding what to do is a model reading customer emails, some of which will eventually contain instructions.

I covered the technical shape of that last risk in the tool schema problem in AI agent security, so I won't repeat it. The small business version is simpler and more boring. Before any automation goes live, it gets its own account, with the minimum permissions it needs, and the owner's credentials come out of the config. If the client says that's too much hassle, that's useful information about whether the automation should exist.

What the small version of governance actually looks like

I don't think a three person company needs a governance team. I do think it needs about a page of writing, and I now refuse to build the automation until that page exists. It has four things on it.

What the automation is allowed to do, in plain sentences. "Read incoming support emails. Draft a reply. Put the draft in a folder. Never send." Not a system prompt, a sentence a non-technical owner can read and agree to.

What it's not allowed to do, which is usually the previous list with the word "never" in front of the scary verbs. Never send. Never refund. Never change a price.

How you'll know it's wrong. Somebody's name, and how often they'll look. "Sara checks the drafts folder every morning before sending." If nobody's name goes here, the automation is unattended, and see above.

How to turn it off. A single switch the owner can flip without calling me. This sounds trivial. It is the most important line on the page. The refund bot took a week to fully stop because it was wired into three places and only I knew where.

That page is the small business translation of everything in the Gilbert + Tobin story: approved tasks, what you can enter, how outputs get reviewed, and who is accountable. It takes an hour to write and it has saved me from at least two projects that would have gone the way of the first one.

Where AI automation does earn its keep

I want to end on the other side, because "don't" is a cheap thing to write and I'd be a hypocrite if I stopped there.

The automations I've built that clients still pay for a year later have all had the same profile. High volume, so mistakes surface fast. A human at both ends. Read-mostly, meaning the model's job is to look at things and produce a draft or a classification, and the effect on the world happens when a person clicks. Invoice data extraction into a review queue. First draft replies that a person sends. Lead notes summarized before a call. Tagging and sorting a backlog so a person can triage it in ten minutes instead of an hour.

None of these are exciting. All of them save real hours, and all of them have a fail state that is "a person spends an extra minute" rather than "a customer gets a promise nobody will keep". The cost side matters too; a review queue that a person clears is far cheaper to run than an agent loop that retries until it's sure, and I've written before about what happened to LLM costs after the free lunch ended.

If you're a small business owner reading this and someone is pitching you an automation, ask them where the human at each end is. If they say there doesn't need to be one, that's your answer. If you're a developer building these things, the way I've come to think about scoping them is written up in more detail on my site, along with the projects where I learned it the hard way.

What to do this week

Pick one automation you're already running, or the one you're about to build, and write the one page. Four sections: allowed, never, who checks and how often, how to turn it off. If you can't fill in "who checks", don't ship it. If you can't fill in "how to turn it off" in under a sentence, you've got a bigger problem than AI, and this is a good week to fix it.

Originally published at abrarqasim.com. I write there about React, PHP, Rust, Go and the AI tooling around them.

Laravel 13.30: chunkBy() and the Storage::path() Line I Had in Production

Qasim Parray — Fri, 11 Sep 2026 11:36:25 +0000

Short version for the impatient: Laravel 13.30 gives collections a chunkBy() method, and it quietly closes a hole where Storage::path() would happily hand you a filesystem path to your .env file. If you have a download endpoint that takes a path from the request, go grep for Storage::path before you read the rest of this.

Now the longer version. I almost skipped this release. Point releases in Laravel land are a weekly thing now, and most of them are the kind of change you nod at and forget. I opened the changelog on Wednesday morning mostly out of habit, saw "chunkBy", thought "nice, another helper", and was about to close the tab. Then I got to the part about Storage::path() and went a bit cold, because I had written the exact vulnerable line in a client project in 2023. I checked. It had been refactored out since, but not because anyone noticed the problem. We just moved to signed URLs for other reasons. That's luck, not engineering.

So this post is two things. A look at chunkBy(), because I like collection helpers and this one replaces a pattern I've typed dozens of times. And a slightly more serious look at what changed in Storage::path(), why it was inconsistent with the rest of the filesystem layer for years, and what it might break when you upgrade.

What chunkBy() replaces

Laravel's collections have had chunk() forever. It splits a collection into pieces of a fixed size. Useful for batching, useless for grouping adjacent items that share something.

For that, the tool since Laravel 8 has been chunkWhile(). It takes a callback and starts a new chunk whenever the callback returns false. Every time I've used it, the callback compared the current item to the last item of the chunk being built:

// Before: chunkWhile with the comparison you always end up writing
$grouped = $orders->chunkWhile(
    fn ($order, $key, $chunk) => $order->customer_id === $chunk->last()->customer_id
);

That works. It also has a shape I have to re-derive every time. Which argument is the chunk? Is it ->last() or ->first() I want? Does == or === matter for the ids coming out of this particular database driver? None of those are hard questions, but they're the kind of questions that cost me a minute of staring each time, and a minute of staring is exactly what a helper should remove.

chunkBy() is that comparison as a method:

// After: chunkBy with a key
$grouped = $orders->chunkBy('customer_id');

// Or a callback, if the grouping value needs computing
$grouped = $orders->chunkBy(fn ($order) => $order->created_at->toDateString());

// Dot notation works, because the key goes through data_get()
$grouped = $addresses->chunkBy('address.city');

Two details from the pull request that I care about. The key resolves via data_get(), so nested keys and objects behave the way they do everywhere else in the framework. And the original keys are preserved inside each chunk, which matters if you're chunking a keyed collection and want to write the results back somewhere by id.

It also lands on LazyCollection, which is the version I'll use most. A common job I write is "walk a big export in order, emit one file per group". With chunkWhile on a lazy collection I always had a nagging feeling that the $chunk->last() lookup was doing more work than it needed to. With chunkBy I stop thinking about it.

The thing to remember: it's adjacency, not grouping

Here's where I'll push back on how I've seen this described online already. chunkBy is not groupBy with a different name. It only groups items that are next to each other.

collect([1, 1, 2, 2, 1, 1])->chunkBy(fn ($v) => $v);
// [[1, 1], [2, 2], [1, 1]]  <- three chunks, not two

collect([1, 1, 2, 2, 1, 1])->groupBy(fn ($v) => $v);
// [1 => [1, 1, 1, 1], 2 => [2, 2]]  <- two groups

That's the whole point of it, and it's the whole trap. If your query doesn't have an orderBy on the column you're chunking by, you'll get fragmented chunks and probably not notice in dev because your seed data happens to be sorted. I have been bitten by this with chunkWhile and I'll be bitten by it with chunkBy. The fix is boring: sort first, then chunk, and put a test in with deliberately shuffled input.

When do you want adjacency instead of a full group? Usually when the order carries meaning. Consecutive log lines from the same request. Runs of the same status in a timeline. Line items on an invoice that should be visually grouped under a parent without being reordered. groupBy would destroy the sequence; chunkBy keeps it.

Storage::path() finally goes through the normalizer

This is the part that made me check old projects.

Every filesystem call on a Laravel disk goes through Flysystem, and Flysystem normalizes the path before doing anything with it. If the normalized path resolves outside the disk root, it throws PathTraversalDetected. So this has been rejected for years:

Storage::get('../../../.env'); // throws PathTraversalDetected

Storage::path() was the one exception. It didn't call into the driver's read or write methods, it just asked the PathPrefixer to glue the disk root onto whatever string you gave it. No normalization, no traversal check. So on the default local disk:

// Before 13.30
Storage::path('../../../.env');
// => "/var/www/app/storage/app/../../../.env"
// which is a perfectly valid native path to your .env

get(), delete() and readStream() all refused that argument. path() returned a string pointing at your secrets. And the string is where it gets dangerous, because path() is the method you reach for when you want to hand a file to something that isn't Flysystem. Like PHP's own file functions. Or a response:

// The line I had in a 2023 project. Do not do this.
return response()->download(Storage::path($request->query('path')));

response()->download() doesn't know or care about disk roots. It just streams whatever native path you give it. So the one Laravel method that skipped the traversal check was also the one most likely to be sitting directly in front of fopen.

As of #61343, path() runs the argument through WhitespacePathNormalizer, the same normalizer that League\Flysystem\Filesystem builds for every other operation. Relative segments now resolve the way the driver resolves them, and anything that escapes the root throws instead of returning a string.

// After 13.30
Storage::path('reports/../invoices/2026-08.pdf');
// => "/var/www/app/storage/app/invoices/2026-08.pdf"  (resolved inside the root, fine)

Storage::path('../../../.env');
// throws League\Flysystem\PathTraversalDetected

I want to be fair here. This was a known inconsistency rather than a hidden zero-day, and a lot of teams never hit it because they don't pass request input to path(). But "a lot of teams" isn't "all teams", and I was on one of the other kind. I wrote about the general shape of this problem, where the tool you use for the review is only as good as the surface it knows about, in my post on running a Laravel security audit with the agent you already use. That audit would have flagged the download() line. It would not have flagged path() itself as the cause, because until this week, path() behaving that way was just how it worked.

What this might break on upgrade

The flip side of a hardening change is that code depending on the old behaviour now throws.

If you have anything that deliberately uses .. in a path given to Storage::path(), it will now either resolve (if the result stays inside the root) or throw (if it doesn't). The first case is silent and probably fine. The second case is an exception in production, and the framework isn't going to guess whether you meant it.

The pattern I've seen most is a "shared" folder one level above the disk root that someone reached with Storage::path('../shared/thing.csv') because configuring a second disk felt like too much ceremony. That will now throw. The fix is to configure the second disk. It was always the right answer, it just wasn't enforced.

The other one to watch is tests. If you have a test that asserts Storage::path() returns a specific string for a path with relative segments, the string changes even when nothing throws, because the segments now collapse. Search your test suite for Storage::path( and ->path( on a disk before you bump.

My own upgrade checklist for this release, in the order I actually did it:

grep -rn "Storage::path\|->path(" app/ and read every hit.
For each hit, ask: can any part of this argument come from a request, a webhook payload, a queued job payload, or a database column a user can edit?
If yes, either validate the path against an allow list before it reaches path(), or stop using path() there entirely and use download() / response() on the disk, which go through the normalizer.
Run the suite. Anything that throws PathTraversalDetected gets fixed, not caught.

Step three is the real one. The 13.30 change means the traversal case throws, which is far better than silently returning a path. But an exception in front of a download endpoint is still a broken endpoint, and I'd rather reject bad input with a 422 than let it reach the filesystem layer at all.

The small one I'll use every day: worker stop reasons

There's a third change in this release that isn't security related and isn't as clever as chunkBy, but I suspect it'll save me more time than either. queue:work now prints why the worker stopped as its last line of output.

Before, a worker that exited just exited. The WorkerStopping event has carried a WorkerStopReason for a while, but you had to register a listener to read it, which is a lot of ceremony for something you usually want to know while staring at a terminal at 11pm.

Now:

2026-09-01 13:20:40 Worker STOPPED Memory limit exceeded

With --json it's a structured record, with the reason, exit code, jobs processed and memory alongside the per-job lines. The WorkerStopReason enum gained a description() method covering nine cases, including memory limit, max jobs, max time, queue empty, restart signal and job timeout. Nothing prints under --quiet or --silent, so it won't spam a supervisor log.

This connects to something I wrote about in the queue defaults I stopped trusting in production. A large fraction of "why did my worker die" investigations end with "it hit the memory limit and Horizon restarted it", and until now the evidence for that was indirect. Having the reason in the log line turns a fifteen minute investigation into a glance.

Was it worth a whole post?

I think so, and not because chunkBy is exciting. It's because the Storage::path() change is the kind of thing that's easy to miss in a weekly release note, and it has a real "go check your code" action attached to it. Most weeks the answer to "should I read the changelog" is "probably not". This week it was yes.

A few other things in the release worth a glance if you're on SQL Server (native sqlsrv: DSN strings now parse instead of getting mangled by parse_url), or if you build Artisan tooling (Artisan::commandNamed() resolves a single command without constructing all of them). The full list is in the Laravel News write-up and the official changelog.

If you want to see how I handle file handling and upgrades on client projects more generally, that's covered on my work page.

What to do this week

Run this in every Laravel project you maintain, even the ones not on 13 yet:

grep -rn "Storage::path\|Storage::disk([^)]*)->path" app/ routes/

For every line it finds, trace where the argument comes from. If any part of it originates outside your own code, wrap it in validation now. Don't wait for the upgrade to throw the exception for you. Then bump to 13.30, run your suite, and swap your first chunkWhile comparison for chunkBy while you're in there. That last one is just for fun.

Originally published at abrarqasim.com. I write there about React, PHP, Rust, Go and the AI tooling around them.

DEV Community: Qasim Parray

How to Handle Scope Creep: The Quick One That Cost Me a Week

Scope creep is mostly a definition problem

What goes in my scope document now

The five-minute rule

When the request is actually a spec bug

Retainers make this easier, projects make it harder

What to change this week

ARC-AGI-3 and GPT-6 Astra: Two Scores, One Model, and the Harness Gap

What ARC-AGI-3 actually tests

The two harnesses, in plain terms

Why the gap is the story, not the 99.9

What Astra was actually doing in there

The part that should worry you about your own evals

What I'm doing about it this week

GitHub Actions Reusable Workflows: The Bug I Fixed Eleven Times

The copy-paste workflow, and why it rots

Reusable workflows vs composite actions

The after: one workflow, one line per repo

Secrets, permissions, and the inherit shortcut

Composite actions for the boring steps

Caching still needs care

What I'd tell someone starting today

Something to do this week

When Not to Use AI Automation: The Refund Bot That Cost Me a Client

What a law firm chose not to automate

The line that transfers: automate the middle, not the ends

When not to use AI automation at all

The permission problem nobody budgets for

What the small version of governance actually looks like

Where AI automation does earn its keep

What to do this week

Laravel 13.30: chunkBy() and the Storage::path() Line I Had in Production

What chunkBy() replaces

The thing to remember: it's adjacency, not grouping

Storage::path() finally goes through the normalizer

What this might break on upgrade

The small one I'll use every day: worker stop reasons

Was it worth a whole post?

What to do this week

Secrets, permissions, and the `inherit` shortcut