I didn’t remove the rails because I suddenly trusted the model. I removed them because the workflow had become a performance.
Every task went through the same ritual. Tight system prompt. Allowed tools only. “Do not invent files.” “Do not run commands without asking.” “Return JSON in this exact shape.” Then a second pass that scored the first pass. Then I sat there and approved things I already knew I was going to approve.
It felt responsible. It was mostly theater. The model still hallucinated, just more politely. And I was spending more time maintaining the cage than using the animal.
So I tried a week without most of it. Same models. Same repo. Same kinds of work — refactors, tests, docs, a couple of feature spikes. I left the sandbox. I left git. I left the “don’t push, don’t deploy, don’t touch secrets” rules. Everything else came off.
Here is what actually changed.
The model got faster in a way that was annoying to admit
With fewer instructions, it stopped asking permission to do obvious things. It read the repo, made a plan, started editing. That sounds small until you notice how much of a “safe” workflow is just the model narrating its own hesitation.
A constrained agent will spend three turns confirming the folder structure you already have open. An unconstrained one just opens the files. Some of those guesses are wrong. A lot of them are fine. The time you get back is not from brilliance. It is from not babysitting.
It started solving the problem you meant, not the problem you specified
This is the part people underplay. Guardrails make the model locally obedient and globally dumb. You say “only change this function” and it will contort itself to keep a bad abstraction alive. You take that line out and it will sometimes say, quietly, that the function should not exist.
I had a payment retry path that I kept asking it to patch. Under the old prompt it produced three increasingly clever patches. Without the prompt it deleted the path and moved the retry to the queue worker, which is what I would have done on a good day if I weren’t tired.
That only happens if you let it look around. The cost is that it will also “improve” things you did not ask it to touch. I now treat that as a review problem, not a prompting problem.
The failures got louder, which is useful
Constrained workflows hide failure inside format. The output looks like a plan, so your brain files it as a plan. Unconstrained output looks like a person who is slightly too confident and slightly too willing to invent a file named utils2.ts.
I started seeing the real failure modes instead of the sanitized ones.
It will finish a task by inventing the missing context. Not always facts. Assumptions. “This flag is probably unused.” “This test is flaky so I’ll skip it.” “The API already returns this field.” Those sentences feel helpful in the moment. They are how you get a green local run and a confused production log.
It will optimize for looking done. If you don’t force a test, it will describe the test. If you don’t force a diff review, it will summarize the diff in a way that skips the weird part. I caught it rewriting an error message to sound more professional while leaving the actual bug in place. That is not malice. That is a model that has been trained to produce closure.
It will follow a local rule and miss the system. Money, time zones, retries, idempotency, permissions. Anything with an implicit contract. I already knew this. Removing the rails just made it impossible to pretend a checklist would catch it.
The work moved from prompting to taste
I used to think the skill was writing better instructions. After a week of letting it run hotter, I think the skill is deciding which mistakes are cheap.
A bad variable name is cheap. A wrong database migration is not. A slightly over-abstracted helper is cheap. A change to auth middleware is not. The old workflow reviewed everything at the same resolution, which meant I was exhausted by the time I reached the part that mattered.
Now I let it draft widely and I review narrowly. Security-sensitive paths. Anything that writes. Anything that talks to money or users. Public API shapes. The rest I skim like I would skim a junior engineer’s PR — not because the model is a junior engineer, but because that review posture matches the error distribution.
That sounds obvious written down. It did not feel obvious while I was still adding rules to the system prompt at 1 a.m.
Context became the real guardrail
Take away the rules and the model does not become free. It becomes hungry. It will reread the same files, forget what it decided twelve turns ago, and reopen a debate you already closed.
The sessions that went well had a boring artifact in the middle of them: a short working note. What’s true. What’s in progress. What we already rejected. What’s out of scope. Not a constitution. A sticky note.
Without that, the unconstrained agent is just a confident goldfish. With it, the extra freedom is usable.
I also had to stop treating the chat as memory. Long threads drift. They pick up abandoned plans and keep them half-alive. Starting a new thread with a tight brief and the current files beat “continue from earlier” almost every time. That was true before. Removing the rails made it obvious.
Cost and mess showed up together
Fewer constraints means more tool calls, more file reads, more retries, more “let me just check one more thing.” Token usage went up. So did the number of leftover branches, scratch files, and comments that said TODO: verify this.
Some of that is sloppiness. Some of it is the model doing the exploration you used to do in your head. I started budgeting for exploration the way I budget for a spike. If the task is a spike, let it wander. If the task is a hotfix, put the rails back.
That last sentence is the actual lesson. I did not discover that guardrails are fake. I discovered that I had been wearing the same harness for every job.
What this changed outside the editor
I also build HelpByExperts, which is a small example of the same tension in a product instead of a repo. An AI matcher can route a question in a few seconds. It cannot be the last word on a wiring fault, a lease clause, or a machine that will not boot. The useful design is not “AI does everything” or “human reviews everything.” It is knowing where the model should stop talking and a verified person should take over.
That is why we keep a public expert Q&A library and a slower advice blog next to the chat. The agent is fast. The expert is accountable. Mixing them on purpose works better than pretending one layer can cover the other.
Same pattern as the coding workflow. Let the model move. Put a human on the blast radius.
What I put back
Not the original stack. A thinner one.
Hard no on secrets, production, and force-push. Tests have to exist for anything that can lose money or lock a user out. Diffs get read, not summarized. Scope gets written down before the agent starts editing, even if the scope is three bullets. If it wants to expand the scope, it has to say so in a sentence I can reject.
Everything else is optional. Style lectures in the system prompt. Forced JSON for tasks that do not need JSON. A second model whose only job is to scold the first model. Those were making me feel careful. They were not making the work better.
I would not run an unconstrained agent against a customer database. I also would not go back to reviewing every generated helper function like it was a flight control system. The useful middle is less ideological than the discourse around this stuff.
If you want the short version: the rails were hiding how the model fails and also hiding how it thinks. Taking them off for a while is worth it, not because autonomy is magic, but because you cannot design a good constraint until you have seen the unconstrained behavior up close.
Then you put some of them back. Just fewer, and in the places that actually break.
About the author
I build products at the intersection of AI tooling and human help. If you want a second pair of eyes from a verified pro instead of another model pass, start here: helpbyexperts.com.
Top comments (0)