DEV Community: Tessl

Cheaper Tokens, Bigger Bills: Token Price Isn't Agent Cost

Tessl — Sat, 29 Aug 2026 06:36:50 +0000

We needed a default model for Tessl Code Review, our new code review tool. The shortlist seemed easy to compare: check the token prices, weigh them against capability, and choose. Then we ran the models on real pull requests through the real review harness. The rate card would have led us to the wrong choice.

Pricing pages quote tokens because tokens are easy for vendors and buyers to count. An agentic workload adds another variable: turns. A turn is one model call to read a file, run a check, or decide what to do next. The model decides how many turns it needs, and you pay for all of them. A rate card cannot show that. Every alternative we tested had cheaper tokens than gpt-5.6-terra, the model we run, yet three cost more per review. One was almost three times cheaper per token and 2.8 times more expensive on the same pull request.

A more useful measure is cost per verified outcome: what you spend for each result that survives an independent check. This captures the full cost of the task and discounts results that do not hold up. As with any other agent evaluation, the check has to be independent. A model cannot be trusted to grade its own work.

Why per-token price stopped predicting your bill

We used ten merged pull requests from our monorepo, pinned at the commits seen by our production reviewer. Four models went through the same harness with the same review lenses, fixing agent, and loop: review the change, apply fixes, then review again for up to four rounds. In August 2026, that produced 100 runs across two conditions. Only the model changed.

The study has two important limits. With ten pull requests and one run per model per subject, we could not measure run-to-run variance. We also chose pull requests where our production reviewer had already found an issue. That raises recall for every model and tells us nothing about false positives. We therefore report multiples rather than percentages and do not rank the models that trailed the baseline against one another.

All costs are relative to gpt-5.6-terra. “From rate card alone” shows what each model would cost if it consumed exactly what terra consumed. “From consumption alone” prices each model's actual usage at terra's rates.

Model	Actual cost gap	From rate card alone	From consumption alone
gpt-5.6-terra, the baseline	1.0x	1.0x	1.0x
A lower-cost open-weight model	0.4x	0.16x	2.33x
A second open-weight model	1.7x	0.54x	3.23x
A third open-weight model	2.8x	0.37x	7.59x

The rate card suggests savings of 46% to 84%, so any of the three alternatives looks attractive in the middle column. Their actual consumption changes the result: they used between 2.3 and 7.6 times more to complete the same job. One model was genuinely cheaper at 0.4 times terra's cost per review, but an independent check found that it also delivered the least useful output.

Turns explain most of the gap. Terra completed a review in 42 turns and used 0.6 million input tokens. The most expensive alternative needed 156 turns and 5 million tokens. Both returned a review, but one spent far longer rereading files and checking its own work.

For a single call with a controlled input and bounded output, token price is a reasonable estimate of cost. An agent with tools and a goal controls much more of its own consumption. Token price still matters, but the model's behavior inside the harness can matter more.

Register to get the early birds discount

A better unit for AI agent evaluation

We use this formula:

`cost per verified outcome =
    price per token
  x tokens consumed per task
  x 1 / (share of output that survives verification)`

The rate card supplies the token price. Running a real task supplies consumption. An independent grader tells you how much of the output survived. Leaving out either of the last two terms can change the ranking entirely.

Consumption: the term that does the damage

Consumption has to be measured on your work and through your harness. Identical inputs in our study produced a fourfold spread in turns and an eightfold spread in token volume. Published benchmarks usually ask whether a model reached the right answer, not what it spent to get there, so they would not have exposed this difference. The measurement also expires: change the model or the harness and consumption can change with it.

Validity: what survives an independent check

The reviewer models assigned severity to their own findings. We regraded all 910 findings with one fixed model that saw only the finding and its diff hunk.

Because each reviewer used and inflated its own scale, we could not compare the original labels directly. The reviewers and grader agreed 59% of the time. When they disagreed, the grader marked 35% of findings down and 6% up. Only 6 of the 70 findings called critical by their authors remained critical, and the grader did not promote a single finding to critical.

Regrading reversed one apparent lead. A competing model reported 73 severity-bearing findings against terra's 62. After grading, terra led 37 to 17. We also compared each model with findings from our production review on the same commits, and that check pointed in the same direction.

The table combines spend with both checks. “Verified severity found” counts severity that survived grading. The next column prices each graded major or critical finding. The final column shows how much of our production review's output the model also caught. All values are relative to terra.

Model	Verified severity found	Cost per graded major or critical finding	Share of our own review's findings also caught
gpt-5.6-terra, the baseline	1.0x	1.0x	1.0x
A lower-cost open-weight model	0.2x	1.4x	0.6x
A second open-weight model	0.4x	4.0x	0.9x
A third open-weight model	0.5x	5.0x	0.9x

The model with the most expensive tokens found the most verified severity and had the lowest cost per graded major or critical finding.

The cheapest model did win one measure: each finding that matched our production review cost half as much as terra. That measure treats a nit and a data-integrity bug as equal, however. The same model produced one fifth of terra's verified severity and reached six tenths of its coverage. Its low cost per match reflects the kind of output it produced, not just efficiency.

Even here, the rate card overstated the saving. Its tokens cost 0.16 times terra's, while a completed review cost 0.4 times as much because the model consumed 2.3 times more.

Reliability: the failure mode you find last

Cost and finding quality were not the only differences. Giving each model the full job exposed failures that a rate card or single-call benchmark would miss.

Our reviewer first finds issues, then reconciles them across rounds. Reconciliation means checking every previous finding against the new code, preserving its identity, and producing a structure that passes validation before publication. When each model handled both roles, two of the four models failed to produce that structure reliably, which killed their runs. With one shared reconciler across every arm, we saw no such failures.

One model failed 6 of its 10 runs and also reported the most severity. Its own labels made it look like the strongest reviewer. The independent grade did not support that claim: only a third of its severity labels held up, the worst agreement in the study, and its 73 severity-bearing findings fell to 17. Because it completed the fewest rounds, its aggregate results also rest on less evidence.

Only one model besides terra completed the supervisor role without a failure. It was the cheapest model at 0.4 times terra's cost, but it settled 8 of its 10 runs by deciding there was nothing left to say and approving the change. Its findings also held up worst under the independent grader. It finished reliably because it attempted less of the job.

Repeatability added another warning. Across both conditions, terra reproduced 71% of its findings on the same code. Another model reproduced 24%. A good result is much less useful when the next run finds something different.

What this changes about how you pick a model

Our results suggest three practical checks for teams choosing a model.

Measure the cost of the task. Run your workload through the harness you will use in production and inspect the bill. The useful number is specific to that model and harness. If the option with cheaper tokens costs 2.8 times more per completed task, the higher task cost is what you will keep paying.

Use an independent grader. Models differed in how much they inflated their own work, from almost no inflation to nearly three quarters of a severity level. There is no constant adjustment that makes those self-reported scores comparable. Use the same independent check for every candidate.

Test roles separately. Finding issues and reconciling them require different capabilities. A model can do the first well and still fail the second. Hold any component that can kill the whole run constant while you compare the rest. In our case, pinning the reconciler let us use specialist models without making the loop unreliable.

The objection worth taking seriously

Newer frontier models may follow instructions about effort and verbosity more closely than older ones. Our prompts do scope the review, so terra's lower turn count probably reflects instruction-following as well as raw efficiency.

That distinction does not change the bill. If one model needs 156 turns because it follows the scope less closely while another needs 42, you still pay for 156. Efficiency belongs to the combination of model and harness, not to the model in isolation. It is also why a model upgrade can move skill performance in either direction. Measure the pairing you intend to run.

Questions we got asked about cost per outcome

Does this only apply to code review? No. Any agent that runs a loop and decides when it is finished controls its own token volume. On those workloads, a rate card prices the input rather than the result. For single-shot classification or extraction, where consumption is bounded, token price remains a reasonable proxy.

Will the next model release change all of this? It will change the numbers, but not the method. Consumption depends on the model and harness together, so each release needs a fresh measurement. A model that was efficient on your workload last quarter may not be efficient today.

What if I cannot build an independent grader? The grader can be small. Ours sees one finding and the relevant diff hunk, without knowing which model produced it. What matters is consistency and independence: every candidate gets the same grader, and no model scores itself.

What would your own numbers say?

What bothered us was how rigorous the wrong comparison looked. It used a price list, a quality claim, and a sensible weighting between them. None of that was careless. It still favored a model that cost more to produce a worse answer.

What is your cost per verified outcome, and would it change the model you chose?

Before your next model swap, pull your own numbers. Turn counts are in your logs, token volume is in your billing, and the third term takes one independent grade over a sample of output. Run them through the formula. If the ranking changes, that is the number to buy on.

Tessl Code Review is free to try, and it records turns and cost per run for you.

The Rise of the Harness Engineer

Tessl — Wed, 26 Aug 2026 06:35:31 +0000

We're in an intense moment as software engineers. On one hand, coding agents have become genuinely amazing and we’re using them all the time with higher and higher throughput. On the other, they build in sloppy, bug-accumulating ways, and we’re under unrelenting pressure to keep them in check.

I'm a researcher and engineer at Tessl, where I've spent a lot of the last while helping build our internal software factory and evaluating what agents actually do when you point them at real work. The conclusion I keep arriving at is this: as agents write more and more of our code, engineering doesn't matter less. It matters more. It just looks different from the engineering many of us grew up doing. I've started calling the person who does this new kind of work a harness engineer, and I think it's a role a lot of us are about to grow into, some as a full-time specialism, many more as a growing slice of the job alongside product engineering.

Let me walk through how I got there, and what new skills are emerging as crucial to staying in control.

The agents got great, but cracks show over time

Start with the good news, because it's genuinely remarkable. The adoption curve for AI coding tools has gone close to vertical. A year ago adoption was spiky and experimental; now, across a lot of organisations, the vast majority of code is AI-authored [see Faros AI, AI whiplash report]. But, hand-in-hand, the number of bugs and production incidents are increasing with it, as more PRs slip through with no human code review.

The benchmarks tell the same story from the other side. Task based benchmarks are largely saturated – if you can define a task cleanly and measure if it was done, an agent can complete it. But if you ask agents to refactor large codebases, or build on their work over time, then we see deterioration, and much lower levels of success.

Better agents will help over time, but also come at a cost

The natural hope is that better models will simply dissolve all of this. But there will be heavy trade offs – much of the issue is reasoning about overall design decisions and reviewing existing code to follow patterns and stay consistent – all of which take time and reasoning effort to do. Expecting agents to do this well from first principals in every session would be incredibly expensive even if they can do it.

If you plot how much it costs to run an agent against its intelligence, the intelligence axis moves roughly linearly while price moves on a log scale. [see Artificial Analysis’s pareto frontier charts] Across the range that's something like a 100× spread, and the difference between a top-tier model and a perfectly capable cheaper one can be around 10× for the same task. Even when the very best model can do the job, that can be an extraordinarily expensive way to solve a problem. And when you start running agents at real throughput, this stops being academic. We closed something like 600 PRs in a single week recently, and we're a small company. When that much is flowing through, cost and efficiency become first-class engineering concerns.

The steering gap

We think a lot about skills at Tessl, so we built a benchmark to understand them properly. We took around a thousand skills from open-source repos, built tasks that would exercise those skills, and then ran agents through the tasks both with and without the skill so we could compare. We graded two different things: did the agent complete the task, and did it actually follow the instructions the skill laid out, not just "did it finish," but "did it do it the way it was told." [see: https://arxiv.org/abs/2606.17819]

What did we learn?

First, well defined skills reduce the difficulty of tasks and let cheaper, smaller agents reach the same level of success as much larger, more expensive ones. If we can make it easy to find the right guidance and break tasks apart well, we’ll win in both consistency and efficiency.

Second, and less comfortably: instructions don't get followed thoroughly. Task completion rates were high across almost everything we studied, but only about 70% of the instructions in a skill were followed on average. When you break a skill down into its individual instructions and check them one by one, even the best models are quietly ignoring a big chunk of what you told them. Skills aren’t enough, they need to be paired with checks that the rules are followed consistently.

There’s an enormous amount of value in designing how we use agents, rather than just using them as they are and hoping they make the right choices.

Engineering matters more, not less

Here’s the shift. When we hand-wrote code, we could review and discuss it, and many of the key decisions we were making could emerge and be shared implicitly as we worked. We’d know to go sketch an architecture diagram on a board when it stopped scaling the way we wanted, or we’d spike on which library to choose at the moment we realised it was important, and we could feed those choices back in. It was easier to mix taking time on the overall approach and the specific task.

Now, the agents are going so fast, it’s easy to fall into only being able to react – being pinged to review the latest PR when it’s ready, and feeling frustrated with the same errors being repeated.

If we want to go really fast, we need to be proactively spending our time designing the system in which the agents run, making our choices explicit and measuring whether the system that is emerging is what we want.

I see three skill sets that matter more now than they used to, and that are a little different from the ones many of us spent years developing.

1. Systems thinking: capturing invariants

Some of what you want from a change is specific to the task in front of you. But a lot of it is really an expression of a general principle you want to stay true across the whole system. If you can identify those invariants, the things you want to keep being true, and then capture and enforce them, you have a real shot at keeping consistency and holding the quality bar high no matter who, or what, is doing the writing.

What do invariants look like in practice? Design systems are a great example, you describe the types of layouts you want, the brand voice, the components, what a button looks like. Individual pages are then built from those reusable components. They show up in architecture, how you want code structured, which libraries you've chosen, opinionated views on how they should be called. Each new feature follows the patterns and choices consistently. And they show up in best practices, how errors should surface through the stack, how things should be named, how data should flow.

In the past, these lived in a code owner's instinct; you could ask them to sketch an architecture diagram, or sit with a designer over the Figma files, and then have them check PR by PR whether things matched.

Now we want to make these choices explicit, and when we do we can enforce them consistently much more cheaply than we could before, effectively steering development from these patterns. Why is it easier? Because just as agents are better at writing code, they are also better at reviewing code – especially when the rules are clear and thoroughly described.

How do we enforce them:

We can write skills that describe exactly what we care about and load them at the right moment.
We can put deterministic checks into CI, linters, and tools like ast-grep, which I'd barely heard of a few months ago and now talk about constantly. (Agents are excellent at writing these, so the thing that used to be a pain is now easy.)
We can add narrow verifiers, tight rules measured on every single file by a quick agent, to enforce the kinds of constraints that are hard to pin down deterministically.
And we can do agentic code review with broader prompts describing the principals we care about.

Between them these form a system of guidance and guardrails that steer the agents when they are working, and review if they did things correctly. They leave a lot less on the table to be caught in ad hoc review, or to slip through and cause problems later.

2. Analytics: reason about the system with data

This one surprised me, because I didn't used to think of analytics as an engineering skill. But as we automate more of the code-generation and review process, we generate more and more data we can actually look at, and there's a wealth of signal in it. This is what enables us to find invariants and measure what’s important.

New signals that we can study:

Agent logs are fascinating. They show where agents are spinning and getting confused. You can see where it wastes time. For example, are agents struggling with a particular API, are they reading files they should, are they puzzled about what we want?
PR comments are another signal. Do comments generalise into invariants that could be applied widely? What types of errors are being caught?
Analysis of the code base itself. Is the complexity of the codebase worsening with spaghetti code importing from everything and god files growing to thousands of lines? Does mutation testing find tests that aren't earning their keep, and gaps in coverage that matter? Are there multiple ways of doing the same thing that could be converged?

Then you turn all of that into hypotheses about where agents are going wrong, and then look for how you could address that – giving the agents better tools, refactoring code, adding skills, defining new invariants and enforcing them so you never have to worry about that failure mode again.

Sometimes that's a human grabbing the data and staring at it; sometimes it's an automated loop where an agent regularly runs the analysis and finds incremental improvements. Either way, it’s a seed for discovering how well the system is working and where improvements could be best made.

3. Risk and operations: grade the blast radius

As we build trust in review mechanisms, then we need to reduce the human review burden. We need to reason about the risk we’re introducing by relying on our processes and how that aligns with the sensitivity of the code base.

At Tessl we use a kind of ladder. Our research codebase is a genuine free-for-all, merge anything you like, no rules. Some of our internal tooling is auto-merged; hundreds of changes flow through, with purely automated review, and nobody looks at any of them. Other parts of the codebase absolutely need an engineer, the person who owns the feature has to take it through and do the final merge themselves. And a few very high-leverage parts should make you stop and go ask someone who's thought hard about them, rather than ramming a change in.

The work is figuring out which parts of the codebase sit at which level of risk, and then codifying that into the approvals flow.

The harness engineer

Put those three together, invariants, analytics, risk, and you get harness engineering. For a lot of organisations this will live in a few specialists who really embody it. But I suspect that for many of us working in engineering, it simply becomes more and more of where we spend our time. That's a real shift in identity: we become tenders and gardeners of our code and our systems, so that when we build features we can focus on whether the behaviour is what we want and trust that the implementation will be consistent and scalable.

Spending your hours on invariants and analytics and risk policy, rather than only on the next feature, feels different, but it has tremendous leverage.

We've leaned all the way into this. Around 90% of our code went through the factory this week, and we've been over 80% for the last month. The shape is consistent: more of the coding moves into remote execution loaded up with skills and context; a layer of guardrails (deterministic checks, LLM verifiers, CI) run through a detailed checklist of rules; automated code review handles new problems and feature specific issues; a triage/gating step reasons about how much risk and care a given change needs; and update loops monitor what's happening and feed learnings back into every layer of the stack.

Where to start

If this isn't your full-time job yet, here's where I'd begin, three small exercises you can try this week.

Build a dataset. Agents are great at this, tell one to grab your last 50 PRs and summarise what kept going wrong: what people repeatedly said in comments, what errors recurred, where things tripped CI. Identify issues that generalise and might come up again.
Distill three invariants for code you work on. Work out what rule would clarify the behaviour you want - in architecture, design, behaviour, or library usage for example.
Turn one invariant into a CI gate, and then make sure you have a way to track what happens with it and watch it. Did it start catching the problem you cared about? How often is it firing? Try implementing it using AST-grep (if it can be expressed deterministically) or try out tessl verifiers (if you want something broader).

For any of these, we’ve loaded guidance into tessl agent, so you can ask it to help you work out where to start and see if you agree with it. Install tessl and run tessl agent to get going. The free tier should be plenty for building out your first invariants.

Here are some examples of rules we have implemented at Tessl to help you get started.

Agents gave us rocket boots. Harness engineering, systems thinking, analytics, and risk, is how we keep from flying straight into a wall. And I think it's about to be some of the most valuable engineering work any of us can do.

Vibe Coding Is Not The Best We Can Do

Tessl — Tue, 25 Aug 2026 07:08:40 +0000

When I talked about vibe coding at AI Native DevCon London, I was not trying to dismiss AI coding assistants. I use them. I think they are a major change in the way software will be produced. But I wanted to ask a harder question: if natural language and agents are becoming part of programming, what still has to remain durable?

My talk was called "Vibe Coding: Is this really the best we can do?" The point was not nostalgia for hand-written code. The point was that software engineering has always depended on specification, verification, feedback, and incremental learning. If AI speeds up code generation but weakens those practices, we have not made engineering better. We have moved the bottleneck somewhere more dangerous.

There are claims around AI programming that are partly true and still risky. Natural language programming has a place. AI agents can write a lot of code. AI-generated tests can sometimes help with characterization or refactoring. But if those ideas are treated as replacements for precise intent and independent verification, they become a bad trade.

Use This Talk As Agent Context

Tessl has turned my AI Native DevCon talk into a skill your agent can use as context. You can also watch the full recording.

Register to get the early birds discount

What Are Tests For?

I started with tests because that is where I usually start. I am an old-school, card-carrying test-driven development developer, and I think tests are often misunderstood in AI coding conversations.

Tests are not mainly there to prove success. They are not there to challenge my genius as a programmer. They are a form of measurement. They are like the carpenter's tape measure: the thing that lets us check whether we are achieving the goal we intended to achieve.

That distinction matters because we cannot infer the goal from the solution. The solution can always be wrong. If I write a function called calculateTax and it returns 50 times the amount I put in, an AI system that only looks at the existing code can generate a test that preserves that behavior. That may be useful if I want to refactor without changing behavior. It is not useful if I want to know whether the tax calculation is correct.

That is the problem with treating AI-generated tests as a substitute for specification. If the code is the only input, the test can only verify that the code continues to do what the code already does. It may reinforce the wrongness. It may give us confidence in precisely the thing we should be questioning.

So yes, AI-generated tests have a place. But they are not the same as tests that express what we wanted before the solution existed. Specifying the goal is part of the job. If we skip that work, the tool cannot recover it for us.

What Is A Program For?

The next question I asked was what a program is for. It is tempting to say that a program exists to define a sequence of instructions, encode algorithms, or implement a design. Those are things a program does, but they are not the whole value.

A programming language has three important goals. First, it helps us organize our thinking about a problem. It gives us a way to explore the surface area of a problem with more precision than informal conversation usually allows.

Second, it communicates our understanding to other humans. A program is not only for the machine. It is also a technical explanation of what we understood, which tradeoffs we chose, and how the solution is supposed to behave.

Third, it tells the computer what to do. That is important, of course, but it is not the only purpose. Assembly language can tell the computer what to do. Most of us do not want to work there because higher-level languages help us think and communicate more effectively.

Programming languages also give us three techniques that matter deeply: a relatively simple and consistent grammar, a precise expression of intent, and repeatable deterministic execution. If we run the same program twice with the same inputs, we expect the same result. That is what makes reasoning, testing, and change possible.

Where Does Natural Language Fall Short?

Natural language is powerful, but it is not a drop-in replacement for programming languages. It is too vague, too open to interpretation, and too hard to execute deterministically.

We have all seen this in ordinary software work. Someone gives a vague instruction from above, a team builds what they understood, and everyone is disappointed because the goal was never precise enough. Natural language can communicate intent, but it often leaves too much room for interpretation.

That is why I do not like the simple claim that English is the programming language of the future. If we only chat with a computer about what we want, we should expect ambiguity. We may get something plausible, but it may not be repeatable. If we version-control the prompt, that does not necessarily mean we can reproduce the same result.

This does not mean we have to inspect every line of generated code forever. I am not arguing that. I have reached the point where AI writes a lot of code for me. But I am more precise than ordinary conversation when I specify what I want. I use a more prescriptive form of natural language, closer to a problem-specific specification than a casual prompt.

Vibe coding alone is not enough for serious systems. We need a way to specify intent more precisely, communicate that intent to humans, and verify that the machine produced the behavior we asked for.

What Problems Does AI Programming Create?

I framed AI programming around three problems.

The first problem is specification: how do we say what we want with enough precision? The second is verification: how do we confirm that we got what we wanted? The third is incrementalism: how do we keep learning and changing in small, controlled steps when agents can produce large changes very quickly?

That third problem matters more than it first appears. Human engineering is constrained by what fits in our heads. We learn by making small changes, evaluating those changes, and adjusting our understanding. Good software development is an incremental process of learning and discovery. I think that is true of engineering more broadly.

AI agents work differently. They can build a large context and then make a large change. If we have not specified the goal and cannot verify the result, the agent may produce a lot of code without helping us learn. Worse, it may regenerate or reshape the system in ways that make incremental understanding harder.

I mentioned someone using multiple agents who estimated that they were producing 12,000 lines of code per day. No human can review 12,000 lines of code per day in a meaningful way. No human can manually test that volume of behavior and reliably decide whether it is correct.

So the bottleneck is not code generation. We have sped up the coding bit, which was often not the hardest part. The bottleneck is specification, verification, and release. If we do not speed up assurance to match generation, we have only moved the constraint.

Executable Specifications Are The Program

The future I described is one where the program moves closer to a precise description of what we want, encoded as executable specifications, with AI translating those specifications into implementation.

In the past, a program was usually a precise solution encoded as algorithms. The problem we were trying to solve was often implicit in the solution and in the heads of the people who built it. In an AI-assisted workflow, that is not enough. The desired behavior needs to become explicit.

That is why behavior-driven development and acceptance testing are so relevant here. A BDD-style specification can describe the behavior we want in a form that is readable by humans and executable by machines. It can act as both the specification and the verification mechanism.

The programmer's job shifts. We design a problem-specific language for the behavior we want. We define the outcomes, edge cases, and non-functional requirements: performance, security, reliability, and any other architecture or design property that matters for this system.

Those details cannot be left for the AI to infer. The right level of security for a single-player game is not the same as the right level of security for a bank. The desired behavior depends on context, and context has to be specified.

In this model, AI generates solutions that satisfy the executable specifications. We can then run those specifications against the result and use test values the model has not seen, so it cannot simply game the visible examples. The point is not that AI disappears from the process. The point is that AI works inside a stronger specification and verification loop.

How Do We Keep Incrementalism?

The answer to the third problem is not new. It is continuous delivery, applied to an AI-native workflow.

In the past, good teams worked iteratively in small steps. They added tests and code, gathered feedback from deployment pipelines, treated each change as an experiment, and valued empirical learning.

In the future, that does not go away. We still work iteratively in small steps. The AI may generate the code, but we still gather feedback, validate every change, build systems incrementally, and treat changes as experiments.

That means version-controlling the specifications, because the specifications are now effectively part of the program. It means running the tests ourselves in a deployment pipeline rather than trusting the AI to tell us it passed. It may also mean keeping some tests hidden from the AI so it cannot shape the implementation only around the examples it has seen.

This is not a rejection of AI. It is the condition that makes AI useful for complex systems. If we can specify what we want precisely, verify that we got it, and preserve incremental learning, then the generated code becomes less important than the behavior we can prove.

That is why I compared this shift to the move from assembly language to higher-level languages. At first, programmers inspected the assembly generated by compilers. Most people do not do that anymore. They trust the compiler because they have other ways to reason about the result.

AI assistants may become similar. We may care less about the code they generate line by line, but only if we have a reliable way to verify the behavior we wanted. Without that, vibe coding is not the best we can do. It is just a faster way to create uncertainty.

I gave the full version of this argument at AI Native DevCon London. To go deeper, watch the full recording.

New in Tessl Academy: Code Review Loops

Tessl — Mon, 24 Aug 2026 04:38:05 +0000

A hands-on course on agentic code review: running your first review, writing a lens that encodes one of your team's own rules, routing lenses by path, and closing the loop on every pull request. Four lessons, about 25 minutes each. Read them on the site or have your agent walk you through them.

Reviewing code is the part of the job that scales worst. It needs someone who knows the codebase, has time, and is willing to read carefully. When agents are writing most of the changes, that person runs out long before the changes do.

Simon covered Tessl Code Review yesterday: what it is, why we built it against whole pull requests rather than diffs, and how re-review keeps track of what a previous round already settled. This post covers the next question, the one you hit about an hour after installing it: your reviewer doesn't know your team's rules yet, and the useful version of it does.

That's Code Review Loops.

npx tessl install tessl-academy/code-review-loops
tessl launch skill --agent tessl-agent -i 01-your-first-code-review

What's in it

Four lessons, in order, each building on the repository state the last one left behind.

Your first code review has you scaffold a small TypeScript service and a branch with three deliberate faults in it, then run a review over that branch three different ways and read what comes back — the outcome, the severities, the JSON. It also settles the naming collision that catches nearly everyone: tessl review run scores a skill, and tessl code review reviews your code. Both commands exist, they sound alike, and picking the wrong one costs a confusing five minutes.

The lesson plants three faults; the reviewer finds two. That's deliberate — the miss is why lesson two exists.

Writing a review lens starts from that missing finding. A lens is a skill: a SKILL.md with a name, a description of when it applies, and a body of review instructions. The four defaults ship in the tessl/code-review plugin and their sources are public, so you can fork one and see how it's put together. You write a lens that encodes a convention only your team keeps, run it beside the defaults, then test it in both directions so it fires on the bad case and stays quiet on the good one. That second half is the part people skip — it's the difference between a lens and a noise generator.

Routing lenses by path moves the configuration off the command line into a YAML profile in the repository, with globs and exclusions so each lens only runs where its rule applies. A review that skipped your files isn't a review that approved them.

Review on every pull request is the capstone: token, caller workflow, Action pinned to a commit SHA, advisory mode first and a gate later. Then a second round with one fix and one reply, so you watch earlier findings resolve as addressed, explained, or declined rather than getting raised again.

The bits you'd otherwise learn the hard way

-skill replaces the profile's default lenses rather than adding to them. Pass two and you get two, not six.
A review takes at most eight lenses, and their order is preserved.
Tessl doesn't go looking for your profile file. A bare tessl code review in a repo containing .tessl-code-review.yml runs the built-in standard profile and ignores your file. Nothing starts judging your code differently just because a file appeared.
In CI, review policy comes from the default branch, not the branch under review — a pull request can change any file in the repository, including the one that decides how that pull request gets reviewed.
@tessl-code-review is text your workflow matches, not an account.

Lesson three needs a CLI newer than 0.96.0, since YAML profiles landed after that. Lessons one, two, and four work fine on older versions.

How the course got built

The Academy is built in a way that's slightly unusual, and the code review course is the clearest example of it so far.

There's no pipeline that compiles curriculum. Lessons are assembled by an agent running a skill, and those skills are what we maintain.

Content lives as versioned components: a concept explained in under 300 words, a hands-on exercise, a starting repository state, a scoring rubric. There are 65 of these across six types, reused across lessons and courses. A human writes the recipe deciding which components a lesson uses and in what order. An agent running compose-lesson turns that recipe into the lesson you read, and generate-lesson-skill turns the lesson into the installable walkthrough that tutors you through it. The only deterministic step is the last one, where a build renders finished markdown into HTML. The site itself is static and knows nothing; the judgement sits upstream in the skills.

Each composed lesson records the component versions it was built from. When the CLI changes underneath us, a maintenance sweep works out which lessons are affected instead of us guessing, and writes a machine-readable verdict per concept. A lesson only counts as verified when every concept in it has a current pass.

That verification has a limit worth stating: a pass means the concept matches the docs, not that anyone ran the command. A wrong doc produces a confidently passing wrong lesson. We check for CLI drift and nothing else yet, and the sweep only runs when a human starts it.

Every skill in the repository also has to clear tessl review run at 80% before it ships. We teach the command, so the tooling that builds the teaching gets reviewed by it — writing a course about review with review switched off would have been a bad look, and more usefully, it caught real problems in our own lens descriptions.

Try it

The course is at tessl.io/academy/code-review. Read it there, or install it and ask your agent to guide me through a first code review and work through it in your own repository.

If you get through it, tell us about your lens — what convention you encoded, and whether the reviewer caught it. That's the feedback that shapes where this goes next. Find us in Discord.

Review Agent-Written Code Against Your Team's Standards

Tessl — Fri, 21 Aug 2026 06:27:13 +0000

Agents can write code faster than any team can review it, and the reviewer that keeps up has to hold your team's bar rather than a generic one. That standard can't sit in a settings screen inside someone else's product. It has to be yours, and you'll keep changing it as you learn how your agents behave.

So we built Tessl Code Review on your team's own standards. You write them once, as a skill you own, version, and can point at another repo tomorrow. From there it reads the whole PR rather than just the diff, and it knows what's still open by the next round.

It’s very easy to use:

tessl code review --pr

With no flags it reviews the current PR. If you give it no PR, it reviews your local uncommitted changes, or hand it a git range. It publishes straight into GitHub, and it runs with whatever agent your team already uses. Tessl Agent is one option, but you can choose the agent if you have a preference.

Built for a world where agents write the code

Most review tools were designed to check human-written code and had AI added later. Tessl Code Review started from the assumption that agents write most of the code.

That changes what the tool needs to be. When agents are producing the volume, the standard you review against can't sit as a setting inside someone else's product, tuned by a vendor and hidden from you. It has to be something your team owns and can change, because you'll be changing it often as you learn how your agents behave. The rest of the product follows from that.

How does it work?

You write your team's review standards once, as a skill, and Tessl applies it to every pull request. We will read the whole PR rather than just the diff. This includes the surrounding codebase, your standards, and everything already said in the thread. And within a pull request, each re-review knows what's fixed, what's explained, what's declined, and what's still open, so you're never arguing a point you settled two rounds ago.

A review that you own

Our bet is different to other code review tools. Tessl Code Review is a review you own, that fits your team, that you can inspect and test, and that can become a trustworthy part of an autonomous pipeline instead of one more opinion shouting from the sidelines. It's a first step toward a software factory rather than the whole thing, but a foundational one. Trusted review is what eventually lets a change reach a settled end state, act as a blocking gate, and merge on its own without a human on every diff.

Why you should consider using Tessl Code Review

Your standards live in a skill you own, not a config screen

You write your review standards once, as a skill. This is simply a plain, readable artifact your team controls, versions, and can reuse elsewhere in your factory. Most tools keep your preferences in a settings page you can't inspect or take with you. A skill is something you can read, edit, version, and point at another repo tomorrow.

Reviews you can inspect

Every review is grounded in your codebase, your team's conventions, and the pull request thread, and you can see exactly what shaped it. You're not sending code into a vendor's system and waiting for a verdict you can't unpack. When the reviewer flags something, you can trace why. That grounding is also the real cure for noise. Irrelevant comments go away when the review understands your standards and your codebase, not when you nudge a sensitivity slider.

Prove your standards work, then govern them

Most teams have no way to check that their review standard is catching what it should, or holding steady as it changes. Since your criteria live in Tessl as a skill, you can run Tessl's own review and eval tooling against them and confirm they meet your org's bar, rather than assuming they do. Your standard becomes something you can test instead of something you hope is working.

One place to manage standards across every repo

Each repo can run its own review skill, tuned to that team. As you add repos, the skill registry is where you see and manage all of them, so your review configuration doesn't turn into a pile of scattered rules as you scale. Review standards become a first-class, versioned thing the whole org can see.

Re-review that remembers the conversation

Update a pull request and the next review already knows what was fixed, explained, or declined, and looks only at what's still open. No repeated findings, no re-arguing settled points, and none of the usual mess of a comment that disappears from the diff the moment the code changes and leaves you juggling tabs to follow one resolved thread.

Under the hood

There are four steps. It builds the context: the diff against main, the PR title and summary, the comment threads and how they were resolved, and the codebase. It runs each lens over the change to produce findings. A lens is just one of your review skills, and each one brings a different perspective: an architectural lens, a code quality lens, a security lens. A stateful supervisor then takes everything that came before, together with the new findings, and works out which are valid, which aren't, and which have already been handled. Finally it publishes inline comments and a summary on the PR, and replies in the existing threads.

We're using it at Tessl, and loving it!

While Tessl Code Review is only recently publicly available in Beta, we've been running it internally across our own monorepo as our daily reviewer, on real changes, for several months. We're still tuning the edges: a full-context review takes a little longer than a shallow one that only glances at the last few commits, and there's always more nit-tuning to do.

We monitor the precision of everything the reviewer flags and how much is a real defect. We took every finding it raised on our own pull requests and had a second, more capable model judge each one against the code on whether it was a real defect, or not. 74% were confirmed as real defects, compared to ~50% we saw in comparable tools.

Try it

Pick whichever fits how you work.

Let your agent set it up. Install the setup skill, then hand it to the agent you already use:

tessl install tessl/code-review-setup

Ask your agent to set up Tessl Code Review in your repository. It reads your existing workflows, asks two questions, when reviews run and whether findings block the merge, then writes the caller workflow and explains the permissions and secret it needs.

Or ask Tessl Agent. If you're running Tessl Agent, ask it to set up code review and it does the same job without you installing anything first.

Then shape the review to your team. Fork the default lenses into something that sounds like your team, or install the lens creator and write your own:

tessl install tessl/code-review-lens-creator

It settles the review question and the bar a finding has to clear, drafts the lens, then backtests it against pull requests that already carry review feedback, so you can see what it catches before you turn it on.

Prefer to do it yourself? Install the Tessl CLI and follow the quickstart.

If you'd rather see it first, watch a single change go through two rounds: a first review, a fix, and a re-review that knows exactly what's fixed, explained, declined, and still open.

Review is one of the most important parts of a software factory, because it's what lets you trust the rest of it to run. This is our first step toward making it one you own, and eventually one you can let run on its own.

Humans Architect The System, AI Writes The Code

Tessl — Thu, 20 Aug 2026 06:12:01 +0000

I have not written a line of code at work since the end of January. That is not a claim about where every team should be tomorrow. It is the way we work, and it has changed what I spend my time on.

The distinction I wanted to make in my talk, "The Humans Architect the System, the AI Writes the Code," is between writing code and building the machine that writes code. In the AI era, that distinction matters. People call this software factories, dark factories, agentic workflows, or something else. The useful part is simpler: humans need to design the process that turns intent into reliable software.

At System Initiative, and in the new company we spun out of that work, this became very concrete. We spent years building a beautifully architected Rust system. At the end of January, we threw the code away because the product itself needed to be rethought for an AI-native world. Five of us stayed behind, started from a Miro board, and rebuilt the way we work around agents from the beginning.

That is the story I wanted to share. Not a recommendation that everybody should ban themselves from typing code, but a concrete example of what changes when you take the idea seriously.

Use this talk as agent context

Tessl has turned my AI DevCon talk into a skill your agent can use as context. You can also watch the full recording.

Why did I stop writing code?

The honest answer is that I had already fallen out of love with writing code line by line.

One of the moments I described was sitting on a call where people were discussing naming conventions for NATS queues. The people on the call cared deeply about getting it right. I understand why. Those details matter in a system. But for me, that was no longer the work I wanted to spend most of my energy on.

What I care about now is architecture, design, constraints, invariants, and the coherence of the product over time. If agents can do more of the implementation, then the human work moves up a level. The question becomes: what does this system have to mean, how should it behave, what must never be violated, and how do we make those expectations executable?

That is why I said the vibes do not scale. A one-line prompt may get you somewhere on a small task. It may even feel impressive. But if there is no explicit idea of what the system is supposed to become, the output will not stay coherent. You can spend a lot of tokens and still end up with a product that has no architectural spine.

We do not accept human-written pull requests

Our rule is blunt: agents write every line of code.

That is not a preference or a guideline. If someone opens a pull request written by a human, we do not merge it. The reason is not that human-written code is bad. The reason is that we are trying to protect the integrity of the system we are building. If the system depends on agents producing the implementation, then the process, constraints, checks, and feedback loops have to be designed around that fact.

The same rule affects open source contribution. We do not accept external pull requests with code. We do accept issues, feature requests, ideas, design proposals, and evidence that something is wrong. That might sound harsh, but in an AI-generated-code world, open source maintainers are already dealing with huge volumes of plausible patches. Some are helpful. Some are careless. Some may be supply-chain attacks.

If I cannot reliably distinguish a well-formed agent-generated contribution from a hostile one, then I do not want that attack surface in the code path. The way to contribute is to help us improve the product and the specification of the work. The agents can then implement it inside the process we control.

The process is a state machine

The important part of the system is not that an agent writes code. The important part is the lifecycle around the work.

Our flow starts with an issue or feature request. The issue is triaged and classified. Then the system generates a plan. That plan goes through an adversarial review loop. One agent proposes the plan, while another challenges it, looking for security problems, architectural holes, missing requirements, and places where the plan does not match the constraints of the system.

That loop is bounded. We allow up to five rounds. If the agents cannot agree, a human becomes the arbiter. The human is not dropped into a vague mess. The disagreements are captured as structured data, so we can ask what the agents agree on, what they disagree on, and why the plan is blocked.

Once the plan is good enough, implementation starts. If it is a bug, the system tries to reproduce and verify it. If it is a feature, the path continues into a pull request. From there, the work moves through review gates, UAT, release, and notification back to the person who requested it.

This is not a pile of shell scripts. It is a state machine inside a skill, backed by a CLI. That makes it maintainable and extendable. The agent is not just improvising. It is moving through a designed workflow.

CLAUDE.md is the executable contract

For us, CLAUDE.md is the center of gravity. It is not passive documentation. It is an executable contract with the agent.

The file contains the constraints that matter in our codebase. TypeScript must be strict. We do not use any. We use named exports. Files carry the right AGPL copyright. We do not fire and forget promises. Endpoints return JSON. Implementation details do not leak into public surfaces.

Those may look like coding standards, but in an agentic workflow they become much more important. They are the rules the agent reads before it acts. If they are vague, stale, or incomplete, the system behaves inconsistently. If they are precise and maintained, they let the team improve the generator instead of repeatedly fixing the output by hand.

The last line matters too. If the agent hits a non-obvious problem that will trip up future sessions, it should record it and propose an update. That is how the system learns from failure. A mistake should become a better constraint, not just a one-off correction.

Tests and gates carry the trust

Trust in this workflow does not come from trusting the model. It comes from trusting the gates.

We run unit tests, integration tests, contract tests, property tests, and architectural tests. After a binary is built, we ship it into a different repository and run it as a user. That user acceptance testing is a release gate. A merged pull request does not automatically mean the code reaches an end user.

We also run adversarial tests. The system tries to inject input, kill processes, remove data, and do the kinds of things real users and broken environments do. Failures get fed back into the process.

Our CI has five merge gates: code review, adversarial review, user experience review, CI security review, and skill checks. The skill checks matter because our product is designed to be driven by agents. If the skill content, format, triggers, or agent experience regress, that is a real product regression.

When every gate passes, the pull request can merge itself. That is only credible because the release path is still constrained by evidence.

The numbers are a result of the loop

The numbers I showed were not there to prove that everyone should copy us exactly. They were there to show what a bounded process can make possible.

In the previous 30 days, 295 issues had been opened. We shipped 217 of them and closed 81 as duplicates or things we were not going to do. Median time to triage was 4.6 elapsed hours. From triage to ship, including the gates, the median was 1.6 elapsed hours.

There are five people in the company. Each of us has a Claude Max Pro subscription, and we spend roughly $1,500 to $2,000 a month on CI review processes, around $3,000 a month in total for this way of working. For us, that is a very different cost profile from the way we used to build software.

But the bottleneck did not disappear. In the Q&A, I said the bottleneck is now deciding what to build. If I can spin up ten agents but point them at the wrong features, I can generate a very fast mess. The human work becomes intent, product judgement, and architecture.

Start with one constraint

If you want to experiment with this pattern, do not point an agent at the whole organization and hope for the best.

Start by turning conventions into constraints. Write down the thing your repository assumes but never says. Encode one piece of knowledge that only one person in the organization seems to know. Run one workflow from issue to plan to review to implementation to verification. Then look at where it breaks and add the next constraint.

That is the smallest useful loop. The point is not to build our whole process on day one. The point is to stop treating agent failures as isolated incidents and start improving the system that caused them.

I ended the talk with the idea that intent is the new architecture. That is where I think the human role is moving. Agents can write more of the code only when humans make the system's intent explicit enough to execute, review, test, and improve.

The full version of this argument was presented at AI DevCon London. To go deeper, watch the full recording.

3 things you can do with Tessl Agent in your first week

Tessl — Tue, 11 Aug 2026 06:49:02 +0000

Tessl Agent is a coding agent that makes your other coding agents better. It reads your PRs, agent session logs, and tickets, then opens pull requests that fix recurring mistakes, add review gates, and turn repetitive chores into automated workflows — so your codebase improves every week without anyone stopping to maintain it. It's in open beta and free to try, and it works alongside Claude Code, Codex, Cursor, or whatever your team already uses.

Teams adopting coding agents keep running into the same problem: the agent can write code, but keeping it consistently effective is unplanned work nobody has time for. Here are three things you can do on your first day, each one a single command.

1. Spend less time reviewing agent PRs

Agent-generated PRs still need a person to babysit them, and review is where a senior engineer's time quietly goes. The more you delegate to agents, the worse that gets.

Tessl Agent sets up a review gate that runs your verifiers and reviewer skills over every diff before a human sees it. It catches the correctness and style problems your team would otherwise flag by hand, and works out when a change genuinely needs a person to look. Run it locally, or wire it in as a GitHub Action so it fires on every PR.

Try it: run tessl agent\ and ask “Set up agentic code review.”

2. Fix the mistakes your agents keep repeating

Agents tend to make the same mistake over and over, PR after PR. Fixing the root cause means updating the context, rules, or skills the agent works from, and that work rarely makes it onto a sprint — so the mistake keeps coming back.

Tessl Agent reads your session logs and PR history and finds those patterns for you. Instead of correcting one more symptom, it opens a PR that fixes the context behind them, so the next agent to touch that code starts from a better place.

Try it: run tessl agent\ and ask “Find optimisations for my agents.”

3. Turn a repetitive chore into an automated workflow

Plenty of recurring chores could run themselves — keeping docs in sync, sweeping for weak tests — but they sit on the backlog because nobody has time to stop and systematise them. They're never urgent enough to prioritise, and never quite gone.

Tessl Agent picks out the tasks worth automating, builds the workflow, and ships it as a GitHub Action. Something you did by hand this week runs on its own the next, and the automations stack up as you go.

Try it: run tessl agent\ and ask “What tasks can I delegate to agents?”

Start in minutes

Each of these is one command from the root of your repo. Tessl Agent is provider-agnostic, so there's no lock-in and it works with the agents and models you already run. It's in open beta and free to try.

Get started: tessl.io/agent

Agents need real sandboxes

Tessl — Fri, 31 Jul 2026 07:13:08 +0000

At AI DevCon London I gave a talk with a slightly silly title: "You're Absolutely Right, It Was Your Home Directory!"

It got a few laughs because the scary version is easy to imagine. You give a coding agent a task, enable the convenient mode where it stops asking for permission every five seconds, and then it confidently does something wrong.

Like deleting your home directory. Hi, gpt-5.6 Sol 👋.

The less funny version is broader than deleting files. It is the agent sitting on the same machine as your source code, API tokens, SSH keys, browser state, local databases, internal docs, test credentials, and the rest of the developer-machine archaeology.

I work at Docker on developer tooling around AI, and one of the things we have been working on is Docker Sandboxes: isolated microVM environments for running coding agents locally. The talk was about local agent isolation: what agents should be able to do, what they should not be able to touch, and why prompt-level guardrails are not enough once the agent can act on your behalf.

Tessl also turned the AI DevCon talk into a skill your agent can use as context: talk-selajev-docker-sandboxes-agents. Or, if you prefer the old-fashioned way, make your agent watch the recording: https://www.youtube.com/watch?v=bFBNXIoLkW4.

Register to get the early birds discount

Autonomy changes the threat model

We want agents to do more work for us. Otherwise we would just keep autocomplete and call it a day.

But the more useful an agent becomes, the more responsibility it takes automatically. The blast radius grows with it, because "helpful" means "able to act."

Productivity is a spectrum. On one end you have autocomplete and chat assistants. They suggest code, you inspect it, and you decide what to paste, run, or commit. The human still performs most of the actions.

On the other end you have agents that get a goal and figure out the path. They inspect files. They run commands. They install packages. They start services. They retry when something fails. They read logs. They rewrite code. They may even coordinate with other agents.

This direction is useful, and it is where the old local-development threat model stops fitting.

More autonomy means more actions taken on your behalf. Fewer approval prompts means fewer chances for a human to notice that something is weird. Local execution means the agent is close to the files, tools, tokens, and environments developers use every day.

In the talk I referred back to Liran Tal's security session, because the same risk model applies. Things get dangerous when three ingredients meet:

private data;
untrusted content;
external communication.

Private data is your code, local files, credentials, API tokens, config, and company context. Untrusted content can come from prompts, issues, pull requests, emails, documents, repositories, or websites. External communication is the agent pushing, uploading, posting, calling APIs, or sending requests somewhere.

Once an agent can combine those three, a sentence in a prompt is not a security boundary.

The agent is not the boundary

A prompt can guide an agent. It cannot enforce filesystem access. It cannot enforce network access. It cannot keep a secret out of a process environment. It cannot prevent a tool from being called if the tool is available and the runtime allows it.

Instructions are useful. Enforcement has to live outside the model.

One part of the demo made this very visible. I gave an agent enough local skills to do dangerous things: edit CLAUDE.md, look for secrets, inspect SSH keys. Then I asked it to write a Python script that did the same thing.

It wrote the script. Then it refused to run it.

Fair enough. The guardrail had noticed the shape of the action. So I changed the shape: put the dangerous code into a module, then write a boring main.py that imports and uses the module.

The agent still built it. It still refused to run it.

Then I cleared the context and asked it to run the program.

It ran very, very fine.

The model did not become evil. It did not need to. The guardrail lived in the conversation, and the conversation changed. The environment still had the files, the tools, and the ability to execute the code.

"Please do not touch this" and "you cannot touch this" are different controls.

For an individual experimenting on a personal machine, maybe this is a risk they knowingly accept. For a company enabling coding agents across many developers, "the agent was told to behave" is not a control.

The agent should be able to work unbothered inside a boundary that does not depend on the agent obeying itself.

Containers are useful, but agents are weird workloads

The obvious first idea is: put the agent in a container.

Docker knows containers pretty well. Containers are a good answer for many software problems. They are excellent for packaging applications, running dependencies, and creating repeatable environments.

Agents are a bit different from packaged applications.

A normal container usually starts from something known. You know what went into the image. You can inspect the Dockerfile. You can produce an SBOM. You can reason about the thing you are about to run.

An agent changes the environment while it works. It installs tools. It writes scripts. It starts services. It edits the workspace. It may build and run more containers. It turns the environment into a temporary development machine.

Then there is the isolation boundary. Containers share the host kernel. That may be a completely acceptable tradeoff for many workflows. But when we talk to enterprise security teams about autonomous agents running near sensitive developer environments, containers alone are not the boundary they usually want.

That is where microVMs come in.

A microVM gives you a stronger isolation boundary while keeping the workflow close to what developers expect. The agent still gets a useful Linux environment. It can still build, test, install packages, and run tools. But the host is no longer one bad tool call away.

A sandbox still has to be useful

This part matters more than security people sometimes like to admit.

If using the sandbox is miserable, developers will bypass it.

A useful agent sandbox has to give the agent enough room to do real software work. It should be able to inspect the project it was given, run builds, execute tests, start dependent services, and use containers inside the isolated environment when the workflow needs that.

The interesting part is what it does not get by default.

It should not get arbitrary access to the host filesystem. The user should choose what is shared. Network requests should be observable and controllable. Secrets should not be copied into random files the model can read. If the agent damages the sandbox, you should be able to throw it away and create a fresh one.

That is the balance I want: useful enough that people keep using it, constrained enough that a weird agent run does not become a weird host-machine incident.

In the live demo I showed the command-line experience of dropping into a familiar agent interface inside the sandbox. Nothing about the UI needed to look exotic. The agent should still feel like the agent you already use, just running somewhere less precious than your host.

It can build, test, explore, and make a mess. It just makes the mess somewhere disposable.

The agent gets capability, not custody

Developer machines are full of real credentials. Sometimes intentionally. Sometimes historically. Sometimes because that one CLI setup guide from 2021 told you to put a token in a file and now nobody remembers it exists.

So the sandbox should not solve auth by copying secrets into the agent-visible workspace.

The pattern I described uses sentinel values and a security proxy. The agent can operate as if it has the capability it needs, but the real credential is injected outside the sandbox boundary when a request goes to an approved service.

The secret does not need to sit in a file the agent can read.

The useful mental model is: the agent gets capability, not custody.

The same idea can apply to larger trusted workflows. Commits, code signing, provenance metadata, and other sensitive operations can happen outside the sandbox or through controlled paths. The agent gets enough power to complete the task. The organization keeps the most sensitive material away from the model-visible environment.

That is how you make an agent useful without handing it everything.

Empty sandboxes do not survive contact with developers

Developers spend years shaping their machines. Compilers, package managers, CLIs, caches, credentials, dotfiles, project conventions, that one tool installed with curl | bash three jobs ago. A clean sandbox can feel like starting from nothing every time.

If that is the experience, people will run the agent outside the sandbox the moment they are under pressure.

One answer is to build a huge base image with everything in it. That works for a while. Then the image gets too large, too slow, too generic, and too annoying to maintain.

The approach I showed is sandbox kits.

A kit is a declarative way to configure a sandbox. It can define commands to run, files to place in the environment, processes to start, network policy, and secret-handling configuration. It can be local, or it can be shared as an OCI artifact.

If you have used devcontainer features, this should feel familiar: reusable environment configuration layered on top of a base. With agents, the kit also has to care about network access and secrets. Convenience is only half the job here.

In the demo I used a Testkube kit to show the idea. The larger point is that vendors and platform teams can provide kits for the tools developers actually need: testing platforms, cloud CLIs, language toolchains, data systems, internal services, whatever makes the sandbox feel like a real place to work.

There are public examples in docker/sbx-kits-contrib, and Docker's docs also have kit examples plus a guide for building an agent kit.

Sandboxing should fit into existing tools

A sandbox should not become a separate ritual.

Developers already live in terminals, editors, issue trackers, CLIs, and pull requests. If running an agent safely means leaving that flow and doing ceremony, people will do it once for the demo and then stop.

Command-line ergonomics matter. IDE integration matters. When a developer opens an agent pane in VS Code, IntelliJ IDEA, Zed, or another editor, the agent should be able to run inside the sandbox rather than directly on the host.

The experience should stay close to normal. The isolation boundary should change underneath.

Containers taught us this already. They became widely adopted because they eventually became practical, repeatable, and integrated into everyday development workflows. Agent sandboxes need the same ergonomics, with a stronger boundary and controls that match autonomous work.

Sandboxes limit blast radius, not every risk

Sandboxing gives you a boundary. I tried to be careful about this in the talk.

A sandbox helps with local-machine blast radius. It gives you a place to enforce filesystem access, network paths, credential handling, and disposable execution.

Application-level permissions still matter.

If you give an agent permission to read and write email, someone may still try to influence the agent through email content. If you give an agent broad access to a SaaS app, the SaaS permissions still matter. If an agent can open pull requests, merge policies still matter. If an agent can call production APIs, please do not pretend the sandbox has solved production governance.

Sandboxing should sit next to tool controls, product permissions, logging, approval policies, and security review.

The useful part is that the boundary moves out of the prompt and into infrastructure.

That was the argument I brought to AI DevCon London: local coding agents need real sandboxes because useful autonomy requires real boundaries.

If you run agents locally, try this exercise: ask what the agent can read, what it can change, what network it can reach, and where its credentials live.

If the answer is "my whole machine," you do not have an agent setup. You have a trust exercise.

Agent Benchmarks Need To Measure The Whole Workflow

Tessl — Thu, 30 Jul 2026 09:08:15 +0000

At AI Native DevCon London, I talked about benchmarking in the agent era because the workloads we care about have changed faster than the benchmarks we use to measure them.

At NVIDIA, we spend a lot of time thinking about inference performance. That means performance in the real world, not only performance on clean single-turn prompts. For a while, many benchmark conversations assumed a chatbot shape: one user input, one model response, a small context window, relatively short outputs, and no tools.

That is not what agentic coding workloads look like anymore.

In "Benchmarking the Agent Era: Measuring Performance Beyond the LLM," I wanted to separate model performance from system performance. The model still matters, but once agents start calling tools, reading files, building context, and looping through many turns, the benchmark has to measure the whole trajectory. You can also watch the full recording.

Register to get the early birds discount

Chatbot Benchmarks Do Not Match Agent Workloads

The old shape was simple. A user asked a question. The model returned an answer. Context length might be one to four thousand tokens. Output length was usually small. There were no tool calls.

In agentic workloads, the shape is different. A single user request can lead to dozens of turns. The model may be called again and again. Sequence lengths can grow from tens of thousands to hundreds of thousands of tokens. Output length can increase. Tools become part of the normal path.

That difference matters because the benchmark that scored well yesterday may not tell you much about the workload you are actually running today.

I used a coding-agent example in the talk: a user asks an agent to fix a flaky test in a codebase. The first model call does not solve the whole task. It decides which file to inspect. A tool reads the file. The result goes back to the model. The model decides where to look next. Another tool runs. That loop can continue for twenty, thirty, or forty turns depending on the complexity of the task.

That is not one inference call. It is a trajectory.

Agents Mix GPU Work And CPU Work

One detail that gets missed in simple benchmarks is where the work actually runs.

In the agentic trajectory, the model calls run on GPUs. The tool calls often run on CPUs. If a tool reads files, searches a repository, runs shell commands, or gathers information from another system, the GPU may be idle while the CPU-side work happens.

That changes capacity planning. If you assume every user keeps the GPU busy all the time, you may underestimate how many concurrent users the system can support. If tool calls create large CPU-side delays, you may have gaps where GPUs are available for other work.

Ignoring tools therefore gives you the wrong concurrency picture. The benchmark may say your hardware can support one number of users, while the real agentic workload could support more or less depending on the shape of the tool calls.

The point is not that tools are free. The point is that they change the system. A real benchmark has to include them.

Prefix Caching Becomes Bread And Butter

The first optimization I discussed was prefix caching.

In an agentic workflow, each new turn builds on the previous history. By the time the agent is deep into the task, much of the context has already been processed before. If the system can cache that work, it does not need to recompute the whole prefix every time.

That is critical for time to first token. If caching is working, the model only has to process the new part of the context. If caching is not working, each turn can become much more expensive because the system keeps reprocessing information it has already seen.

This is why agent benchmarks need to test caching behavior directly. A benchmark with single-turn fixed input and output shapes does not expose whether the serving system can handle the repeated, growing context that agents create.

For agentic coding workloads, prefix caching is not a small optimization. It is foundational.

Routing Has To Find The Cache

Caching is not enough if the next turn lands on the wrong replica.

In production, the same model may be served across multiple replicas. If turn one lands on replica A, that replica has the relevant cache. If turn two is routed round-robin to replica B, the cache may not be there. The system loses the benefit.

That is why KV cache-aware routing matters. The serving layer needs to send later turns to the place where the useful cache exists, or use another strategy that preserves the cache benefit.

There are different ways to do this. A system can use session IDs to keep related requests together. It can also use more explicitly cache-aware routing. The important thing is that the benchmark should reflect this issue. A benchmark that turns this off or ignores multi-turn trajectories misses one of the major performance properties of agentic workloads.

Speculative Decoding Changes Output Speed

The third optimization I discussed was speculative decoding.

The high-level idea is that a smaller draft model proposes several tokens ahead. The larger target model then verifies those tokens. If the draft model is sufficiently aligned with what the target model would have produced, the system can generate multiple tokens in one pass instead of producing them one by one.

This can create a meaningful speedup on the decode side, where the model is producing output. Newer models may include multi-token prediction components that make this easier to turn on.

But there is a measurement trap. Speculative decoding changes the distribution of token speeds. If you look at the wrong percentile, you may accidentally penalize a useful optimization. In the talk, I showed how one percentile view could make the optimized run look worse even though the broader distribution was better.

The lesson is that agent benchmarking has to look at distributions, not only averages.

Means And Medians Are Not Enough

In agentic systems, user experience often breaks in the tails.

If most requests are fast but a meaningful tail is slow, users will still feel the system as unreliable. For time to first token, left is good and right is bad. For token speed, the interpretation changes because slow generation is the problem. Either way, the mean can hide what users actually experience.

That is why I focused on percentile-based thinking. P95, P25, and similar distribution metrics can be more useful than mean or median, but only if you choose the percentile that matches the user experience and the optimization you are evaluating.

The benchmark should help you answer questions such as: how many users can this hardware sustain while meeting a time-to-first-token SLO? How fast are tokens generated for the important part of the distribution? What happens when caching is warm? What happens before it is warm?

Those questions are closer to production reality than "how fast is one model response?"

Measure Steady State, Not Only Startup

Agentic workloads also take time to stabilize.

At the beginning of a run, caches may be cold. Routing may not yet have enough useful history. The system may still be in a transient state. If you measure performance too early, you might conclude that a deployment fails its SLO even though it would pass once the workload stabilizes.

This is especially important for caching-heavy agent workloads. The performance curve can improve as the system warms up. So the measurement needs to show whether the metrics have flattened out over time.

If the benchmark only captures a transient window, it can lead teams to overprovision hardware or reject a deployment that would have worked.

A Better Benchmark Starts With Real Trajectories

I pointed to the Artificial Analysis agent benchmark as an example of the field moving in the right direction.

The important shift is that it tries to include the properties I discussed: multi-turn trajectories, cache behavior, speculative decoding, SLO-based measurement, and real-world workload shape. The question becomes: for a given hardware setup and a given SLO, how many users can the system sustain?

That is a much more useful question for production planning than a single model score.

It also opens the door to per-watt and per-dollar views, which matter when teams are making infrastructure decisions. Performance is not only speed. It is speed under cost, power, concurrency, and quality constraints.

The Missing Work Is Still Large

This field is still early.

In the talk, I focused mainly on single-agent coding trajectories. That is already more realistic than chatbot benchmarks, but it is not the end state. Multi-agent workflows create additional complexity. Long-running sessions can continue for days, with caches and memory growing over time. Heterogeneous workloads may behave very differently from coding agents. CPU coupling needs deeper treatment. Quality under load has to be measured alongside speed.

The final metric may not be time to first token or token speed at all. For many teams, task completion time will matter more. For others, the right question will be how quickly the agent completes the task while still producing a correct result.

That is why I think benchmarking the agent era means measuring beyond the LLM. The model call is only one part of the system. The real workload includes context growth, tool calls, caching, routing, decode behavior, CPU and GPU interaction, SLOs, tail latency, and quality.

Benchmarks for the chatbot era have caught up to chatbot workloads. Benchmarks for the agent era are still being built. The next step is to make them look like the systems we actually run.

The full version of this argument was presented at AI Native DevCon London. To go deeper, watch the full recording.

What GitHub learned when better tools made Copilot code review worse

Tessl — Tue, 14 Jul 2026 10:08:57 +0000

TL;DR: GitHub gave Copilot code review better shared tools, but reused generic instructions — reviews got pricier and less accurate until they rewrote the instructions for how a reviewer actually works, cutting cost ~20% with no quality loss.

Shared tooling is supposed to be the easy win: less duplicated code, fewer things to maintain, improvements that carry automatically across products. GitHub's own account of an internal migration – moving Copilot code review onto its shared CLI toolset – makes the case for treating that assumption with at least a little suspicion.

Migrating to shared tools made Copilot's reviews pricier and less accurate

Copilot code review previously ran its own code-exploration tools — list directories, search files, search directories, read code — purpose-built for earlier, less capable models. GitHub's Copilot CLI, meanwhile, runs a broader Unix-style toolset — grep, glob, view — that several other Copilot products draw on too.

GitHub decided to migrate Copilot code review onto that shared CLI toolset — retiring its own tools in favour of the same grep, glob, and view already used elsewhere. The appeal was, essentially, less duplicated engineering effort, and a single toolset that could be improved once and inherited everywhere it was used.

In offline benchmarks, the opposite happened. Review cost went up and fewer useful issues got flagged. Napalys Klicius, software engineer at GitHub, notes that moving to the shared CLI toolset was expected to improve results by giving the agent more flexible code-exploration tools, that didn't hold up once they looked at what the agent was actually doing.

"The tools weren’t the problem, the instructions were," Klicius writes – meaning the prompt-level guidance that tells the agent when and how to use each tool.

"Once we rewrote them for the way a reviewer actually reads a pull request, the regression flipped into a win."

Cost per review fell by around a fifth, without the quality of the reviews slipping.

Klicius likens tool descriptions and system instructions to API documentation — when that documentation is muddled, a developer ends up making worse calls, not because the underlying tool is flawed, but because the guidance around it failed them.

“Unclear tool prompting can do the same for an LLM; a small wording change can affect cost, quality, and the shape of the investigation because it changes how the agent spends its attention,” Klicius writes.

Tool traces showed the agent exploring code instead of reviewing a diff

What made the benchmarks useful here wasn't the score itself — it was that GitHub could pull up exactly which tools the agent reached for, in what order, and how much came back each time. What that record showed was an agent acting less like a reviewer and more like someone poking around a codebase for the first time — casting a wide net, taking guesses at where relevant code might live, and pulling back far more than any single review question called for.

Before — a simplified illustration of the general-purpose behavior we observed: widening the search, guessing paths, and accumulating context. (GitHub)

None of that extra material got discarded — it sat in the agent's working memory for the rest of the review, driving up cost without necessarily helping the agent reach a better answer.

None of that was irrational — it's exactly how you'd want an assistant to behave if its job was to get oriented in a codebase before touching it. But reviewing a pull request is a different task. The goal isn't to build a broad understanding of the codebase, it's to gather just enough context to determine whether a specific change introduced a problem.

New guidance narrowed the agent's search and cut review cost by a fifth

Nothing changed about the tools themselves. What changed was the order the agent was told to reach for them — start from the diff, narrow candidates with grep and glob, and only call view once it actually knew which file or line range mattered. Even failure handling got more specific: a search that came back empty should be retried once with simpler terms, not treated as a cue to start guessing at neighbouring files.

After — a simplified illustration of the review-shaped behavior the prompt guided toward: stay anchored to the diff, narrow with grep and glob, then read focused ranges with view. (GitHub)

In production, that shift held: a roughly 20% drop in average review cost, with review quality unchanged. Worth flagging that this is GitHub's own reported figure from its own benchmarking, not an independently verified number.

The more interesting result came from testing the same fix somewhere it didn't help. GitHub tried applying the same review-shaped guidance inside Copilot CLI itself and saw no equivalent gain, because a CLI session has no single pull request anchoring it — a developer might redirect the whole task halfway through, so there's no diff to narrow around in the first place. The tool was never the variable that mattered. What mattered was whether the guidance around it matched the job the agent was actually being asked to do.

The benchmarks that proved the fix

None of this would have been visible without a way to test it. GitHub could only identify the regression — and prove the rewrite worked — because it had a benchmark suite that could replay the same reviews before and after, measuring both cost and quality. Without that evidence, the new tools would have made an easy scapegoat, and the actual cause — instructions that no longer matched the job — could have gone unnoticed indefinitely.

That's the same discipline behind Tessl's evals model: testing and measuring a skill's instructions before and after every change, treating them as something that needs continuous verification. GitHub built that evaluation infrastructure internally for Copilot code review. Teams managing skills across many agents and many tools need the same kind of repeatable evidence to separate a genuine improvement from a change that simply altered agent behaviour.

The wider lesson here is that any team consolidating tools, upgrading models, or standardising instructions across agents is making the same bet GitHub made: that shared components will behave the same way everywhere they're used. That bet doesn't announce itself when it fails — it just shows up as slightly worse output that nobody's measuring closely enough to catch, which is the argument for building that measurement in before a change ships.

Reflection Before Augmentation

Tessl — Mon, 13 Jul 2026 09:17:28 +0000

Last week was conference week for me. I spent the start of it at Tessl's AI DevCon and the end of it at Muslim Tech Fest, where I hosted a design roundtable.

AI DevCon was filled with discussions about agents, workflows, evaluation, and the future of software. Muslim Tech Fest was filled with discussions about AI too, but through a different lens: community, responsibility, and building meaningful careers. On the surface, they felt unrelated.

By the end of the week, I wasn't sure they were discussing different problems at all.

Why individual AI gains don't translate to teams

One of the recurring themes at AI DevCon was the challenge of scale. Many teams have now experienced what AI can do for an individual contributor. A developer paired with the right tools, whether Claude Code, Cursor, or Copilot, can move faster, explore more options, and ship a working prototype in an afternoon that would have taken a week not long ago.

The harder question is what happens next. How do those gains translate beyond the individual? How do teams share context when everyone has their own workflows, prompts, and agents? How do organisations maintain standards, govern behaviour, and build systems that multiple people can contribute to and trust? The conversations were less about whether AI works and more about how it fits into the reality of organisations.

This is where context engineering becomes the real work. Individual gains stay individual until a team can capture and share the context that produced them. Reusable, evaluated instructions for agents are one way teams turn one person's good workflow into a shared standard the whole organisation can rely on.

Reflection comes before the tool

A few days later, I found myself facilitating a design roundtable at Muslim Tech Fest. The discussion quickly moved away from tools and towards a more personal set of questions. People spoke about feeling overwhelmed by the pace of change, uncertainty around where to begin, and wanting to make better use of AI without always knowing how.

What struck me was that the most useful answers rarely started with the technology itself. Instead, they started with reflection. If someone understood where they created value, they could identify opportunities for amplification. If they were clear about their weaknesses, they could identify opportunities for AI to support them. If they knew which parts of their work depended on experience, judgement, or taste, they could make more informed decisions about what to delegate and what to retain. The challenge was not simply learning how to use AI.

It was understanding yourself well enough to use it intentionally.

What organisations should understand before adopting AI

The more I reflected on those conversations, the more relevant they felt to many of the challenges being discussed at AI DevCon. Before an individual can decide how AI should augment their work, they need to understand where they create value. Before an organisation can decide how AI should transform its operations, it needs to understand what makes it effective in the first place.

What are its strengths? Where does its advantage come from? What knowledge is unique to it, and what standards does it want to uphold?

Without those answers, adoption becomes reactive. Without those answers, the conversation starts with the tool rather than the problem.

The practical version of this for an engineering leader is unglamorous: before deciding which tools to buy or where to apply agents, audit where your team actually creates value and which standards you are unwilling to compromise. That audit, not the tooling decision, is the thing that makes everything after it work.

Reflection as a discipline, not an abstraction

One of the reasons this idea stayed with me is that it feels surprisingly familiar. Reflection occupies an important place within the Islamic tradition. Not reflection as an abstract exercise, but as a means of examining one's intentions, actions, strengths, shortcomings, and responsibilities. The goal is not simply greater self-awareness. The goal is growth. Reflection is valuable because it creates the conditions for more intentional action.

That framing gave me a different way of thinking about many of the conversations I heard throughout the week. Much of the discourse around AI focuses on capability: what the technology can do, which tasks it can automate, and how quickly it is improving. These are important questions, but they are not the only questions. An equally important question is what we choose to amplify.

If someone lacks clarity about where they contribute value, AI will not solve that problem. If a team lacks shared standards, more capable tools will not create them. If an organisation does not understand what makes it successful, adding AI to the equation is unlikely to provide the answer.

Technology can accelerate direction. It cannot provide direction.

What stays human

Perhaps this is why so many conversations about AI eventually become conversations about judgement. What should remain human? What deserves deeper care? What standards are worth preserving? What kind of work is worth striving for? These are not new questions. But they feel newly important in a world where capable tools are becoming increasingly abundant.

The most memorable conversations I heard last week were not really about AI. They were about understanding ourselves. As individuals, understanding where we create value. As teams, understanding how we work together. As organisations, understanding what makes us effective. Only then can we make informed decisions about what to automate, what to delegate, and what to amplify.

The tools will continue to improve. That much seems certain. The harder challenge may be understanding ourselves well enough to use them wisely.

Sandboxing AI Coding Agents with lincubate

Tessl — Sun, 12 Jul 2026 07:08:04 +0000

At Tessl, we spend a lot of time working with agent skills. Writing them, testing them, tweaking them, running evals to see if they actually do what we think they do. You probably do something similar if you're spending any serious time with Claude Code, Codex, or any of the other AI coding agents that have colonised our terminals lately.

Here's a problem that crept up on me: my ~/.claude/ directory is a mess of skills, settings, and commands accumulated from a dozen different projects. When I sit down to test a new skill I've been writing, the agent is already carrying all that baggage. Skills from unrelated projects bleed in. Results are hard to interpret. Is this behaviour because of my new skill, or something else lurking in my config? It's the software equivalent of debugging with the wrong environment, except the environment is invisible.

What I needed was a clean room — somewhere I could run an agent with exactly the context I chose to give it, and nothing else.

That's the main reason I built lincubate.

The clean room problem

If you're writing and evaluating agent skills, reproducibility matters. Stray configuration from other projects — skills you installed last week for a completely different codebase — can skew your results in ways that are genuinely hard to spot. The agent might be doing something because of your carefully crafted new skill, or it might be doing it because of an old skill you'd forgotten about. Good luck figuring out which.

lincubate solves this by running the agent inside an LXD container, isolated from your host. Your project files are bind-mounted in so the agent can actually work on your code, but your ~/.claude/ directory? Not there unless you specifically ask for it.

The --allow-claude-skills flag opts you into sharing your host skills with the container. Without it, the container is a blank slate. The agent sees your project and nothing else — no accumulated config, no borrowed skills, no surprises. It turns sandboxing from a vague security concern into a practical focus tool.

Why LXD and not Docker?

Fair question, and I've been asked it a few times already.

Honestly? I'm an LXD person. I've been using it for years on my Ubuntu ThinkPad. It gives you a full system container — proper init, systemd, real user accounts — rather than wrapping a single process. It feels like a lightweight VM rather than a process in a box, which matters when an AI coding agent expects to operate in something that resembles a normal Linux system. Agents do all sorts of things: install packages, run build tools, start services. A full system container handles all of that without friction.

There's also a practical angle: on my ageing ThinkPad, LXD is noticeably lighter than Docker. Not enormously so, but enough that I notice it over the course of a day's work.

If you're a Docker person, none of this is meant as a dig. I'm just not, and I wrote the tool that fits my workflow.

What it actually does

The core usage is simple. From inside your project directory:

`lb claude`

That launches Claude Code in an LXD container. Your project files are bind-mounted in at /home/ubuntu/project with UID mapping, so file ownership works the way you'd expect. API keys get (optionally) forwarded as environment variables. Auth tokens can be copied or mounted read-only if you need them — it's all off by default.

One container per project directory. The container name is derived from your working directory, so running lb from the same place always targets the same container. You're not spinning up a fresh environment on every launch; if you stopped work yesterday and pick it up today, the container's already there waiting.

`lb         # Drop into a shell (no agent, useful for poking around)
lb destroy # Tear down the current project's container when you're done`

First run takes a few minutes — lincubate builds a base image with Node.js, common packages, and all supported agents pre-installed. After that, launches are fast.

Supported agents

Claude Code is the one I use most, but lincubate isn't opinionated about which agent you bring:

Agent	Command
Claude Code	lb claude
OpenAI Codex	lb codex
Aider	lb aider
Gemini CLI	lb gemini
GitHub Copilot	lb copilot
OpenCode	lb opencode
Cursor (CLI)	lb cursor
Cursor (IDE / GUI)	lb cursor-gui

The GUI option forwards X11/Wayland into the container, which feels like a minor miracle the first time you try it.

The rewrite

lincubate started as a Bash script. It worked, but it had the kind of accumulated jank that (my) shell scripts tend to develop when they get complicated — lots of string manipulation, too many calls out to the lxc binary, config files sourced as shell variables. I rewrote it in Go, using the LXD Go client library directly, with TOML for configuration. The result is a single static binary with no runtime dependencies and a lot less jank.

There were some fun gotchas along the way. su -l resets the environment, which broke credential forwarding in ways that took a while to track down. LXD occasionally returns a non-zero exit code on a successful container start, which had me questioning my sanity for longer than I'd like to admit. UID mapping (raw.idmap) isn't supported in every LXD configuration, so there's a graceful fallback. These are the kinds of things you only find out by running something properly for a while.

Try it

The code and release binary is at github.com/popey/lincubate. Build from source with just build, drop the binary somewhere on your path, and you're ready. I assume you already have LXD installed and configured locally or remotely.

Zero configuration required to get started. If you want to customise things — add packages to the base image, control which agents get pre-installed, add extra environment variables — generate a config file and edit it:

`lb generate-config`

The generated file is fully commented, which I think is the bare minimum you should expect from any tool that writes config files on your behalf.

I talked about lincubate in more detail on Linux Matters episode 78.

If you're spending real time writing and evaluating agent skills, having a proper clean room makes a surprising difference. Give it a Go.