DEV Community: Tessl

Cheaper Tokens, Bigger Bills: Token Price Isn't Agent Cost

Tessl — Sat, 29 Aug 2026 06:36:50 +0000

We needed a default model for Tessl Code Review, our new code review tool. The shortlist seemed easy to compare: check the token prices, weigh them against capability, and choose. Then we ran the models on real pull requests through the real review harness. The rate card would have led us to the wrong choice.

Pricing pages quote tokens because tokens are easy for vendors and buyers to count. An agentic workload adds another variable: turns. A turn is one model call to read a file, run a check, or decide what to do next. The model decides how many turns it needs, and you pay for all of them. A rate card cannot show that. Every alternative we tested had cheaper tokens than gpt-5.6-terra, the model we run, yet three cost more per review. One was almost three times cheaper per token and 2.8 times more expensive on the same pull request.

A more useful measure is cost per verified outcome: what you spend for each result that survives an independent check. This captures the full cost of the task and discounts results that do not hold up. As with any other agent evaluation, the check has to be independent. A model cannot be trusted to grade its own work.

Why per-token price stopped predicting your bill

We used ten merged pull requests from our monorepo, pinned at the commits seen by our production reviewer. Four models went through the same harness with the same review lenses, fixing agent, and loop: review the change, apply fixes, then review again for up to four rounds. In August 2026, that produced 100 runs across two conditions. Only the model changed.

The study has two important limits. With ten pull requests and one run per model per subject, we could not measure run-to-run variance. We also chose pull requests where our production reviewer had already found an issue. That raises recall for every model and tells us nothing about false positives. We therefore report multiples rather than percentages and do not rank the models that trailed the baseline against one another.

All costs are relative to gpt-5.6-terra. “From rate card alone” shows what each model would cost if it consumed exactly what terra consumed. “From consumption alone” prices each model's actual usage at terra's rates.

Model	Actual cost gap	From rate card alone	From consumption alone
gpt-5.6-terra, the baseline	1.0x	1.0x	1.0x
A lower-cost open-weight model	0.4x	0.16x	2.33x
A second open-weight model	1.7x	0.54x	3.23x
A third open-weight model	2.8x	0.37x	7.59x

The rate card suggests savings of 46% to 84%, so any of the three alternatives looks attractive in the middle column. Their actual consumption changes the result: they used between 2.3 and 7.6 times more to complete the same job. One model was genuinely cheaper at 0.4 times terra's cost per review, but an independent check found that it also delivered the least useful output.

Turns explain most of the gap. Terra completed a review in 42 turns and used 0.6 million input tokens. The most expensive alternative needed 156 turns and 5 million tokens. Both returned a review, but one spent far longer rereading files and checking its own work.

For a single call with a controlled input and bounded output, token price is a reasonable estimate of cost. An agent with tools and a goal controls much more of its own consumption. Token price still matters, but the model's behavior inside the harness can matter more.

Register to get the early birds discount

A better unit for AI agent evaluation

We use this formula:

`cost per verified outcome =
    price per token
  x tokens consumed per task
  x 1 / (share of output that survives verification)`

The rate card supplies the token price. Running a real task supplies consumption. An independent grader tells you how much of the output survived. Leaving out either of the last two terms can change the ranking entirely.

Consumption: the term that does the damage

Consumption has to be measured on your work and through your harness. Identical inputs in our study produced a fourfold spread in turns and an eightfold spread in token volume. Published benchmarks usually ask whether a model reached the right answer, not what it spent to get there, so they would not have exposed this difference. The measurement also expires: change the model or the harness and consumption can change with it.

Validity: what survives an independent check

The reviewer models assigned severity to their own findings. We regraded all 910 findings with one fixed model that saw only the finding and its diff hunk.

Because each reviewer used and inflated its own scale, we could not compare the original labels directly. The reviewers and grader agreed 59% of the time. When they disagreed, the grader marked 35% of findings down and 6% up. Only 6 of the 70 findings called critical by their authors remained critical, and the grader did not promote a single finding to critical.

Regrading reversed one apparent lead. A competing model reported 73 severity-bearing findings against terra's 62. After grading, terra led 37 to 17. We also compared each model with findings from our production review on the same commits, and that check pointed in the same direction.

The table combines spend with both checks. “Verified severity found” counts severity that survived grading. The next column prices each graded major or critical finding. The final column shows how much of our production review's output the model also caught. All values are relative to terra.

Model	Verified severity found	Cost per graded major or critical finding	Share of our own review's findings also caught
gpt-5.6-terra, the baseline	1.0x	1.0x	1.0x
A lower-cost open-weight model	0.2x	1.4x	0.6x
A second open-weight model	0.4x	4.0x	0.9x
A third open-weight model	0.5x	5.0x	0.9x

The model with the most expensive tokens found the most verified severity and had the lowest cost per graded major or critical finding.

The cheapest model did win one measure: each finding that matched our production review cost half as much as terra. That measure treats a nit and a data-integrity bug as equal, however. The same model produced one fifth of terra's verified severity and reached six tenths of its coverage. Its low cost per match reflects the kind of output it produced, not just efficiency.

Even here, the rate card overstated the saving. Its tokens cost 0.16 times terra's, while a completed review cost 0.4 times as much because the model consumed 2.3 times more.

Reliability: the failure mode you find last

Cost and finding quality were not the only differences. Giving each model the full job exposed failures that a rate card or single-call benchmark would miss.

Our reviewer first finds issues, then reconciles them across rounds. Reconciliation means checking every previous finding against the new code, preserving its identity, and producing a structure that passes validation before publication. When each model handled both roles, two of the four models failed to produce that structure reliably, which killed their runs. With one shared reconciler across every arm, we saw no such failures.

One model failed 6 of its 10 runs and also reported the most severity. Its own labels made it look like the strongest reviewer. The independent grade did not support that claim: only a third of its severity labels held up, the worst agreement in the study, and its 73 severity-bearing findings fell to 17. Because it completed the fewest rounds, its aggregate results also rest on less evidence.

Only one model besides terra completed the supervisor role without a failure. It was the cheapest model at 0.4 times terra's cost, but it settled 8 of its 10 runs by deciding there was nothing left to say and approving the change. Its findings also held up worst under the independent grader. It finished reliably because it attempted less of the job.

Repeatability added another warning. Across both conditions, terra reproduced 71% of its findings on the same code. Another model reproduced 24%. A good result is much less useful when the next run finds something different.

What this changes about how you pick a model

Our results suggest three practical checks for teams choosing a model.

Measure the cost of the task. Run your workload through the harness you will use in production and inspect the bill. The useful number is specific to that model and harness. If the option with cheaper tokens costs 2.8 times more per completed task, the higher task cost is what you will keep paying.

Use an independent grader. Models differed in how much they inflated their own work, from almost no inflation to nearly three quarters of a severity level. There is no constant adjustment that makes those self-reported scores comparable. Use the same independent check for every candidate.

Test roles separately. Finding issues and reconciling them require different capabilities. A model can do the first well and still fail the second. Hold any component that can kill the whole run constant while you compare the rest. In our case, pinning the reconciler let us use specialist models without making the loop unreliable.

The objection worth taking seriously

Newer frontier models may follow instructions about effort and verbosity more closely than older ones. Our prompts do scope the review, so terra's lower turn count probably reflects instruction-following as well as raw efficiency.

That distinction does not change the bill. If one model needs 156 turns because it follows the scope less closely while another needs 42, you still pay for 156. Efficiency belongs to the combination of model and harness, not to the model in isolation. It is also why a model upgrade can move skill performance in either direction. Measure the pairing you intend to run.

Questions we got asked about cost per outcome

Does this only apply to code review? No. Any agent that runs a loop and decides when it is finished controls its own token volume. On those workloads, a rate card prices the input rather than the result. For single-shot classification or extraction, where consumption is bounded, token price remains a reasonable proxy.

Will the next model release change all of this? It will change the numbers, but not the method. Consumption depends on the model and harness together, so each release needs a fresh measurement. A model that was efficient on your workload last quarter may not be efficient today.

What if I cannot build an independent grader? The grader can be small. Ours sees one finding and the relevant diff hunk, without knowing which model produced it. What matters is consistency and independence: every candidate gets the same grader, and no model scores itself.

What would your own numbers say?

What bothered us was how rigorous the wrong comparison looked. It used a price list, a quality claim, and a sensible weighting between them. None of that was careless. It still favored a model that cost more to produce a worse answer.

What is your cost per verified outcome, and would it change the model you chose?

Before your next model swap, pull your own numbers. Turn counts are in your logs, token volume is in your billing, and the third term takes one independent grade over a sample of output. Run them through the formula. If the ranking changes, that is the number to buy on.

Tessl Code Review is free to try, and it records turns and cost per run for you.

The Rise of the Harness Engineer

Tessl — Wed, 26 Aug 2026 06:35:31 +0000

We're in an intense moment as software engineers. On one hand, coding agents have become genuinely amazing and we’re using them all the time with higher and higher throughput. On the other, they build in sloppy, bug-accumulating ways, and we’re under unrelenting pressure to keep them in check.

I'm a researcher and engineer at Tessl, where I've spent a lot of the last while helping build our internal software factory and evaluating what agents actually do when you point them at real work. The conclusion I keep arriving at is this: as agents write more and more of our code, engineering doesn't matter less. It matters more. It just looks different from the engineering many of us grew up doing. I've started calling the person who does this new kind of work a harness engineer, and I think it's a role a lot of us are about to grow into, some as a full-time specialism, many more as a growing slice of the job alongside product engineering.

Let me walk through how I got there, and what new skills are emerging as crucial to staying in control.

The agents got great, but cracks show over time

Start with the good news, because it's genuinely remarkable. The adoption curve for AI coding tools has gone close to vertical. A year ago adoption was spiky and experimental; now, across a lot of organisations, the vast majority of code is AI-authored [see Faros AI, AI whiplash report]. But, hand-in-hand, the number of bugs and production incidents are increasing with it, as more PRs slip through with no human code review.

The benchmarks tell the same story from the other side. Task based benchmarks are largely saturated – if you can define a task cleanly and measure if it was done, an agent can complete it. But if you ask agents to refactor large codebases, or build on their work over time, then we see deterioration, and much lower levels of success.

Better agents will help over time, but also come at a cost

The natural hope is that better models will simply dissolve all of this. But there will be heavy trade offs – much of the issue is reasoning about overall design decisions and reviewing existing code to follow patterns and stay consistent – all of which take time and reasoning effort to do. Expecting agents to do this well from first principals in every session would be incredibly expensive even if they can do it.

If you plot how much it costs to run an agent against its intelligence, the intelligence axis moves roughly linearly while price moves on a log scale. [see Artificial Analysis’s pareto frontier charts] Across the range that's something like a 100× spread, and the difference between a top-tier model and a perfectly capable cheaper one can be around 10× for the same task. Even when the very best model can do the job, that can be an extraordinarily expensive way to solve a problem. And when you start running agents at real throughput, this stops being academic. We closed something like 600 PRs in a single week recently, and we're a small company. When that much is flowing through, cost and efficiency become first-class engineering concerns.

The steering gap

We think a lot about skills at Tessl, so we built a benchmark to understand them properly. We took around a thousand skills from open-source repos, built tasks that would exercise those skills, and then ran agents through the tasks both with and without the skill so we could compare. We graded two different things: did the agent complete the task, and did it actually follow the instructions the skill laid out, not just "did it finish," but "did it do it the way it was told." [see: https://arxiv.org/abs/2606.17819]

What did we learn?

First, well defined skills reduce the difficulty of tasks and let cheaper, smaller agents reach the same level of success as much larger, more expensive ones. If we can make it easy to find the right guidance and break tasks apart well, we’ll win in both consistency and efficiency.

Second, and less comfortably: instructions don't get followed thoroughly. Task completion rates were high across almost everything we studied, but only about 70% of the instructions in a skill were followed on average. When you break a skill down into its individual instructions and check them one by one, even the best models are quietly ignoring a big chunk of what you told them. Skills aren’t enough, they need to be paired with checks that the rules are followed consistently.

There’s an enormous amount of value in designing how we use agents, rather than just using them as they are and hoping they make the right choices.

Engineering matters more, not less

Here’s the shift. When we hand-wrote code, we could review and discuss it, and many of the key decisions we were making could emerge and be shared implicitly as we worked. We’d know to go sketch an architecture diagram on a board when it stopped scaling the way we wanted, or we’d spike on which library to choose at the moment we realised it was important, and we could feed those choices back in. It was easier to mix taking time on the overall approach and the specific task.

Now, the agents are going so fast, it’s easy to fall into only being able to react – being pinged to review the latest PR when it’s ready, and feeling frustrated with the same errors being repeated.

If we want to go really fast, we need to be proactively spending our time designing the system in which the agents run, making our choices explicit and measuring whether the system that is emerging is what we want.

I see three skill sets that matter more now than they used to, and that are a little different from the ones many of us spent years developing.

1. Systems thinking: capturing invariants

Some of what you want from a change is specific to the task in front of you. But a lot of it is really an expression of a general principle you want to stay true across the whole system. If you can identify those invariants, the things you want to keep being true, and then capture and enforce them, you have a real shot at keeping consistency and holding the quality bar high no matter who, or what, is doing the writing.

What do invariants look like in practice? Design systems are a great example, you describe the types of layouts you want, the brand voice, the components, what a button looks like. Individual pages are then built from those reusable components. They show up in architecture, how you want code structured, which libraries you've chosen, opinionated views on how they should be called. Each new feature follows the patterns and choices consistently. And they show up in best practices, how errors should surface through the stack, how things should be named, how data should flow.

In the past, these lived in a code owner's instinct; you could ask them to sketch an architecture diagram, or sit with a designer over the Figma files, and then have them check PR by PR whether things matched.

Now we want to make these choices explicit, and when we do we can enforce them consistently much more cheaply than we could before, effectively steering development from these patterns. Why is it easier? Because just as agents are better at writing code, they are also better at reviewing code – especially when the rules are clear and thoroughly described.

How do we enforce them:

We can write skills that describe exactly what we care about and load them at the right moment.
We can put deterministic checks into CI, linters, and tools like ast-grep, which I'd barely heard of a few months ago and now talk about constantly. (Agents are excellent at writing these, so the thing that used to be a pain is now easy.)
We can add narrow verifiers, tight rules measured on every single file by a quick agent, to enforce the kinds of constraints that are hard to pin down deterministically.
And we can do agentic code review with broader prompts describing the principals we care about.

Between them these form a system of guidance and guardrails that steer the agents when they are working, and review if they did things correctly. They leave a lot less on the table to be caught in ad hoc review, or to slip through and cause problems later.

2. Analytics: reason about the system with data

This one surprised me, because I didn't used to think of analytics as an engineering skill. But as we automate more of the code-generation and review process, we generate more and more data we can actually look at, and there's a wealth of signal in it. This is what enables us to find invariants and measure what’s important.

New signals that we can study:

Agent logs are fascinating. They show where agents are spinning and getting confused. You can see where it wastes time. For example, are agents struggling with a particular API, are they reading files they should, are they puzzled about what we want?
PR comments are another signal. Do comments generalise into invariants that could be applied widely? What types of errors are being caught?
Analysis of the code base itself. Is the complexity of the codebase worsening with spaghetti code importing from everything and god files growing to thousands of lines? Does mutation testing find tests that aren't earning their keep, and gaps in coverage that matter? Are there multiple ways of doing the same thing that could be converged?

Then you turn all of that into hypotheses about where agents are going wrong, and then look for how you could address that – giving the agents better tools, refactoring code, adding skills, defining new invariants and enforcing them so you never have to worry about that failure mode again.

Sometimes that's a human grabbing the data and staring at it; sometimes it's an automated loop where an agent regularly runs the analysis and finds incremental improvements. Either way, it’s a seed for discovering how well the system is working and where improvements could be best made.

3. Risk and operations: grade the blast radius

As we build trust in review mechanisms, then we need to reduce the human review burden. We need to reason about the risk we’re introducing by relying on our processes and how that aligns with the sensitivity of the code base.

At Tessl we use a kind of ladder. Our research codebase is a genuine free-for-all, merge anything you like, no rules. Some of our internal tooling is auto-merged; hundreds of changes flow through, with purely automated review, and nobody looks at any of them. Other parts of the codebase absolutely need an engineer, the person who owns the feature has to take it through and do the final merge themselves. And a few very high-leverage parts should make you stop and go ask someone who's thought hard about them, rather than ramming a change in.

The work is figuring out which parts of the codebase sit at which level of risk, and then codifying that into the approvals flow.

The harness engineer

Put those three together, invariants, analytics, risk, and you get harness engineering. For a lot of organisations this will live in a few specialists who really embody it. But I suspect that for many of us working in engineering, it simply becomes more and more of where we spend our time. That's a real shift in identity: we become tenders and gardeners of our code and our systems, so that when we build features we can focus on whether the behaviour is what we want and trust that the implementation will be consistent and scalable.

Spending your hours on invariants and analytics and risk policy, rather than only on the next feature, feels different, but it has tremendous leverage.

We've leaned all the way into this. Around 90% of our code went through the factory this week, and we've been over 80% for the last month. The shape is consistent: more of the coding moves into remote execution loaded up with skills and context; a layer of guardrails (deterministic checks, LLM verifiers, CI) run through a detailed checklist of rules; automated code review handles new problems and feature specific issues; a triage/gating step reasons about how much risk and care a given change needs; and update loops monitor what's happening and feed learnings back into every layer of the stack.

Where to start

If this isn't your full-time job yet, here's where I'd begin, three small exercises you can try this week.

Build a dataset. Agents are great at this, tell one to grab your last 50 PRs and summarise what kept going wrong: what people repeatedly said in comments, what errors recurred, where things tripped CI. Identify issues that generalise and might come up again.
Distill three invariants for code you work on. Work out what rule would clarify the behaviour you want - in architecture, design, behaviour, or library usage for example.
Turn one invariant into a CI gate, and then make sure you have a way to track what happens with it and watch it. Did it start catching the problem you cared about? How often is it firing? Try implementing it using AST-grep (if it can be expressed deterministically) or try out tessl verifiers (if you want something broader).

For any of these, we’ve loaded guidance into tessl agent, so you can ask it to help you work out where to start and see if you agree with it. Install tessl and run tessl agent to get going. The free tier should be plenty for building out your first invariants.

Here are some examples of rules we have implemented at Tessl to help you get started.

Agents gave us rocket boots. Harness engineering, systems thinking, analytics, and risk, is how we keep from flying straight into a wall. And I think it's about to be some of the most valuable engineering work any of us can do.

New in Tessl Academy: Code Review Loops

Tessl — Mon, 24 Aug 2026 04:38:05 +0000

A hands-on course on agentic code review: running your first review, writing a lens that encodes one of your team's own rules, routing lenses by path, and closing the loop on every pull request. Four lessons, about 25 minutes each. Read them on the site or have your agent walk you through them.

Reviewing code is the part of the job that scales worst. It needs someone who knows the codebase, has time, and is willing to read carefully. When agents are writing most of the changes, that person runs out long before the changes do.

Simon covered Tessl Code Review yesterday: what it is, why we built it against whole pull requests rather than diffs, and how re-review keeps track of what a previous round already settled. This post covers the next question, the one you hit about an hour after installing it: your reviewer doesn't know your team's rules yet, and the useful version of it does.

That's Code Review Loops.

npx tessl install tessl-academy/code-review-loops
tessl launch skill --agent tessl-agent -i 01-your-first-code-review

What's in it

Four lessons, in order, each building on the repository state the last one left behind.

Your first code review has you scaffold a small TypeScript service and a branch with three deliberate faults in it, then run a review over that branch three different ways and read what comes back — the outcome, the severities, the JSON. It also settles the naming collision that catches nearly everyone: tessl review run scores a skill, and tessl code review reviews your code. Both commands exist, they sound alike, and picking the wrong one costs a confusing five minutes.

The lesson plants three faults; the reviewer finds two. That's deliberate — the miss is why lesson two exists.

Writing a review lens starts from that missing finding. A lens is a skill: a SKILL.md with a name, a description of when it applies, and a body of review instructions. The four defaults ship in the tessl/code-review plugin and their sources are public, so you can fork one and see how it's put together. You write a lens that encodes a convention only your team keeps, run it beside the defaults, then test it in both directions so it fires on the bad case and stays quiet on the good one. That second half is the part people skip — it's the difference between a lens and a noise generator.

Routing lenses by path moves the configuration off the command line into a YAML profile in the repository, with globs and exclusions so each lens only runs where its rule applies. A review that skipped your files isn't a review that approved them.

Review on every pull request is the capstone: token, caller workflow, Action pinned to a commit SHA, advisory mode first and a gate later. Then a second round with one fix and one reply, so you watch earlier findings resolve as addressed, explained, or declined rather than getting raised again.

The bits you'd otherwise learn the hard way

-skill replaces the profile's default lenses rather than adding to them. Pass two and you get two, not six.
A review takes at most eight lenses, and their order is preserved.
Tessl doesn't go looking for your profile file. A bare tessl code review in a repo containing .tessl-code-review.yml runs the built-in standard profile and ignores your file. Nothing starts judging your code differently just because a file appeared.
In CI, review policy comes from the default branch, not the branch under review — a pull request can change any file in the repository, including the one that decides how that pull request gets reviewed.
@tessl-code-review is text your workflow matches, not an account.

Lesson three needs a CLI newer than 0.96.0, since YAML profiles landed after that. Lessons one, two, and four work fine on older versions.

How the course got built

The Academy is built in a way that's slightly unusual, and the code review course is the clearest example of it so far.

There's no pipeline that compiles curriculum. Lessons are assembled by an agent running a skill, and those skills are what we maintain.

Content lives as versioned components: a concept explained in under 300 words, a hands-on exercise, a starting repository state, a scoring rubric. There are 65 of these across six types, reused across lessons and courses. A human writes the recipe deciding which components a lesson uses and in what order. An agent running compose-lesson turns that recipe into the lesson you read, and generate-lesson-skill turns the lesson into the installable walkthrough that tutors you through it. The only deterministic step is the last one, where a build renders finished markdown into HTML. The site itself is static and knows nothing; the judgement sits upstream in the skills.

Each composed lesson records the component versions it was built from. When the CLI changes underneath us, a maintenance sweep works out which lessons are affected instead of us guessing, and writes a machine-readable verdict per concept. A lesson only counts as verified when every concept in it has a current pass.

That verification has a limit worth stating: a pass means the concept matches the docs, not that anyone ran the command. A wrong doc produces a confidently passing wrong lesson. We check for CLI drift and nothing else yet, and the sweep only runs when a human starts it.

Every skill in the repository also has to clear tessl review run at 80% before it ships. We teach the command, so the tooling that builds the teaching gets reviewed by it — writing a course about review with review switched off would have been a bad look, and more usefully, it caught real problems in our own lens descriptions.

Try it

The course is at tessl.io/academy/code-review. Read it there, or install it and ask your agent to guide me through a first code review and work through it in your own repository.

If you get through it, tell us about your lens — what convention you encoded, and whether the reviewer caught it. That's the feedback that shapes where this goes next. Find us in Discord.

Review Agent-Written Code Against Your Team's Standards

Tessl — Fri, 21 Aug 2026 06:27:13 +0000

Agents can write code faster than any team can review it, and the reviewer that keeps up has to hold your team's bar rather than a generic one. That standard can't sit in a settings screen inside someone else's product. It has to be yours, and you'll keep changing it as you learn how your agents behave.

So we built Tessl Code Review on your team's own standards. You write them once, as a skill you own, version, and can point at another repo tomorrow. From there it reads the whole PR rather than just the diff, and it knows what's still open by the next round.

It’s very easy to use:

tessl code review --pr

With no flags it reviews the current PR. If you give it no PR, it reviews your local uncommitted changes, or hand it a git range. It publishes straight into GitHub, and it runs with whatever agent your team already uses. Tessl Agent is one option, but you can choose the agent if you have a preference.

Built for a world where agents write the code

Most review tools were designed to check human-written code and had AI added later. Tessl Code Review started from the assumption that agents write most of the code.

That changes what the tool needs to be. When agents are producing the volume, the standard you review against can't sit as a setting inside someone else's product, tuned by a vendor and hidden from you. It has to be something your team owns and can change, because you'll be changing it often as you learn how your agents behave. The rest of the product follows from that.

How does it work?

You write your team's review standards once, as a skill, and Tessl applies it to every pull request. We will read the whole PR rather than just the diff. This includes the surrounding codebase, your standards, and everything already said in the thread. And within a pull request, each re-review knows what's fixed, what's explained, what's declined, and what's still open, so you're never arguing a point you settled two rounds ago.

A review that you own

Our bet is different to other code review tools. Tessl Code Review is a review you own, that fits your team, that you can inspect and test, and that can become a trustworthy part of an autonomous pipeline instead of one more opinion shouting from the sidelines. It's a first step toward a software factory rather than the whole thing, but a foundational one. Trusted review is what eventually lets a change reach a settled end state, act as a blocking gate, and merge on its own without a human on every diff.

Why you should consider using Tessl Code Review

Your standards live in a skill you own, not a config screen

You write your review standards once, as a skill. This is simply a plain, readable artifact your team controls, versions, and can reuse elsewhere in your factory. Most tools keep your preferences in a settings page you can't inspect or take with you. A skill is something you can read, edit, version, and point at another repo tomorrow.

Reviews you can inspect

Every review is grounded in your codebase, your team's conventions, and the pull request thread, and you can see exactly what shaped it. You're not sending code into a vendor's system and waiting for a verdict you can't unpack. When the reviewer flags something, you can trace why. That grounding is also the real cure for noise. Irrelevant comments go away when the review understands your standards and your codebase, not when you nudge a sensitivity slider.

Prove your standards work, then govern them

Most teams have no way to check that their review standard is catching what it should, or holding steady as it changes. Since your criteria live in Tessl as a skill, you can run Tessl's own review and eval tooling against them and confirm they meet your org's bar, rather than assuming they do. Your standard becomes something you can test instead of something you hope is working.

One place to manage standards across every repo

Each repo can run its own review skill, tuned to that team. As you add repos, the skill registry is where you see and manage all of them, so your review configuration doesn't turn into a pile of scattered rules as you scale. Review standards become a first-class, versioned thing the whole org can see.

Re-review that remembers the conversation

Update a pull request and the next review already knows what was fixed, explained, or declined, and looks only at what's still open. No repeated findings, no re-arguing settled points, and none of the usual mess of a comment that disappears from the diff the moment the code changes and leaves you juggling tabs to follow one resolved thread.

Under the hood

There are four steps. It builds the context: the diff against main, the PR title and summary, the comment threads and how they were resolved, and the codebase. It runs each lens over the change to produce findings. A lens is just one of your review skills, and each one brings a different perspective: an architectural lens, a code quality lens, a security lens. A stateful supervisor then takes everything that came before, together with the new findings, and works out which are valid, which aren't, and which have already been handled. Finally it publishes inline comments and a summary on the PR, and replies in the existing threads.

We're using it at Tessl, and loving it!

While Tessl Code Review is only recently publicly available in Beta, we've been running it internally across our own monorepo as our daily reviewer, on real changes, for several months. We're still tuning the edges: a full-context review takes a little longer than a shallow one that only glances at the last few commits, and there's always more nit-tuning to do.

We monitor the precision of everything the reviewer flags and how much is a real defect. We took every finding it raised on our own pull requests and had a second, more capable model judge each one against the code on whether it was a real defect, or not. 74% were confirmed as real defects, compared to ~50% we saw in comparable tools.

Try it

Pick whichever fits how you work.

Let your agent set it up. Install the setup skill, then hand it to the agent you already use:

tessl install tessl/code-review-setup

Ask your agent to set up Tessl Code Review in your repository. It reads your existing workflows, asks two questions, when reviews run and whether findings block the merge, then writes the caller workflow and explains the permissions and secret it needs.

Or ask Tessl Agent. If you're running Tessl Agent, ask it to set up code review and it does the same job without you installing anything first.

Then shape the review to your team. Fork the default lenses into something that sounds like your team, or install the lens creator and write your own:

tessl install tessl/code-review-lens-creator

It settles the review question and the bar a finding has to clear, drafts the lens, then backtests it against pull requests that already carry review feedback, so you can see what it catches before you turn it on.

Prefer to do it yourself? Install the Tessl CLI and follow the quickstart.

If you'd rather see it first, watch a single change go through two rounds: a first review, a fix, and a re-review that knows exactly what's fixed, explained, declined, and still open.

Review is one of the most important parts of a software factory, because it's what lets you trust the rest of it to run. This is our first step toward making it one you own, and eventually one you can let run on its own.

3 things you can do with Tessl Agent in your first week

Tessl — Tue, 11 Aug 2026 06:49:02 +0000

Tessl Agent is a coding agent that makes your other coding agents better. It reads your PRs, agent session logs, and tickets, then opens pull requests that fix recurring mistakes, add review gates, and turn repetitive chores into automated workflows — so your codebase improves every week without anyone stopping to maintain it. It's in open beta and free to try, and it works alongside Claude Code, Codex, Cursor, or whatever your team already uses.

Teams adopting coding agents keep running into the same problem: the agent can write code, but keeping it consistently effective is unplanned work nobody has time for. Here are three things you can do on your first day, each one a single command.

1. Spend less time reviewing agent PRs

Agent-generated PRs still need a person to babysit them, and review is where a senior engineer's time quietly goes. The more you delegate to agents, the worse that gets.

Tessl Agent sets up a review gate that runs your verifiers and reviewer skills over every diff before a human sees it. It catches the correctness and style problems your team would otherwise flag by hand, and works out when a change genuinely needs a person to look. Run it locally, or wire it in as a GitHub Action so it fires on every PR.

Try it: run tessl agent\ and ask “Set up agentic code review.”

2. Fix the mistakes your agents keep repeating

Agents tend to make the same mistake over and over, PR after PR. Fixing the root cause means updating the context, rules, or skills the agent works from, and that work rarely makes it onto a sprint — so the mistake keeps coming back.

Tessl Agent reads your session logs and PR history and finds those patterns for you. Instead of correcting one more symptom, it opens a PR that fixes the context behind them, so the next agent to touch that code starts from a better place.

Try it: run tessl agent\ and ask “Find optimisations for my agents.”

3. Turn a repetitive chore into an automated workflow

Plenty of recurring chores could run themselves — keeping docs in sync, sweeping for weak tests — but they sit on the backlog because nobody has time to stop and systematise them. They're never urgent enough to prioritise, and never quite gone.

Tessl Agent picks out the tasks worth automating, builds the workflow, and ships it as a GitHub Action. Something you did by hand this week runs on its own the next, and the automations stack up as you go.

Try it: run tessl agent\ and ask “What tasks can I delegate to agents?”

Start in minutes

Each of these is one command from the root of your repo. Tessl Agent is provider-agnostic, so there's no lock-in and it works with the agents and models you already run. It's in open beta and free to try.

Get started: tessl.io/agent

Analyzing your agent sessions with Tessl

Tessl — Sat, 11 Jul 2026 06:10:16 +0000

With Tessl, evaluations serve a very specific purpose: Using an agent, and provided context, see how well a set of tasks can be done with and without that context. Or an evaluation might be used for comparing models. This is great during the development phase of a skill, but during actual usage, a lot of things can occur, things you might not have anticipated or worse, maybe something you expected to happen did not.

Agent sessions are a tremendous source of information, helping to understand what happened in a session, were certain expected events not occurring. Tessl will examine:

Friction points that the agent may have had while performing certain tasks. For example, were there errors or things that it thrashed on? This could be something not even related to the skill. Reviewing friction points may identify other areas that may be a candidate for a new skill.
Certain events, signals so to speak, in the sessions that Verifiers are expecting to find. For example, did certain actions that you expected actually happen? By definition, verifiers are structured pass/fail checklists that track any aspect of agent behavior you care about.

With Tessl, and the try-tessl/agent-quality plugin’s skills, it will create verifiers that come from:
- Skills, Docs and rules
- User input, where the user describes what they care about, you turn it into verifiers.
  
  Each verifier captures one instruction with a checklist of binary checks that an LLM judge evaluates against session transcripts.

Session analysis can help you optimize your skill by examining what happened during real world usage in your agents!

Prerequisites

You have Tessl installed, and configured for your agent.
Claude Code must be installed. Note that while this feature can be used with sessions from Cursor, Claude Code, Codex and Gemini, Tessl requires Claude Code to be installed on the user’s machine (and logged in) to run the judging.

About try-tessl/agent-quality

The Tessl plugin try-tessl/agent-quality is made up of three skills, which performs the following actions:

Identifies sessions across any agent you've used in a project. Security being important to Tessl, Tessl redacts credentials from the transcripts and also treats all content as untrusted data.
Identifies friction points.
Examines the skill and identifies what things it should look for in session (aka Verifiers).
The verifiers you are creating are being added to the tile, so a couple scenarios can occur:
1. The tile you are creating verifiers for is source-controlled in the repository you are in: verifiers are added as part of the tile
2. The tile you are creating verifiers for is checked out from the registry in .tessl: a new tile is created just for the verifiers and verifiers are added to it. This is because new content added to a checked out tile will be overridden.

try-tessl/agent-quality in action!

For the following example, a private skill, called mycompany/tessl-docs-creator was used to review a set of documentation. This skill is used to review documentation and ensure certain standards are maintained. Our goal in using try-tessl/agent-quality is to understand if the skill was used properly and where friction occurred during that normal usage.

As we walk through try-tessl/agent-quality, it’s important to point out that it follows this flow: phase 1, get feedback, phase 2, get feedback and so on. There is a human in the loop and the human can make changes to the skill and verifiers with each step.

Step 1 - Install plugin

In a project that has Tessl initialized, ask your agent:

I need you to install try-tessl/agent-quality

Step 2 - Start the process

Ask your agent to review your agent session sessions with:

Analyze my sessions

If you already have verifiers for your skills, skip straight to Step 5.

Step 3 - Create verifiers

The session will be identified, and ask if you want to create verifiers.

Remember, verifiers are created from Skills, Docs, Rules, and user input, generating checklists that the LLM will judge against. While Tessl automates it, similar to how Tessl generates scenarios in evaluations, it’s recommended you review what’s generated to determine the intent of the skill vs what verifiers are being created.

If you've not already done so, indicate you wish to proceed with generating the verifiers.

If there are no verifiers, ask your agent:

Create the verifiers

You might get asked if you want to create verifiers and/or review friction. In this step, focus on creating verifiers so that you can review them, you will create both in Step 5, which focuses on generating results.

Step 4 - Review verifiers

Your agent will create the verifiers, a summary will be created. Review them to determine if they match the intended purpose of the skill.

Note that if you use verifiers on your skill, Tessl will create a new tile that you can edit if it's not in a workspace you have permissions for.

A verifiers folder will be created with related files.

Once the generation is complete, a set of verifiers are generated for review;

Step 5 - Review agent session

Generate the analysis by asking your agent:

Review the agent sessions

You may be asked if you want to run verifier and friction review, Tessl would recommend running both.

Step 6 - Results review

Once analyis is complete, a summary will be presented.

Review and accept, or modify, any guidance that is provided.

Step 7 - Loop

So now you’ve seen how to create verifiers, and run an analysis, but over time it’s natural to improve your skill, or want to update your verifiers as you observe things while troubleshooting issues. The following guidance will help determine what you should update or which steps to skip to above, when you make these changes:

a) You implement the guidance in step 6: Start an agent session, using your skill, demonstrating normal use over a few sessions, or, over a day or two. After enough data is collected, rerun the analysis to see if it has impact. No verifier modification is required because the guidance that was generated was based on the verifier(s), so you won’t need to update them.

b) You did an analysis, and identified verifiers are not performing as expected: It could be that your verifiers are too wide, resulting in too many things being flagged, or too narrow, where they are not flagging issues you're aware of. In such cases, return to step 4 to modify the verifiers and then rerun the proceeding steps.

c) During your normal workflow you update your skill; Return to step 4 to update your verifiers so they match the new expectations of your skill.

Summary

Ultimately, this is trying to get you data on how your agent is actually doing vs just vibes so that you can iteratively improve it! And when your skill is working well, you’ll have the data to confirm it!

Reviewing agent sessions are a very powerful capability to review what happened in a session, identify friction points and verify if what you expected to occur actually occurred when using the skill. Tessl is building out a powerful toolkit that allows you to evaluate your skill from its packaging, against scenarios, comparing your skill against different models and now providing data on what actually happened during use of a skill.

I Spent a Week Fixing the Wrong Skill (And Other Lessons from Evaluating an AI PR Reviewer)

Tessl — Wed, 08 Jul 2026 08:19:17 +0000

TLDR

The baseline model (Claude Opus, no guidance) already catches ~65% of textbook bugs. The plugin's value comes from false positive suppression and risk classification, because the baseline already catches most bugs on its own.
The plugin had been classifying risk correctly all along. I just wasn't measuring it. One eval weight change, zero code changes, and the gap widened 9 percentage points.
I spent four versions rewriting the reviewer's prompt to fix a false positive. The actual fix was one line in a completely different skill, upstream.

In Part 1, I described the PR review plugin: evidence-first architecture, six skills, risk lanes. It hit 97.7% accuracy across 43 eval scenarios. This post is about how it got there, because the eval journey taught me more than the final number.

How I evaluated the AI PR reviewer

I built four test repos from scratch: data-service, payments-api, web-dashboard, deploy-infra. Each has planted bugs of varying subtlety, from "you forgot to sanitize this input" to "this session TTL is set to zero, which means sessions never expire, which means stolen session tokens are valid forever."

The baseline is Claude Opus reviewing the same PRs with no plugin guidance. Just the model, the diff, and a generic "review this code" prompt. I started with 33 scenarios and ended with 43.

First surprise: the baseline scored ~70% on the initial 33 scenarios. On textbook bugs (missing input validation, obvious SQL injection, unhandled error paths) the baseline catches most of them. The model is smart. This isn't 2023 anymore.

That ~70% is important context for everything that follows. It means any AI reviewer that just adds more bug-finding instructions on top of a capable model is competing for the remaining 30%. And if it generates false positives along the way, it might be net negative. The firehose problem the research warned about.

It also means the baseline's score will drop as the test gets harder, because those easy wins that inflate the 70% start counting for less once you add scenarios the baseline can't handle. Watch the baseline column in the table below. It goes down, not up. That's by design.

Where the gap actually comes from

Version 14, my first serious eval run: plugin 87.8% against the baseline's ~70%. Real gap. Here's what created it.

The plugin found roughly the same bugs with far fewer false positives and better risk classification. The evidence builder's lane system meant the reviewer wasn't hallucinating security findings on docs-only PRs. That's the difference between a review a developer reads and one they close after the second paragraph.

Improving AI review accuracy: domain knowledge, harder tests, better scoring

The first lever was domain knowledge. I taught the plugin about CSV formula injection in export fields (a cell starting with = gets executed by Excel; ask any security team that's dealt with this), Glacier storage cost traps, stale auth cache interactions. The kind of bugs a human reviewer with domain expertise catches because they've been burned before. That took the plugin from 87.8% to 94.5%.

Then I made the test harder. Ten new scenarios, tougher bugs, and I reweighted scoring so the gimme scenarios (where both plugin and baseline score 100%) counted for less. The gap blew open: plugin 94.1%, baseline 64.6%. A 29.5 percentage point spread. The harder I made the test, the wider the gap got.

The most interesting version bump barely touched the plugin at all. I changed the eval's scoring weights: risk classification went from 5 points to 10 points per scenario. The gap widened another 9 percentage points. Same plugin code, same scenarios. The plugin had been classifying risk correctly the whole time; I'd been underweighting the thing it was best at.

Final run, version 21: plugin 97.7%, baseline 66.6%.

`Version  What changed                        Plugin  Baseline  Gap
──────── ──────────────────────────────────── ─────── ──────── ─────
v14      First serious eval (33 scenarios)    87.8%   ~70%     ~18pp
v15      Domain-specific hotspots             94.5%   ~70%     ~25pp
v17      +10 harder scenarios, reweighted     94.1%   64.6%    +29.5pp
v20      Risk classification weight 5→10      ----    ----     +9pp wider
v21      Evidence builder fix (route guards)  97.7%   66.6%    +31.1pp`

Here's what a scenario looks like. This is the session TTL zero eval (one of the "high subtlety" bugs I expected to stump the baseline):

`Task: "Review pull request #5 in the repository ai-pr-reviewer-tests/payments-api."

Criteria (weighted checklist):
{
  "context": "session_data cache TTL set to 0 means sessions persist
    in Redis indefinitely",
  "checklist": [
    {
      "name": "Catches session never-expire risk",
      "description": "Identifies that TTL=0 means sessions stored
        with no expiry, creating stale/orphaned sessions if the
        auth layer fails to explicitly delete them.",
      "max_score": 15
    },
    {
      "name": "Catches unbounded Redis memory growth",
      "max_score": 5
    },
    {
      "name": "Risk classified yellow or higher",
      "max_score": 10
    }
  ]
}`

The task is one sentence. The rubric is weighted: catching the core security risk (session never-expire) is worth 15 points, the memory growth consequence 5, and risk classification 10. The baseline caught this one at 100%.

When fixing the reviewer prompt doesn't work

One scenario gave me the most trouble: a PR adding authorization middleware to three API routes that previously had none. Correct code, good security practice. The plugin kept flagging it as HIGH severity: "potential security misconfiguration in route handling."

I rewrote the reviewer's instructions four times. Version one: I told the reviewer to consider whether route guards are additive security measures. Still flagged. Version two: three sentences with examples explaining that adding a guard is a security improvement. Flagged. Version three: I restructured the entire reviewer prompt section on security findings. Same result. Version four: I got specific. "If the change adds authorization checks to routes that previously had none, this is a hardening change, not a vulnerability."

Still flagged it.

The reviewer wasn't broken. The evidence builder upstream had classified the route change as "red lane": high risk, security-relevant, requires deep scrutiny. By the time the reviewer saw the code, the framing was already set. I'd been tuning the wrong skill for a week.

The fix: I changed the evidence builder's classification logic to recognize that adding guards to unguarded routes is a hardening pattern, not a risk pattern. The evidence pack now classified it as green-lane. The reviewer read the same diff, saw a green-lane classification, and correctly identified it as a security improvement.

4% accuracy on that scenario became 100%. I never touched the reviewer. The only thing that changed was what the evidence builder told it before it started reading the code.

Upstream evidence quality determines downstream review quality. The reviewer is only as good as the evidence pack it's handed. Fixing the reviewer's prompt is like arguing with a judge after the prosecution already presented tainted evidence. The bias is baked in before the verdict.

Here's the actual text I added to the evidence builder's risk classification logic:

`Auth risk requires call-site analysis. Do not classify a PR as red
solely because it touches permission-checking code. Read the call
sites to determine whether the effective access policy changed.

For example, a switch from every() to some() on a role array changes
behavior — but if every call site passes OR-style role lists, some()
is the correct semantic and the change is a bug fix, not a regression.
Classify based on whether the access policy actually changed.`

That's it. One paragraph of guidance in the evidence builder, telling it to check call sites before panicking about auth changes. The reviewer's prompt didn't change at all.

AI catches more bugs than the research predicted

I designed several "high subtlety" scenarios expecting them to stump the baseline. Session TTL set to zero. A crash in an authentication provider that fails open instead of closed. The baseline caught both at 100%.

Models are more capable than the 2025 research estimated. The window for "bugs only AI-guided review can find" is narrower than I assumed, which is exactly why the plugin's value lives in the evidence pipeline (risk classification, false positive suppression, structured handoff) rather than in raw bug detection.

LLM variance, though, is real. One scenario (correlation ID propagation) scored 88% in one run and 36% in another. Same scenario, same plugin, same model. The difference is just... the model having a different day. Single-run evals can lie to you. I learned this the hard way in the Good OSS Citizen work, and I still almost got burned by it here.

The gap we haven't closed: developer trust

I validated one thing: does the plugin find the right problems and classify them correctly? Yes. 97.7% across 43 scenarios says yes.

I did not validate the thing that actually matters: do developers trust what it finds and act on it?

The 2025 research says AI review comments get adopted 1-19% of the time. My plugin produces better-structured, higher-signal findings. Maybe that adoption rate is higher. Maybe it isn't. I have zero data.

The retrospective skill exists. It's designed to compare the plugin's findings against human decisions and feed the results back. I never ran it. Not once. The plugin has a feedback loop that has never looped.

I designed for human handoff because the research told me to, and I still haven't tested whether the handoff actually works. Finding the right bugs is solved. Whether a developer reads the brief and actually changes their merge decision, that's the question this plugin can't answer yet, and it's the one that decides whether any of this matters.

Try it yourself

`tessl install tessl-labs/pr-review-guardrails`

The eval corpus is in the GitHub repo. Forty-three scenarios across four test repos with rubrics. Fork it, add scenarios from your own domain, run the eval. If you use the retrospective skill on a real PR, you'll have more adoption data than I do.

Further reading: Part 1 (what the plugin does and how to use it) | Research brief and eval corpus

I Built an AI PR Reviewer That Catches Bugs by Not Looking for Bugs

Tessl — Tue, 07 Jul 2026 06:43:42 +0000

TLDR

Humans don't want to review AI-generated code (why spend an hour reading something that took 30 seconds to generate?), and AI reviewers get ignored 81-99% of the time. PR review is broken from both sides.
The plugin that hit 97.7% accuracy doesn't hunt for bugs. It builds an evidence pack, classifies risk into lanes, and hands a structured brief to a human who makes the actual call.
Install it with tessl install tessl-labs/pr-review-guardrails and point it at a real PR. You'll know in five minutes whether this approach works for your codebase.

PR review is broken from both sides.

Humans don't want to do it. The effort asymmetry is brutal. An agent generates a PR in 30 seconds, and now a human is supposed to spend an hour carefully reading code they didn't write, didn't design, and can't ask clarifying questions about. That's a hard sell even when the code is good. When the code is AI-generated, the motivation drops further. Who wants to be a proofreader for a glorified word-guessing monkey?

So hand it to another AI? The 2025 research says that doesn't work either. AI code review comments get adopted 1-19% of the time, depending on the study, while human reviewer comments land at significantly higher rates. The gap is signal-to-noise. AI reviewers flood PRs with findings, most of them either obvious (the linter already caught it) or wrong (the code is fine, the reviewer hallucinated a vulnerability). Developers learn to ignore the firehose.

I built a Tessl plugin to try a different approach. A Tessl plugin (used to be called a "tile") is a context artifact: a bundle of skills, rules, and scripts that gives an AI coding agent domain-specific context. Think npm packages, but for agent behavior instead of code. Mine doesn't try to be a better bug finder. It builds a dossier of evidence about the PR, classifies the risk, and hands a structured brief to a human who makes the actual call.

Where it started

Earlier this year, I spent some time researching how AI-generated PRs are wrecking open source maintainers. That became the Good OSS Citizen plugin, teaching agents how to contribute. But while studying the flood of AI-generated PRs, I kept circling back to the other side: who reviews all this code?

The research said something useful: AI is good at local, checkable problems: buffer overflows, missing null checks, SQL injection in a query builder. Things where you can point at a specific line and say "this is wrong because X." What AI is bad at is intent, architecture, and trade-offs. The stuff that requires understanding why the code exists, not just what it does.

So the design question became: what if the AI reviewer's job isn't to find bugs? What if its job is to gather evidence and let the human make the call?

Build the dossier first. Let the opinions follow from that.

How the evidence-first review pipeline works

The plugin has six skills. The first one matters most.

The evidence builder reads the diff, maps which files changed, figures out what kind of change this is, and classifies risk into lanes: green (routine), yellow (needs attention), red (security-relevant, requires deep review). Everything downstream flows from this classification. A README fix gets a green lane and a light pass. A change to the auth middleware gets red and the full treatment.

Then the fresh-eyes reviewer gets the evidence pack and the code. It hunts for problems, but only problems the evidence supports. If the evidence builder classified a PR as green-lane, the reviewer isn't going to invent an exotic attack vector in a README change. If I enabled the optional challenger (a second model checking the first reviewer's work), that runs next. The research says cross-model review works as a verification layer, and I wanted to test that claim.

After the review, a synthesizer compresses everything into a single brief with findings, evidence, confidence levels, and a recommendation for what a human should focus on. The human handoff formats that brief for the person who actually decides whether to merge.

There's also a retrospective skill that's supposed to run after the human makes their call, comparing the plugin's findings against the human's decision. A feedback loop that's supposed to improve the plugin over time.

What an AI code review brief looks like

When you run the plugin on a PR, the human reviewer gets a brief. It looks like this:

The brief starts with risk classification (green, yellow, or red) so you know immediately how much attention this PR needs. A green-lane config change gets a one-paragraph summary. A red-lane auth change gets the full breakdown: which files are security-relevant, what data flows through them, what the specific risks are, and what to look for when you read the code.

Each finding comes with evidence: the specific lines, why the plugin flagged them, and a confidence level. A finding that says "this user input reaches the SQL query on line 47 without sanitization" is something a developer acts on. A finding that says "potential security concern in this module" gets ignored before the developer finishes reading it. The plugin is built to produce the first kind.

The brief also tells you what it didn't check. If the PR touches areas outside the plugin's domain knowledge, it says so instead of pretending it reviewed everything.

Here's what the plugin produced for a real PR that changes Redis cache TTL configuration in a payments API:

`PR: #5 — Update Redis cache TTL configuration
Risk lane: RED
  - Cache invalidation logic changes with auth-adjacent session_data prefix
  - TTL=0 introduces keys that never expire (memory and security implications)
  - Mandatory human review required (auth/security, cache invalidation)

Finding 1 [HIGH / verify]: session_data TTL config entry has no consumer
  File: src/cache/cache_layer.py:17
  "session_data": 0,  # sessions managed by auth layer, no TTL needed
  Evidence: grep for session_data across src/ returns zero results.
  src/auth/sessions.py manages its own Redis keys with 24h TTL,
  bypassing the cache layer entirely.

Finding 2 [HIGH / fix]: Zero-TTL cache entries persist forever
  File: src/cache/cache_layer.py:47
  if ttl == 0: r.set(key, json.dumps(value))
  Evidence: No background cleanup, no maxmemory-policy safeguard,
  no monitoring for key count growth. Gradual Redis memory leak.

Finding 3 [MEDIUM / discuss]: payment_details staleness window widened to 5min
Finding 4 [MEDIUM / fix]: New ttl==0 branch in set_cached is untested

Questions for human reviewer:
1. Is there a planned follow-up PR that routes session data through cache?
2. Are Stripe webhooks invalidating cached payment details on status changes?
3. What is the Redis maxmemory-policy in production?`

Four findings, each with the specific file, line, code, and evidence trail. The human reviewer knows exactly what to focus on and why.

What AI code review still can't do

The plugin doesn't replace the human reviewer. The research is clear on this: AI review catches local, checkable problems. Intent, architecture, trade-offs: those are still yours. The plugin's job is to do the tedious forensic work (trace this data flow, check this input path, verify this config isn't exposed) so the human can focus on the questions only a human can answer: should this feature exist? Does this design make sense? Is this the right trade-off?

It also doesn't have real-world adoption data yet. I validated that it finds the right problems across 43 eval scenarios (97.7% accuracy against a 66.6% baseline). I did not validate whether developers trust what it finds and act on it. That's the honest gap. If you run the retrospective skill after a real review, you'll have more data than I do.

In Part 2, I'll show how I built the eval, what I learned from eight rounds of iteration, and the debugging story where I spent a week fixing the wrong skill.

Try it

`tessl install tessl-labs/pr-review-guardrails`

The plugin, the eval corpus, and the research brief are all in the GitHub repo. Point it at a PR you've already reviewed and compare its brief against what you found. That's the fastest way to know if the evidence-first approach works for your codebase.

Further reading: Good OSS Citizen Part 1 (the research that started this) | Research brief and eval corpus

Tessl Academy is live (in preview) — and there are two ways in

Tessl — Sat, 04 Jul 2026 07:13:53 +0000

Tessl Academy is live (in preview) — and there are two ways in

We just shipped the first version of Tessl Academy, a hands-on curriculum for building, evaluating, and running skills for coding agents. It's early. Two courses are up — Skill Foundations and Tuning Your Agent — with more on the way. We'd rather get it in front of you now and shape it with your feedback than polish it in private for another month.

Here's the idea. Most of us are already using coding agents, but the results swing between magic and mess. The Academy is about closing that gap: moving from one-off AI coding experiments to workflows you can repeat and trust. Skills are the thread running through every lesson — small, reusable instructions your agent loads on demand.

Two ways to take it

We built the Academy so you can learn whichever way suits you right now:

Read it. Every lesson works as a plain read on the site. No install, no setup — open a lesson and go. Good for a commute, a coffee, or deciding whether the hands-on version is worth your time.
Run it. Install a course once, then ask your agent — Claude Code, Cursor, Codex, or Tessl Agent — to walk you through a lesson. It guides you one step at a time, waits while you work, and hands off to the next lesson when you're done. You learn skills by building one.

Same content, two speeds. Start by reading and switch to hands-on whenever you like — the Quickstart gets you running in about four steps.

It's a preview, and your feedback shapes it

This is genuinely a first cut. Some lessons will land, some won't, and the roadmap past these two courses is still open. That's where you come in: tell us what's confusing, what's missing, and what you'd want to learn next.

Join the conversation in our Discord
Or email me directly: alan@tessl.io

I'll be reading everything. Expect the Academy to move quickly over the coming weeks, and the fastest way to influence where it goes is to try it and tell me what you think.

Start with the Quickstart →

Your agents keep making the same mistakes. Nobody has time to fix it.

Tessl — Wed, 01 Jul 2026 07:18:31 +0000

Your agents keep making the same mistakes. Nobody has time to fix it.

AI coding agents are getting better at the tasks you give them direct feedback on. Everything else stays broken.

You leave the same comment in code review three sprints in a row. There's a recurring task that could run as an automation but it's on the backlog because no one has time to stop and systematize it. The context your agents need to do better work — updated conventions, patterns from past PRs, recurring fixes — exists in your commit history and session logs. Nobody has time to extract it and package it up.

Agent enablement is real work. It just never gets done.

What teams usually do

Most teams handle this one of three ways: rely on PR review to catch the same errors week after week, schedule occasional cleanup sprints to update skills and conventions (that never actually get scheduled), or accept that their agents plateau.

All three require engineers to stop building to maintain the thing that's supposed to help them build faster.

Introducing Tessl Agent — open beta

Today we're launching Tessl Agent.

Point it at a repo. It scans your PRs, coding agent session logs, and tickets continuously. When it spots a recurring error pattern, it creates a skill to address it and opens a PR. When it finds a task your team runs manually every week, it turns it into a GitHub Actions workflow. Then it asks if you want it to keep doing that automatically; daily, weekly, on a schedule you set.

Tessl Agent is built to get you to stop using it interactively. You work with it, and at the end of each session it says: I could set some of these up as recurring actions. I could create a CI/CD check for this. The goal is that most of the recurring work — finding optimizations, catching agent mistakes, updating context — runs on a trigger and files issues without you having to ask.

What it looks like in practice

The use case we use most at Tessl: setting up an agentic code review harness.

You type something like set up agentic code review or I want to spend less time reviewing code. Tessl Agent scans your PRs, your issue tracker, and your coding agent session logs. It surfaces what's there: your style guide, common agent failure patterns, comments your team leaves repeatedly in review. Then it walks you through building on that.

First, it creates a code review skill that maps to your team's best practices. Unlike a one-click tool you forget about, this is a skill you own; you can update it, augment it, share it across workflows. From that point, every PR gets agentic review automatically. Then it sets up a recurring loop that optimises that review over time, so the quality of automated review improves as your codebase evolves.

You spend time reviewing code and shipping features, knowing the routine work is handled.

It works alongside your coding agent, not instead of it

Tessl Agent is not a replacement for Claude Code, Codex, or whatever you're using. It runs in the background. You don't context-switch to it mid-session.

It's also provider-agnostic — it works with CodeRabbit, GitHub Actions, and your existing stack. It's not tied to any one coding agent, which matters when you want something that works across your whole development workflow, not just within one tool.

The compounding effect

This is what loop engineering looks like in practice. Each automated improvement creates the conditions for the next one.

A skill that encodes a common pattern means your agent makes that class of error less often. An automated workflow that runs weekly means recurring tasks get systematised instead of repeated. At some point you look up and 40, 50% of your PRs don't have a human looking at them. You never had to run a big initiative to make that happen. You got started, kept building, and over time delegated more to the agent.

That's the path toward a software factory. Not a big-bang platform migration, but incremental agent enablement that compounds week over week.

Try it

Tessl Agent is in open beta and free to try. Download the Tessl CLI, run tessl, and open a session. A good starting point: pull up the last month of your team's coding agent sessions and ask what's broken, what's taking a lot of your time. The findings tend to be immediately useful.

Try Tessl Agent for free or book a demo.

Why Warp is betting engineering leaders are done picking a favourite coding agent

Tessl — Mon, 29 Jun 2026 06:48:20 +0000

Engineering leaders have spent the past year trying to get their teams to adopt AI coding tools as quickly as possible. Now, a new set of questions has taken over: how do you measure whether any of it is worth the money, and how do you stop agents from running unchecked on production systems?

Developer tooling company Warp, an open agentic development environment built from the terminal up, thinks the answer isn't picking a single agent and standardising on it — it's giving teams a way to run several at once, compare them, and govern all of them from a single control plane.

As Tessl wrote back in February, orchestration has emerged as a discipline in its own right — a dedicated layer of tooling for coordinating, supervising and directing multiple agents running in parallel. Back in February, Warp launched Oz as a cloud platform for running and managing coding agents at scale.

Now, Warp is taking things a step further. In May, the company expanded Oz into what it's calling the first multi-harness control plane — meaning teams can now run Claude Code, Codex and Warp Agent simultaneously through a single interface, rather than committing to any one of them.

Tessl caught up with Warp CEO Zach Lloyd to discuss how engineering leaders are thinking about agent fleets, what the harness layer actually changes, and where the lines between autonomy and human oversight are really being drawn.

"The wild west": how the agent gold rush became a budget problem

Zach spent several years at Google, leading engineering on Docs and Sheets before co-founding photo-editing startup SelfMade. He later served as interim CTO at Time, before founding Warp in 2020, raising north of $70 million in funding from the likes of Sequoia, Google Ventures, Figma co-founder Dylan Field, and Salesforce’s co-founder Marc Benioff.

That background — building collaborative tools at Google scale, then navigating the startup world — gives Zach a particular vantage point on how quickly the engineering tooling landscape has moved. A year and a half ago, he says, most companies were still trying to get developers to use AI autocomplete tools. Then, about a year ago, the conversation moved to interactive agents — Claude Code, Codex, Warp — where engineers were directing tools to build features and fix issues end to end.

Now, he says, that phase too has largely passed — and the CFO's arrival in the conversation is perhaps the clearest sign of it.

"Companies right now have moved from a 'can we get people to adopt' mindset to a 'how do you measure ROI' mindset," Zach explained. "They're paying a lot of money for these tools, and the CFO has gotten involved. All these costs are showing up, and so they are thinking through how to go from the wild west, where every engineer is just spending as much as they can on different agents, to a world where they're still creating as much productivity as possible. But they want to measure it, they want to put quotas and budgets in place, and they also want to use different agents for different types of tasks."

That last point is central to Warp's multi-harness bet. Rather than standardising on a single agent, Zach argues that engineering teams want the ability to route different tasks to different agents depending on what each does best — while keeping the governance layer consistent across all of them.

"The biggest trend that we see is: can you use open-weight models for some tasks when you have to be at the frontier?” Zach said. "The way that we're positioning Oz is that you can basically not lock into one source of intelligence. You can use Claude Code, you can use Codex, you can use open-weight models — but you can still confidently invest in a layer of infrastructure for governance that is not tightly coupled to any one particular agent."

The economics driving that are already visible. Open-weight models — DeepSeek, Kimi, Qwen — have gone from lagging well behind the frontier to matching it on many tasks, and at a fraction of the inference cost. Tessl also recently switched its default eval model from Claude Sonnet 4.6 to GLM 5.1 for exactly this reason — finding that for skill evaluation work, a cheaper open-weight model produced near-identical signal at meaningfully lower cost.

Elsewhere, AI agent startup Lindy recently moved 100% of its traffic from Anthropic to DeepSeek v4, with CEO Flo Crivello claiming the company would be saving millions in the process.

It's worth noting that Warp has been doubling down on openness more broadly, open-sourcing its client earlier this year and using Oz itself to manage the repo — agents handle the implementation, community contributors handle direction and verification.

“We now have a lot of confidence in code that is generated by Oz with our rules, context and verification, so anyone contributing should have a high chance of success coding a feature correctly,” Zach said at the time.

The move also serves as a live test of Warp's own thesis — if the orchestration layer is good enough to run a public repo at scale, it's good enough for enterprise teams to trust with their own.

“Leaning on agents creates pressure for us to nail orchestration, memory, handoff, and all of the other parts of agentic engineering that are core to our business,” Zach continued. “There’s a virtuous loop here.”

That loop extends to customers too. The things that matter most — context management, memory, audit logs — can all be separated from the agent itself, Zach argues. That's the point of Oz: a container layer for all of it, so that when the best model or harness changes — and Zach is clear that it will, every few months — teams aren't starting from scratch.

The model isn't enough: why the harness and context matter just as much

The natural question is whether multi-harness is a solution in search of a problem. If Claude Code and Warp Agent can both run on Anthropic models, what is the harness actually changing?

Zach's answer is that performance is a function of three things working together: the model, the harness, and the context.

"The harness is what feeds the context in," Zach said. "You want a harness that is good at managing the context window — when do you take different sources of external context and put them in? If you put too much context in, the model has to summarise and it loses information on the current task. How you manage that context window is really important. Different harnesses excel at different things — Claude Code is a great harness, Codex is a really good harness, Warp's agent harness is [also] really good."

The model and the harness are table stakes. The third element — organisational context — is where Warp is investing most heavily right now, through what it calls cross-harness memory. The idea is that as agents complete tasks, the system captures what worked and surfaces it automatically in future runs, across whichever harness is being used.

"Every time one of these agents runs, it does some task, and maybe in the course of figuring out some problem, with the guidance of a human, they arrive at some solution," Zach said. "What you don't want to do is throw that away and start from scratch next time. If you have a memory system, think of it as a layer that is observing what all of your agents are doing and being like: this seems like an important thing to remember."

Cross-harness agent memory is currently in research preview with a small number of pilot customers.

More autonomy, more controls: Warp's answer to an uncomfortable balancing act

The tension at the heart of Oz's pitch is one that Zach doesn't try to resolve so much as manage. On the one hand, the platform promises agents that can handle complex, long-running tasks — migrations, production deployments — with less human oversight. On the other, the same release adds approval gates, per-user authentication, and least-privilege permissions.

Those two things pull in opposite directions.

"I think there's a fundamental tension, but I think it's necessary," Zach said. "From talking to our customers, I don’t think companies are ready to be fully hands off. The ideal system at this moment looks like a factory floor, where you want to put stuff that can be automated through an automation process, but then you want a human to step in and say: ‘was this done right’?"

The logic Zach applies is essentially risk-tiering. The parts of the stack where errors are cheapest get automated first; the parts where they are most costly stay human-supervised longest.

"The parts that can be most automated are the parts where the risks are lowest — this is common sense," Zach said. "Making changes to our website is way lower risk than making changes to our data. So you'll see more and more of the guardrails go away on the low risk things before they go on the high risk things."

As for who inside an enterprise actually draws those lines, Zach says it's rarely one team. Platform teams or dedicated AI developer productivity functions tend to lead, with security always involved and finance increasingly so.

"The security team is always involved — probably the team that's most scared," Zach said. "Increasingly there is a cost management component. What's the budget for this? What's the token budget per engineer? What's the way that you see ROI? It's starting to become a significant line item for all of these customers."

Evals: measuring the factory floor

Which brings the conversation to evals — how teams actually know whether any of this is working. Zach's framing here draws again on the factory floor analogy: what you want, ultimately, is a bird's eye view of how work flows from idea to shipped product.

Warp has built a live version of this for its own open-source repository at build.warp.dev, where anyone can pull up a view of how issues move through the agent pipeline. Zach uses it as a reference point for what enterprise teams should be aiming for.

"The things you can measure are throughput of code as one basic measurement," Zach said. "Ideally, in a more sophisticated world, you would go all the way from measuring throughput of code to throughput of user or customer impact — be able to tie back: ‘a ticket came in asking for this feature, an agent was able to build it, it cost this number of dollars or tokens, and in production it was used by XYZ customers’. That's the dream loop. The code part is not that hard — that's where we can just deliver."

Token efficiency per PR is the baseline metric Warp currently offers. The harder problem — tying agent output to business outcomes — remains what Zach calls the “holy grail.”

The agent builder: a new role that doesn't require an engineering background

One of the more striking parts of the conversation is what Zach describes happening to engineering teams themselves as agent fleets become the norm — at Warp and at the companies it works with.

The background profile of engineers Warp hires hasn't changed much, he says. What has changed is what they do.

"The day to day of a software engineer now is not about writing code," Zach said. "It's about: can you accurately specify a user requirement to an agent? Can you make sure that the technical plan an agent comes up with makes sense? Is it building in the right part of the codebase? Is it repeating a bunch of code? Is it using the same quality of abstraction that a human would use?"

Beyond that shift in existing roles, Warp has also introduced a new function it calls the agent builder — a full-time role focused on building internal automations using agents. Notably, the people filling it don't come from engineering backgrounds.

"The people who are in this role are people with product and design backgrounds," Zach said. "They are not engineers by training, and I don't think you need that. For internal tooling use cases you can hire people who are more generic builders. One of the cool things that's come out of all this new technology is a democratisation of who gets to build stuff."

The caveat is that this only holds where the stakes are low — customer-facing product, he implies, is a different matter. "As long as it's not customer-facing, I think it's pretty much fine for that to work that way," Zach said.

Among the companies Warp works with, Zach sees two distinct camps emerging. Larger organisations with dedicated developer productivity teams are building their own internal software factories from scratch — the complexity is manageable if you have the headcount. Smaller ones are buying, because the build cost simply doesn't justify the investment. What they share, he says, is the destination: a centralised system where agents handle the routine work and humans focus on the exceptions.

What that means in practice for engineering leaders is less about which agent to pick and more about building the layer around it — the governance, the memory, the measurement — that makes any agent trustworthy enough to run at scale.

For all the variation in how companies are approaching this — different tools, different team structures, different risk tolerances — Zach sees them all heading toward the same place.

"The goal of most companies right now is to get to what I would call an internal software factory — a centralised system where agents are taking in issues, judging, building, verifying, pushing," Zach said. "They don't want to do that for 100% of the issues, and they don't want to take humans out of the loop. But they're all trying to stand up this same kind of machine. And different companies are further along on this journey than others.”

See You at AI Engineering World's Fair 2026

Tessl — Sun, 28 Jun 2026 07:51:55 +0000

Next week, the Tessl team is heading to AI Engineering World's Fair 2026, and we couldn't be more excited to spend a few days with the community talking about the future of AI engineering.

If you're attending, come and find us at Booth L-G48. We'll be demoing our latest product, sharing what we've been building, and talking all things agentic development with engineering teams from around the world.

Come and meet the team

At Tessl, we believe skills are the new code. Treat them that way.

Tessl enables development teams to continuously build, test, distribute and optimize agent skills with the security and governance of enterprise software.

Throughout the event, our technical team will be running live demos at the booth and chatting with attendees about everything from coding agents and agent workflows to evaluation, context management and harness engineering. Whether you're just getting started or already deploying agents in production, we'd love to hear what you're building.

We're also running a competition throughout the conference, with prizes including:

🎁 Ray-Ban Meta Smart Glasses
🎟️ A ticket to AI DevCon in New York this November

Unveiling Tessl Agent

AI agents shouldn't just write software—they should continuously improve how software gets built.

At AI Engineering World's Fair, we'll be unveiling Tessl Agent.

Build your software factory, one workflow at a time.

Tessl Agent makes your agents more autonomous over time. It continuously scans your pull requests, session logs and tickets for recurring mistakes and opportunities, automatically opens improvement PRs, turns repeated patterns into automated workflows, and ships them through GitHub Actions—creating a software factory that compounds week after week without slowing feature delivery.

If you'd like to see it in action, stop by the booth for a live demo.

The conversation we're most excited about: Harness Engineering

Every conference has a theme. This year, we think it'll be Harness Engineering.

AI models are getting smarter every month. The challenge is everything around them.

Agents need context. They need evaluation, testing, guardrails, observability and workflows that help them operate reliably in production. In short, they need a harness.

We believe Harness Engineering is becoming one of the defining disciplines of modern AI engineering, and we're looking forward to hearing how the community is tackling these challenges.

Catch our talks

We're delighted to have two Tessl speakers presenting on Thursday, July 2.

Coding Agents Don't Scale Themselves. Neither Do Your Teams: The Rise of Agent Enablement

🕜 1:30–1:50 PM

Patrick Debois, AI Product Engineer

Coding agents are transforming software development, but the context that drives them is still managed with ad hoc prompts, copied rule files and undocumented practices.

Patrick introduces the Context Development Lifecycle—a framework for treating context with the same engineering discipline we've spent decades applying to code—and explores how teams can build a feedback loop that continuously improves agent performance over time.

Harness Engineering: The New Core Skill for Agentic Developers

🕝 2:50–3:10 PM

Dru Knox, Head of Product & Design

As coding agents become more capable, success depends less on writing code and more on upgrading your codebase so agents can reliably succeed.

Dru introduces the core loop of Harness Engineering, the common improvements teams are making today, and how Tessl's Harness Engineering Agent helps developers scale those improvements across their software factory.

Join our community event

We're also hosting an evening fireside discussion:

Harness Engineering: Building Reliable AI Systems

📅 Wednesday, July 1 | 6:00 PM

Featuring Steve Yegge and Dru Knox, this conversation explores the emerging discipline of Harness Engineering and what it takes to move AI systems beyond experimentation into reliable production software.

Together they'll discuss the systems surrounding AI models—from context and evaluation to testing, observability and guardrails—followed by audience Q&A and networking with the AI engineering community.

👉 Reserve your place: https://luma.com/7f31tcht

Leadership dinner

Alongside the conference, we're also hosting an invite-only leadership dinner, bringing together engineering leaders and AI practitioners for an evening of conversation about the future of agentic development.

We're looking forward to sharing ideas with some of the people helping define where this industry goes next.

See you next week

AI Engineering World's Fair has become one of the best places to connect with the people shaping the future of software engineering, and we can't wait to be part of it.

Whether you want to see Tessl Agent in action, chat about Harness Engineering, attend one of our talks, or simply swap ideas about building reliable AI systems, we'd love to meet you.

Come and see us at Booth L-G48.

Or, if you'd like to guarantee some time with the team, book a meeting with us through the AI Engineering World's Fair app.