DEV Community: jamilxt

Anthropic's $895 Billion Compute Commitments Change How Teams Should Buy AI

jamilxt — Tue, 03 Nov 2026 16:07:35 +0000

A software team can compare AI APIs by model quality, latency, and price per token. Anthropic's November 2, 2026 IPO amendment adds another question: what happens to your product if the provider's ability to serve you depends on infrastructure deals measured in hundreds of billions of dollars?

The filing reports about $895.4 billion in aggregate committed spend under purchase commitments and lease agreements, substantially all related to compute and infrastructure. It does not say Anthropic must pay that whole amount now. About $447.5 billion is expected before the end of 2030, and nearly $70 billion is due in the 12 months after September 30, 2026. The distinction between a headline total and the timing of cash obligations matters. Anthropic's SEC filing lays out the detail; Associated Press reporting independently confirms the quarterly results and near-term commitment figure.

This does not mean Anthropic is about to fail, or that companies should stop using Claude. It is a reminder that AI vendor risk now includes the physical supply chain behind a model. Engineering teams cannot predict a provider's future, but they can design systems that handle a change in price or product terms.

What the $895 billion figure actually means

The figure combines several types of agreements. Anthropic describes purchase commitments and lease agreements, including short-term commitments and leases that have not yet started. The total is an aggregate committed-spend figure, not a bill due today, not annual spending, and not a forecast of compute that has already been delivered. The filing says roughly half is expected before 2031, with most of the rest expected from 2031 through 2045. The SEC filing's risk discussion gives those time ranges.

That time horizon says something about the business model. A frontier AI provider cannot treat compute like a small, flexible cloud add-on if it needs to reserve data-center capacity years ahead. It may sign long agreements to secure access before customers arrive. If demand grows, those reservations can support service expansion. If demand, model efficiency, or the business changes, fixed commitments can become a constraint.

The filing itself describes both sides. Anthropic says access to large-scale compute is important to train and serve models, and that its results may fluctuate with substantial compute-related spending. It also reports that in the third quarter, $5 billion of short-term compute cost significantly more than comparable capacity under multi-year commitments. The extra compute reduced revenue per dollar of compute compared with the previous quarter. That is the operating tradeoff: short-term capacity can cost more, while long-term commitments create obligations that may last well beyond current product cycles. This is the company's disclosure, not a prediction that its contracts will go wrong.

Revenue is growing fast, but that is not the same as an easy business

The amendment shows third-quarter revenue of about $17 billion, while Anthropic still reported a quarterly net loss of $23.3 billion. For the first nine months of 2026, revenue was $33.2 billion and the reported net loss was $104 billion, including a $94 billion accounting charge related to financing agreements, according to the Associated Press's breakdown of the filing.

Those numbers need context. A large reported net loss does not mean the company paid that same amount in cash during the period. The filing identifies a major non-cash accounting charge. Fast revenue growth also does not erase the need to pay for infrastructure and customer acquisition. AP reports that revenue for the first nine months rose to $33.2 billion from $2.6 billion a year earlier. That growth is striking, but it does not prove that each product or customer segment has durable margins.

The engineering lesson is narrower than an investor verdict: model access can be commercially valuable and still depend on a costly, capacity-constrained supply chain. A software team buying an API is buying a service layered on top of that chain.

Why application teams should care

Capacity is part of reliability. An API can be reachable while a provider still changes rate limits, queues, available model options, or regional capacity. Contracted infrastructure can help a provider meet demand, but it does not guarantee every customer a fixed service level. Your product needs a plan for timeouts and temporary model unavailability.

Price is connected to utilization and efficiency. Providers can change prices as hardware supply, model efficiency, and usage patterns shift. A workload that is affordable at today's rates may become expensive when users send longer prompts or agents make more calls. Measure cost per completed user task, rather than per API request.

A model name is not a stable interface. Model versions can be retired, behavior can change, and product features may be tied to one provider. If important business logic depends on a particular model's response style or tool behavior, a migration may take more than changing a URL. The system needs tests that capture what the application actually expects.

Cloud relationships can affect distribution and supply. Anthropic's filing discusses its reliance on outside compute providers and the risk that those relationships may not always fit its long-term interests. This is not unique to Anthropic. AI services depend on links between model developers and infrastructure suppliers. Teams should understand which parts of their application rely on a single supplier.

A practical AI-provider review for engineering teams

You do not need to build a universal multi-model platform on day one. You do need to know what would break if a provider became unavailable or changed its terms. Use this checklist before putting a model at the center of a product:

Name the critical path. List which user journeys require the model, which can degrade gracefully, and which must stop safely. A support-summary feature and an account-access decision should not share the same fallback policy.
Measure by completed task. Track task outcomes and cost for representative user requests. A cheaper response that creates more retries or human correction may cost more overall.
Keep a versioned evaluation set. Save representative inputs and expected output properties. Re-run it when the model or prompt changes. Measure task success and harmful errors, not whether output merely looks fluent.
Separate business rules from model choice. Keep authorization and irreversible actions in ordinary application code. Let a model suggest or classify; use deterministic checks to decide what the system is allowed to do.
Build a tested degraded mode. Decide whether the product should queue work or ask the user to retry. Test the behavior rather than assuming a fallback will work under load.
Read the commercial terms. Check rate limits, notice periods, model retirement policies, data handling, support commitments, and any minimum-spend terms that apply to your account. A public pricing page is not a substitute for the contract you signed.
Estimate the cost of leaving. Identify provider-specific prompts, tool calls, embeddings, stored data, and evaluation work. Put a rough migration estimate in the architecture record, even if you decide the current integration is worth it.

A second provider can help, but only when it has been tested against real tasks. Different models may return different formats, refuse different requests, or need different context. A routing layer without shared evaluations can make it harder to notice regressions. For a small application, a clear primary provider plus a documented manual fallback may be safer and cheaper than a complex abstraction that nobody exercises.

The larger shift: AI is an infrastructure buying decision

For years, teams could treat many software APIs as replaceable services. AI makes that assumption less reliable because quality, inference cost, data-center capacity, and model behavior are tied together. The provider is not simply returning a static lookup. It is allocating scarce compute to produce each result while also investing in future models and infrastructure.

Anthropic's filing makes the scale visible, but it does not tell us which provider will win or whether its commitments will produce a return. The company has disclosed its view of the opportunity and its risks in a preliminary securities filing. It is not an independent forecast. What teams can take from it is a more practical point: the vendor's capacity plan is part of the environment your application runs in, even when your code only calls an API.

That means procurement and architecture should meet earlier. Before adopting an AI feature, product owners should say what user value justifies its ongoing cost. Engineers should define what happens when the model is slow or unavailable. Finance and procurement should understand the service terms and exposure to usage growth. Those are normal software reliability questions, now applied to model services.

A decision rule worth saving

Before making a model provider a critical dependency, ask three things:

Can the product remain safe and useful during an outage or rate-limit event?
Can the team detect a quality or cost change before users bear the impact?
Can the team move the workflow, or narrow its use, without rebuilding the whole application?

If the answer to all three is no, the immediate problem is not that the provider has large infrastructure commitments. It is that the application has no plan for dependence on an external service. Fix that while the integration is still small.

Subscribe for practical, source-based analysis of AI infrastructure and software engineering. It is free.

Have you made a model provider a critical dependency in a product? What would you test first if its capacity or pricing changed?

Sources: Anthropic's November 2, 2026 S-1/A filed with the SEC, Associated Press coverage of Anthropic's amended filing, Axios coverage of the compute commitments.

Supabase MCP vs Agent Skills: Give Coding Agents Context Without Overexposing Data

jamilxt — Tue, 03 Nov 2026 12:07:16 +0000

A coding agent can produce valid SQL and still make a poor Supabase change. It might create a view that bypasses Row Level Security, use an outdated CLI command, or guess its way through a migration. Giving it a live connection to your project solves only part of that problem. Giving it a checklist without a connection solves a different part.

The distinction is newly relevant because Supabase announced an MCP server for apps on October 22, 2026. That lets an app expose selected actions to a customer's agent. It is not the same connection as Supabase's developer MCP server, which lets a coding agent work with a developer's Supabase project. The names are similar, but the users and permission boundaries differ.

That is the practical difference between Supabase Agent Skills and the Supabase MCP server. One teaches an agent how to approach Supabase work. The other lets it inspect and act on a Supabase project through structured tools. They are often presented as alternatives, but they address different failure modes.

This comparison is based on Supabase’s current documentation and published evaluation, not on a hands-on test of these tools. The short version: start with skills when your problem is unreliable reasoning or stale knowledge; add MCP when the task needs live project context or action. For consequential work, combine them with a scoped, preferably read-only connection first.

The short version: instructions versus access

Agent Skills provide procedural knowledge. They are portable instruction files that an agent loads when a task calls for Supabase or Postgres expertise. They can tell the agent to check the current docs, use a particular migration workflow, or remember a security rule. They do not connect to your project by themselves.

MCP provides a live tool connection. Once configured, an agent can call tools to inspect tables and migrations, query logs, generate types, or perform other actions allowed by the connection. It does not automatically teach the agent when a tool is appropriate or which project conventions to follow.

Supabase describes the developer MCP connection as access to the project and Agent Skills as reusable, on-demand procedural knowledge. Its current AI tooling guide also offers a plugin that bundles both for supported agents. That framing is useful: skills are instructions, MCP is capability, and a plugin is a convenient package containing both.

A model with a database tool is not automatically a careful database engineer. Likewise, a model with excellent instructions cannot report what is in a live database unless it has some way to inspect it.

Two Supabase MCP servers, two different trust models

The developer MCP server is for building and debugging your project. It connects an AI coding tool to Supabase resources under the developer's account permissions. The tool can access data the developer can access, so project scoping, read-only mode, and limiting enabled tool groups matter. Supabase specifically warns against giving this developer connection to customers or end users.

The new app MCP server is for letting your customers use your product through their own agents. In Supabase's October 22 announcement, the example server uses OAuth to identify the signed-in customer. Tool calls use that customer's Supabase client, so the app's Row Level Security policies determine which rows that user's agent can reach. The server can expose product actions, not a general developer database console. Supabase's example avoids a service-role key and recommends keeping business rules in the same functions the app already uses.

That difference changes the design question. If you are building the product, Agent Skills can guide your coding agent, and developer MCP can give it scoped project context. If you want your customers to use your product from ChatGPT or Claude, the app's own MCP server is a separate product feature: define narrow tools, authenticate each user, and enforce access through the same authorization rules as the application. Do not expose the developer MCP connection as a shortcut for customer access.

What Agent Skills do well

They can correct repeated mistakes before a tool call happens. Supabase says it observed agents skipping Row Level Security policies, inventing CLI commands, and creating views without security_invoker = true. Its skill includes those kinds of rules and tells the agent to consult current documentation rather than relying on training data alone.

That distinction matters because documentation and implementation facts change. A skill should not try to copy an entire product manual into a prompt. The useful form is a compact workflow: identify the task, find current documentation, apply security rules, and verify the result.

The installation is straightforward for agents that support the Agent Skills standard:

npx skills add supabase/agent-skills --all

The repository provides Supabase-specific instructions, while the format is designed to work across several coding agents. That portability makes skills useful when a team uses more than one agent or wants its instructions checked into a project workflow.

Skills do not require project credentials. That is a meaningful advantage during early experimentation. You can test whether an agent follows updated docs and safer implementation guidance without first connecting it to a production project. The agent may still need an ordinary web or CLI route to look up documentation, but the skill itself is not a database credential.

Their limitation is that instructions are not enforcement. A skill can tell an agent to inspect a migration and preserve a security rule. It cannot guarantee that the agent follows the instruction, and it cannot make an unsafe action impossible. For stronger boundaries, use permissions, read-only access, isolated databases, and review gates outside the prompt.

What MCP does well

It supplies current project context. Instead of asking the agent to infer the schema from a few files, a live connection can expose tools to inspect tables, migrations, project settings, and logs. The agent can use actual project state to answer a question or prepare a change.

It supports work that ends in an action. Depending on the enabled tools and permissions, the agent may run SQL, apply a migration, or deploy a function. That can reduce handoffs for development and debugging tasks. Structured tools also give the model a defined interface rather than requiring it to invent terminal commands or parse arbitrary shell output.

But access is not a safety policy. Supabase’s own security documentation warns that data retrieved from a project can contain prompt-injection instructions. A support ticket, comment, or database row could contain text that tries to redirect the agent. If the agent has broad access, that untrusted text may influence what it attempts to do with the connection.

MCP therefore needs deliberate configuration. Supabase documents options to scope a connection to a project, make it read-only, and restrict enabled tool groups. Its guidance recommends keeping manual approval enabled for interactive work and treating approval prompts as a guardrail, not a security guarantee. Some clients can automate approvals, and some tool calls may not trigger a confirmation.

A sensible starting configuration for inspection is project-scoped and read-only:

https://mcp.supabase.com/mcp?project_ref=YOUR_PROJECT_REF&read_only=true

That URL is an example of the documented configuration shape, not a complete account-security plan. Use the current Supabase setup guide and your agent’s client documentation before connecting. Do not put tokens, secret values, or production credentials into prompts or source files.

The comparison that matters in practice

Choose Agent Skills when the problem is “the agent keeps getting the approach wrong.” Examples include stale API knowledge, forgotten RLS requirements, inconsistent migration conventions, or a team-specific verification checklist. Skills can improve the agent’s decision process without granting it access to the database.

Choose MCP when the problem is “the agent cannot see the facts it needs.” Examples include checking which migration reached a project, investigating recent logs, inspecting a schema, or performing a scoped development action. Without a live tool, the agent may be forced to reason from incomplete or outdated files.

Choose both when the task requires context and judgment. A debugging task may need live logs and schema inspection, plus instructions about which security checks to perform and how to propose a fix. In this arrangement, the skill guides the sequence while MCP supplies project-specific evidence.

There is a useful caveat in Supabase’s own evaluation. The company compared a baseline, MCP-only access, and MCP plus its Agent Skill across four agent/model configurations. It reported results over six Supabase scenarios per condition. MCP alone did not improve every baseline: Claude Code Opus 4.6 scored 58% at baseline and 50% with MCP only, while Codex GPT-5.4 stayed at 71% in both conditions. Adding the skill raised those reported scores to 67% and 88%, respectively. Supabase labels the sample small and the results early, so these numbers are evidence about those tested scenarios, not a universal benchmark or proof that skills always improve every agent.

The more useful takeaway is about task design. A tool connection can make more information available without making an agent use that information well. An instruction file can steer the agent toward the right checks, but it cannot supply missing live state. The two parts are complementary, though still not a substitute for tests and permissions.

A safe rollout you can actually use

Treat the first setup as an experiment in behavior, not as a production automation launch.

1. Start with the instructions. Install the skill in a disposable or development project, then give the agent a small task involving a schema change or a known Supabase pattern. Ask it to explain which documentation it consulted and which security requirements apply. Check its output rather than assuming that the skill loaded correctly.

2. Add a read-only MCP connection. Scope it to one non-production project and enable only the tool groups needed for the test. Ask the agent to inspect a table or diagnose a log event. Verify that its answer matches the dashboard or CLI, and confirm that it cannot write through that connection.

3. Test the failure cases. Try a task where a row contains misleading instructions. Ask for an action beyond the connection’s permission. Confirm that the agent reports the boundary instead of finding another path. Test what happens when a tool is unavailable, a query returns no rows, or the documentation and project state disagree.

4. Add write access only for a narrow workflow. If the use case is development, prefer a local or staging database and keep changes reviewable. Have the agent prepare a migration or pull request before considering automatic application. Production access should be an explicit exception with project scoping, the smallest practical permissions, appropriate approval, and a recovery plan.

5. Keep an evaluation record. Save the task prompt, expected result, observed tool calls, generated changes, tests, and human corrections. Compare the same scenarios with and without the skill or MCP connection. That will tell you whether either component improves your actual workflow, rather than relying on a vendor’s benchmark or a polished demo.

The decision checklist

Before choosing a setup, answer these questions:

Does the task need live project data, or can the agent work from the repository and current documentation?
Is the recurring failure missing knowledge, missing context, or missing permission boundaries?
Can you begin with a project-scoped, read-only connection?
Which tool groups are necessary, and which can stay disabled?
Are approvals genuinely manual in your client, or can hooks and rules accept them automatically?
Can the agent use a local or staging database instead of production?
What tests and human review must pass before a change is applied?

If the first two questions point to knowledge, start with a skill. If they point to live context, add MCP with narrow access. If both are true, use both, but keep the actual security boundary in database permissions and infrastructure rather than in prompt wording.

Supabase's MCP and Agent Skills tools reflect a broader shift in coding agents: reliable work depends on both good instructions and the right tools, while user-facing agents need a separate identity and authorization design. The hard engineering choice is not “Which one makes the agent autonomous?” It is “What is the smallest combination that gives it enough context to help without granting more authority than the task requires?”

I write practical guides on software engineering and AI tools. Subscribe for free if you want more comparisons and implementation checklists.

Have you connected an AI coding agent to a live backend, or do you keep it limited to repository files? What boundary would you insist on before granting write access?

Save this rule of thumb: Skills guide how the agent should work; MCP determines what project information and actions it can reach. Begin with instructions, then add scoped read-only context, and grant writes only when a tested workflow truly needs them.

Sources: Supabase Agent Skills announcement and evaluation, Supabase Agent Skills documentation, Supabase MCP security documentation, Give your Supabase app its own MCP server.

When Coding Agents Watch GitHub: A Practical Safety Checklist Before They Open PRs

jamilxt — Tue, 03 Nov 2026 03:06:02 +0000

A coding agent that waits for a prompt is easy to understand. You give it a task, watch the run, and review the result. An agent that watches issues and comments changes the arrangement. It may notice a request, decide whether it is allowed to act, create a separate work session, and return a pull request later.

The deepseek-ai GitHub organization hosts a public dsh-superagent research prototype, but its README explicitly says it is not an official DeepSeek product. The README describes it as reference research code, not production-ready software. Its value here is as a design case: the project makes permission and review questions visible.

This is not a setup guide for that prototype. I have not installed or tested it. The checklist below is a safer way to evaluate any system that turns GitHub activity into coding work.

A new input is also a new trust boundary

The prototype's flow is more than "issue in, code out." Its documentation describes sources that receive events, policies that decide who may request work, tickets that hold tasks, and executors that do the coding. Work can return to the conversation where it started. Its setup guide recommends a dedicated agent account and an initial observe mode for work inferred from activity, rather than immediately executing it.

That separation matters. A public issue is not automatically an authorized instruction. A comment can include misleading requests, hostile text, or copied instructions from an untrusted source. If that text reaches a model connected to a shell and a write-capable account, it is part of the system's input whether or not the author intended it as a command.

GitHub's own Actions security guidance makes a related point: workflow credentials should have the least privilege needed, and default token permissions should be read-only where possible. The same principle applies to an agent. Its model may be capable, but its permissions should assume that some inputs will be wrong or malicious.

What the prototype gets right, and what it does not prove

The project draws a useful line between a direct request from an allowed person and work inferred from ambient activity. In its documented setup, inferred work begins in observe mode. The operator can see what the system would have done before enabling that behavior. The guide also advises using a dedicated GitHub identity, keeping it separate from the operator's personal account, and requiring human review rather than enabling automatic merging as the default.

Those are good starting choices, not a security certification. The same deployment documentation warns that its worker sessions run with broad operating-system access under the operator's user. The account wrappers select GitHub credentials, but they do not isolate the process. A worker may still run programs by path and read files in that user's home directory. The docs recommend a dedicated OS account for isolation.

This is the important distinction: separate GitHub identity is not the same as a sandbox. A bot account limits how the agent appears and what it can do in the repository. It does not, by itself, limit what code running on the host can read or execute.

The checklist: start with observation, not autonomy

Before connecting any always-on coding agent to a real repository, answer these questions in order.

1. Which events can create work?

Start with explicit mentions from a small set of trusted accounts. Avoid treating every new issue, every comment, or every conversation as permission to run code. If the tool supports inferred work, keep it in a mode that records candidate tasks without executing them until you have inspected its decisions.

Write the rule down in plain language. For example: "Only a new comment that mentions the agent, from an approved account, can start a task." That is easier to test than a vague instruction such as "help with relevant issues."

2. Who is allowed to ask?

Use an explicit allowlist. Do not assume that repository visibility, team membership, or an issue label is enough to establish intent. Review who can add accounts to the list and how quickly removals take effect.

Try negative tests before real work: an unapproved account mentions the agent; a bot posts a comment; a user edits an old issue to add a mention. The expected result should be no execution and a clear record of why the request was rejected.

3. What can the agent's GitHub identity change?

Create a separate account for the agent, then grant only the repository access required for its task. Keep merge rights with people. Make the agent's branches easy to identify, and ensure its comments and commits are visibly attributed to the agent rather than to a human maintainer.

Review token scopes and repository permissions directly. If the agent only needs to read issues and create branches or pull requests, do not give it organization-wide administration or access to unrelated repositories. Recheck permissions when the tool or workflow changes.

4. What can the process read on the host?

This is easy to miss when setup instructions focus on GitHub accounts. Find out which operating-system user runs the agent, which directories it can read, whether its shell can access environment variables, and where credentials are stored. Check whether its tools can reach internal network services.

Use a dedicated OS account or a properly isolated execution environment when the agent handles untrusted issue text. Keep secrets out of the agent's checkout and home directory where possible. A wrapper that changes which GitHub account a command uses is helpful, but it is not a process sandbox.

5. Can text in the task change what the agent is allowed to do?

Prompt instructions are not a permission boundary. Define limits outside the model too: allowed repositories, allowed commands, network access, branch rules, and whether secrets are available. Treat instructions quoted inside issues, pull requests, logs, and linked pages as untrusted data unless a trusted person explicitly confirms the task.

6. What has to pass before a pull request is ready?

Require the same checks you expect from a human contribution: tests, formatting, dependency review, and a human reading the diff. The agent should explain what it changed, which tests it ran, and what it could not verify. A green check means the configured checks passed; it does not mean the request was safe or correct.

Keep automatic merge disabled while you learn how the agent behaves. A pull request is a review boundary, not a guarantee. If the agent can merge its own changes, review whether that permission is genuinely necessary and what happens when tests are incomplete or the change touches sensitive code.

7. How will you stop and investigate it?

Know how to pause event intake, stop active workers, revoke credentials, and inspect logs before enabling the service. Decide who receives alerts for stuck tasks, repeated retries, or unexpected repository writes. Test the stop path before the agent has access to valuable credentials.

A low-risk rollout in three stages

A small rollout makes failures easier to understand.

Stage one: observe. Connect a test repository or a narrow set of events. Let the system record what it would do. Compare its decisions with what maintainers actually want.
Stage two: prepare changes. Permit a dedicated account to create branches and draft pull requests in a low-risk repository. Keep a person responsible for every review and merge.
Stage three: expand one permission at a time. Add more requesters or repositories only after you have checked false triggers, quality, noisy behavior, and recovery. Do not change event access, host access, and merge rights together. If something goes wrong, you need to know which new permission caused it.

Keep a short record of expected behavior for test cases: who requested work, what event started it, what the agent was allowed to do, which checks ran, and who approved the result. This creates evidence for a decision to widen access, rather than relying on a few impressive demonstrations.

The useful idea is the workflow, not the word "autonomous"

DeepSeek's prototype points to a real design problem: a team may want one agent to carry work across messages, code changes, and review comments without making people restate the task in every session. That can save coordination effort. It also means the agent becomes a long-running participant with access to event streams and execution tools.

The project itself is careful to label its public implementation as a research reference, not production-ready software. Its documentation also exposes tradeoffs that a launch headline can hide, including broad host access and the need for a separate operating-system boundary. Those caveats are reasons to test it as an operations system, evaluating both its code and the safeguards around it.

A useful rule is simple: automate the preparation of a change before automating permission to apply it. Start with narrow inputs, limited credentials, isolated execution, visible drafts, and human approval. Expand only when the logs show that the system follows those limits consistently.

Before enabling an issue-to-PR agent, check:

Only approved people can create an executable request.
Inferred work is observed before it is allowed to run.
The agent uses a dedicated, narrowly scoped GitHub identity.
Its operating-system access is isolated from personal files and secrets.
Pull requests require human review and do not merge themselves.
You can pause the service, revoke access, and inspect its actions.

I write about software engineering and AI systems. Subscribe if you want practical, source-based guides like this one.

Have you let an agent act on GitHub issues or pull requests? What permission limit would you set first?

Sources: DeepSeek dsh-superagent prototype, DeepSeek setup guide, DeepSeek deployment and permissions guide, DeepSeek project philosophy, GitHub Actions secure use reference.

AI Coding Agents Do Not Make Framework Choice Irrelevant. They Make Its Costs Measurable

jamilxt — Mon, 02 Nov 2026 16:05:48 +0000

A familiar argument is making the rounds in web development: if an AI agent writes most of the code, developer experience matters less, so teams should choose the framework that works best for agents. It sounds plausible. It also skips the hard question: what does “works best” mean, and how would you know?

Nolan Lawson tried to answer that with a small benchmark. He asked Claude Opus 5.5 to build the same Pokémon catalog app in 12 JavaScript stacks, tested each result against a shared conformance suite, and measured cost, time, code size, and selected Lighthouse metrics. He ran five iterations per stack, except Angular, which had three measurable results for some metrics. The benchmark write-up and its open-source harness make the experiment inspectable.

The headline is not that one framework won. The useful finding is that “agent-friendly” is not a single score. In this test, Hono with htmx was among the least expensive and shortest-code options, while Astro sent no JavaScript on the tested page. Angular cost more and generated more code, but this is one content-oriented application, one model, one benchmark specification, and a small number of runs. The evidence is enough to question vague claims. It is not enough to declare a universal winner.

That distinction matters because framework decisions are long-lived. If the agent saves a few minutes today but creates more code to inspect and maintain tomorrow, the initial speedup may be a poor trade. The better response to AI-assisted development is not to pick a framework based on intuition about what models prefer. It is to extend the measurement window from “how fast did the agent finish?” to “what does this choice cost across the life of the application?”

What the experiment actually measured

A shared task, not a framework tutorial. Lawson specified a small but non-trivial Pokémon app using local data from PokéAPI. Agents had to implement the same user-visible features and pass the same conformance suite. They were allowed to use official framework MCP tools or skills where available, and the harness recorded whether the agent actually used them. The benchmark repository says each run used a fresh Podman container and that the generated app was tested separately from the agent environment. The repository README describes the controls and reproduction setup.

More than token count. Lawson reported API-equivalent cost, wall-clock time, tokens, lines of code, and Lighthouse measurements for performance and accessibility. That is a useful choice: an agent can spend more tokens to produce code that is smaller, more accessible, or faster for users. Optimizing only for the first completion time would miss those trade-offs.

The median reported cost ranged from $2.67 for Hono plus htmx to $4.97 for Angular. These are not the experimenter's cash charges. Lawson says he used a Claude Max plan and reported the API prices quoted by Claude Code. The cost range is therefore a way to compare the reported usage, not a billing receipt or a forecast of what a team will pay.

The time and token figures also varied. Astro's median run took 8.2 minutes, while Angular's took 17.2 minutes. Median token use ranged from 3.85 million for Nuxt to 9.76 million for Angular. But those figures came from five runs per framework, with Angular's reduced sample in some measurements. The author's own write-up notes stochastic results and overlapping ranges. A median is a helpful summary, not proof that the same ordering will hold on your task.

The counterintuitive result is the important one

Raw JavaScript was not automatically the cheapest route for the agent. The benchmark's “Vanilla” condition used JavaScript or TypeScript with Vite and no UI framework. It was not simply “write the smallest possible plain JavaScript app.” The agent made architectural choices inside the allowed setup, and the result used a median of 5.00 million tokens and took 10.1 minutes. Several framework conditions used fewer tokens or completed sooner.

That does not prove frameworks are better for agents. It demonstrates that removing a framework does not automatically remove complexity. A framework can supply conventions, routing, data loading, and a structure an agent can follow. Conversely, it can introduce conventions the model does not know well. Which effect dominates depends on the framework, task, tool setup, and model version.

The web payload and the agent's build cost are different budgets. Astro's tested page transferred a median of 0 KiB of JavaScript, while Next.js transferred 142 KiB. Hono plus htmx used 18 KiB, and Angular 93 KiB. On that metric, the result separates framework bundles more clearly than the agent's cost rankings do. But a zero-JavaScript page is not a complete performance verdict: the application still has server work, network requests, content, and user interactions. The benchmark also reports different results for blocking time and largest contentful paint, with caveats around variance and the Lighthouse setup.

If a team collapses those measurements into one “best” framework score, it will hide the very trade-offs that make the exercise useful. Agent usage cost is an engineering input. Browser payload and responsiveness are user-facing outputs. Maintainability is a future cost. They should not be treated as interchangeable just because they came from the same run.

Code size is a proxy, not maintainability itself. Hono plus htmx generated a median of 1,370 non-blank source lines; Angular generated 2,009. That is a measurable difference, and smaller implementations may mean fewer tokens for later agent edits. Lawson explicitly connects code volume with the cost of future changes. Yet line count cannot tell you whether code is understandable, well-factored, secure, or easy for a human team to change. A short, opaque implementation is not automatically maintainable.

Why “better for agents” is a weak architecture rule

Framework advocates can point to model familiarity, official agent skills, MCP servers, and established examples. Those can matter. But the benchmark found that agents did not consistently use the available tooling. For example, the report says some frameworks' tools were used in only a subset of runs, and Nuxt's MCP was not used in the observed attempts. Installing an integration is not the same as proving it changed the result.

There is also a training-data trap. Lawson suggests that SvelteKit's higher measured cost may partly reflect its recent release: the agent repeatedly read its migration guide, possibly because the latest version was not represented well in the model's training data. That explanation is plausible, but it is not isolated as a causal factor. A follow-up would need to vary model version, tool access, and framework documentation exposure independently to determine whether unfamiliarity caused the extra work.

The result is a moving target. Framework versions change, models change, tools change, and the same agent can make different choices across runs. A one-time benchmark can help choose what to investigate. It cannot permanently rank a technology stack. That is especially true when official AI tooling is unevenly available or the agent ignores it.

How to use this kind of benchmark without overfitting

Start with your application shape. A content catalog is not a collaborative editor, admin dashboard, offline-first mobile web app, or real-time interface. Lawson says the sample is closer to a content site than a highly interactive product. If your application is dominated by complex client-side state, this result is a poor proxy. Pick a representative slice of your own product before treating any framework ranking as relevant.

Define acceptance criteria before the run. Specify user-visible behavior, accessibility expectations, performance thresholds, supported browsers, and operational constraints. Keep the same tests across candidates. If one framework gets a different prompt or extra hints, the result measures prompt quality as much as framework choice.

Measure failed attempts, not just successful output. In the benchmark, Angular failed to create a measurable app in two of five runs. Lawson retained the three usable iterations for some metrics and explained the difference. That transparency is important. In your own evaluation, record failures, retries, timeouts, test results, and cleanup effort. Do not quietly discard the runs that make a preferred tool look worse.

Separate repeatability from generalization. Run enough iterations to see whether differences exceed run-to-run noise. Then repeat with a second task and, if relevant, a second model. The original author notes variance in both agent output and Lighthouse. Re-running one task reduces some uncertainty; it does not make that task representative of every product.

Track the whole cost surface. At minimum, keep separate measures for:

Agent spend or token use per accepted change.
Human review and correction time.
Test pass rate and severity of defects.
Browser performance and accessibility.
Code size and complexity signals.
The effort required to add a second feature after the first implementation.

That last measure is easy to skip and potentially more revealing than initial generation time. A framework that lets an agent produce a first screen quickly may not help when the next request changes data flow, permissions, or navigation. Add a follow-up task that tests whether the generated structure supports change without a rewrite.

A decision checklist for framework evaluation

Before you switch stacks because a model appears to prefer one, ask:

Is the benchmark task representative of our actual product?
Did each candidate receive the same specification, model, resource limits, and acceptance tests?
Were official agent tools installed, and did the agent use them?
Are the reported differences larger than the run-to-run spread?
Did we count failed runs and human repair work?
Are user outcomes measured independently from agent speed and cost?
Does the generated code remain workable after a second, realistic change?
Would we make the same choice if the current model or tool integration disappeared next month?

The last question is a useful stress test. A framework should not be selected solely because one model has recently seen more examples of it. Model familiarity can improve today's result, but application needs, ecosystem support, and maintainability still matter after the next model update.

The takeaway: benchmark claims, not vibes

This experiment is valuable because it exposes its scope, shares its harness, reports multiple measures, and discusses limitations. Its results are not a framework league table. They are a reminder that the cost of a stack now includes both the work an agent spends producing code and the work people spend checking, operating, and changing that code.

For a team choosing a framework, the practical move is modest: use public benchmarks to form hypotheses, then test those hypotheses on a small, representative feature with your own requirements. Keep the agent's cost, user-facing quality, reliability, and maintenance effort separate. If the decision changes when you add a second task or another model, you have learned something important about how fragile the original ranking was.

I write about developer tools, infrastructure, and AI. Subscribe if you want evidence-based engineering analysis without treating a benchmark as a universal answer.

Have you measured how your framework changes an AI agent's cost or the quality of the code it produces? What would you include in a fair test?

Sources

Kwi vs Konsole: Should KDE Developers Switch Terminals?

jamilxt — Mon, 02 Nov 2026 12:04:01 +0000

A terminal can feel slow for two very different reasons. Sometimes your command is doing expensive work. Sometimes the terminal is struggling to paint the output quickly enough to keep up. Those problems look similar when a screen fills with text, but replacing the terminal only helps with the second one.

That distinction is why KDE's new terminal, Kwi, is interesting. Announced on November 1, 2026, Kwi is not simply a faster Konsole release. It is a new implementation built around GPU rendering, Qt Quick, and Ghostty's terminal parsing library. Its developer reports results from 3x to 35x faster than Konsole depending on the benchmark, including 870 frames per second in a DOOM-fire benchmark compared with 23 in Konsole. Those numbers come from the project's developer, not an independent benchmark, and the same announcement says Kwi has not reached a tagged 0.1 release yet. KDE's announcement explains the architecture, benchmark, and early status.

So should a KDE user replace Konsole with Kwi? For most people today, no. Keep Konsole as the dependable default, and treat Kwi as an early project worth testing if your workload exercises the areas it is designed to improve. The more important story is architectural: KDE is exploring a shared terminal component for its newer Qt Quick applications, while Konsole remains the mature, feature-rich application that people already use.

This comparison is based on public project information, not hands-on testing. Kwi is still in active development, so this is a guide to deciding what to evaluate rather than a claim that it is ready for everyday production use.

What is different under the window?

Konsole is the established application. KDE describes it as a terminal for running a command shell. Its documented features include tabs, multiple profiles, activity and silence monitoring, bookmarks, search, and saving output. It is also used by other KDE applications as an embedded terminal. Its long history matters: Kwi's developer says Konsole has had 29 years of continuous development and is used by millions of people. KDE's Konsole page lists its features and supported installation routes.

Kwi is a newer architecture, not just a visual refresh. Its developer says Kwi uses libghostty-vt for parsing terminal output and maintaining terminal state, KPty for pseudo-terminal integration, and Qt Quick's scene graph for a GPU-accelerated renderer using Vulkan or OpenGL. Its interface is built with Kirigami, KDE's UI framework for modern KDE applications. In practical terms, the project is trying to reuse a dedicated terminal parser and put more of the display work on the graphics stack used by newer KDE software. The Kwi announcement describes these components and their roles.

That design difference helps explain why Kwi's headline performance number needs context. A terminal emulator receives bytes from a program, interprets escape sequences and terminal state, lays out text and graphics, then draws the result. A workload that produces a constant stream of visual updates can stress the rendering path. A command that waits on a database or downloads a file may barely exercise that path at all. A benchmark result on a display-heavy workload is evidence about that workload, not a universal multiplier for every shell command.

Kwi vs Konsole, by the decisions that matter

Rendering and high-volume output: Kwi is the one to test. Kwi's GPU renderer is the central architectural bet. KDE's announcement reports the 3x to 35x range and the specific 870 versus 23 fps result. That is a large enough claim to justify testing on the sort of output that makes your current terminal visibly lag. But the project has not published a broad, independently reproduced benchmark suite in that announcement. Hardware, graphics drivers, resolution, terminal settings, and the exact workload can all change the outcome. Do not convert one benchmark into a promise that builds, tests, or remote sessions will run faster.

Everyday reliability and feature coverage: Konsole has the advantage. Konsole is already a released KDE application, with profiles, tabs, bookmarks, search, and saved output documented on KDE's software page. Kwi is explicitly described as young, still in development, and not yet at version 0.1. The Kwi page in KDE's application directory also says that it has not been released yet. If a terminal is part of your daily workflow for deployments, incident response, or long-running sessions, maturity is a real feature. KDE's current Kwi listing confirms its unreleased status.

Modern terminal protocols: Kwi has a promising direction, not yet a maturity win. The Kwi project lists more complete support for modern terminal features, including Kitty image and animation protocols, clipboard protocols, keyboard behavior, progress reporting, and color scheme queries. It also highlights improved text rendering, ligatures, fractional scaling, accessibility, and screen-reader text extraction. Those capabilities may matter to users of modern terminal user interfaces, but a feature list is not the same as broad compatibility testing. Check the applications and workflows you actually rely on before treating support as complete in practice. The project announcement has the detailed feature list and caveats.

Integration into KDE apps: Kwi could fill a gap. The project is not only trying to build another terminal window. Its developer says Kwi provides a public Qt Quick/QML module that other KDE apps can embed. Konsole already has an embeddable component used by QWidget-based applications, but the announcement identifies a missing equivalent for the newer Qt Quick and Kirigami stack. If that component works as intended, Kwi could make it easier for new KDE applications to include a consistent terminal without each app solving the problem independently. That potential is for application developers, not a reason by itself for every desktop user to switch.

Configuration and migration: Konsole is safer today. Kwi already supports Konsole color scheme files, but the project notes that migration is not finished. Existing QWidget applications that embed Konsole's component need a code change to request Kwi's component, and the project has not settled on a central way for users to express a terminal preference. That is an important distinction between technical compatibility and a smooth user migration. KDE's announcement calls out both the color scheme support and the unresolved transition path.

Availability: Konsole is ready, Kwi is for evaluation. KDE lists Konsole as installable through Linux distributions and Flathub. For Kwi, the announcement points to nightly Flatpak builds from KDE's CI and warns that its fresh dependencies can make building difficult. Nightly packages are useful for testing, but they are not the same promise as a stable release. Do not put a changing nightly build into a workflow that needs predictable upgrades unless you are comfortable diagnosing regressions. KDE's Konsole page gives its distribution and Flathub options, while the Kwi announcement describes its nightly-build route.

The performance claim needs a fair test

If you want to know whether Kwi is faster for your work, avoid comparing a single dramatic demo with a vague impression. Pick one repeatable workload that currently makes Konsole struggle. For example, use the same terminal size, font, theme, machine, and output command in both terminals. Measure wall-clock completion time separately from perceived smoothness. A faster animation does not necessarily mean the command itself completed sooner.

Try at least three kinds of work:

A rendering-heavy workload: a continuously updating TUI, large output stream, or terminal graphics demo. This is the kind of case most likely to expose renderer differences.
A normal developer session: editing in a terminal UI, running tests, navigating history, and using the tabs or profiles you depend on. This checks whether the rest of the experience is usable.
A reliability check: run your actual shell setup, SSH workflow, clipboard path, screen-reader or accessibility tools if relevant, and any terminal applications that use images or custom keyboard protocols.

Keep these questions in the notebook:

What exact command or TUI is slow, and is the delay in rendering or in the program itself?
Does Kwi improve that same workload on your hardware, or only the project's illustrative benchmark?
Does it handle your shell, fonts, keyboard shortcuts, clipboard, and terminal applications correctly?
Can you recover your existing color scheme and workflow without rebuilding settings by hand?
Is a nightly build acceptable for this machine, or should you wait for a tagged release and distribution package?

That checklist is more useful than picking the terminal with the biggest advertised number. It turns a benchmark claim into a personal, repeatable decision and keeps an early experiment out of a critical workflow until it earns trust.

Who should use which?

Choose Konsole for now if you want a stable, familiar KDE terminal with established features, or if you depend on an integrated terminal in an existing KDE application. KDE itself describes Kwi as an option under development, not a decision to replace Konsole. That distinction matters: a new project can be valuable without making the existing one obsolete.

Try Kwi in a separate test setup if you use KDE Plasma, care about GPU-rendered terminal output, rely on modern terminal protocols, or build Qt Quick/Kirigami applications that could benefit from an embeddable terminal component. Treat it as a project to evaluate, not as the default replacement for a production shell environment. If the first result is compelling, retest after updates because the code and dependencies are changing quickly.

Wait if you need a stable release, complete migration support, or evidence from benchmarks that cover your workload. The project is young, and its maintainer explicitly asks readers to expect bugs. Waiting is not resistance to progress; it is matching the maturity of a tool to the cost of a failure.

The verdict is straightforward: Konsole wins on readiness and established workflows. Kwi is the more interesting experiment for rendering performance and Qt Quick integration. The 870 fps result is a reason to pay attention, not a reason to uninstall a terminal that already works.

I write about developer tools, infrastructure, and AI each week. Subscribe if you want practical comparisons that separate a promising claim from a dependable tool.

Have you tested Kwi or another GPU-rendered terminal? What workload would convince you to switch from Konsole?

Sources

CVE-2026-98432: How to Find and Patch Vulnerable Linux Hosts

jamilxt — Mon, 02 Nov 2026 03:04:57 +0000

A server can have every application package updated and still be running the old kernel. That gap matters when the vulnerability is in the kernel's network path, and it is why “the package manager says everything is current” is not a complete incident response.

CVE-2026-98432 is a use-after-free flaw in Linux's IPv4 networking code. Red Hat describes a race during TCP connection-request processing that can free socket options while another operation is still using them. Its current bulletin rates the issue Critical and says a remote, unauthenticated attacker may cause a crash or potentially execute code. AWS describes the affected path more narrowly as IPv4 TCP handshakes involving crafted packets with IP options, and says researchers demonstrated remote code execution against hosts with an MPTCP listener. AWS also says it is not aware of exploitation in the wild. These descriptions are not identical, so follow the advisory for your exact distribution and product rather than treating one vendor's summary as a universal exploit guarantee. Red Hat's advisory and AWS's bulletin were checked on November 2, 2026.

This is a hands-on response guide, not an exploit walkthrough. It shows how to build an inventory, check vendor status, update the host kernel, verify the kernel that is actually running, and avoid the common mistake of patching an image while leaving existing nodes untouched. The steps are drawn from vendor advisories and standard Linux administration commands.

First, understand what “patched” means

A vulnerability record is not a universal list of safe kernel version numbers. Linux distributions routinely backport security fixes into a vendor kernel while keeping the version string's upstream-looking base unchanged. A kernel numbered 5.15.x can contain a security fix that is absent from another 5.15.x build. Compare your installed package against the vendor's fixed package or advisory status, not against an upstream version guessed from a headline.

There is a second distinction: the kernel package installed on disk is not necessarily the kernel executing now. Linux loads the kernel during boot. Updating it generally requires a reboot or a controlled replacement of the node before the fix is active. AWS explicitly calls for updating and rebooting self-managed EC2 hosts, and Red Hat says a kernel update is the complete remediation; its filtering rule is only a mitigation. AWS's bulletin and Red Hat's mitigation section explain those steps.

Keep three states separate in your records:

Package available: the vendor has published a fixed kernel package or image.
Package installed: the host has downloaded and installed that package.
Fixed kernel running: the host has booted the fixed kernel, or has been replaced by a node that did.

Only the last state confirms that the host is actively using the kernel fix.

Step 1: Find the machines you actually own

Start with your asset inventory, not a single SSH session. Include virtual machines, physical servers, Kubernetes workers, container hosts, appliances you manage, and old instances that are stopped but may return. A container image scan alone will not tell you which kernel the container host is running. Containers share the host kernel, so assess and patch the nodes beneath them.

For each host, record its cloud or data-center location, distribution and release, kernel package, current running kernel, workload role, and whether untrusted IPv4 TCP traffic can reach it. Do not infer reachability from an application's intended design. Check security groups, load balancers, firewall rules, and listening sockets.

On a Linux host, collect a starting snapshot:

printf 'hostname: '; hostname -f 2>/dev/null || hostname
printf 'os: '; . /etc/os-release && printf '%s %s\n' "$ID" "$VERSION_ID"
printf 'running kernel: '; uname -r
printf 'architecture: '; uname -m

If you administer a group of machines, gather the same fields through your existing configuration-management or cloud-inventory system. Avoid building a one-off scanner that silently misses nodes. A report should identify unreachable hosts and failed commands as unknown, not as patched.

Step 2: Check the right vendor advisory

Search for CVE-2026-98432 in the security tracker for the operating system and kernel flavor actually installed. That includes specialized cloud kernels, real-time kernels, HWE kernels, immutable images, and vendor-managed container nodes. Read both the affected-status and fixed-package fields. “Under investigation” is not the same as “not affected,” and a fix for one release does not establish the fix status for another.

The vendor pages can change while fixes roll out. As of November 2, 2026, AWS says fixed Amazon Linux 2 and Amazon Linux 2023 kernel packages are available and instructs customers to update and reboot. AWS also lists updated EKS-optimized Amazon Linux 2023 AMIs, while its page describes the Auto Mode rollout as still progressing. On the same date, Red Hat's bulletin says it is expediting fixes, and notes that traffic filtering can reduce exposure but cannot fully prevent the issue. Those are examples of why a single “Linux is patched” status is misleading: the answer depends on distribution, product, node type, and the exact time the advisory was checked. AWS security bulletin; Red Hat security bulletin.

Capture the advisory URL and the time you checked it in the change record. Recheck it before deployment if the remediation spans several days. Do not copy a fixed package version from another distribution or cloud image and assume it applies to your fleet.

Step 3: Update the host, not just the workload

For Amazon Linux 2, AWS currently documents:

sudo yum update kernel
sudo reboot

For Amazon Linux 2023, the bulletin documents:

sudo dnf update kernel --releasever=latest
sudo reboot

AWS notes that instances pinned to a particular Amazon Linux 2023 release need a release containing the fixed kernel. Follow the live bulletin and the relevant release advisory, especially if you pin repository versions. AWS security bulletin.

For other distributions, use their supported package manager and upgrade path. For example, a Debian-family system commonly uses apt, while a Red Hat-family system uses dnf or yum, but the command syntax is not the key decision. The decisive check is whether the distribution tracker says the installed kernel package for that release contains the fix. If no fixed package is available yet, do not substitute an unrelated kernel build on a production host without a supported migration plan.

Before rebooting, confirm that you have a recovery path: console access, a tested maintenance window, a healthy replacement node, or a rollback plan. Kernel changes affect the whole machine. For a cluster, drain and replace nodes in controlled batches, preserve capacity, and confirm that the new node image includes the fixed kernel before sending workloads to it.

Step 4: Verify what is running after remediation

After reboot or node replacement, check the live kernel:

uname -r

Then verify the installed kernel package using the distribution's package tools and compare it to the vendor's fixed package information. Keep both results. uname -r answers “what is running now?” Package queries answer “what is installed?” Neither alone answers whether the vendor considers that exact build fixed.

For a systemd-based host, verify uptime or boot time if the change record needs evidence that a reboot occurred:

uptime -s

For a fleet, collect the result from every in-scope node and reconcile it against your inventory. A successful update command on one representative machine is not fleet-wide proof. Include nodes that failed to reboot, were cordoned, or were replaced from an older image. If an automated scanner reports a vulnerable package after the reboot, investigate whether it is detecting the on-disk package, a stale image, or a separate kernel flavor rather than suppressing the finding without evidence.

Step 5: Treat cloud and Kubernetes layers separately

A cloud provider may patch its own managed control plane or service infrastructure while leaving customer-managed virtual machines for the customer to update. The provider's bulletin spells out that boundary. AWS, for example, says self-managed EC2 Linux instances remain the customer's responsibility and distinguishes them from services where AWS is applying infrastructure updates. It also has separate steps for EKS node AMIs, ECS on EC2, Fargate, and Bottlerocket. Read the section for the exact service you use. AWS security bulletin.

For Kubernetes, the pod's image is not the kernel boundary. Check the node operating system, node image release, and the mechanism that replaces nodes. Updating a launch template or node group does not necessarily replace already-running workers immediately. Confirm rollout status, drain or recycle old nodes, and verify the running kernel on each new node. If your cluster uses a managed automatic-update mode, confirm the provider's rollout status and your disruption-budget behavior instead of assuming that “managed” means “already complete.”

Also include edge cases in your inventory: autoscaling groups that can recreate an old image, warm pools, disaster-recovery regions, build agents, and long-lived development machines. A newly created node can reintroduce the old kernel if its image source was not refreshed.

What about the temporary firewall mitigation?

Red Hat describes an nftables rule that drops incoming IPv4 TCP packets containing IPv4 options. The vendor explicitly warns that the filter reduces exposure but cannot fully prevent the vulnerability, and that it may affect workloads. It should be treated as a temporary risk-reduction measure, reviewed and tested against your network, not as proof that the host is safe. Remove the rule after the fixed kernel is active, following the vendor's procedure. Red Hat's advisory includes the rule, validation, and removal commands.

Avoid applying network filters blindly across a fleet. First establish whether your traffic uses IPv4 options, whether the rule is compatible with your existing firewall framework, and how you will persist, monitor, and later remove it. Test in a representative non-production environment. If the vendor's exact mitigation is not relevant to your distribution or firewall manager, ask that vendor for its supported guidance rather than translating a rule from memory.

A compact response checklist

[ ] Inventory hosts, cloud nodes, container hosts, and rebuildable images.
[ ] Identify distribution release and kernel flavor per host.
[ ] Check the distribution or cloud vendor's current CVE-2026-98432 status.
[ ] Separate package availability, package installation, and the running kernel.
[ ] Prioritize hosts reachable by untrusted IPv4 TCP traffic.
[ ] Apply the vendor-supported fixed kernel or image.
[ ] Reboot or replace nodes using a controlled rollout.
[ ] Verify uname -r and the vendor package status on every in-scope host.
[ ] Refresh autoscaling, Kubernetes, and disaster-recovery images.
[ ] Track any temporary mitigation and remove it after patch verification.

The useful operational lesson is not “every Linux version is vulnerable” or “one firewall rule fixes it.” It is that a kernel security response has to trace the fix all the way from the vendor advisory to the kernel executing on each machine. The package manager is one checkpoint in that chain, not the finish line.

I write practical guides on software engineering, infrastructure, and AI. Subscribe if you want more walkthroughs like this.

How do you verify that a kernel fix reached every node in your fleet: package inventory, image rollout checks, or both?

Sources

Amazon’s AI Developer Memo Gets One Thing Right: Measure Outcomes, Not Tokens

jamilxt — Sun, 01 Nov 2026 16:05:49 +0000

A team can double the number of AI-generated lines in a week and still deliver no more useful software. It might even deliver less: more review work, more defects, more rework, and a larger bill. The uncomfortable part is that a dashboard full of activity can make the opposite story look convincing.

That tension sits at the center of Amazon’s newly published guidance on the software developer role in the age of AI. The memo, written by Terk Terkowitz, vice president and distinguished engineer at Worldwide Amazon Stores, asks engineers to learn and use AI. It also warns that token consumption, adoption rates, code changes, and reviews are deeply flawed if managers treat them as targets rather than imperfect signals. The company says those inputs can still correlate with valuable outputs, but should be used with care and judgment. Amazon’s memo is more nuanced than a simple “stop using AI” headline suggests.

The important question is not whether developers should use AI. It is whether a company can tell the difference between useful adoption and performative activity. Amazon’s guidance offers a good starting point, but the measurement problem is harder than swapping one dashboard for another.

What Amazon is actually saying

The memo makes two claims that need to be held together.

First, AI is likely to become central to software engineering. Amazon encourages engineers to learn the tools, apply them in daily work, and keep improving their practice. The memo calls for hands-on understanding, not secondhand opinions.

Second, the company says measures such as tokens consumed, adoption rates, code changes, and reviews are “deeply flawed” when detached from outcomes. As targets, they can encourage higher costs or unnecessary complexity. The memo gives “tokenmaxxing” as the case where increased consumption is treated as a goal in itself. It also warns that measuring lines of code can conflict with simplicity.

That does not mean Amazon considers those measurements useless. Its memo says input metrics have correlated with outputs that matter, and recommends looking carefully at both the 10th and 90th percentiles of adoption. The first can reveal teams that are not finding effective uses. The second can show whether teams with heavy usage are seeing delivery gains to match.

This distinction matters. A metric can be informative as a signal and damaging as a quota. If a manager asks why adoption is low on a particular team, the answer may expose a training gap or a workflow where the tools genuinely do not help. If the manager demands that every engineer hit a token target, the measure has become the work.

Why usage metrics are tempting

Usage is easy to count. Outcomes are not.

A provider can report tokens, requests, active seats, or generated suggestions almost immediately. A team can count pull requests, changed lines, or review comments. These numbers are legible, comparable, and available before a release has reached customers.

But the distance between “the tool was used” and “the customer received more value” is long. A generated patch may be rejected. An accepted patch may need extensive revision. A merged change may not solve the intended problem. A feature may ship faster and still introduce a reliability issue that takes weeks to surface.

That gap is a measurement problem, not an argument against using AI. It is also not solved by measuring only a different proxy. If token count is a weak stand-in for productivity, replacing it with lines of code or pull request count may merely move the incentive to another easy-to-game number.

The 2025 METR randomized controlled trial is a useful caution about scope. It studied 16 experienced open-source developers completing 246 tasks in mature projects they knew well. With the early-2025 AI tools in the study, developers took 19% longer on average, even though they expected AI to make them faster. METR’s paper does not prove that current tools slow every developer or every task. It shows why broad adoption claims cannot substitute for measuring the specific work, people, and tools involved.

METR later ran a follow-up with late-2025 tools, but said the newer data was an unreliable signal because developers who most wanted AI were less likely to participate or submit tasks they preferred to do with AI. In that follow-up, METR estimated an 18% slowdown among returning participants, with a confidence interval spanning from 38% slower to 9% faster, and a 4% slowdown among newly recruited developers, with a confidence interval spanning from 15% slower to 9% faster. Both intervals include no effect. METR said the selection effects made the results difficult to interpret and likely understated benefits for the developers and tasks missing from the study. METR’s 2026 design update explains those caveats. The takeaway is not that AI still slows developers; it is that a single metric or study cannot settle the question for every workflow. That is exactly the kind of nuance a usage dashboard hides. It can show that a team used an assistant more. It cannot, on its own, tell you whether the assistant helped with that team’s actual task mix.

A better unit of analysis: the task, not the token

A practical evaluation should start with a meaningful unit of work. That could be a bug fix, a small feature, a migration step, a test suite improvement, or a support issue. The definition should be stable enough that “done” means something consistent before and after AI enters the workflow.

For each task, record a small set of signals:

Outcome: Was the intended user or operational problem resolved?
Time to an accepted result: Include prompting, checking, editing, and rework, not only the first generated draft.
Quality: Did tests pass? Were review findings, regressions, or follow-up fixes needed?
Human effort: How much focused developer and reviewer time did the task consume?
Context: What kind of task was it, how familiar was the code, and what tool or workflow was used?

This is not a universal productivity formula. It is a way to avoid collapsing different work into a single count. A one-line configuration change and a complex production incident should not be treated as equivalent just because both produce one pull request.

Teams can then compare like with like. For example, they might examine several similar maintenance tasks completed with and without AI assistance, while noting differences in developer familiarity and review requirements. The comparison will not be a perfect experiment. It can still be more informative than asking whether AI usage went up.

The key is to preserve enough context to explain the result. If AI helps draft repetitive tests but slows work in a deeply familiar legacy module, that is not contradictory. It is evidence that usefulness depends on the task and workflow.

Keep adoption data, but do not turn it into a quota

There is a reasonable place for usage data. If nobody on a team has tried an approved tool, that may be worth investigating. Maybe access is broken. Maybe the team has not had time to learn. Maybe its work is unusually constrained. Usage can identify a question to ask.

It should not answer the question by itself.

A manager can use adoption data as a prompt for a conversation: “Where did this help? Where did it create extra work? Which task would you not use it for again?” That makes usage a diagnostic input. A required weekly token number makes it a performance target, even if leadership says otherwise.

Amazon’s attention to the 10th and 90th percentiles points toward a better practice: look at the outliers and investigate why they differ. Low usage might reveal a workflow that has not been adapted or a legitimate lack of fit. High usage should invite the same scrutiny: are those teams delivering more, or merely generating more activity? Do not assume the answer from the chart.

Averages can conceal both situations. A team-wide mean may look healthy while a few groups are struggling to adopt the tools, and a few others are spending heavily without a corresponding benefit. Percentiles do not explain the cause, but they help identify where a closer look is worthwhile.

A lightweight review cycle for engineering teams

A team does not need a measurement department to test whether AI helps. It does need a review cycle that resists easy gaming.

1. Pick one workflow. Choose a recurring task with a clear completion condition. Avoid beginning with “all engineering work.” Narrow scope makes the result interpretable.

2. Define the quality bar first. Specify the tests, review expectations, or operational checks that make the task acceptable. Do not lower the bar after seeing a faster draft.

3. Count the whole loop. Include time spent describing the task, waiting, reading generated output, correcting it, reviewing it, and handling rework. A fast first response is not the same as a fast accepted outcome.

4. Record negative cases. Note when the tool was not useful and why. Amazon’s memo explicitly treats an engineer’s explanation of where current AI does not help as useful judgment, not failure. Without this record, a team may remember only the impressive examples.

5. Review outcomes with context. Compare similar tasks and separate changes in scope, experience, and codebase familiarity. Treat a small sample as a learning signal, not a company-wide proof.

6. Revisit the measure. If people start optimizing the metric instead of the work, stop using that metric as a target. Keep it only if it still helps explain outcomes.

This cycle creates a practical decision: expand the workflow, adjust it, or leave it alone. The result can be “AI helps here,” “AI needs a different review pattern,” or “the current tool adds no value for this task.” All three are useful findings.

The “Renaissance Developer” is more than a tool user

Amazon describes the desired developer as a “Renaissance Developer,” combining AI tools with human judgment and domain expertise. The label is broad, but the memo gives it a concrete meaning: understand the product and customers, learn the tools directly, connect inputs to outputs, and exercise judgment about where AI belongs.

That is a stronger standard than simply asking engineers to use more AI. More importantly, it puts responsibility on leaders as well as individual contributors. Managers need to understand the technology well enough to recognize when a workflow is useful, and to connect adoption to delivery rather than treating usage as proof of progress.

There is a tension in the memo worth noticing. Amazon argues that its input metrics correlate with output, while also warning that they can cause harm when made targets. That is plausible: a measure may track a trend at the organizational level and still be a poor individual performance score. A company-wide relationship does not establish that every engineer should have a personal target for tokens, code changes, or tool usage.

The memo does not supply a complete measurement method, and its recommendations are written for Amazon’s own SDE and SDM organizations. Other teams should not copy its percentile thresholds or assume they will observe the same correlation. The transferable idea is more modest: inspect where AI use is low or high, ask what explains it, and verify whether outcomes moved too.

A checklist before adopting an AI productivity metric

Before putting a number on a dashboard, ask:

Does this measure customer or operational value, or only activity?
Can an engineer increase the number without improving the result?
Does it account for review, correction, and rework?
Are we comparing similar tasks and codebase familiarity?
Could the metric reward unnecessary code, requests, or complexity?
Do we record cases where AI was not useful?
What decision will we make if the number rises but outcomes do not?

If the answers are unclear, the metric is not ready to become a target. It may still be useful for exploration, but label it as an input and say what it cannot prove.

Amazon’s memo is a welcome correction to the idea that more AI activity automatically means more productivity. Its strongest point is not that tokens do not matter. It is that tool usage is only a clue. The work still has to solve the right problem, pass the quality bar, and deliver something useful.

I write about software engineering and AI tools. Subscribe if you want more practical, evidence-led breakdowns like this. Have you seen AI usage metrics help a team improve, or push people toward activity instead of outcomes? Share what happened.

Save the checklist for the next time someone proposes a token, adoption, or code-volume target. Start with the result you want, then choose the smallest measure that can help you understand whether the tool contributed.

Sources

HackerTube vs. YouTube Recommendations: Curated Signal or Personal Fit?

jamilxt — Sun, 01 Nov 2026 12:05:06 +0000

A video can be excellent and still never appear on your screen. That is the problem HackerTube tries to solve, but not by making a better version of YouTube's recommendation model. It changes the source of the recommendations.

HackerTube starts with videos that people recommended in Hacker News comments over almost two decades. YouTube's homepage, by contrast, is personalized around your own viewing history and other signals. One system asks, in effect, "What has this community found worth sharing?" The other asks, "What might this particular viewer want next?"

That difference makes HackerTube an unusually clear comparison of two discovery strategies: a shared human-curated corpus versus an individualized platform feed. Neither is automatically better. Each has a different blind spot, and the choice depends on whether you want a common catalog of recommendations or a feed tuned to your own habits.

I have not audited HackerTube's dataset or run a controlled comparison against YouTube. This is an analysis of the project creator's account, the site's stated features, YouTube's own description of its recommendation system, and the developer discussion around the launch.

What HackerTube changes

Andrej Karpathy introduced HackerTube on October 31, 2026. He says it draws from 39,834 Hacker News comments, narrowed to roughly 40,000 comments that recommend videos, and contains 6,454 videos across 150 topics from 2007 through 2026. Those are creator-reported figures, not an independent audit. The project presents those videos through a familiar video-browsing interface. Karpathy's launch post describes the collection and workflow; the Hacker News launch thread summarizes the catalog.

The source of its signal is the important part. A Hacker News user had to encounter a video, decide it was worth mentioning, and place it in a discussion. That is not a formal quality review, and a comment recommendation can be a joke, a disagreement, or a niche reference. But it is a trace of human selection that exists outside your own watch history.

The site's design also makes a few deliberate choices. Karpathy says it uses a frame from the middle of a video instead of the creator-uploaded thumbnail, offers recommendations without accounts, and keeps browsing state in the local browser session. He also says users can export the site's data and use it to build a different interface. These are descriptions from the creator, not guarantees about long-term availability or a substitute for inspecting the implementation. The launch post lists these features.

The project is not a new video host. The catalog points people toward YouTube videos. The discovery layer changes; the underlying video source does not. That distinction matters when assessing what the project can and cannot control.

What YouTube optimizes for

YouTube describes its recommendations as a personalized system. Its official explanation says the homepage primarily uses watch history and considers signals including search history, subscriptions, likes, dislikes, feedback such as "Not interested," and satisfaction surveys. The company says the system compares your viewing habits with those of people with similar habits.

That design can be powerful. If you have specific interests, a long viewing history, and channels you return to, the system can make useful suggestions that a general-purpose catalog would miss. It can also react to your feedback and viewing patterns rather than relying on what an outside community liked years ago.

The system's complexity is not proof that it serves every individual well. YouTube's explanation describes the signals it uses and its stated goal; it does not establish that every viewer experiences the feed as satisfying. The HN launch discussion contains both reactions: one commenter said the unpersonalized feed felt unusually close to their interests, while another said HackerTube resembled an HN-shaped algorithm and included videos they did not want. Those are personal reports, not representative user research. The thread is useful precisely because it shows that a community-derived feed can feel refreshing to one person and narrow to another.

HackerTube vs. YouTube: the practical trade-offs

Discovery signal: HackerTube uses historical links people chose to recommend in Hacker News discussions. YouTube uses personal and contextual signals, including viewing and search history. If you want to step outside your established viewing pattern, HackerTube's external corpus is attractive. If you want suggestions based on what you actually watch, YouTube has the more direct feedback loop. HackerTube's creator describes its source; YouTube describes its signals.
Breadth versus fit: HackerTube's reported 6,454 videos and 150 topics are a finite, shared collection. Its selection may surface a talk or documentary you would not find through your usual recommendations, but the collection reflects what was linked in one community. YouTube's catalog and personalization can cover far more viewing contexts, but a personalized feed may keep reinforcing patterns already present in your history. The latter is a possible trade-off, not proof that the system inevitably creates a filter bubble.
Freshness: HackerTube's corpus reaches back to 2007. A historical recommendation can remain useful, especially for foundational talks, but a decades-long window can also make old material prominent. YouTube can use current behavior and context to suggest recent uploads, though the results are still ranked for each viewer. HackerTube has since added a "Recently recommended" sort, according to a creator update in the launch thread. That helps distinguish a new HN mention from an old upload, but it does not make every video itself new.
Thumbnail incentives: HackerTube's middle-frame thumbnail is meant to avoid relying on the creator's selected thumbnail. It changes the preview, not the video's content, title, or reputation. YouTube's recommendation system considers whether viewers choose to watch, stay, and report satisfaction, according to its official overview. The two interfaces therefore expose different cues, but HackerTube has not demonstrated that one thumbnail strategy produces better viewing outcomes.
Personalization and privacy: HackerTube says it does not require accounts and stores browsing state in the local browser session. YouTube says its recommendations use account-associated signals such as watch history and subscriptions, and it provides controls to pause or delete that history. The right choice depends on the experience you want: minimal personalization or a feed that adapts to your behavior. Check each service's current settings and privacy explanation before treating these descriptions as a complete privacy comparison. HackerTube's creator post and YouTube's explanation describe the respective approaches.
Control and escape hatches: YouTube offers controls including "Not interested," "Don't recommend channel," and watch-history management. HackerTube's exportable catalog offers a different kind of control: a dataset someone can inspect or reuse to create another interface. One adjusts a personal feed; the other allows people to fork the collection's presentation, subject to the data and service remaining available. YouTube's help page lists its feedback signals; HackerTube's About link is linked from the launch discussion.

A recommendation feed is a product decision

The deeper lesson is not that one algorithm is honest and the other is manipulative. The two systems answer different questions because they start from different objectives and evidence.

HackerTube uses a community's past recommendations as a shared taste map. That map can help people discover material outside their own viewing history. But it inherits the limits of that community: who participates, which links get posted, which discussions survive, and what a commenter meant by sharing a video. A popular recommendation can be a useful clue, not a quality certificate.

YouTube's system personalizes suggestions from a broader stream of signals. That can make discovery more relevant to an individual, but it also means the experience is shaped by a platform's ranking choices and the data available about that viewer. YouTube says it uses satisfaction surveys alongside watch behavior. That is a meaningful distinction from a simplistic claim that the system only maximizes time watched. The actual outcome for any viewer still needs to be judged from their experience, not inferred from the system description alone.

HackerTube's smaller scale is a feature for some people and a limitation for others. A curated catalog makes the selection legible: the underlying videos were surfaced in public discussions, and the project reports the collection size and topics. But the process that filtered comments and grouped content is not the same as a transparent editorial review. Karpathy says AI models helped filter and classify the comments; that does not make the resulting list neutral or objectively high quality. His post explains the broad workflow, while the HN discussion surfaces questions about curation and coverage.

Which one should you use?

Choose HackerTube when you want a finite, community-shaped browsing session, especially if you are curious about technical talks and videos recommended in Hacker News discussions. Treat the catalog as a discovery index, then decide for yourself whether each video deserves your time.

Choose YouTube when you want a personalized feed, recent uploads from channels you follow, or recommendations that can adapt to your interests and feedback. If the homepage has drifted away from what you want, review your watch history, subscription list, and recommendation controls before deciding the whole system is useless. YouTube's own recommendation controls guide explains how to manage those inputs.

You can also combine them. Use HackerTube as a source of candidates, and YouTube as the player and subscription tool. That avoids pretending either source is universally superior. It also gives you a practical test: do you find videos worth watching that you would not have found through your normal feed?

A five-minute evaluation checklist

Pick a goal. Are you looking for a specific topic, a new creator, a long-form talk, or something recent? A discovery tool can only be judged against a concrete job.
Set a short session. Browse for ten minutes and save candidates instead of letting autoplay decide the length of the experiment.
Track useful finds. Count videos you actually want to watch, not just thumbnails you click. Note whether you already knew the creator or topic.
Check provenance. For a HackerTube result, inspect the discussion context when available. A recommendation's reason can be more informative than the fact that it appeared in a list.
Compare the cost. Consider time spent, interruptions, account requirements, and how easy it is to leave with a short watchlist.
Repeat once. A single session is anecdotal. Try a different topic or day before deciding one approach consistently works better for you.

The best feed is not necessarily the one with the most advanced model. It is the one whose recommendations match your intention without turning every visit into an unplanned session.

HackerTube is an interesting counterexample to the assumption that discovery must begin with a platform's own personalization. It asks whether a public archive of human recommendations can be useful enough to browse on its own. YouTube answers a different question: given the signals available about this viewer, what might they want next? For some viewers, switching between those answers may be more useful than choosing a permanent winner.

I write about developer tools, AI systems, and practical software engineering. Subscribe if you want more comparisons that focus on how tools behave, not just how they are announced.

Have you tried HackerTube or another community-curated video catalog? What did it help you find that your usual feed missed?

Sources

DoorDash Has Seen Hundreds of AI-Agent Orders. Build a Checkout That Knows When to Stop

jamilxt — Sun, 01 Nov 2026 03:07:25 +0000

A shopping agent can find a restaurant, assemble a cart, and call an ordering API. The hard question comes one step later: when is it allowed to spend the user's money?

DoorDash's latest earnings discussion offers a useful reality check. CEO Tony Xu said the company had seen “maybe several hundred orders in total” from some third-party AI agents. That is a cumulative count for those agents, not all orders and not a quarterly total. The same report says customers still predominantly order through DoorDash's own app. The company has also built its own agent experiences and an MCP connector for corporate ordering. The signal is not that agents cannot order. It is that connecting an agent to a real fulfillment network does not automatically make people use it. TechCrunch reported Xu's remarks and context.[1] The SEC-filed transcript of DoorDash's Q3 earnings call records Xu's cumulative figure and describes its broader strategy around digital assistants and physical fulfillment.[2]

For developers, the practical lesson is smaller and more actionable: do not let a language model turn a suggestion into an irreversible purchase. Build a deterministic boundary between “the agent found an option” and “the merchant accepted an order.” This tutorial develops that boundary as a small checkout state machine. It is a teaching example, not a production payment integration.

The order should be data before it becomes an action

A conversation is a poor source of authority. “Get the usual, but cheaper” is not a payment authorization. It leaves open which items, substitutions, delivery address, tip, fees, and final amount the user accepted.

The agent can help interpret intent and propose a basket. The application should then turn that proposal into a structured quote from the merchant. The quote is the object the user reviews. It should include exact line items, quantities, merchant, address, total, currency, and an expiration time. If inventory or price changes, create a new quote and require a new decision rather than silently editing the old one.

This is not merely a prudent UI convention. The Agent Payments Protocol (AP2) specification separates the shopping agent, merchant, credential provider, payment processor, and a trusted surface for user consent. It says transaction validation must happen in deterministic code. Its direct flow has a person approve the closed checkout and payment, while its autonomous flow uses user-approved constraints that the completed checkout must satisfy. Those are different authorization modes, not interchangeable versions of a model prompt.[3]

A compact state machine

Start with explicit states:

DRAFT: the agent is still assembling a proposed order.
QUOTED: the merchant has returned a concrete, expiring quote.
AWAITING_APPROVAL: the user is shown that exact quote.
APPROVED: the user approved the quote, and the approval is bound to its identifier and amount.
SUBMITTING: the server is making one idempotent order request.
CONFIRMED: the merchant accepted it and returned an order identifier.
REJECTED, EXPIRED, or FAILED: a terminal result that cannot be mistaken for success.

The key invariant is simple: no transition from DRAFT directly to SUBMITTING. The server checks the quote, the approval, and the merchant's current terms first. A model may request a transition, but it cannot grant itself the permission.

Here is a TypeScript sketch of the policy check. It deliberately stops before payment processing. Production systems should use the merchant and payment provider's supported APIs rather than treating this function as a substitute for them.

type Quote = {
  id: string;
  userId: string;
  merchantId: string;
  currency: string;
  totalMinor: number;
  expiresAt: number;
  items: Array<{ sku: string; quantity: number }>;
};

type Approval = {
  quoteId: string;
  userId: string;
  approvedAt: number;
  maxTotalMinor: number;
};

function canSubmit(
  quote: Quote,
  approval: Approval | undefined,
  now: number,
): boolean {
  if (!approval) return false;
  if (approval.userId !== quote.userId) return false;
  if (approval.quoteId !== quote.id) return false;
  if (now >= quote.expiresAt) return false;
  if (quote.totalMinor > approval.maxTotalMinor) return false;
  if (quote.items.length === 0) return false;
  return true;
}

Use integer minor units for money, such as cents, not floating-point dollars. The approval should refer to the immutable quote ID, not just “approved: true.” That makes a changed basket a different transaction. In a real service, the quote and approval records belong in durable storage with a transaction or compare-and-set transition so two workers cannot submit the same order concurrently.

Make retries safe, not merely convenient

Networks fail at awkward moments. Your server may send an order request, the merchant may accept it, and the response may time out before your application receives it. Retrying with a new request can create a duplicate dinner order.

Generate an idempotency key from the approved quote and persist it before contacting the merchant. Reuse that same key for every retry of the same logical submission. Record the request state, merchant response, and returned order ID. If the provider does not support idempotency, use its documented lookup or reconciliation mechanism before retrying. Do not invent a new order just because the first response was lost.

The state transition should be guarded server-side, roughly like this:

async function submitApprovedQuote(quoteId: string, userId: string) {
  const attempt = await db.transaction(async tx => {
    const order = await tx.orders.lockByQuoteId(quoteId);
    if (!order || order.userId !== userId) throw new Error("not found");
    if (order.status === "CONFIRMED") return { existing: order };
    if (order.status !== "APPROVED") throw new Error("not approved");
    if (order.expiresAt <= Date.now()) throw new Error("quote expired");
    return tx.orders.markSubmitting(order.id, order.idempotencyKey);
  });

  if ("existing" in attempt) return attempt.existing;
  return merchantApi.createOrder(attempt.payload, {
    idempotencyKey: attempt.idempotencyKey,
  });
}

This snippet assumes a transaction, row lock, and merchant API behavior that your actual stack must implement and test. It is not enough to put an idempotency header in a request if your application forgets which key belongs to which approved quote.

Approval must be meaningful

A confirmation screen is not meaningful if it hides the details that could change the decision. Show the merchant, exact items and substitutions, delivery destination, fees, tax, tip, total, and the deadline for the quote. Make the action label explicit: “Place this order for $X” is clearer than “Continue.” If the amount rises, an item is substituted, or the address changes, return to approval.

The authorization must also have a scope. “Spend up to $40 at this merchant for these listed items before 7:15 PM” is narrower than “buy groceries for me.” The AP2 specification describes open mandates for autonomous purchases, where a user approves constraints and the agent produces a specific checkout that is evaluated against them. It also recommends short expirations and requires that an agent not reuse an open mandate for another checkout until the previous one is resolved. That is a useful design pattern even if your implementation does not adopt AP2: bind permission to limits, merchant, duration, and the transaction being made.[3]

Payment credentials need similar boundaries. Stripe's agentic commerce guide describes Shared Payment Tokens scoped to a retailer, order amount, and time window, so an agent can initiate a payment without seeing the underlying payment details. That is a vendor-specific product description, not a general guarantee or a drop-in integration. The general principle is to avoid exposing reusable credentials to a model or agent process.[4]

Keep the merchant authoritative

The agent's cart is a proposal. The merchant remains the source of truth for inventory, current price, taxes, discounts, fulfillment availability, and order acceptance. Revalidate the quote at the commit boundary. If the merchant returns a changed total, do not silently charge the difference. Move the order back to a state where the user can review the new terms.

This also gives the system a clean failure vocabulary. “Agent suggested a basket,” “merchant returned a quote,” “user approved,” “payment authorized,” and “merchant confirmed” are separate events. Do not display “Your order is placed” after only a tool call or payment authorization. Wait for the authoritative confirmation and show the order ID or a clear pending status.

Tests worth writing before connecting a payment provider

Approval for one quote cannot authorize a different quote, even if the totals match.
A quote past its expiration is rejected.
A total increase or item substitution invalidates the previous approval.
Two concurrent submissions produce one merchant order.
A retry after a lost response reuses the same idempotency key.
A merchant timeout produces PENDING_RECONCILIATION, not a false success or a blind second order.
An agent cannot alter the shipping address after approval.
The model's textual statement that a user approved something is never accepted as proof of approval.

These tests target the boundary between probabilistic planning and deterministic execution. They do not prove the whole checkout is safe, but they make the most consequential transitions explicit and reviewable.

A practical decision rule

Use a human-present confirmation when the order is new, high value, sensitive, or likely to involve substitutions. Consider bounded autonomous purchasing only when the user has explicitly configured a narrow policy, the merchant provides stable structured quotes, credentials are scoped, and you can reconcile uncertain outcomes. Keep the two paths separate in both code and audit records.

DoorDash's “several hundred” figure is not a verdict on agentic commerce. It is a reminder that a demo, an integration, and a habitual customer behavior are three different milestones. A useful system has to do more than call tools. It has to earn permission, preserve the exact terms that were approved, and recover safely when the network does something ordinary and inconvenient.

Checkout review checklist:

Is the final merchant quote immutable and expiring?
Does approval bind to that exact quote, user, and maximum amount?
Can retries create duplicate orders?
Who is authoritative when price or availability changes?
Can the audit trail explain what the user saw and what the merchant accepted?
Does the user have a genuinely trusted surface for confirmation?

I write about developer tools, backend systems, and AI engineering. Subscribe if you want more practical pieces like this.

Have you built an agent flow that can spend money or submit a real order? Where do you put the approval boundary?

Sources

[1] TechCrunch: DoorDash said it’s only received “several hundred” orders from AI agents
[2] DoorDash Q3 earnings-call transcript, filed with the SEC
[3] Agent Payments Protocol specification
[4] Stripe: Agentic Commerce, A Getting Started Guide

When Java 27 Removed JVMCI, TornadoVM Built Its Own Compiler

jamilxt — Sat, 31 Oct 2026 16:08:40 +0000

A JDK upgrade can turn an implementation detail into a product-level problem.

For TornadoVM, the detail was JVMCI, the JVM Compiler Interface. The project used a relocated copy of Graal through that interface to translate Java kernels into GPU code. When JDK 27 removed JVMCI, TornadoVM had a choice: keep building around an integration that no longer existed in the new JDK, or take ownership of the compilation path. Oracle's JDK 27 migration guide documents the removal. [2]

With TornadoVM 8.0.0, its maintainers chose the second route. They replaced the bundled Graal compiler with TC, a Java bytecode-to-kernel compiler that emits CUDA C++, OpenCL C, or Metal code. Their release report says the new path preserves application APIs, passes the previous compiler's passing tests, and reaches comparable or higher throughput in selected workloads. Those are project-reported results, not an independent benchmark, but the architectural change is real and worth studying. [1]

The interesting story is not simply that a GPU framework wrote a compiler. It is that the team used a breaking platform boundary to simplify its own system, improve diagnostics, and make more deployment options possible.

The dependency was doing more than compiling

TornadoVM lets Java programs offload suitable computation to accelerators. Its users express work using APIs such as TaskGraph, TornadoExecutionPlan, KernelContext, @Parallel, and @Reduce. Under those APIs, a compiler must understand Java bytecode, map supported operations to accelerator execution, and produce code that a device driver can compile.

Previously, TornadoVM shipped a modified and relocated copy of Graal for this job. Graal is an optimizing compiler, but the TornadoVM team says its pipeline was designed to generate CPU machine code, not source kernels for a GPU driver. It also created a close dependency on JVMCI and internal JDK module exports. The project says this prevented it from reading newer class-file versions and kept its builds pinned to --release 22.

JDK 27's official migration guide lists the removal of JVMCI among that release's removed features. That's a narrow change in the JDK, but it had a large effect on software whose internals were built around the interface. [2]

The scale of the replacement is striking. In its release account, the TornadoVM team says the compiler replacement touched 1,218 files, adding about 40,166 lines and deleting about 119,699. It removed the relocated Graal module, vendored jdk.internal.vm.ci, and backend-specific Graal packages. This was not a small adapter written to make the next JDK compile. It was a redesign of a critical subsystem. [1]

TC narrows the compiler to the job

TC is a compiler written in Java, targeting JDK 21 and later. It reads class files directly, builds its own typed intermediate representation, performs optimization passes, lowers TornadoVM-specific operations, and prints code in a shared C-like form with target-specific dialects for CUDA, OpenCL, and Metal.

That design matters because the output is source code for a device-driver compiler. The team describes TC as having a front end, an SSA-based intermediate representation, optimization passes such as constant folding, dead-code elimination, global value numbering, loop-invariant code motion, and loop unrolling, followed by TornadoVM lowering and structured source emission.

The key distinction is not that TC has more compiler features than Graal. It is that TC can represent and expose the transformations this use case needs. The compiler can preserve Java variable names, source lines, and bytecode positions as it translates a kernel. The generated CUDA shown in the project article carries comments mapping emitted operations back to the Java source line.

That is practical debugging value. A GPU performance tool can identify a costly generated instruction, and the source annotation helps a developer trace it back to the Java kernel. When TC cannot compile a method, the project says it emits categorized error codes and points to the Java line and bytecode index. Compare that with diagnosing a failure deep inside an imported compiler pipeline: the new approach makes the boundary between application code and generated code more visible.

TC also runs without a GPU or the full TornadoVM runtime for its standalone emission mode. That allows a developer or CI job to inspect a generated kernel without having the target accelerator installed. It does not eliminate the need to test on actual hardware. It does make some compiler feedback easier to obtain and reproduce.

What the reported results say, and what they do not

A compiler replacement is only credible if it preserves behavior. The TornadoVM team's release article reports these test comparisons:

CUDA on an RTX 4090: 1,516 tests passed with Graal and 1,539 with TC.
OpenCL on an RTX 4090: 1,027 with Graal and 1,057 with TC.
OpenCL on Intel UHD 770: 932 with Graal and 934 with TC.
Metal on an Apple M3 Pro: 942 with Graal and 941 with TC. The team notes this Metal result was measured before the old Graal code was deleted and has not been rerun after removal.

The team says every test that passed before still passes, while TC handles some cases that the former compiler rejected. The Metal caveat should stay attached to the number. It would be misleading to present every backend as equally revalidated after the replacement landed.

For an application workload, the report uses jitLLM with Llama 3.2 1B and Qwen3 0.6B. It says the generated output was token-identical, with throughput between 1.06 and 1.14 times Graal on CUDA, and 1.04 to 1.09 times on OpenCL. On Metal, the two were within measurement noise. These are useful project measurements, but they are not a broad claim that all Java GPU workloads become faster. Results depend on the workload, device, model, precision, and measurement setup.

The honest takeaway is narrower: the replacement appears to have met its compatibility and selected-performance targets in the workloads and test matrix the project published. Teams evaluating it should look at the raw specification, reproduce their own kernels, and test every target backend they plan to support.

The surprising benefit is deployment choice

Removing JVMCI did not only restore compatibility with a newer JDK. According to the project, it also removed a blocker for Project Leyden AOT caches and GraalVM Native Image.

In a companion report, the team measured a small saxpy program that compiles and runs one CUDA kernel. It took about 0.6 seconds from launch to exit on a regular JVM, compared with about 0.3 seconds using an AOT cache or a Native Image in the stated RTX 4090 setup. The compiler's own kernel compilation reportedly dropped from about 30 milliseconds to 11 milliseconds with the cache, and to under 1 millisecond in the Native Image. The project notes that JNI-based libraries such as cuDNN, cuDF, and CUTLASS are not supported in Native Image yet. [3]

These figures are startup measurements for a specific small program, not a promise of doubled throughput for a long-running service. The project itself says startup optimizations matter most for command-line tools, tests, and short-lived workloads; long-running services gain less once warm. But the additional deployment paths are meaningful. An application may be able to choose a conventional JVM, a cached startup path, or a native executable according to its runtime environment.

This is a useful systems lesson: architectural boundaries can constrain features beyond the one they were introduced to support. A dependency that ties compilation to an internal JVM interface may also complicate startup optimization, native packaging, build flags, and future JDK compatibility. Replacing it changes more than the compiler.

What Java teams can borrow from this change

Most teams do not need to write a compiler after a library dependency breaks. The transferable idea is to use a compatibility crisis to inspect the shape of the dependency, rather than automatically rebuilding the same coupling around a replacement.

When an internal platform boundary changes, ask:

What capability do we actually consume? Name the narrow function, not just the library. TornadoVM needed to translate Java kernels into accelerator code, not to carry all of Graal's CPU JIT architecture.
Which constraints come from the dependency itself? The team points to class-file limits, internal module exports, backend code duplication, and poor visibility into intermediate code.
Can we preserve the public contract while replacing the implementation? TornadoVM says its application-facing APIs and kernel ABI remain stable in version 8.
What is the cost of owning the new layer? A compiler means ongoing semantic compatibility, optimization work, target-specific behavior, and a larger testing burden. One successful release does not make that maintenance disappear.
What evidence would make the migration safe? Define a test-parity gate, representative application benchmarks, explicit platform matrices, and a visible caveat list before declaring success.
Can the new design provide a second benefit? Here, the project links independence from JVMCI with AOT caches and Native Image support. A redesign can pay for itself through capabilities that the immediate compatibility patch would not provide.

This is not an argument to reject established dependencies. Graal served TornadoVM for years, and rewriting compiler infrastructure is an expensive choice. The lesson is to treat the break as an opportunity to evaluate the architecture, not as automatic proof that a full rewrite is justified.

What I would verify before adopting it

I have not run TornadoVM 8.0.0 or reproduced its benchmarks. This article is based on the project's release materials and Oracle's JDK migration documentation, so treat the performance numbers as vendor-reported until independently reproduced.

If you work with Java GPU computing, a practical evaluation can stay small:

Select a kernel your application actually needs, not only a sample vector operation.
Run it on every accelerator backend and JDK version you intend to deploy.
Compare output correctness first, then warm throughput and cold-start behavior separately.
Inspect the emitted code and error messages for a kernel you understand.
Test any Native Image or AOT configuration against your real libraries, especially if JNI is involved.
Record the device, driver, JDK build, compiler settings, and workload shape so a later comparison is meaningful.
Keep a rollback path until your test suite covers the kernels and launch configurations that matter to you.

That checklist is more useful than treating the release's strongest benchmark as a general verdict. The project has made a substantial engineering claim; a user should validate it against their own compute path.

A compiler rewrite can be a platform strategy

TornadoVM 8 is a good example of a team turning a JDK removal into a chance to own the piece of infrastructure closest to its product. The reported outcome is more than “it builds on Java 27”: the project says it now has a compiler designed for GPU kernel emission, clearer diagnostics, less dependence on internal JDK interfaces, and new startup options.

The cost is also real. TC is now a compiler the project must evolve and support. The published figures are promising, but they come from the project itself and include a backend caveat. That combination calls for interest, not blind adoption.

For Java teams, the broader question is worth keeping: when a platform removes a hook your system depends on, do you need a smaller adapter, a new dependency, or a narrower subsystem you can own? The answer depends on what the dependency was doing and what capabilities the current architecture prevents.

I write about Java, Spring Boot, and AI every week. Subscribe if you want more engineering breakdowns like this.

Sources

Have you had a platform change force a rethink of a core dependency? What did you replace, and what did you keep?

Before an AI Rewrites Legacy Code, Build a Parity Gate

jamilxt — Sat, 31 Oct 2026 12:05:11 +0000

A team can generate a new implementation faster than it can decide whether the new implementation is still the same product.

That is the important tension behind a report that Cisco president Jeetu Patel gave a 27-year-old engineer unlimited AI tokens and a goal to rewrite 98% of Cisco's legacy code within 24 months, with a four-week proof of concept. The account comes from Fortune reporting summarized by Tom's Hardware. The headline is striking. The engineering problem underneath it is less glamorous: how do you know a rewrite still does what users and dependent systems expect?

This article is a source-based engineering guide, not a report from a Cisco project or a hands-on legacy migration. The public coverage describes an ambition and a proof-of-concept window, not completed results. It does not establish that 98% has been rewritten, that the effort will meet the deadline, or that token supply is the limiting factor. The useful takeaway is a method you can apply to a small, reversible change: create a parity gate before asking an agent to replace behavior.

Why generated code is the easy part

A legacy system often has no single complete specification. Its behavior is distributed across code, database constraints, production data, integrations, error handling, and assumptions that nobody wrote down. A rewrite can look cleaner and still change a default, rounding rule, ordering guarantee, or failure mode that another system depends on.

The first question is not, “Can an agent reproduce this function?” It is, “What observable behavior must remain stable, and what evidence would show that it changed?”

The distinction is practical. A test that merely calls the new code and checks that it returns a plausible value proves little about equivalence. A parity test runs the same representative inputs through the old and new implementations and compares the outcomes. A reviewer can then inspect disagreements instead of trying to infer correctness from a large diff.

Microsoft's guidance for migrating AI agents recommends defining acceptance gates before a model change, capturing a baseline from the current system, using the same test data and environment assumptions for both versions, and investigating individual regressions rather than trusting an average score. Those ideas apply just as well to a code rewrite. See Microsoft's migration-validation guidance. For coding-agent work, Visual Studio Code's testing guide also says to preserve assertions, report what ran, and treat unrun tests as unverified.

Build the smallest useful parity harness

Start with a behavior that is bounded, deterministic, and important enough to matter. A pricing rule, date conversion, permission decision, or serialization function may be a better first slice than an entire service. Do not begin with the hardest code in the repository. Begin with a seam where you can call both implementations under controlled conditions.

For illustration, imagine an old and new implementation that both accept JSON inputs and return JSON results. The harness below reads one case per line, sends each input to both commands, and records exact output mismatches. It intentionally treats a process error as a failure rather than silently skipping it.

#!/usr/bin/env python3
"""Compare legacy and candidate commands on newline-delimited JSON cases."""
import json
import subprocess
import sys
from pathlib import Path

CASES = Path(sys.argv[1])
OLD_COMMAND = ["python3", "legacy_impl.py"]
NEW_COMMAND = ["python3", "candidate_impl.py"]


def run(command, case):
    completed = subprocess.run(
        command,
        input=json.dumps(case),
        text=True,
        capture_output=True,
        check=False,
        timeout=10,
    )
    if completed.returncode != 0:
        raise RuntimeError(
            f"{command[1]} exited {completed.returncode}: "
            f"{completed.stderr.strip()}"
        )
    return json.loads(completed.stdout)


def main():
    cases = [json.loads(line) for line in CASES.read_text().splitlines() if line]
    mismatches = []
    for index, case in enumerate(cases, start=1):
        old = run(OLD_COMMAND, case)
        new = run(NEW_COMMAND, case)
        if old != new:
            mismatches.append({"case": index, "input": case,
                               "legacy": old, "candidate": new})

    print(json.dumps({
        "cases": len(cases),
        "mismatches": len(mismatches),
        "details": mismatches,
    }, indent=2, sort_keys=True))
    return 1 if mismatches else 0


if __name__ == "__main__":
    raise SystemExit(main())

Save representative inputs in cases.jsonl, one JSON object per line, then run python3 parity.py cases.jsonl. Replace the two command arrays with real, isolated entry points. The example is intentionally strict: if the output formats differ, the harness reports a mismatch. Normalize only fields that are genuinely nondeterministic, and document each normalization so it cannot hide a meaningful regression.

This small script is not a migration framework. It is a starting gate. Production systems may need database snapshots, service stubs, message queues, feature flags, and stronger isolation. The key is that each version sees the same input and that you retain the evidence for every comparison.

Choose cases that reveal contracts

A collection of ordinary happy-path examples is not a useful characterization set. Include cases that make hidden behavior visible:

Typical input: common values from real use, with sensitive data removed or safely synthesized.
Boundaries: empty strings, minimum and maximum values, timezone transitions, rounding edges, and large payloads where relevant.
Invalid input: missing fields, malformed values, and unsupported combinations.
Failure behavior: timeouts, downstream errors, unavailable dependencies, and retry limits.
Compatibility details: field ordering if consumers rely on it, exact error codes, null versus omitted values, and stable identifiers.
Previously fixed bugs: regressions that the old code now avoids should be explicit cases.

Do not ask an agent to invent the expected result for every case and then treat its own answer as the oracle. Where possible, derive expected outcomes from existing tests, approved requirements, recorded behavior, or a domain owner. When the old implementation is the reference, it can reveal current behavior, but it cannot tell you whether that behavior is correct. Mark known defects as exceptions with an explicit decision rather than preserving them accidentally.

Make mismatches reviewable

A raw “17 tests failed” message is not enough. Each mismatch should show the input, the old output, the candidate output, and a stable case identifier. Group related mismatches by behavior: rounding, validation, authorization, serialization, and so on. A reviewer can then decide whether a difference is a bug, an intended improvement, or an old defect that should not be carried forward.

Keep three outcomes distinct:

Equivalent: the difference is not observable to supported callers, and the reason is documented.
Approved change: behavior is intentionally changing, with an owner and a migration or compatibility plan.
Unresolved: the difference blocks promotion until someone decides what the contract should be.

Never turn a mismatch into a pass by weakening the test just to make the candidate green. Microsoft's guidance explicitly recommends investigating individual regressions rather than relying on an aggregate rate. A 99% pass rate can conceal the one authorization or billing case that matters most.

Put the agent inside the gate, not above it

A coding agent can help inventory call sites, propose test cases, explain mismatches, and implement a candidate behind a feature flag. Those are useful tasks. It should not be allowed to change the acceptance criteria and approve its own output in the same unreviewed step.

A safer task boundary looks like this:

Ask the agent to map the selected behavior and identify callers, tests, data assumptions, and failure paths. Require file and line references so a human can verify the map.
Have a person review the map and choose the initial cases. Add cases for critical requirements that are missing from the existing suite.
Ask the agent to implement the replacement without editing the parity cases or their expected values. Protect those files through repository permissions or a separate review boundary where possible.
Run both implementations on the same fixed corpus. Save the command, version identifiers, environment, results, and mismatch report with the change.
Review every mismatch, inspect the final diff, and run the relevant existing test suite before merging.
Roll out gradually behind a reversible switch. Compare production signals where safe, and define a rollback trigger before the rollout begins.

This also makes “unlimited tokens” less important as a planning metric. More generation can increase the amount of code to inspect. The scarce resource may be trustworthy test cases, time from domain owners, or a review path that catches an unintended behavior change. That is an engineering inference, not a reported finding about Cisco's internal project.

A promotion checklist worth saving

Before replacing a legacy component, ask:

Is the selected slice small enough to revert independently?
Does the old implementation run in a controlled baseline environment?
Are cases drawn from real contracts and known incidents, not only agent-generated examples?
Are critical paths weighted separately from low-risk cases?
Can the harness reproduce each mismatch from a saved input?
Are nondeterministic fields normalized narrowly and transparently?
Are test files and expected outputs protected from the implementation agent?
Does every accepted behavior change have an owner and compatibility plan?
Do performance, security, privacy, and operational checks supplement functional parity?
Is rollout gradual, observable, and reversible?

A parity harness is not proof that a rewrite is safe. It is a way to turn a vague claim of equivalence into inspectable evidence. It tells you where to focus human attention, and it makes the decision to proceed, pause, or roll back more concrete.

Start with evidence, then scale

The public Cisco account is a striking example of how AI changes the perceived cost of attempting a large rewrite. But a deadline and token budget are not evidence of preserved behavior. Until results are independently reported, the 98% figure belongs to the stated ambition, not the outcome.

For an engineering team considering a similar effort, begin with one narrow behavior and a fixed corpus. Make the old and new outputs comparable. Let the agent help produce the candidate, but keep the contract, mismatch decisions, and promotion gate reviewable by people responsible for the system. Expand only when the gate catches the kinds of mistakes you care about and the rollback path has been exercised.

I write about software engineering, developer tools, and practical AI workflows. Subscribe if you want more guides like this. Have you used a parity harness for a rewrite or migration? What behavior did it catch that ordinary unit tests missed?

Pull Requests vs Agent Trails: Does AI Code Need a New Review Interface?

jamilxt — Sat, 31 Oct 2026 03:04:34 +0000

A pull request was designed around a simple unit of work: a person writes a change, proposes a diff, and asks another person to review it. Coding agents change the volume and shape of that work. They can produce multiple commits quickly, repeat a task across repositories, and revise code in response to automated feedback. The familiar review page can become a queue of diffs and comments while the real question remains unanswered: did the change accomplish what was intended?

That is the problem Entire is trying to address with Trails, a review workflow in its new Git forge. Rather than treating the pull request diff as the only review surface, Entire puts the task, agent activity, evaluations, and build results together. The company says agents can score customized evaluations on each push, while a human validates progress and merge gates determine when the change is ready.[1]

It is an ambitious proposal, but it should be judged as a workflow hypothesis, not as proof that human code review is obsolete. The useful comparison is not “old GitHub versus a magical agent forge.” It is diff-centered review versus evidence-centered review, and whether the second can preserve the strengths of the first.

What a pull request is good at

A pull request is more than a diff viewer. In GitHub’s documented workflow, reviewers inspect files and changes, leave comments or suggestions, and submit a decision such as approve or request changes.[4] Branch protection can also require reviews and passing status checks before changes merge.[3]

That model has several durable advantages:

The change is concrete. Reviewers can inspect the exact lines that will be merged.
The conversation has history. Questions, responses, suggestions, and decisions stay attached to the proposed change.
Policy is explicit. Required reviews and checks can block a merge rather than merely offering advice.[3]
It works across tools. Git is the shared version-control layer, while the pull request gives teams a familiar place to coordinate.

These are not incidental features. A repository needs a clear record of what changed, who approved it, and which checks passed. A faster code generator does not remove that need.

The weakness appears when the patch is no longer the whole story. A reviewer may need to understand the original request, the agent’s plan, which tests were selected, what changed after a failure, and whether the latest push invalidated an earlier review. A diff can show what changed without explaining whether the code solves the right problem.

What Entire’s Trail changes

Entire launched its forge on October 26, 2026. Its announcement describes Trails as a new review surface where the idea, agent trace, and outcome sit alongside the code. It says customizable evaluations can score each push, agents can address findings in a loop, and humans can validate progress.[1]

The company also describes a broader platform, not just a new pull request screen: Git-compatible hosting, integrations with Buildkite and Depot for CI, and “Continuous Memory” that connects codebase information and previous agent sessions.[1][2] Its blog says failing CI blocks merging on a Trail, while the announcement says the service can mirror GitHub repositories as well as host repositories natively.[2]

That distinction matters. Teams do not have to interpret the launch as “replace GitHub today.” Entire says repositories can be mirrored, and Trails can be used over pull requests while GitHub remains the source of truth.[2] For a cautious evaluation, mirroring a low-risk repository is a more reversible experiment than migrating a production codebase and its governance all at once.

Entire also advertises pushes up to 89 times faster than GitHub and up to six times faster than Cursor Origin, based on its own open-source benchmark.[1][2] Those are company-reported benchmark results, not an independent finding that the forge will make a team’s delivery 89 times faster. Push throughput is not review quality, merge safety, CI reliability, or time-to-correctness. Keep those measurements separate.

The practical comparison

Pull requests optimize for human-readable change review. They make diffs, comments, approvals, and required checks the center of the workflow. This remains a strong fit when changes are small enough to inspect, maintainers need a clear discussion record, or the team’s approval process is built around reviewing code line by line.

Trails try to optimize for intent and evidence around agent work. The proposed unit includes the task and the agent’s work history, with machine-generated evaluations and CI results providing more context than a diff alone.[1][2] This may help when agents make repeated pushes and the reviewer needs to follow progress rather than re-open a separate comment thread for every iteration.

The tradeoff is that an evaluation score is not a substitute for understanding the change. An agent can produce a plausible score against an incomplete or poorly designed evaluation. Passing tests demonstrate that selected tests passed, not that the system has no defect. A session trace can explain the agent’s process, but length does not guarantee that the reasoning was sound.

The best version of an agent-first review system would therefore make evidence easier to inspect without hiding the source code. A reviewer should be able to move from task intent to relevant tests, from test results to the actual diff, and from an evaluation failure to the affected behavior. If the UI presents a green score more prominently than the code and the evaluator’s limits, it risks creating a new kind of automation bias.

How to evaluate it without betting the repository

A useful trial should answer a workflow question, not simply test whether the interface looks modern. Pick one low-risk repository and a handful of representative changes. Keep the existing source of truth and required merge rules intact during the trial.

For each change, record four separate signals:

Review time: How long did a human need to understand the task and inspect the final change?
Defect discovery: Did the workflow surface meaningful issues before merge, and were they issues a reviewer could verify?
Rework: How many agent pushes or human corrections were needed before the change was acceptable?
Governance: Can the repository still enforce the required human approvals, status checks, and merge restrictions?

Do not combine these into a single “agent review score.” A tool might reduce review time while increasing missed defects, or improve context while adding operational work. Decide in advance which outcomes matter to your team.

For every evaluation that can affect merge readiness, ask:

What exact requirement is being tested?
Which code and test inputs did the evaluator inspect?
Can a developer reproduce the result locally or in CI?
What happens when the evaluator is unavailable or returns an ambiguous result?
Can a human override it, and is that override recorded?
Does a new push invalidate the previous evaluation and approval?

These questions apply to any forge that adds AI scoring, not just Entire. The goal is to make checks reviewable, repeatable, and appropriately scoped. An opaque “9/10” should not be treated as stronger evidence than a failing test with a clear reproduction.

A decision checklist

Stay with pull requests for now if:

Your work is already reviewed reliably through diffs, comments, and required approvals.
Your audit or compliance process depends on established approval records.
Agent-generated changes are occasional, and the review queue is not a real bottleneck.
The alternative cannot demonstrate that it preserves your branch rules and CI protections.

Run a bounded Trail trial if:

Agents make frequent revisions and reviewers lose the task context between pushes.
You want task intent, session history, evaluations, and CI results in one place.
You can test by mirroring a repository without changing the current source of truth.[2]
You are willing to evaluate the quality of the evidence, rather than trusting a score at face value.

Do not migrate just because of a speed claim. First verify what the benchmark measured, reproduce it for your own workload where possible, and separately measure review outcomes. Push speed may matter for some distributed workflows. It does not establish that the resulting code is easier or safer to merge.

The interface may change, but the responsibility does not

Entire’s launch is a useful signal that the pull request is no longer the only plausible interface for reviewing software. Agent-heavy development creates more activity around a change: prompts, plans, repeated edits, evaluation runs, and test results. A single diff-and-comment page may not organize all of that particularly well.

But a new surface does not eliminate the basic engineering obligations. Teams still need to know what was requested, what code will ship, which checks ran, who is accountable for approval, and what evidence supports the decision. Trails could make that evidence more visible. Pull requests already provide a mature way to inspect code and apply governance. The right choice depends on whether a new workflow measurably improves the review process without weakening its controls.

The takeaway: evaluate review systems by the quality and inspectability of the evidence they put in front of a human, not by how many agent-generated changes they can push per second.

I write about developer tools, software engineering, and AI. Subscribe if you want more practical analysis like this.

Have you tried an agent-first code review workflow, or does the pull request still fit your team? What evidence would make you trust an automated evaluation?

Sources

[1] Entire launches an independent Git forge for agent-human collaboration

[2] The Git Forge for Agent-Human Collaboration

[3] About protected branches - GitHub Docs

[4] Review pull requests - GitHub Docs