DEV Community: Aditi Gupta

Claude Opus 5.5 Explained: Coding, Cost, and What Developers Need to Change

Aditi Gupta — Wed, 30 Sep 2026 08:22:14 +0000

An AI coding model can generate a convincing patch in seconds. The more useful question is how long it takes to produce a patch you can confidently merge.

Did it find the relevant code? Preserve existing behavior? Test the failure case? Notice that its fix breaks under concurrent requests?

That is the lens through which to evaluate Claude Opus 5.5. Anthropic’s release promises improvements in coding capability, speed, and efficiency. Early community feedback is enthusiastic, but the practical value depends on the complete workflow.

This guide explains what Opus 5.5 is, how its pricing works, what changes in the API, and how to evaluate it on a realistic coding task.

Find full Review here :

What is Claude Opus 5.5?

Claude Opus 5.5 is an AI model from Anthropic, released on September 22, 2026, designed for long-running coding and knowledge work.

It processes text and images and produces text. When connected to tools through an application, it can inspect files, request code changes, run commands, and use the results to decide what to do next.

Specification	Opus 5.5
Claude API model ID	`claude-opus-5-5`
Context window	1 million tokens
Standard maximum output	128,000 tokens
Input	Text and images
Output	Text
Thinking	Adaptive, always enabled
Default effort	`medium`
Knowledge cutoff	June 2026

These are the specifications in Anthropic’s official model overview.

The model and the coding application play different roles. Opus generates responses and tool requests. The application provides repository access, executes permitted actions, and maintains the conversation.

Calling the API alone does not give Claude access to your filesystem or turn it into an autonomous coding agent.

What does “agentic coding” mean in practice?

Consider a webhook handler that occasionally processes the same payment event twice.

A single code-generation request might produce a replacement function. An agentic workflow can investigate the surrounding system:

Read the webhook handler and database schema
    ↓
Inspect queue behavior and existing tests
    ↓
Reproduce duplicate processing
    ↓
Implement a fix
    ↓
Run sequential and concurrent delivery tests
    ↓
Inspect failures and revise the patch

The challenge is broader than writing syntax. The agent must discover whether duplicate delivery comes from retries, simultaneous requests, or a failure between the side effect and the acknowledgement.

It also needs to understand what the system means by “processed.”

That makes this a useful example for evaluating a coding model: a plausible local edit can still be an incorrect system-level fix.

What has improved over Opus 5?

Anthropic reports that Opus 5.5 generates output more than 30% faster than Opus 5 and costs approximately 40% less on typical workloads at default settings. The cost estimate combines lower token prices with reduced token consumption.

Its published coding results include:

Evaluation	Opus 5	Opus 5.5
Terminal-Bench 4.0	52.3%	66.4%
FrontierCode v1.1, Main	48.0%	54.4%
CursorBench 4.0	46.6%	57.8%

These are Anthropic-reported results under specified configurations. Most listed Opus 5.5 results use maximum effort; Terminal-Bench uses xhigh. Anthropic also discloses fallback-model use when safeguards intervened in certain evaluations. Source: launch announcement.

The results justify testing the model. They do not predict how reliably it will fix your webhook handler.

Also distinguish output speed from task completion time. Faster text generation helps, but a coding session includes repository searches, commands, tests, and repair attempts.

What developers are noticing

Early discussions emphasize faster responses, lower subscription usage, and clearer explanations. Some developers also report stronger bug fixes and more effective delegation to subagents. Community discussion.

These observations suggest useful evaluation questions:

Does the model need fewer correction prompts?
Does it explain the actual behavioral change?
Does it inspect relevant dependencies before editing?
Are its final reports easier to verify?

Treat reports such as “this used 4% of my allowance instead of 10%” as individual experiences. Subscription allowances are not a direct measurement of API spending.

Creative demos provide another perspective. An interactive 3D project reportedly consumed about $1,874 in tokens, while comments identified existing assets and prebuilt systems used in the scene. Such examples show what a combined workflow can produce, but do not isolate the model’s contribution or establish typical costs. Project discussion.

Pricing: why “half the token price” is not half the task cost

Standard API pricing is:

Token category, per million tokens	Opus 5.5	Sonnet 5.5
Uncached input	$4	$2
Output	$20	$10
Cache reads	$0.20	$0.20
Five-minute cache writes	$5	$2.50
One-hour cache writes	$8	$4

Opus 5.5 fast mode has premium input/output rates of $8/$40 per million tokens. Batch processing discounts standard input and output pricing by 50%. Source: official pricing.

Notice that cache reads cost the same for both models. A session that repeatedly reads cached context will not necessarily become half as expensive when switched to Sonnet.

For an illustrative Opus request without caching:

100,000 input tokens × $4 / 1,000,000  = $0.40
 10,000 output tokens × $20 / 1,000,000 = $0.20

Model cost = $0.60

That excludes tool charges and other pricing modifiers.

For the webhook task, count the investigation, implementation, testing, and repair attempts together:

Cost per accepted task =
    total spending across all evaluation runs
    ÷ number of runs that meet acceptance criteria

Track human review time separately. A low API bill is less impressive if a developer spends an hour correcting the patch.

Opus 5.5 versus Sonnet 5.5

Sonnet 5.5’s published Terminal-Bench 4.0 score is 70.6%, compared with Opus 5.5’s 66.4%. Opus leads on other coding evaluations, including CursorBench 4.0.

Anthropic positions Sonnet for well-scoped everyday tasks and Opus for complex, open-ended work requiring sustained judgment. It also notes that higher Sonnet effort settings can produce task costs closer to Opus. Source: Sonnet 5.5 announcement.

A benchmark score does not cleanly separate “reasoning” from “implementation.” It measures performance on a particular set of tasks under a particular setup.

For our example, I would test Sonnet on implementing a clearly specified deduplication mechanism. I would test Opus on investigating an ambiguous duplicate-processing incident that crosses the handler, database, and queue.

Those are evaluation hypotheses, not guaranteed model rankings.

A practical API example: reviewing the webhook handler

Here is a simplified handler with a concurrency problem:

def handle_webhook(event, db, payments):
    if db.was_processed(event["id"]):
        return {"status": "already_processed"}

    payments.apply_credit(
        customer_id=event["customer_id"],
        amount=event["amount"],
    )

    db.mark_processed(event["id"])
    return {"status": "processed"}

Two requests can both pass was_processed() before either records completion. A process can also crash after applying credit but before marking the event as processed.

Ask Opus to review these failure modes before requesting a patch.

Install the SDK:

pip install -U anthropic

Set ANTHROPIC_API_KEY in your environment, then save the handler as webhook.py and run:

from pathlib import Path
import anthropic

client = anthropic.Anthropic()
source = Path("webhook.py").read_text()

review_request = """
Review this webhook handler for duplicate side effects.

Analyze:
- Sequential delivery of the same event.
- Concurrent delivery of the same event.
- A crash after applying credit but before recording completion.

Identify what depends on the database and payment API.
Do not invent transaction guarantees or available methods.
Propose a minimal design and focused regression tests.
Explain why the design handles each failure case.
"""

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=4096,
    output_config={"effort": "medium"},
    messages=[
        {
            "role": "user",
            "content": (
                f"{review_request}\n\n"
                f"```
{% endraw %}
python\n{source}\n
{% raw %}
```"
            ),
        }
    ],
)

for block in response.content:
    if block.type == "text":
        print(block.text)

print("Stop reason:", response.stop_reason)
print("Usage:", response.usage.model_dump())

The request follows Anthropic’s documented API pattern. Responses must be read by block type, and max_tokens includes thinking plus visible output. Inspect the stop reason for truncation. Source: migration guide.

This example requests analysis. It does not execute tests or implement a fix.

The design review should establish whether credit updates and deduplication can share a database transaction. If the side effect occurs in an external service, determine whether that service supports an idempotency key. A unique event record alone does not resolve every crash-recovery scenario.

That distinction is a useful test of the model’s engineering judgment.

API migration changes to check

Existing integrations may need more than a new model ID.

Change	What can break	What to do
Thinking is always enabled	Disabled thinking or manual thinking-budget requests	Omit `thinking` or use adaptive thinking; tune effort
Forced tool selection is unsupported	`tool_choice` values `any` or `tool`	Use `auto` or `none`; handle the possibility of no tool call
Thinking blocks are bound to conversation context	Replaying blocks after editing earlier instructions or tools	Preserve blocks and follow documented history-update rules
Progress updates arrive in thinking blocks	Interfaces rendering only text may appear silent	Configure a supported display mode and update rendering
Older computer-use tool unsupported on Claude API and Google Cloud	Requests using `computer_20251124`	Migrate to the supported computer toolset
Refusals can return HTTP 200	Applications treating every HTTP success as task success	Inspect `stop_reason` and refusal details

See Anthropic’s API and behavior changes. When migrating from older models, also review sampling parameters and assistant-prefill restrictions in the migration guide.

Build completion checks into the agent

A coding agent can end a turn while work remains.

Anthropic specifically documents that Opus 5.5 may return a progress report with stop_reason: "end_turn" during a longer task. It recommends tracking open work, using bounded continuations, and waiting for pending commands or subagents. Source: unattended-agent guidance.

For the webhook fix, define completion before execution:

Task: prevent duplicate webhook side effects.

Inspect:
- Handler and persistence code.
- Payment-service interface.
- Queue retry behavior.
- Existing tests.

Completion requires:
- Sequential duplicate-delivery coverage.
- Concurrent duplicate-delivery coverage.
- A tested recovery path for interrupted processing.
- Existing public response behavior preserved.
- Relevant tests executed and results reported.

Identify blockers explicitly.
Avoid unrelated refactors.

Your application should verify test outcomes and pending work before accepting completion. Keep runtime, spending, and continuation limits as well.

For a small fix, one agent may be sufficient. If you delegate test implementation to a second agent, provide an explicit contract, isolate file changes, and validate the integrated result. Include coordination and repair costs when comparing that approach with a single-agent run.

A reproducible evaluation you can run

The following is a proposed experiment, not a report of measured results.

Use one small repository with a webhook handler, a persistent test database, and a controllable fake payment service. Prepare three task variants:

Task	Agent receives	Acceptance criteria
Sequential duplicates	Reproduction showing the same event delivered twice	One credit applied; retry receives the expected response
Concurrent duplicates	A test triggering overlapping deliveries	One credit applied under forced overlap
Interrupted processing	A fault injected after the external side effect	Retrying safely completes recovery without a second credit

Keep acceptance tests outside the agent’s editable workspace. For the concurrency test, use synchronization to force overlap; merely launching two requests may fail to exercise the race.

Compare Opus 5.5 at medium effort with Sonnet 5.5 at medium effort. Repeat each task three times per configuration from a clean checkout. That gives 18 runs—a small exploratory evaluation, not enough to establish a universal ranking.

Hold these conditions constant:

Repository state, instructions, tools, and permissions.
Dependencies and test environment.
Per-run time and spending limits.
Acceptance criteria and review rubric.

Record the results without filling gaps with estimates:

Configuration	Accepted runs	Total cost	Median completion time	Median review time
Opus 5.5, medium	—	—	—	—
Sonnet 5.5, medium	—	—	—	—

Inspect rejected patches as carefully as successful ones. Did the agent miss the race, invent a database capability, weaken a test, or stop before validation?

If you publish the experiment, include the repository commit, prompts, execution dates, SDK version, effort settings, and failed runs. That gives other developers enough context to challenge or reproduce your findings.

Opus 5.5’s reported gains make it worth evaluating. The strongest evidence for adopting it will be a change in your own workflow: more accepted patches, fewer repair rounds, or less time spent verifying the result.

Jev by TypeSafe AI: The Complete Developer Guide

Aditi Gupta — Thu, 24 Sep 2026 17:32:23 +0000

Jev is a model from TypeSafe AI that returns typed decisions with probabilities instead of text. You send data plus questions with fixed answer options, and get back one answer per question in roughly 70-500 ms, at $0.042 per million input tokens with output free. It's great for routing, classification, scoring, guardrails, and LLM-as-judge work. It can't write, can't do math, has no vision, and isn't open weights. Treat it as a smart if statement, test it on your own data, and ignore the hype posts.

In the past ten days, Jev has taken over r/LocalLLaMA, r/LLMDevs, r/AI_Agents, and half of dev Twitter. Some call it the biggest thing since ChatGPT. Others call it "just a classifier" with a $40M marketing budget. Both camps are partly right, and the useful answer sits in the details.

This guide covers what Jev is, how the API works, real code, pricing, where it breaks, what people are building, and straight answers to the questions that keep coming up on Reddit.

Find full video here:

What Jev actually is

Jev is a model that makes decisions and does not write text. You send it some data (the "state") plus a set of typed questions, and it sends back one answer per question with probabilities attached.

It comes from TypeSafe AI, a San Francisco lab that came out of stealth on September 15, 2026 with $40M in seed funding led by DCVC. The founder is Diogo Almeida, who worked at OpenAI on RLHF and InstructGPT. Jev is their first public model, and TypeSafe calls it a "System One model."

The best mental model is a smart if statement. Normal code branches on things it can compute:

if (order.total > 100) applyDiscount(order);

That falls apart when the condition needs judgment. Is this ticket angry? Is this email about billing? Which of these 40 buttons continues checkout? Before Jev you had three options: write brittle rules, train a custom classifier with labeled data, or ask an LLM for JSON and hope. Jev takes questions you define at runtime like an LLM, but returns a constrained probability distribution like a classifier.

The name has two references. "Jev" comes from William Stanley Jevons, whose paradox says that cheaper resources lead to more consumption. "System One" comes from Daniel Kahneman's fast, intuitive System 1 thinking, as opposed to slow, deliberate System 2 reasoning. TypeSafe's bet is simple: make an AI decision cost a tiny fraction of a cent and you'll put decisions in places you'd never call an LLM.

If you want the ELI5 version for your non-dev friends: Jev is a very fast multiple-choice test taker. You hand it some information and a few questions with fixed answer options, and it circles one answer per question and tells you how sure it is. It can't write an essay, but it can take thousands of these tests per second for pennies.

How it works under the hood

TypeSafe hasn't published much about internals, so be careful with anyone claiming to know the exact architecture. What's public from the launch post: a new model architecture, a parallel sampler, and a training method called Reinforcement Learning for Calibrated Decisions (RLCD). Jev is transformer-based but not an LLM. No weights, parameter count, or architecture paper have been released.

The key difference from an LLM with structured outputs is how the answer gets produced. An LLM generates {"category": "billing"} token by token, and a schema only constrains that process. Generation can still fail, stop early, or drift. Jev evaluates every question independently and in parallel against the same state, and outputs a probability distribution over the options you defined. A successful response can never contain a value outside your schema.

RLCD is the training side. TypeSafe argues that RLHF tuned models for human preference, which produced good chat but also overconfidence and mode dropping. RLCD instead aims for probabilities that match outcomes: across many predictions, answers given 90% probability should be right about 90% of the time. That calibration claim is the most important one, and it's also the least independently tested so far.

The "it's just Qwen plus logits" theory from r/LocalLLaMA deserves a fair hearing. Yes, you can approximate the behavior by taking an open LLM, doing a single forward pass, and reading the probabilities over a fixed answer set. People have done this for years, and several "Jev in 25 lines of Python" posts show it. Whether Jev does something meaningfully better depends on RLCD and the parallel sampler, and without weights nobody outside TypeSafe can verify that. Test it on your data instead of arguing about it.

The API: three primitives, one endpoint

Everything goes through one endpoint:

POST https://api.typesafe.ai/v1/systemone
Authorization: Bearer <API_KEY>
Content-Type: application/json

The body carries model, state, and a map of questions. There are three question types:

Type	What it asks	What comes back
`noul`	Is this true?	`noul`: probability from 0 to 1
`choice`	Which of these options?	`choice`, `probabilities`, `confidence`
`score`	Where on this ordered scale?	`score`, `legend`, `probabilities`, `confidence`

"Noul" is TypeSafe's name for a yes/no question. The Vercel AI SDK calls the same thing boolean.

Here is a raw request for support ticket triage:

curl -s https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "jev-latest",
    "state": {
      "ticket": "Our webhook stopped firing after the update. We are losing orders. Please fix today."
    },
    "questions": {
      "team": {
        "type": "choice",
        "instructions": "Which team should handle `ticket`?",
        "criteria": {
          "billing": "Charges, invoices, refunds",
          "engineering": "Bugs, outages, integrations",
          "sales": "Pricing, upgrades, new accounts",
          "other": "Anything else"
        }
      },
      "severity": {
        "type": "score",
        "instructions": "How severe is the problem in `ticket`?",
        "criteria": [
          "Cosmetic, nothing is blocked",
          "Feature broken but a workaround exists",
          "Blocking, customer is losing money or data"
        ]
      },
      "is_urgent": {
        "type": "noul",
        "instructions": "Does `ticket` ask for action today?"
      }
    }
  }'

The response always has the same shape: an answers object keyed by your question IDs, the versioned model that answered, and usage with token counts. A Choice answer looks like this:

{
  "type": "choice",
  "choice": "engineering",
  "probabilities": { "billing": 0.02, "engineering": 0.95, "sales": 0.0, "other": 0.03 },
  "confidence": 0.91
}

A Score returns the probability-weighted mean of the level indexes. A 1.7 on the severity scale above means "mostly blocking, some weight on workaround exists."

A few things trip people up:

The question ID is not sent to the model. team means nothing to Jev, so the full question must live in instructions.
Use backticks around field paths like `ticket` or `order.items[0]` so the model knows which part of the state you mean.
Always add an other option to a Choice when your list might not cover every input. Jev has to pick something.

Python SDK

pip install typesafe-sdk   # Python 3.10+

from typesafe_sdk import Choice, Noul, TypeSafeClient

client = TypeSafeClient()  # reads TYPESAFE_API_KEY

resp = client.system_one(
    state={"ticket": ticket_text},
    questions={
        "team": Choice(
            instructions="Which team should handle `ticket`?",
            criteria={
                "billing": "Charges, invoices, refunds",
                "engineering": "Bugs, outages, integrations",
                "other": None,
            },
        ),
        "is_urgent": Noul(instructions="Does `ticket` ask for action today?"),
    },
)

team = resp.answers["team"]
if team.confidence < 0.5:
    send_to_human(ticket_text)
elif team.choice == "engineering" and resp.answers["is_urgent"].noul > 0.8:
    page_oncall(ticket_text)

There's also an AsyncTypeSafeClient with configurable retries.

JavaScript / TypeScript SDK

npm install @typesafe-ai/sdk   # Node 20+

import { choice, noul, TypeSafeClient } from "@typesafe-ai/sdk";

const client = new TypeSafeClient();

const { answers } = await client.systemOne({
  state: { ticket },
  questions: {
    team: choice("Which team should handle `ticket`?", {
      billing: "Charges, invoices, refunds",
      engineering: "Bugs, outages, integrations",
      other: null,
    }),
    is_urgent: noul("Does `ticket` ask for action today?"),
  },
});

// answers.team.choice is typed as "billing" | "engineering" | "other"

The TypeScript SDK infers answer types from your question definitions, so you get autocomplete and exhaustive switch checks for free. Keep calls on the server, since the SDK blocks browser use to protect your API key.

Other ways to call it

Vercel AI Gateway: available as typesafe-ai/jev at the same price, no waitlist. AI SDK 7 has experimental_evaluate for this kind of model. Note that TypeSafe's confidence lives in result.providerMetadata.typesafe.confidence on that path.
LangChain: exposed through TypeSafeClassifier.
Cloudflare Workers AI: users report it listed as typesafe/jev.

Limits and errors

Limit	Value
State + all questions	~64,000 tokens
State + longest single question	~32,000 tokens
Choice options	up to 255
Score levels	2 to 10
Input	text only (string, JSON object, or JSON array)
Rate limits (early access)	250,000 tokens/sec, 1,200 requests/min

Errors: 401 bad key, 422 validation failure (the response names the field), 429 rate limited, 529 overloaded. Retry the last two with exponential backoff; the SDKs do this for you.

The current model is jev-1.13.0, and jev-latest points at the stable release. Log the versioned model ID from every response, and pin it once you've tuned thresholds against it.

Confidence, fan-out, and the patterns that matter

The probabilities are the real product. Every Choice and Score answer carries a confidence from 0 to 1, computed from the shape of the distribution. If one option dominates, confidence is high. If probability is spread across options, it's low. An answer can win at 0.84 and still have confidence around 0.6 because a second option holds meaningful weight.

That gives you a clean three-band pattern:

const { team } = answers;

if (team.confidence >= 0.9) {
  autoRoute(team.choice);          // act
} else if (team.confidence >= 0.5) {
  routeWithReview(team.choice);    // act, but flag for review
} else {
  sendToHuman();                   // don't trust it
}

Set thresholds by the cost of being wrong. Labeling a dashboard event can run at 0.5. Deleting a file should need 0.9 and probably a human anyway. Your risk tolerance lives in plain numbers in your code, which is far easier to review than a prompt.

Speculative fan-out. With LLMs you ask one question, then decide the next. With Jev, questions run in parallel and each extra one only costs its own tokens, so ask everything you might need in one call and let code decide which answers to use. In one TypeSafe cookbook, 13 questions in a single call were 12.2x cheaper and 10x faster than 13 sequential calls. Most of that saving comes from sending a large state once, so it matters most with big inputs.

Composite scoring. When a judgment depends on several factors, ask one Score per factor and combine them in code:

// normalize each score to 0..1 by dividing by (number of levels - 1)
const norm = (ans, levels) => ans.score / (levels - 1);

const priority =
  0.6 * norm(answers.severity, 3) +
  0.3 * norm(answers.frustration, 3) +
  0.1 * norm(answers.report_quality, 4);

If rankings feel off, change a coefficient and rerun. No prompt rewriting.

Routing. This is where most real savings show up. Put Jev in front of your stack and let it decide whether a request needs a database lookup, a cheap model, an expensive model, or a person. The order-status lookup never touches an LLM, and only the hard cases reach your frontier model. The same idea works as a model router inside an agent.

Using coding agents to integrate it. Install TypeSafe's agent skill first, because agents trained on LLM APIs tend to ask one question per call and invent request fields.

# Claude Code
claude plugin marketplace add typesafe-ai/skills
claude plugin install typesafe@typesafe-ai

# Cursor, Codex, others
npx skills add typesafe-ai/skills --skill typesafe-ai

Pricing and speed, with the fine print

Jev costs $0.042 per million input tokens (TypeSafe markets it as $42 per billion), and output tokens are free because there's almost nothing to meter. For comparison, TypeSafe quotes typical LLM input prices at $0.20 to $10 per million. A 300-token support ticket costs about a hundredth of a cent. TypeSafe reports 70-500 ms end-to-end, measured from the US West Coast, so add your own network latency if you're elsewhere.

The famous "193.6x faster, 444.6x cheaper" numbers need context. They come from TypeSafe's own workflow evals, where the reference answers were generated by frontier models and the workflows were written by TypeSafe's team. TypeSafe itself says these gains sit at the high end of real use, and that it can't prove the price isn't subsidized.

Independent checks are more modest but still strong:

An r/accelerate test on multiple-choice benchmarks found Jev roughly Terra-tier on System 1 tasks at about 18x lower cost.
Bryo AI's CTO found Gemini slightly more accurate for email triage, but 10-20x more expensive.
One r/LocalLLaMA user got 94.6% with a model trained on their own data versus 68.1% for Jev over the API.

That's the realistic picture. Jev is a strong general-purpose default, and a narrow model trained on your data can still beat it.

For your own bill, the math is simple:

savings = (share of AI spend on decisions) x (how much cheaper Jev is on those calls)

If 60% of your spend is classification, routing, scoring, and verification, you might cut the bill by more than half. If it's 10%, you'll save at most 10%. Include retries, human review, and any LLM steps before or after Jev in the calculation.

Where Jev breaks

TypeSafe is fairly honest about limits and publishes a "jaggedness" page per model version. These are the failure modes that matter in production.

It can't do math. Counting, arithmetic, date comparisons, and hex color distances are unreliable. Do those in code and pass in the result. To count items that match a meaning, ask one Noul per item and sum in code:

const questions = Object.fromEntries(
  items.map((_, i) => [`item_${i}`, noul(`Is \`items[${i}]\` a fruit?`)])
);
const { answers } = await client.systemOne({ state: { items }, questions });
const count = items.filter((_, i) => answers[`item_${i}`].noul > 0.5).length;

It reads you literally. Scoping words, negations, and implied conditions get taken at face value. If you catch yourself explaining what you "really meant" after a wrong answer, that explanation belongs in the instruction.

Score is not a measurement. A 1.4 doesn't mean "40% of the way between levels." Use scores for thresholds and ranking only. Describe situations in your criteria ("blocking, no workaround exists") instead of degrees ("very severe"), or the model spreads probability across levels.

Context rot. Accuracy drops as the state fills with irrelevant content. Filter in code first, or ask a Noul per chunk ("is this relevant?") and drop the rest.

Prompt injection. State is treated as data, but adversarial text can still shift answers. Test with hostile inputs before exposing it publicly.

"Zero hallucinations" is a narrow claim. It means Jev can't return an option you didn't define. It can still pick the wrong one. The 0% figure is a structural guarantee about the output shape, not an accuracy number.

No vision, no writing. Images, audio, and video need converting to text first. People have forced it to "chat" by picking one character at a time, but it's slow and bad at it. The working rule: with Jev you pick a card from the deck, you don't ask it to name one. If your instinct is "extract X," rephrase it as "here are the candidates for X, which one is it?"

It's closed. Hosted API only, early access behind a waitlist, no weights and no self-hosting option.

What people are building with it

The community moved fast. shipwithjev.com, an independent catalog, already lists hundreds of builds. Some highlights with real numbers:

Browser and phone agents. Browser Use's jev-ultrafast ran a Zurich to London Google Flights search in 7.1 seconds. A planner LLM sets the goal, and Jev picks the next element to click as a Choice over the page's interactive elements. Droidrun's mobile-jev drove Uber on a real Android phone.

browser-use / jev-ultrafast

Fastest and cheapest web agent

Jev Ultrafast ⚡

Important

The Browser Use Cloud waitlist is open. Get early access to ultrafast browser agents in the cloud Join the waitlist →

A browser agent with a dynamic, indexed action space.

Give it one goal. TypeSafe's Jev picks an operation and an element. A small LLM writes text only when the operation is TYPE_TEXT.

Zürich → London on Google Flights in 7.1 seconds. One natural-language goal, actual text generation, and loading waits included.

Watch the MP4 · Measurements · Read the loop

The action space

Every observation produces a new element table:

[1] button    Change ticket type · Round trip
[2] combobox  Where from?        · San Francisco
[3] combobox  Where to?          · empty
[4] textbox   Departure          · empty
...

The operations are CLICK, TYPE_TEXT, SELECT, SCROLL_UP, SCROLL_DOWN, WAIT, DONE, and BLOCKED. Only supported operations and targets are offered.

…

View on GitHub

Coding agent guardrails. jev-guard rates each tool call as deny, ask, or allow. Vercel CEO Guillermo Rauch reported Jev up to 18x faster at p95 than GPT Luna for command safety checks, and more accurate. Others use it for Pi's auto mode, instant context compaction (score each old tool call and drop the irrelevant ones), and as a "subconscious" filter that handles small checks before the main model sees anything.

leepokai / jev-guard

Auto mode for every coding agent, built on Jev: risk-scores every tool call with session context (deny / ask / allow), flags prompt injection in results, checks skills and plugins. Claude Code, Codex, Copilot, Gemini, Cursor, pi, OpenCode, ACP.

jev-guard

A security hook for coding agents, powered by Jev.

Auto mode, for every coding agent

Claude Code's auto mode is described as: "A separate classifier model reviews actions before they run, blocking anything that escalates beyond your request, targets unrecognized infrastructure, or appears driven by hostile content Claude read." That is exactly the job jev-guard does — as three typed questions to Jev (risk, user_requested, from_untrusted) instead of a proprietary classifier — and it does it for Codex, Copilot, Gemini, Cursor, pi, OpenCode and ACP editors too, with the same policy and the same session memory everywhere. If you want auto mode outside Claude Code, or a second opinion inside it, this is the build.

Why Jev: price and speed, with sources

Figure	Source
Price	$0.042 per 1M input tokens, $0 output — a typical jev-guard call is ~1k tokens, so ≈ $0.00004 per

…

View on GitHub

Production usage. Metaview, a recruiting platform, says it shipped Jev into every agent on its platform over a weekend, and candidate searches went from minutes to seconds at the same accuracy and lower cost.

Bulk labeling. A teardown of 724 live ads from 37 brands took about 40 seconds and nine cents. Resume screening, email triage, listing classification, and feature engineering for classical ML models all fit the same "label every row" shape.

Search and databases. JevQL and pg-jev let you add plain-language filters to Postgres queries, like WHERE jev(meetings, 'could have been an email'). jevsearch re-ranks keyword hits by intent with no vector database. For memory retrieval in agents, the pattern is similar: pull a larger candidate set with vector search, then ask Jev per item whether it's relevant. Results in r/Rag were mixed, so expect precision gains, not magic.

kylemclaren / jevql

Semantic SQL for Postgres, powered by Jev

jevQL

Semantic SQL for vanilla PostgreSQL. One extra family of functions, jev(), works in any query, from the CLI, from a shared HTTP and MCP node, or from the Go, TypeScript and Python SDKs.

SELECT name, city, jev_prob(people, 'could work from home') AS p
FROM people
WHERE jev(people, 'could work from home')
  AND country = 'PT'
ORDER BY p DESC
LIMIT 20;

The database only ever sees ordinary SQL. jev_* calls are evaluated by the CLI using TypeSafe's System One model (Jev). No CREATE EXTENSION, no superuser, no wire-protocol proxy.

Install

Homebrew (macOS and Linux):

brew install kylemclaren/tap/jevql

Prebuilt binaries for macOS and Linux (arm64 and amd64) are attached to each GitHub release as jevql_<version>_<os>_<arch>.tar.gz with a checksums.txt.

From source, with Go 1.23+ and a C compiler (libpg_query is bundled via pg_query_go and needs CGO):

CGO_ENABLED=1

…

View on GitHub

LLM-as-judge replacement. People use Jev to grade agent traces against rubrics, check whether a cited source supports a claim, and verify RAG answers. One r/Rag benchmark found it tied on accuracy while being 187x cheaper, but it let through 23% of unsupported claims. Good as a first pass, risky as the only check.

Real-time apps and games. Doom at about 10 decisions per second, Pac-Man, StarCraft, Minecraft, NPC combat decisions, live tone scoring while you type, a debate "BS meter," and Home Assistant control through HA-Jev. Voice agents are a natural fit too, since turn-level decisions like intent routing, escalation, and "did the caller confirm?" need answers in well under a second.

Security and moderation. Jailbreak and PII detectors under 300 ms, anti-phishing checks, secret detection, spam and auto-reply detection in the Laravel AI SDK, and moderation against a site's own rules.

Two warnings. The trading bot threads (including one that reportedly lost $31,680 overnight) prove that fast decisions aren't good decisions. Jev doesn't understand markets any better than an LLM. And most of these are first-week demos, not production case studies. Every cost and latency figure above is self-reported by the builder.

Every Reddit question, answered

Isn't it just a classifier?
Mostly yes, and that's the point. Normal classifiers need labeled data and training per task, while Jev takes new labels at runtime. "A zero-shot classifier with calibrated probabilities behind a fast API" is an accurate description. Whether that deserves the hype depends on how many of your LLM calls are secretly classification. The r/AI_Agents hot take nailed it: we've been using LLMs as very expensive if/else statements.

Why not just use normal automation or JSON parsing?
If a rule can decide correctly, use the rule. It's free and never wrong. Jev is for conditions a rule can't express, like "is this customer about to churn" or "does this message sound like a scam." Parsing tells you what fields exist. Jev tells you what the text means.

Jev wasn't first. Same work existed months earlier.
Correct. Using LLM logits for classification is old, and several researchers posted earlier open work. TypeSafe never claimed to invent the concept. Its claim is a new training method and a production-grade API, which is more of a product achievement than a research one.

They raised $40M and an open clone matched the API in 3 days. Where's the moat?
Copying the API shape is easy, since it's just state in and probabilities out. Copying the quality of the probabilities is the hard part. If RLCD really produces better-calibrated answers than a logit-grabbing hack, that's the moat. If it doesn't, the Christensen-disruption crowd is right and this becomes a commodity fast. Nobody outside TypeSafe has proven which is true yet.

How is RLCD even RL if the outputs are differentiable?
Fair question, and TypeSafe hasn't published a paper to settle it. Choice, Score, and Noul outputs could be trained with plain supervised loss if you had perfect labels. The RL framing likely comes from rewarding calibration against outcomes rather than matching fixed labels. Until details are public, treat "RL" as TypeSafe's description, not a verified method.

Can I run it locally or self-host it?
Not Jev itself. Open alternatives appeared within days: Laya (an open-weights System One model some users report beating Jev on speed), OpenJev, Nokia's AnyJev (a training-free layer that turns open LLMs into calibrated decision models), and Contrastive-LM's CLM-8B. If data residency or cost at huge scale matters, benchmark these on your task. As for "Laya came first, why is everyone talking about Jev?", distribution and polish usually beat being first.

Is it a scam?
No. The API works, pricing checks out in independent tests, and the founder's track record is real. The "scam" posts mostly point to a single bias example or to aggressive marketing.

Jev said Taiwan isn't a country / shows sexist or racist tendencies.
Jev always picks one of your options, so it gives an answer on sensitive questions where an LLM might hedge or refuse. That makes underlying biases more visible. It doesn't prove it's a Chinese model or uniquely biased, but it does mean you must test hiring, moderation, lending, and anything involving people for fairness before shipping.

Why is Reddit flooded with Jev posts?
Part genuine excitement, part low-effort SEO spam (the "use Jev for FREE" posts). The r/LocalLLaMA mod request got 1.2K upvotes for a reason. Trust posts with code and numbers.

The demos are misleading (Tesla FSD in an hour, car wash question).
Partly true. The "FSD" demo fed Jev clean simulated data instead of camera input, which is the hard part of self-driving. Game and driving demos show latency, not real-world readiness. The car wash cost comparison ($6.86 for Jev vs $57.84 for DeepSeek) shows a real gap on that task, but one benchmark isn't your workload.

Can I use it with OpenAI, ChatGPT, Codex, or Claude?
Alongside them, not as a replacement. Jev decides, the LLM writes. Common setups use Jev to route between models, gate tool calls, or filter work before an expensive agent runs.

Is it good for coding?
It can't write code. It's useful around coding agents: command safety checks, "is the task done?" checks, picking which files matter, semantic linting, and routing tasks to the right model.

People used it to generate pixel art and game levels. I thought it can't generate?
It can't on its own. Those builds use an LLM or code to propose options, and Jev picks between them fast and repeatedly. The generation comes from the loop, not the model.

Is it good for roleplay?
As a helper, maybe. It can decide mood, which character speaks next, or whether a reply stays in character, while a writing model produces the text.

Jev controlling a Mac in real time: better than Astra or Fable?
Faster, not smarter. Computer control is mostly a string of small choices (which button, which menu), which suits Jev. It needs a text view of the screen like an accessibility tree, since it has no vision. For multi-step planning, reasoning models still lead, so the common pattern is LLM plans, Jev clicks.

Can Jev detect AI-written posts?
It can score text against criteria you write, but it has the same problem as GPTZero and other detectors: there are no reliable signals for AI writing. Don't auto-reject posts on its score alone.

Will 400x cheaper inference unlock ad-supported AI apps?
For decision-heavy features, the cost per user drops low enough that ads could cover it. But most consumer AI value is still generated text, which Jev can't produce.

How do I get access? Is there a free tier?
Join the waitlist at typesafe.ai (people report getting in within a day or two), or use the Vercel AI Gateway at the same price with no waitlist. There's no known free tier beyond that, so be suspicious of "free Jev" posts.

Where to start

Pick one boring decision your code handles with a fragile regex or a costly LLM call. Replace it with one Noul or one Choice. Run it in shadow mode next to your current logic for a week, log the confidence, compare against real outcomes, and only then let it act on the low-risk path. That's how a smart if statement should be adopted: one branch at a time.

Useful links: Launch post · Docs · Agent skills · Community builds

What are you using Jev for, or what's stopping you? Drop it in the comments.

OmniRoute AI Explained: Features, Installation, Claude Code Setup

Aditi Gupta — Wed, 23 Sep 2026 11:03:19 +0000

OmniRoute is a free, open-source AI gateway that connects applications to multiple AI providers through a shared API endpoint. It centralizes provider credentials, model selection, fallback routing, token compression, and usage monitoring.

You can use it with coding tools such as Claude Code and OpenCode, connect it to your own applications, or run it in front of local models.

This guide explains what OmniRoute does, whether it is free, how to install it, and how it compares with OpenCode and 9router. It also addresses questions raised in OmniRoute Reddit discussions about latency, free providers, token savings, and safety.

Find a summary here:

What does OmniRoute do?

OmniRoute receives AI requests from a client, selects an eligible provider or model, forwards the request, and returns the response. Depending on your configuration, it can apply compression, track usage, and fall back to another connection when the first one fails.

A typical workflow looks like this:

Claude Code, OpenCode, or your application
                   ↓
               OmniRoute
                   ↓
       Selected AI provider or local model

The OmniRoute GitHub repository contains the source code, installation instructions, documentation, and release information.

The main benefit is centralized control. Several applications can use the same gateway while you manage provider connections and routing policies in one place.

OmniRoute AI features

Multiple providers and local models

OmniRoute supports several types of connections, including API keys, supported OAuth integrations, public endpoints, and local inference servers.

Its provider reference includes local options such as Ollama, LM Studio, and vLLM alongside cloud providers.

Provider support does not automatically grant model access. Availability still depends on your account, credentials, region, quota, and the provider’s requirements. Provider reference.

Automatic model routing

OmniRoute provides routing aliases that express what you want to prioritize:

Routing alias	Purpose
`auto`	Balanced routing with preference for the last successful provider
`auto/coding`	Quality-focused selection for coding
`auto/fast`	Favor lower latency
`auto/cheap`	Favor lower token costs
`auto/smart`	Quality-focused routing with greater exploration

These aliases represent selection policies, rather than specific models. Actual results depend on the eligible models and connections available to your installation.

Despite its name, auto/offline prioritizes quota availability; it does not restrict requests to local models. Auto-routing documentation.

Combos and automatic fallback

A combo groups models or connections under a routing policy. You can configure an ordered fallback chain or select strategies based on cost, load, quota, context size, and other factors.

For example, a coding combo could try your preferred model first and then use a compatible backup when necessary.

OmniRoute also includes circuit breakers that temporarily stop routing requests to repeatedly failing providers. This improves resilience, but requests can still fail when no eligible destination is available. Resilience guide.

Token compression

OmniRoute includes compression engines that reduce selected content before it reaches a model:

RTK targets terminal, build, test, and other tool output.
Caveman condenses natural-language content.
Stacked compression combines engines in a configured sequence.

Compression can help with verbose coding sessions, but savings vary with the input and settings. Aggressive compression also needs quality checks because removing information can affect the answer. Compression documentation.

Dashboard and usage visibility

The dashboard provides controls for providers, combos, analytics, health, and costs. It gives users a central place to inspect their setup and investigate problems. OmniRoute dashboard overview.

APIs beyond chat

OmniRoute’s documented API surface includes chat, embeddings, image generation, audio, transcription, reranking, search, video, music, and OCR, as well as Batch and Files APIs.

Support depends on the selected provider and model. A unified endpoint does not make every model capable of every task. API reference.

MCP, A2A, memory, and skills

OmniRoute also supports more advanced agent workflows:

Capability	What it adds
MCP server	Structured tools for interacting with gateway functions
A2A interface	Agent discovery and task interactions
Optional memory	Retrieval and reuse of stored context
Skills	Configurable reusable instructions

Memory and A2A are disabled by default in the reviewed documentation. These features are optional; basic routing does not require them. See the MCP, A2A, and memory documentation.

Is OmniRoute free?

Yes. OmniRoute’s self-hosted software is free and MIT-licensed. AI model usage and hosting can still cost money.

There are three separate costs to understand:

Component	Cost
OmniRoute software	Free, open-source software
Upstream AI usage	Depends on provider pricing, subscriptions, or free allowances
Hosting	Your computer’s resources or a server bill

OmniRoute can route requests to providers offering free access, but it does not create unlimited credits or remove upstream restrictions.

Its free-tier catalog distinguishes recurring allowances, temporary signup credits, and shared quota pools. Multiple models may share one allowance, so their advertised limits cannot always be added together. Free-tier documentation.

A free setup is possible when your selected providers offer suitable free access and your usage stays within their conditions. It is not a guarantee that every model or workload will be free.

OmniRoute install: how to set up OmniRoute

1. Install the package

For the npm installation, use a supported Node.js version. The project documents Node 24 LTS as an option.

Run:

npm install -g omniroute
omniroute

The -g flag installs the command globally. Using only npm install omniroute installs the package in the current project instead.

By default, the dashboard opens at:

http://localhost:20128

The OpenAI-compatible API base URL is:

http://localhost:20128/v1

Follow the setup guide for platform-specific installation details.

2. Connect a provider

Open Providers in the dashboard and configure a supported connection using the required credentials or sign-in flow.

Start with one provider and confirm that it works before adding a large fallback chain.

3. Create your gateway API key

Open Endpoints and create or copy your OmniRoute API key.

There are two different credentials involved:

Provider credentials: allow OmniRoute to call the upstream service.
OmniRoute API key: allows your client to call the gateway.

They serve different purposes and should not be confused. Authorization guide.

How to use OmniRoute

Once the gateway is running, configure a compatible client with:

Base URL: http://localhost:20128/v1
API key:  Your OmniRoute API key
Model:    auto, or an available provider/model identifier

To test the connection, set OMNIROUTE_API_KEY in your environment and run:

curl http://localhost:20128/v1/chat/completions \
  -H "Authorization: Bearer $OMNIROUTE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [
      {
        "role": "user",
        "content": "Explain what an API gateway does."
      }
    ]
  }'

You can inspect the model list with:

curl http://localhost:20128/v1/models \
  -H "Authorization: Bearer $OMNIROUTE_API_KEY"

These requests use OmniRoute’s documented API interfaces. API reference.

For readers searching “how to use omni route,” the essential sequence is: install the gateway, connect a provider, configure your client, and test a request. Add routing policies and compression after that basic connection works.

OmniRoute Claude Code integration

OmniRoute can act as the backend gateway for Claude Code, allowing it to use configured, compatible model connections.

With OmniRoute running and Claude Code installed, the documented interactive configuration command is:

omniroute configure claude

Follow the prompts to select the connection and model. OmniRoute also provides process-based launchers for supported coding tools. CLI integration guide.

It helps to distinguish three terms:

Claude Code: the coding application.
Claude: Anthropic’s model family.
OmniRoute: the gateway handling the configured connection.

Using Claude Code through OmniRoute does not necessarily mean a Claude model is answering. That depends on your selected route.

Likewise, an “OmniRoute Claude” integration does not automatically provide free Claude access. Check which provider serves the model, what access your account includes, and whether the connection method is supported.

OmniRoute Docker installation

OmniRoute can also run in Docker. This example keeps the published port accessible only from the host computer:

docker run -d \
  --name omniroute \
  --restart unless-stopped \
  --stop-timeout 40 \
  -p 127.0.0.1:20128:20128 \
  -v omniroute-data:/app/data \
  diegosouzapw/omniroute:latest

The named volume preserves application data across container replacement.

Memory requirements depend on the workload. Long coding-agent sessions can require substantially more memory than light dashboard or chat use. The documentation includes larger memory settings and Compose deployment options; its single-container quick-run path is intended for users who already run Redis separately. Docker guide.

Also remember that localhost refers to the current machine or container. A client inside another container needs an address that can reach the OmniRoute service.

What are the key differences between OmniRoute and OpenCode?

OmniRoute is primarily an AI gateway. OpenCode is an AI coding agent. They can be used together.

Question	OmniRoute	OpenCode
Main purpose	Manage and route model requests	Help users understand and modify code
Main interaction	Gateway API, dashboard, and management commands	Coding interface in a terminal, desktop app, or IDE
Provider handling	Centralizes connections for multiple clients	Connects the coding agent to its chosen provider
Typical responsibility	Routing, fallback, compression, and monitoring	Working with project files and carrying out coding tasks
Can it work independently?	Yes, with compatible clients	Yes, with directly configured providers
Can they work together?	Serves as OpenCode’s gateway	Sends model requests through OmniRoute

OpenCode’s documentation describes workflows for explaining code, planning features, making changes, and undoing changes. OpenCode documentation.

A combined setup looks like:

You → OpenCode → OmniRoute → AI provider

For the supported configuration flow, use:

omniroute configure opencode

OmniRoute also documents a separate plugin integration. Plugin compatibility depends on the OpenCode major version, so follow the appropriate instructions for your installation. OpenCode integration details.

OmniRoute vs 9router

OmniRoute began as a fork of 9router, a relationship acknowledged in its repository. Both projects provide AI gateway functionality. OmniRoute project history.

The current 9router documentation lists fallback routing, multiple accounts, format translation, custom combos, token-saving features, quota tracking, and analytics. Those capabilities should not be presented as exclusive to OmniRoute. 9router features.

OmniRoute’s documentation additionally describes its particular auto-routing modes, compression pipelines, MCP server, A2A interface, and memory system.

For an OmniRoute vs 9router evaluation, compare the exact workflow you need:

Does your provider connection work reliably?
Does your coding client support the integration?
Are the routing controls sufficient?
How much operational complexity does each setup introduce?
Which project resolves issues affecting your environment?

Provider counts alone do not establish which gateway will work better for you.

OmniRoute Online and cloud OmniRoute: what is the difference?

OmniRoute Online commonly refers to the project website, omniroute.online. Visiting the website is separate from running your own gateway.

The website currently presents self-hosting alongside Cheaper Inference as a hosted gateway option. Hosted offerings have their own access and billing arrangements.

A cloud OmniRoute setup can also mean deploying your own instance on a VPS or server. OmniRoute documents remote contexts and scoped tokens for managing a remote installation from a local command line. Remote-mode guide.

Use the endpoint issued by your chosen service or your own deployment. Do not assume an address copied from an older tutorial remains a supported public endpoint.

OmniRoute Reddit questions: practical answers

The questions below reflect themes in community discussions, including the Reddit examples reviewed for this guide. They are not a statistical ranking of the most frequently asked questions.

Is OmniRoute safe?

OmniRoute is open source and can run locally, but those facts alone do not establish that every configuration is safe.

Cloud-routed prompts still reach the selected provider. Data handling also depends on logging, optional memory, upstream services, and gateway access controls.

The project documents authorization controls and guardrails, including optional credential masking. These are useful protections, not proof that sensitive information can be sent through any provider without review. Security and guardrails.

Can using OmniRoute get my provider account restricted?

The answer depends on the provider and connection method. An integration being technically available does not establish permission to use it under every subscription or account type.

Check the provider’s current rules for third-party clients, automated access, and credential use. Treat Reddit reports of bans or successful access as individual reports rather than universal outcomes.

Does OmniRoute actually reduce token usage?

It can reduce tokens in eligible content through compression. The size of the reduction depends on your prompts, tool output, conversation history, and compression mode.

Routing to a cheaper model can reduce spending without reducing tokens. These are different forms of savings.

Measure input tokens, output tokens, cost, and task success separately. Claims such as “95% savings” should not be treated as a guaranteed reduction in your entire bill. Compression modes.

Why is OmniRoute slow?

Possible causes include a slow upstream model, exhausted quotas, request queues, repeated retries, long context, or insufficient server resources.

A useful diagnostic sequence is:

Test one explicitly selected model.
Inspect provider health and request failures.
Check whether retries or fallback attempts explain the delay.
Temporarily simplify the combo.
Compare a short request with the slow workload.

OmniRoute’s resilience documentation describes queues, cooldowns, and circuit breakers that can affect routing behavior. Resilience guide.

What are the best free providers for heavy coding?

There is no permanent best free provider. Availability, quotas, model quality, and tool-calling support change.

For heavy coding, evaluate a provider on a small real task before adding it to a fallback chain. Check whether it can handle your context size, call tools correctly, and sustain the required request volume.

Use OmniRoute’s current free-tier catalog to find candidates, then verify the provider’s own terms and limits. Free-tier catalog.

Why does a listed model fail?

A catalog listing does not prove that your account can currently use that model.

Check the credentials, exact model identifier, quota, region, endpoint compatibility, and whether the provider still offers the model. A model that answers plain-text prompts may also fail a coding-agent request requiring tool calling.

Does OmniRoute provide unlimited AI usage?

No. Routing across eligible connections can improve availability, but each provider’s limits still apply.

If every eligible connection is exhausted or unavailable, the request can still fail.

How to uninstall OmniRoute

For a global npm installation, stop the running process and remove the package:

npm uninstall -g omniroute

Removing the package does not necessarily remove stored configuration, credentials, or usage history.

For the Docker container shown earlier:

docker stop omniroute
docker rm omniroute

These commands preserve the named data volume. Deleting stored data is a separate step and permanently removes the configuration it contains. Follow the instructions for your installation method in the OmniRoute uninstall guide.

Who should use OmniRoute?

OmniRoute is useful for developers who want shared provider management across tools, configurable fallback, usage visibility, or a combination of local and cloud models.

A direct provider connection may be sufficient for a simple application. A gateway becomes more valuable as the number of clients, credentials, routing requirements, and operational questions grows.

To evaluate it, start with one working provider and one client. Then add a compatible backup and test whether the routing, monitoring, and compression features improve your actual workflow.

GPT-6 Astra for Developers: When to Use It and When to Save Your Tokens

Aditi Gupta — Mon, 14 Sep 2026 17:28:56 +0000

I came away from testing GPT-6 Astra with a practical recommendation: use it when the difficult part of a task justifies the extra cost.

For my video, I gave it three game-building prompts, with one attempt per game and no follow-up fixes. It produced a space shooter, a racing game, and a physics stacking game. All three ran at the Light setting, and together they used only a small portion of my weekly allowance.

That made me interested in Astra for substantial first drafts and prototypes. It didn't tell me to replace every model in my workflow.

Here's how I'd decide where to use it.

A quick decision guide

Your task	Where I'd start	What would justify Astra
Small edits, boilerplate, or routine transformations	A cheaper model	The cheaper option repeatedly misses an important requirement
A substantial prototype with several interacting requirements	Astra at a lower effort setting	A usable first version saves significant implementation and correction time
A difficult bug or architectural decision	Your usual model, then Astra if needed	Better diagnosis or reasoning changes the outcome
Work across browsers, files, and applications	Astra with the necessary tools connected	It can complete a meaningful workflow and leave a result you can verify
3D scenes and interactive visual concepts	A focused Astra trial	You need editable spatial output and can inspect it
Repetitive tasks at high volume	A smaller model or deterministic code	Measured quality gains outweigh the additional cost

These are my starting recommendations, not results from a controlled comparison across all those tasks.

What my test actually showed

The useful finding from my three builds was how little intervention they required. I didn't need to repair the main interactions through several additional prompts.

For a developer exploring an idea, fewer correction cycles can be valuable. A working first version gives you something concrete to evaluate.

But my test was limited: three prompts, three first attempts, and no equivalent runs against competing models. It supports trying Astra for prototyping. It doesn't establish long-term maintainability, production readiness, or a universal coding advantage.

Video walkthrough:

Where I'd give Astra a harder assignment

Problems with several constraints

Astra is positioned for complex reasoning, coding, research, computer use, and document creation. I'd consider it when a task combines several of those capabilities. OpenAI's model documentation.

For example, I'd try it on a bug that crosses multiple components, a migration plan with compatibility constraints, or an analysis that requires reconciling conflicting evidence.

My evaluation would be specific: did it identify the problem, respect the constraints, and produce reasoning I can check?

I wouldn't choose it just because an assignment contains code or mathematics. The difficulty and the value of getting it right matter more.

Work that spans applications

Computer use lets models operate browser and desktop interfaces through a connected environment. That makes tasks such as testing a user flow or completing a sequence in an application possible, provided the necessary tools and access are available. Computer-use documentation.

I'd consider Astra when the work involves understanding a goal and carrying it through several steps. I'd still want an inspectable result: a saved artifact, completed checks, or a clear account of what changed.

3D and spatial prototyping

This is another promising area. OpenAI has published a walkthrough of Astra building editable Blender scenes, inspecting renders, and transferring a scene into Unreal Engine. Architectural visualization example.

That gives me a reason to test it on a visual prototype with geometry, materials, and interactions. It doesn't establish that the result is physically accurate or suitable for engineering use.

Where I wouldn't spend the premium

I wouldn't make Astra the default for renaming variables, formatting JSON, summarizing a short document, or producing routine boilerplate.

I'd also avoid repeatedly asking a premium model to explore an idea while I'm still deciding what I want. I'd clarify the brief first, then give Astra a task with a defined finish line.

If Sol or another model already handles your regular development work well, I don't see a reason to switch without a comparison on your own repository. Review the quality of the diff, the tests, the unnecessary changes, and the time you spend correcting it.

For repeated structured work, I'd also consider whether a script would solve the problem more predictably.

Token efficiency doesn't automatically mean lower cost

At the published base API rates:

Model	Input per 1M tokens	Cached input per 1M tokens	Output per 1M tokens
GPT-6 Astra	$10	$1	$50
GPT-5.6 Sol	$4	$0.40	$20

Source: OpenAI's model comparison.

Here's an illustrative calculation for 100,000 uncached input tokens and 10,000 total billed output tokens:

Astra: $1.00 input + $0.50 output = $1.50
Sol: $0.40 input + $0.20 output = $0.60

These are calculated examples, not measurements from my games. They exclude tool fees and use base rates without caching or speed adjustments.

Astra can still be economical if it needs fewer attempts or substantially less work to complete the task. OpenAI reports lower estimated API cost per task in several evaluations because Astra used fewer output tokens while achieving stronger results. That result is specific to those evaluations. Model guidance.

There is also a hidden part of the output count: reasoning tokens are billed as output tokens even though they aren't shown as the final answer. A concise response can still involve substantial reasoning. Reasoning-token documentation.

The number I care about is the cost of reaching an acceptable result, including retries and review.

Subscription limits need a separate check

Those API calculations are not a conversion formula for your subscription's usage meter.

In Codex and Work, allowance consumption depends on the model, context, complexity, reasoning, tools, retrieval, and caching. A five-hour allowance window does not promise five hours of continuous execution. Subscription usage documentation.

This is why I wouldn't use my inexpensive game runs to predict the cost of a long session inside a large repository.

I would check the allowance before and after a representative task, while accounting for any other sessions running at the same time. Then I'd compare the consumption with how often I need to repeat that work.

Fast mode deserves a separate decision, too: faster execution can consume credits at a higher rate. Speed and usage details.

Where Astra still needs improvement

Calibrating effort to the assignment. OpenAI's guidance notes that Astra can perform broader testing than a small coding change requires. I'd like stronger judgment about when another check will improve confidence and when the task is already complete.

Knowing when to proceed. The documentation also describes clarification pauses and sensitivity to conflicting instructions. Those behaviors can interrupt a task even when the user expects it to continue.

Producing cleaner prose by default. Astra can favor detailed formatting and recurring phrases. If the output is documentation, a technical explanation, or a report, that can create editing work. Documented behaviors.

Making consumption easier to understand. I'd like clearer explanations of which parts of a run consumed the allowance. That would make it easier to distinguish expensive reasoning from unnecessary repetition.

My game test did not measure these failure modes. They are documented behaviors and product improvements I'd watch for when evaluating longer-term use.

The workflow I'd recommend trying

Define the deliverable. State the result, constraints, relevant files, and acceptance criteria.
Start with lower effort. My builds succeeded on Light. Increase effort when the task exposes a need for deeper reasoning.
Keep context relevant. Supply enough information to solve the problem without filling the session with unrelated history.
Use Astra for a bounded role. A difficult plan, implementation, or review can be a sensible assignment. Splitting work across models is worth testing, but handoffs aren't free.
Ask for evidence of completion. For code, that might mean relevant test results and an explanation of unresolved issues.
Compare total effort. Include model cost, corrections, review time, and whether you could actually use the result.

I'd use Astra more often if it consistently reduced the time needed to get work into an acceptable state. If it produced roughly the same result as my usual model at a higher cost, I'd keep the usual model.

That's the comparison I'd encourage other developers to make on a few tasks they understand well.

Gemini 3.8 Flash for Coding: Settings, Prompts, and Debugging Tips

Aditi Gupta — Wed, 09 Sep 2026 17:17:14 +0000

Gemini 3.8 Flash can help you get from an idea to a working prototype quickly. Getting reliable results, though, depends on how you define the task, manage revisions, and test the output.

This guide covers a practical workflow for using it: where to start, how to write useful prompts, and what to do when a debugging conversation stops making progress.

The observations come from a session building three small games in Google AI Studio. The prompt templates below are reusable examples based on those lessons.

The quick version

Start with a small feature you can test.
Try medium thinking first; evaluate whether higher thinking improves your particular task.
Include expected behavior and constraints in the prompt.
Describe visual choices explicitly.
When a fix fails repeatedly, ask for a diagnosis before another patch.
Check functionality separately from appearance.

The video walkthrough shows the builds and the issues that came up:

1. Start with a task you can verify

Gemini 3.8 Flash is available through Google’s developer tools, including Google AI Studio. For current capabilities and availability, see the official model documentation.

During my session, I could build using free AI Studio access without entering a credit card. Treat that as an account of the session: access limits and API billing are separate things to check before relying on it for ongoing work.

For an initial project, choose something with a clear finish line:

A form that validates a few inputs.
A searchable list using sample data.
A single interactive page.
One feature in an existing application.

“Build a project management app” leaves many decisions unresolved. “Build a task list with add, complete, and filter actions” gives you something you can inspect.

You can add complexity after the basic behavior works.

2. Use medium thinking as a starting point

Higher thinking sounds like the obvious choice for coding. My debugging session gave me a reason to question that default.

A racing prototype registered a collision when there was no other car on the road. On high thinking, the model repeatedly attempted a similar unsuccessful fix. After I switched to medium, it changed approach and resolved the issue.

That is one observation, not proof that medium is universally better. It also doesn’t establish a fixed percentage of token savings.

My practical recommendation is to start with medium for small, well-defined tasks and inspect the result. Try higher thinking when a task needs more analysis, then compare whether it produces a better outcome.

Watch for repeated failure as well as usage. A long response doesn’t tell you whether the diagnosis is improving.

3. Give the model acceptance criteria

A useful coding prompt explains how you will decide whether the result works.

Here’s an example:

Build a task list using HTML, CSS, and JavaScript.

Features:
- Add a task.
- Mark a task complete.
- Filter by all, active, or completed.
- Save tasks in localStorage.

Acceptance criteria:
- Reject empty or whitespace-only tasks.
- Preserve tasks after a page refresh.
- Keep each task's completion state when switching filters.
- Show an empty state when a filter has no matching tasks.

Constraints:
- No external dependencies.
- Support keyboard interaction.
- Make the layout usable on mobile.

Explain how to run it and how to check each acceptance criterion.

This makes several otherwise hidden decisions explicit.

It also improves follow-up requests. If filtering breaks, you can point to a specific requirement instead of asking the model to “make it work.”

Keep the first request focused. A small feature with clear checks is easier to evaluate than a large application with loosely defined behavior.

4. Specify the design instead of asking for “modern”

AI-generated interfaces can look convincing while feeling surprisingly similar.

My three game prototypes shared visual tendencies despite being different genres. Describing the palette, viewing angle, and mood more precisely helped create variety.

For a web interface, try instructions like:

Visual direction:
- Warm white background and dark gray text.
- Muted blue for primary actions.
- Compact layout with readable spacing.
- Clear borders around inputs.
- Minimal decoration.
- Visible keyboard focus states.

You can also describe the intended user experience:

This is an internal tool used repeatedly throughout the day.
Prioritize scanning, readable tables, and quick access to actions.

That gives the model a reason for the design choices.

Use an actual screenshot when you have a specific visual target. When you don’t, a few concrete constraints are more useful than several broad adjectives.

5. When debugging stalls, ask for evidence

Repeatedly asking “try again” can keep a conversation circling around the same assumption.

A better follow-up includes:

The steps that reproduce the issue.
What you expected.
What happened instead.
What the previous fix failed to change.

For example:

The previous patch did not resolve the issue.

Steps to reproduce:
1. Add two tasks.
2. Mark the first task complete.
3. Switch to the active filter.
4. Switch back to all tasks.

Expected:
The first task remains complete.

Actual:
Both tasks appear active.

Before editing:
- Trace where completion state is stored and updated.
- Identify the likely cause and the code supporting that diagnosis.
- Explain why the previous patch did not address it.

Then propose the smallest relevant fix.

The purpose is to get a diagnosis you can assess.

If the model claims a fix works, ask what it actually checked. A suggested test and an executed test are different pieces of evidence.

6. Keep the current project state clear

As you revise a project, the conversation accumulates obsolete instructions, failed patches, and decisions you have changed.

When the model starts repeating earlier work or losing track of requirements, summarize the current state:

Current goal:
Add filtering to the existing task list.

Already working:
- Creating tasks.
- Toggling completion.
- Saving and loading tasks.

Current issue:
Switching filters changes completion state.

Preserve:
- Existing storage format.
- Existing keyboard interactions.

Relevant files:
- app.js: state and event handlers.
- index.html: filter controls.

A short summary helps make the next request unambiguous.

If you start a fresh chat, include the relevant current code too. The summary explains the task; the code lets the model investigate the implementation.

7. Test behavior separately from appearance

The collision bug in my racer was a useful reminder: a polished screen can still contain incorrect logic.

For a small web application, check a few categories:

Area	Useful checks
Main flow	Can the user complete the intended task?
Input	What happens with empty, invalid, or unusually long values?
State	Does behavior remain correct after filtering, editing, or refreshing?
Errors	Does the interface handle a failed request or missing data?
Interaction	Can you use the main controls with a keyboard and on a small screen?

Choose checks that fit the feature. You don’t need an elaborate test suite for every prototype, but you do need evidence that its important behavior works.

If a bug returns after a later change, a focused regression test can help catch that specific failure.

Where Flash and Pro fit

For my workflow, Flash is useful when the task is concrete and I can inspect the result quickly. I still prefer Pro for longer planning discussions and explanations.

That led to a simple personal rule: plan with Pro, build with Flash.

It isn’t a universal model ranking. Choose based on the next task and the quality of the results you’re getting.

For a small build, the most useful loop is:

Define one behavior.
Generate an implementation.
Run it.
Report a specific failure.
Verify the fix before expanding the scope.

Gemini 3.8 Flash made the first implementation easy to reach in my session. Clear requirements and careful feedback made that implementation more useful.

What has helped your AI coding sessions most: better initial prompts, smaller tasks, or more specific debugging feedback?

Archify : Honest Review

Aditi Gupta — Thu, 03 Sep 2026 07:20:29 +0000

Architecture diagrams become unreliable when the code changes but the diagram does not.

Archify approaches this problem by generating diagrams through your coding agent. The agent analyzes a repository, creates a typed JSON description of the system, and Archify validates and renders it as an interactive HTML/SVG diagram.

I tested it on an unfamiliar repository to answer four practical questions:

How accurately does it represent real code?

Can the diagram be regenerated when the code changes?

Is its architecture comparison useful during reviews?

What should you verify before trusting the output?

This post explains how to install Archify, generate a focused architecture diagram, check its claims against the repository, and avoid wasting model usage on an overly broad scan.

What Archify actually does

Archify is an agent skill for Claude Code, Cursor, Codex, and OpenCode.

The coding agent examines your repository or system description and writes a typed JSON representation of the architecture. Archify validates that representation and compiles it into a self-contained interactive HTML/SVG diagram.

It currently supports five types of diagrams:

Architecture
Workflow
Sequence
Data flow
Lifecycle

This distinction matters: the coding agent interprets the repository, while Archify validates and renders the resulting structure.

Archify can catch malformed data, invalid references, layout problems, and several kinds of misleading visual routing. It cannot guarantee that the agent understood every architectural detail correctly.

You still need to review the result.

How to try it

Archify requires Node.js 18 or newer.

Install the skill globally:

npx skills add tt-a1i/archify -g

Then open your project in a supported coding agent and use a scoped prompt such as:

Analyze this repository, then use Archify to create a high-level
runtime architecture diagram.

Show 8–12 core components, the primary request path, external
dependencies, storage, and trust boundaries.

Include source evidence where it is supported. Put secondary
details in cards instead of adding more edges.

The 8–12 core components constraint is useful. Without it, a large repository can produce a technically detailed diagram that is difficult to read.

For a system that does not exist as code yet, you can also start with plain English:

Use Archify to draw this system:

Browser -> API gateway -> authentication service -> application API
-> Redis cache -> PostgreSQL fallback.

Show the trust boundary around the private services and database.

1. Regenerating a diagram is easier than maintaining one

Every architecture diagram I have created manually was accurate for a limited time.

Then somebody renamed a service, moved a responsibility, or introduced another queue. The code changed, but the diagram did not.

An Archify diagram is not automatically synchronized with your repository. You must rerun the agent when the architecture changes.

The difference is that regeneration is much cheaper than manually redrawing the system.

Instead of deciding whether an old diagram is still trustworthy, I can regenerate it from the current repository and review the differences.

That makes architecture documentation feel more like a build artifact and less like a drawing somebody has to remember to maintain.

2. Source evidence makes the diagram easier to challenge

In evidence-backed architecture mode, components can reference repository files and line ranges pinned to a specific commit.

This is the part I found most valuable.

When I select a component, I can inspect the source evidence behind it instead of accepting a convincing-looking box. If a component is missing, I can search the repository and determine whether the agent overlooked it or whether the code genuinely is not there.

There is an important limitation, though.

Validation does not make hallucination impossible. The agent still authors the JSON structure, and repository evidence is optional and subject to supported repository conditions.

My rule is simple: I do not trust the diagram because it looks polished. I trust it only after I have checked the important nodes and relationships against the code.

A useful verification pass is:

Do the main nodes correspond to real folders, modules, or services?
Do the important edges represent actual calls or data movement?
Are external dependencies shown?
Are security and deployment boundaries based on evidence?
Is anything suspiciously absent?
Does the primary path match the application’s real entry point?

The diagram accelerates understanding. It does not replace engineering judgment.

3. Guided views explain the system without changing it

Archify diagrams can contain guided views that focus the reader on existing nodes and relationships.

A view might highlight:

How a request enters the system
Where authentication happens
How tools are selected
Where untrusted execution is isolated
How the response returns to the caller

These views are presentation layers over the authored topology. They do not silently create additional components or connections.

This makes the output useful for more than private exploration. I could see it working well for onboarding, design discussions, and explaining an unfamiliar service during a review.

4. The delta view could be useful in pull requests

Archify can compare two validated architecture snapshots and produce a Before, Delta, and After view.

From a checkout containing the Archify CLI, the command looks like this:

node archify/bin/archify.mjs compare architecture \
  base.json \
  head.json \
  architecture-delta.html \
  --json

Adjust the CLI path to match your installation.

The comparison can identify authored additions, removals, changes, movement, and rerouted relationships. It also produces a machine-readable receipt.

What it does not do is equally important.

The delta view does not determine:

Runtime impact
Operational risk
Whether a change is safe
Whether a pull request should be merged

It compares two authored architecture descriptions. It does not replace tests, observability, or human review.

Even with that limitation, seeing architectural additions and removals beside a pull request is something a static diagram rarely provides.

Where I would be careful

Archify itself is open source and free to run locally. The potentially expensive part is the coding agent analyzing your repository.

On a small application, that may be negligible. On a large monorepo, an unbounded request can consume significant model context and usage.

I now scope repository runs by:

Naming the application or package I want analyzed
Limiting the diagram to the most important components
Asking for one primary runtime path
Excluding generated files and vendored dependencies
Requesting additional diagrams only when the first one exposes a real question

I would not begin with “diagram the entire monorepo.”

I would start with something narrower:

Map the runtime architecture of packages/payments only.

Include its public entry points, database access, queues, external
providers, and calls to other workspace packages.

Limit the result to 12 primary components.

Smaller diagrams are cheaper to generate and easier to verify.

When I would use Archify

I would use it for:

Exploring an unfamiliar repository
Preparing an onboarding walkthrough
Documenting a request or data path
Reviewing a proposed architectural change
Creating a diagram that needs to be regenerated regularly
Turning a system description into a shareable technical artifact

I would not rely on it alone for:

Security audits
Production dependency discovery
Runtime performance analysis
Merge-safety decisions
Proving that an architecture is complete

Those require evidence beyond a diagram.

Video walkthrough

I tested Archify on both a plain-English system and a real repository. I also show two failure cases and how I checked the output.

REPLACE_WITH_YOUTUBE_URL

Final verdict

Archify did not eliminate the work of understanding an unfamiliar codebase.

It changed the order of that work.

Instead of reading everything before I could form a useful mental model, I started with a structured map. I then used the code to confirm, correct, and deepen that map.

That was much faster than starting from a blank whiteboard.

The most useful feature was not that the diagrams looked good. It was that the underlying structure could be validated, regenerated, inspected, and compared.

Have your architecture diagrams ever survived longer than a month, or have you accepted that they eventually lie?

DeepSeek Harness Explained: What It Is, When to Use It, and When Not To

Aditi Gupta — Fri, 21 Aug 2026 08:44:36 +0000

AI models can generate code, explain repositories, and suggest fixes. But a model alone cannot safely inspect your project, execute commands, remember a long-running task, or coordinate multiple tools.

That surrounding infrastructure is called an agent harness.

DeepSeek Harness also called dsh is DeepSeek’s open-source implementation of that infrastructure. It provides the runtime that connects a model to files, tools, sessions, sandboxes, approval policies, workflows, and a user interface.

Here is the breakdown:

This article explains what DeepSeek Harness is, what makes it interesting, how to try it, and—just as importantly—when it may be the wrong tool.

DeepSeek Harness is currently a developer preview. Expect breaking changes, unfinished edges, and rapidly evolving APIs.

The short version

DeepSeek Harness is not:

A new DeepSeek model
A model-training framework
A replacement for Node.js, Python, or your IDE
A guarantee that an AI-generated change is correct or safe

It is an agent runtime.

A useful mental model is:

AI agent = model + harness

The model provides reasoning and language capabilities. The harness gives that model a controlled way to interact with the outside world.

Your request
    ↓
DeepSeek Harness
    ├── builds the model context
    ├── exposes approved tools
    ├── manages the workspace
    ├── executes tool calls
    ├── records the session
    ├── applies approval and sandbox policies
    └── returns results through the UI
    ↓
Configured AI model

The model decides what it wants to do. The harness decides how that action is represented, executed, recorded, and controlled.

Why do agents need a harness?

Suppose you ask a regular chat model:

Find the authentication bug in this repository, fix it, and run the tests.

The model needs more than intelligence to complete that request. It needs a way to:

Discover the repository structure
Read the relevant files
Search for related code
Edit the implementation
Execute the test suite
Inspect failures
Make another change
Preserve a record of what happened

A harness supplies these capabilities.

Without one, you have a model that can tell you what code might work. With one, you have an agent that can potentially inspect and modify a real environment subject to the permissions you give it.

That last part matters. A harness makes a model more useful, but it also makes the model more capable of causing damage. Workspace boundaries, approvals, sandboxes, and human review remain essential.

What makes DeepSeek Harness different?

The main design principle is:

Everything is a plugin.

Models, tools, skills, sessions, storage, sandboxes, agent loops, scheduling, and even the UI are provided through plugins.

At the center is Cordis, a plugin kernel responsible for mounting plugins, resolving their dependencies, and letting them communicate through services and events.

This has an important practical consequence: capabilities can be replaced or recomposed without maintaining a permanent fork of the harness.

For example, a developer could theoretically swap:

One model provider for another
A local shell backend for a remote sandbox
The default storage implementation for a custom store
One approval policy for a stricter policy
The standard agent loop for a specialized workflow
The browser UI for another client

This architecture is most valuable when you want to build or study agent infrastructure, not merely chat with an AI model.

DeepSeek Harness does not require a DeepSeek model

The name can be misleading.

DeepSeek Harness includes direct support for configuring DeepSeek, but it can also work with other catalog providers and custom OpenAI-compatible endpoints. The model and the harness are separate layers.

That means you can evaluate questions such as:

How does the same model behave with different tools?
How do two models perform inside the same agent environment?
What happens when the sandbox or approval policy changes?
Can an internal model endpoint be connected to a reusable agent runtime?

For custom providers, you supply details such as the provider ID, base URL, API protocol, credentials, and model list.

Be aware that “OpenAI-compatible” does not always mean perfectly compatible. Different gateways may use different roles, token-limit fields, reasoning formats, or image capabilities. DeepSeek Harness exposes compatibility settings for these cases, but connecting an unusual endpoint may require experimentation.

Every run is traceable

DeepSeek Harness uses an append-only session log.

The log records the model-visible history of a run, including prompts, messages, tool calls, tool results, context injections, and agent activity. Features such as resuming, forking, searching, replaying, and inspecting a trajectory are built from this event stream.

This is useful for debugging questions like:

Why did the agent edit this file?
Which tool result changed its direction?
What context did the model receive?
Where did a multi-step task begin to fail?
Did the problem come from the model, a tool, or the harness configuration?

Traceability is especially helpful when developing an agent system. Looking only at the final answer often hides the real failure.

It also has a privacy implication: session logs may contain code, prompts, tool output, file contents, or other sensitive context. Treat stored trajectories as potentially sensitive data.

The four runtime modes

DeepSeek Harness provides several modes for different kinds of work.

Mode	What it provides	Best suited for
Standard	File editing, shell access, search, skills, planning, goals, subagents, and workflows	General agent-assisted development
Code	Standard capabilities exposed through a code-based orchestration SDK	Multi-step tool orchestration
Minimal	A persistent shell and file editor	Benchmarking models with minimal harness influence
Creator	Runtime inspection and plugin experimentation in addition to standard capabilities	Building presets and extending the harness

Standard mode

Start here if you want to understand the normal user experience.

It provides the familiar capabilities expected from a coding agent: reading files, editing code, searching, running commands, planning work, and delegating subtasks.

Code mode

Code mode lets the model combine several tool operations in a generated TypeScript program.

This can reduce the overhead of repeatedly moving between the model and individual tools. It is useful for complex orchestration, but it also increases the importance of execution controls and careful review.

Minimal mode

Minimal mode intentionally removes most of the surrounding machinery.

It is useful when comparing models or studying how much the harness itself influences performance. It is less convenient for everyday development because many higher-level capabilities are absent by design.

Creator mode

Creator mode is for developers experimenting with the harness itself.

Use it to inspect the runtime, test plugins, and compose custom presets. If your goal is simply to fix an application bug, Creator mode is probably unnecessary.

When should you use DeepSeek Harness?

DeepSeek Harness is a strong candidate in the following situations.

1. You are building an agent platform

If your product needs interchangeable tools, model providers, storage systems, sandboxes, or agent loops, the plugin architecture gives you an existing composition model to study or extend.

2. You need inspectable agent runs

The session event stream and trajectory view make it easier to reconstruct what an agent saw and did. This is valuable for debugging, evaluations, and failure analysis.

3. You want to compare models inside the same environment

Model comparisons are difficult when each model uses a different set of prompts, tools, and execution rules. A configurable harness helps keep more of the environment consistent.

4. You are experimenting with custom tools or policies

Because tools and execution policies are extension points, the project is relevant when testing a custom capability, approval flow, sandbox backend, or internal integration.

5. You want an open-source base you can inspect

DeepSeek Harness is released under the MIT license. You can examine the implementation, modify it, and build on it within the terms of that license.

Remember that MIT-licensed does not mean zero operating cost. A configured model provider may charge for API usage, and remote infrastructure can introduce additional costs.

When should you not use it?

A new open-source agent system can be exciting, but it is not automatically the right choice.

1. You only need a simple model call

If your application sends a prompt and receives an answer, a model SDK may be enough. Adding a full harness introduces plugins, sessions, configuration, storage, and operational complexity you may not need.

2. You need a stable production API today

The project is explicitly marked as a developer preview and warns that compatibility-breaking changes will occur.

That makes it suitable for learning, prototyping, and experimentation. Production adoption requires version pinning, migration planning, testing, and a willingness to follow upstream changes.

3. You cannot isolate the working environment

An agent that can edit files and execute commands should not receive unrestricted access to a sensitive machine.

If you cannot provide a narrow workspace, suitable approval policies, secret isolation, and preferably a disposable environment, do not use it for autonomous changes.

4. Your workflow requires deterministic results

An agent loop combines model decisions with changing context and tool output. Even with the same request, the exact path may vary.

Use conventional scripts, tests, and workflow engines when deterministic execution is the primary requirement.

5. Your team does not need harness customization

If a mature coding assistant already meets your needs, adopting an extensible agent runtime may create maintenance work without delivering meaningful value.

6. You plan to trust the output without review

A traceable agent can still make incorrect changes. Logs help explain a decision; they do not make that decision correct.

Treat generated code like a contribution from an unfamiliar developer: review the diff, run tests, inspect security-sensitive changes, and verify the behavior.

How to try DeepSeek Harness

The fastest path is through its local Web UI.

Step 1: Check Node.js

The repository currently declares support for Node.js ^22.19.0 or >=24.0.0.

node --version

Because the project is changing quickly, verify the current requirement in the repository before installing.

Step 2: Start the Web UI

npx @deepseek-ai/dsh web

By default, this starts a local server at:

http://127.0.0.1:3080

Run the command from the project directory you want to work with. The process uses its starting directory as the default filesystem location, although you still need to select a workspace in the UI.

Step 3: Configure a model

Open:

Settings → Models

You can enter a DeepSeek API key, add another supported provider, or configure a custom provider.

Credentials are stored separately from normal settings, and the UI receives a redacted credential descriptor after saving rather than the literal key.

You still need to protect the machine and the harness home directory. Never commit credential files to a repository.

Step 4: Choose a workspace

Select Choose workspace, add the relevant project directory, and select it.

Use the smallest practical directory. Do not select an entire home folder or a directory containing unrelated secrets and projects.

Step 5: Begin with a read-only task

A good first request is:

Summarize this repository. Identify its main packages, test commands,
and likely entry points. Do not modify any files.

This lets you evaluate how the agent explores the project before allowing it to make changes.

A reasonable next task is:

Find one small, well-contained issue in this repository.
Explain the proposed fix and wait for approval before editing files.

Only after you understand the permission flow should you try a full implementation task.

Step 6: Inspect the trajectory

Do not judge the harness only by the final response.

Inspect:

What context was sent to the model
Which tools were called
Which files were accessed
Whether commands required approval
How tool output affected later decisions
Whether the agent repeated unnecessary work

This is where a traceable harness becomes more useful than a simple chat interface.

A safer evaluation workflow

For early experiments, use a disposable branch, worktree, container, or test repository.

A practical workflow looks like this:

Create an isolated copy of a small project.
Remove production credentials and customer data.
Start DeepSeek Harness from that directory.
Select only that directory as the workspace.
Use a read-only repository-summary task first.
Ask for a plan before permitting edits.
Review every requested command.
Inspect the resulting diff manually.
Run the project’s tests yourself.
Review the trajectory for surprising behavior.

Do not let a successful demo convince you to skip these controls on the next run.

Building from source

If your goal is to inspect or modify the harness itself, clone the repository and build it with its configured package manager:

git clone https://github.com/deepseek-ai/deepseek-harness.git
cd deepseek-harness
pnpm install
pnpm run build
pnpm dsh web

Use this route when you want to:

Read the architecture alongside the implementation
Develop or modify plugins
Test changes to the runtime
Contribute upstream
Pin your work to a specific commit

For a first evaluation, the npx command is simpler.

What to evaluate before adopting it

A successful installation only proves that the harness starts. Before using it for real work, evaluate the following.

Model quality

Does your chosen model use tools reliably? Can it recover from failed commands? Does it stop when it lacks information?

Permission behavior

Which actions require approval? Are writes and command execution constrained appropriately?

Workspace isolation

Can the agent access files outside the intended project? Are secrets, SSH keys, cloud credentials, and production configuration isolated?

Trace quality

Can you reconstruct why a change happened? Does the log contain enough information to debug failures without exposing more sensitive data than necessary?

Plugin trust

A plugin can add substantial capabilities. Review its source, dependencies, permissions, maintenance status, and network behavior before installing it.

Upgrade cost

Since the project is in preview, test upgrades against pinned configurations and plugins. Do not assume a newer release will preserve every API or behavior.

Cost

The harness is open source, but model requests, hosted sandboxes, storage, and other providers may not be free. Measure token usage and infrastructure costs with realistic tasks.

Final perspective

DeepSeek Harness is best understood as infrastructure for constructing and studying agents.

Its value does not come from making a model magically correct. It comes from giving developers a composable way to connect models with tools, sessions, workspaces, policies, storage, orchestration, and observability.

Use it when you need that control or want to experiment with agent architecture.

Avoid it when a simple API call is enough, when stability is more important than extensibility, or when you cannot safely isolate what the agent can access.

Most importantly, keep the model and the harness conceptually separate:

The model decides.
The harness enables, constrains, executes, and records.
The developer remains responsible for the system.

Official resources

Grokbot Honest Review: Is xAI and Cursor's Computer-Use Agent Worth $200 a Month?

Aditi Gupta — Mon, 17 Aug 2026 18:03:19 +0000

Grokbot is xAI and Cursor's attempt at an AI agent that doesn't just chat, it uses a computer. Each bot you create gets its own machine in the cloud, with Chrome, a file manager, and its own operating system, and it keeps working after you close your laptop. I spent a week building three bots to see whether that idea holds up in practice.

Prefer the quick version? I covered the three bots I built, the shared-login trap, and my full verdict in this video:

What Grokbot actually is
You download it from x.ai/bot as a desktop app, not a website. You make a bot with a name and a short description of its job, and it spins up its own virtual computer to get started. Because that computer lives in the cloud, the bot runs on its own, 24/7, without touching your machine.

Why I tested it
I wanted to know if computer-use is real yet or still a demo. So I built three bots with very different jobs: one to find startups that raised seed funding recently and pull the founder names and amounts, one to compare flight prices from Delhi to Tokyo, and one to check the price of a phone across five stores. Same setup each time.

Where it provides value
The genuinely new part is that Grokbot skips the API problem. A huge chunk of the internet has no API an AI can plug into, and even when there is one it rarely lets you do everything the website can. Grokbot gets around that by using the site directly, logging in, clicking through pages, and downloading files the way a person would. It ran with my laptop shut, and I could take over the virtual computer from the mobile app. The teach-a-task feature is the standout: you record yourself doing something fiddly once, it watches the recording frame by frame, and turns it into a reusable private skill.

The main catch
Two things stood out. First, reliability. It got stuck on simple clicks, brought back half the data, and sometimes just froze on easy tasks. Second, and more important, the virtual computer and its logins seem shared across all your bots. I signed into Google on one bot and the others could suddenly reach every site tied to that login. Convenient, but you should be deliberate about which accounts you hand it.

The settings and habits to change first
Always require approval before the bot sends an email, publishes anything, books a flight, buys a product, or deletes something. For anything complicated, record a demo with teach-a-task instead of hoping it figures the task out alone. And keep sensitive logins on bots you trust, given the shared-computer behavior.

How it compares to Claude Code and Codex
Claude Code and Codex can drive a browser too and do more flexible work, but they usually run on your machine, you kick them off yourself, and they stop when your computer is off. Grokbot's lane is different: watching web pages, checking prices on a schedule, moving data between apps, and grinding through repetitive screen work in software you already use. It is not a replacement for a coding agent.

Who should and shouldn't try it
The cheapest plan is $200 a month, and that does include Cursor Ultra. If you already burn through a lot of AI tokens and you're fine switching your subscription to Cursor Ultra, the price makes more sense. If you just want the bot, it's a lot to pay for something this early. Try it on the free week if you have one repetitive task nothing else can automate.

My verdict
Grokbot is a real step in the right direction. Instead of waiting for every website to build an API, it lets an AI use a computer the way we do, and if that gets reliable enough it could automate almost anything you can do on a screen. Right now the automation isn't consistent, the trial is tiny, and the price is steep, so I wouldn't tell most people to pay for it yet.

Have you tried Grokbot or another computer-use agent yet? Tell me in the comments. If you like quick, honest AI-tool reviews, follow my YouTube channel for the next one.

Links and sources

Grokbot (xAI + Cursor): https://x.ai/bot
Full video review: https://youtu.be/rUfvWFPYGMo
More AI tool breakdowns + weekly news: https://goventure.live https://youtu.be/rUfvWFPYGMo
My testing notes: the three bots, the shared-login behavior, and the reliability misses are all shown in the video above.

Tags: #ai #agents #cursor #aitools

I Tried Kimi K3 for Free in VS Code - Can It Replace Claude or GPT?

Aditi Gupta — Wed, 12 Aug 2026 17:35:03 +0000

My coding assistant subscription has one job: save me more time than it costs.

So when Moonshot AI released a 2.8-trillion-parameter model with a one-million-token context window—and I found a way to connect it to VS Code for free—I wanted to answer one practical question:

Is Kimi K3 good enough to replace a paid coding assistant?

After looking past the headline numbers, comparing the benchmark results, and testing it on a complete browser game, my short answer is:

Kimi K3 is good enough to become a serious coding workhorse for many developers. But it is not a universal replacement for every paid model, and the free access comes with an important catch.

Prefer the video walkthrough? I cover the test and both setup methods here:

First, what exactly is Kimi K3?

Kimi K3 is Moonshot AI's newest open-weight, native multimodal model. According to the official release, it has:

2.8 trillion total parameters
104 billion active parameters per token
A Mixture-of-Experts architecture that activates 16 of 896 experts
A 1,048,576-token context window
Native image and video understanding
Kimi Delta Attention and Attention Residuals
Built-in reasoning for coding, research, and agentic work

That “104 billion active” detail matters. Kimi K3 does not use all 2.8 trillion parameters for every token. Its sparse architecture routes each token through a small subset of experts, making inference more efficient than the headline size suggests.

Open-weight is not the same as free to run

You will see Kimi K3 called “open source,” but open-weight is the more precise description. Moonshot has released the model weights under its own Kimi K3 license, so developers can inspect, deploy, and build on the model within those terms.

However, downloading the weights does not make inference free. A 2.8T model is far beyond the practical local setup of most developers. Unless you have access to serious GPU infrastructure, you will use a hosted provider—and that provider pays the compute bill.

That distinction becomes important when we get to the free setup.

The benchmark results are genuinely competitive

Kimi K3's coding scores are the main reason I took it seriously.

Here are selected results from Moonshot's published evaluation:

Benchmark	Kimi K3	Claude Fable 5	GPT-5.6 Sol	Claude Opus 4.8
DeepSWE	67.5	70.0	73.0	59.0
ProgramBench	77.8	76.8	77.6	71.9
Terminal-Bench 2.1	88.3	88.0	88.8	84.6
FrontierSWE	81.2	86.6	71.3	66.7
SWE-Marathon	42.0	35.0	39.0	40.0
BrowseComp	91.2	88.0	90.4	84.3

The pattern is more interesting than a simple “Kimi wins” headline:

It nearly matches GPT-5.6 Sol on Terminal-Bench 2.1.
It beats GPT-5.6 Sol and Opus 4.8 on FrontierSWE, although Fable 5 scores higher.
It leads this comparison on ProgramBench, SWE-Marathon, and BrowseComp.
It falls behind both GPT-5.6 Sol and Fable 5 on DeepSWE.

That is not a clean sweep. It is something more useful: evidence that an open-weight model now belongs in the same coding conversation as the leading proprietary models.

One benchmark warning most reviews skip

These numbers come from Moonshot's evaluation, and some models were tested with different agent harnesses. Kimi used Kimi Code on several tests, GPT used Codex on some, and Claude used Claude Code or other harnesses on others. All Kimi results also used maximum reasoning effort.

In other words, treat the table as a strong signal—not a perfectly controlled, apples-to-apples contest. The technical report and repository document the methodology if you want to inspect it.

My practical test: build a browser game from one prompt

Benchmarks tell me whether a model deserves a test. They do not tell me whether I want it editing my project.

So I gave Kimi K3 a long, single prompt to generate a complete browser game. This forced it to plan the interface, write the game logic, connect the files, and keep the result coherent over a longer generation.

It produced a complete, playable first version in one pass.

What impressed me was not one clever function. It was the model's ability to sustain a multi-part implementation without losing the original goal.

But a one-shot demo has limits. A generated game can look impressive while still hiding brittle state management, accessibility problems, or code that becomes painful on the second revision. The better test is whether the model can explain its decisions, respond to bug reports, make targeted changes, and run verification without rewriting unrelated code.

That is how I would evaluate Kimi K3 on a real repository.

How to try Kimi K3 for free

I found two practical routes: one in the browser and one inside VS Code.

Availability note: The free Kimi K3 routes I tested are promotional and may be rate-limited, renamed, or removed. Check the provider's model page before following the steps.

Method 1: Use Kimi K3 in your browser

When I tested it, GenSpark included Kimi K3 in its model menu after a free signup.

Create a free GenSpark account.
Open the AI chat interface.
Select Kimi K3 from the model menu.
Start with a real task—not “write hello world.”

Try asking it to:

Explain an unfamiliar module and identify risky dependencies.
Build a small feature with clear acceptance criteria.
Review a pull request and separate bugs from style preferences.
Turn a screenshot into a working frontend component.

In my usage, Kimi consumed fewer GenSpark credits than the premium Claude and GPT options, so the free allowance lasted longer. Credit rules can change, so verify the current rate in the interface.

Method 2: Connect Kimi K3 to VS Code

For coding, this is the more useful setup because the model can work with your files through an agent extension.

You can use Kilo Code or Cline. Both support OpenAI-compatible providers, which means you normally need only three pieces of information:

A provider Base URL
An API key
The exact model ID

Step 1: Install the extension

Install Kilo Code from the VS Code Marketplace. If its provider setup gives you trouble, install Cline instead; Cline documents the same OpenAI-compatible fields.

Step 2: Create a provider API key

Use a current provider that lists a promotional Kimi K3 route. At the time of writing, the following OpenAI-compatible configuration works with ZenMux:

Provider: OpenAI Compatible
Base URL: https://zenmux.ai/api/v1
Model ID: moonshotai/kimi-k3-free
API key: <your provider key>
Reasoning: enabled

The free model is explicitly listed as a limited-time route, so confirm that the model ID still appears in the provider's catalog before setting up the extension.

Step 3: Configure Kilo Code or Cline

Open the extension's model settings and:

Choose OpenAI Compatible as the provider.
Paste the provider's Base URL.
Paste your API key.
Enter the exact Kimi K3 model ID.
Enable reasoning or thinking mode if the client exposes that setting.
Save and send a small test prompt.

If you receive “model not found,” do not keep changing random settings. Check the provider's live model list first. Promotional model IDs change frequently.

Step 4: Test it safely

Start with a disposable branch or a small personal project. Ask the agent to explain its plan before editing, review the diff after each task, and keep automatic command approval off until you trust the workflow.

Also remember that a hosted endpoint can receive the prompts and code you send through it. Do not upload credentials, private customer data, or proprietary source code until you have reviewed the provider's privacy and retention policies.

The real catch behind “free”

The free access is real, but “free” can mean three different things:

Free weights: You can download the model under its license, but you supply the hardware.
Free web access: A product absorbs the inference cost and applies its own limits.
Free promotional API access: A provider offers a zero-cost route temporarily, usually with rate limits and no service guarantee.

The VS Code method falls into the third category. It is excellent for testing and personal projects. I would not build a production workflow around the assumption that the endpoint will stay unlimited or free forever.

There is another practical catch: the official Kimi API expects clients to preserve reasoning content across multi-turn tool calls. If your coding extension drops that state, you may see weaker follow-up behavior even when the first response looks good.

Should you replace your paid coding assistant?

Here is my honest recommendation.

Kimi K3 could become your default if:

Most of your work is coding, browser automation, or agentic web tasks.
You regularly need to reason across a large repository.
You are a student, indie developer, or early-stage builder minimizing subscriptions.
You are comfortable switching providers if a free route disappears.
You review diffs and verify generated code instead of accepting it blindly.

Keep a paid model as your primary if:

Reliability and support matter more than saving the subscription fee.
You work with sensitive code and need clear enterprise data controls.
Your tasks are ambiguous, high-stakes, or difficult to verify.
You depend on stable throughput, predictable latency, or a service-level agreement.
You want one assistant for coding, writing, analysis, and specialized professional work.

For many developers, the best answer is not a dramatic switch. It is a two-model workflow: use Kimi K3 as the high-context coding workhorse and keep a paid model as the fallback for difficult or high-stakes tasks.

Final verdict

Kimi K3 does not make every paid coding assistant obsolete.

What it does is more significant: it narrows the gap enough that “open model” no longer automatically means “second-tier coding model.” Its benchmark performance is competitive, its million-token context is genuinely useful, and the hosted free routes make it easy to test inside a real coding workflow.

My verdict:

Do not cancel your paid assistant because of one leaderboard. But if you write code, Kimi K3 deserves a permanent slot in your model picker while the free access lasts.

Have you tried Kimi K3 on a real repository? Share the task, what it did well, and where it failed. That comparison is more useful than another benchmark screenshot.

If this walkthrough helped, follow my YouTube channel, GoVenture, for more practical and honest AI-tool tests.

Links and sources

Kimi K3 official repository and benchmark table
Kimi K3 technical report
Kimi API documentation
Cline: configuring an OpenAI-compatible provider
ZenMux OpenAI-compatible API quickstart
GenSpark (browser method): http://genspark.ai/
Free API provider Token Router (VS Code method) : https://www.tokenrouter.com/

Meta Muse Spark 1.2 Honest Review: Loses the Benchmarks, Wins as an Agent

Aditi Gupta — Sat, 08 Aug 2026 08:49:34 +0000

Meta just shipped two things at once: Muse Spark 1.2, the model, and Muse Code, an agentic harness that wraps around it. The interesting part is that they pull in opposite directions. On raw coding the model is a clear underdog. As an agent it is suddenly near the top. Here is what actually held up when I tested it.

Prefer the quick version? I covered the benchmark split, the broken games, the harness features, and a head to head against Qwen 3.8 in this video:

What Muse Spark and Muse Code are
Muse Spark 1.2 is Meta's new coding model. Muse Code is the harness you actually run it in. The harness keeps a local log of every tool call and every edit, so if a run crashes it picks up exactly where it left off instead of starting over. It ships with built-in skills too: /plan turns a task into an approval-gated plan, /grill stress-tests that plan before you run it, and /goal keeps the agent pushing toward the objective.

Where it loses
On the coding benchmarks it never comes first. Second on Terminal Bench behind Opus 5, third on DeepSWE, and behind Opus 5 on Meta's own internal benchmark. The games I had it build showed the same weakness. They were laggy and half-finished, one dragon bounced on the spot, one character's arms were missing, another spun around when you tried to walk backwards. On the same prompts, Qwen 3.8 and Fable 5 built noticeably cleaner, more playable versions.

Where it wins
Point the same model at tools inside Muse Code and the picture flips. On agent and tool-use benchmarks it jumps to first. The crash-resume log is the standout feature for anyone running long agent jobs. To test it properly I gave both Muse Spark and Qwen 3.8 the same task: read a guide and turn it into a reusable skill. Muse Spark replied faster and produced the sharper result. It analyzed the guide, built a proper table, and got specific instead of generic.

Who should run it
If you want the best raw coding model, Opus 5 still wins and Qwen 3.8 still builds cleaner. If you care about agent workflows, tool use, and not losing progress when a long run dies, Muse Code is worth a serious look even though the model underneath loses the benchmark race.

My verdict
The model is not the story. The harness is. Muse Spark 1.2 is a reminder that in 2026 the wrapper around a model can matter as much as the weights.

Have you tried Muse Spark or Muse Code yet? Tell me in the comments. If you like quick, honest AI-tool reviews, follow my YouTube channel for the next one.

Tags: #ai #metaai #musespark #aiagents #coding``

Qwen3.8-Max Beat Claude

Aditi Gupta — Tue, 04 Aug 2026 11:34:00 +0000

Alibaba has released Qwen3.8-Max, its largest and most capable AI model to date.

The headline specifications are absurd: 2.4 trillion total parameters, 95 billion activated per token, multimodal input, and a one-million-token context window.

But the specification sheet is not the most interesting part.

Alibaba says Qwen3.8-Max operated autonomously for roughly 16 days, starting with an empty repository and building a working software project through issues, code changes, testing, pull requests, and self-correction.

That sounds impressive. It also sounds suspiciously like the sort of claim that deserves more inspection than a celebratory repost.

So I examined Alibaba's announcement, the public repository, and its benchmark results to answer a more useful question:

Does Qwen3.8-Max actually beat Claude, or did the benchmark department simply have an excellent week?

Prefer the two-minute version?

TL;DR

Qwen3.8-Max is a mixture-of-experts model with 2.4 trillion total parameters and 95 billion active per token.
Alibaba reports a one-million-token context window and support for text, images, and agentic workflows.
It beats the Claude models in Alibaba's table on Terminal Bench 2.1, PaperBench, and OSWorld-Verified.
It trails Claude Fable 5 on demanding repository-level engineering benchmarks such as SWE-bench Pro and FrontierSWE.
Alibaba says the model weights will be released next week. At the time of writing, the API is available, but the weights are not yet downloadable.
The 16-day coding trace is public, but the results still come from Alibaba's own evaluation setup and need independent replication.

What is Qwen3.8-Max?

Qwen3.8-Max is Alibaba's new flagship mixture-of-experts model.

It contains 2.4 trillion parameters in total, with approximately 95 billion active during each forward pass. This architecture allows Alibaba to scale the model's capacity without paying the full inference cost of a dense 2.4-trillion-parameter model on every token.

The model also supports a one-million-token context window, making it suitable for large repositories, long documents, persistent agent sessions, and other tasks where context compression usually arrives carrying a shovel.

Alibaba calls it the first Qwen model at Max scale that will receive an open-weight release. However, there is an important distinction:

The API is available now. The model weights are scheduled for release next week.

So calling it an "open-weight model" is reasonable when discussing Alibaba's release plan, but saying the weights are already available would be inaccurate.

You can find the specifications and release details in the official Qwen3.8-Max announcement.

The 16-day autonomous coding run

Alibaba asked Qwen3.8-Max to create a project called oh-my-cli from an empty repository.

Instead of responding to a single prompt and stopping, the model worked through a continuous engineering loop:

Convert feedback and requirements into GitHub issues.
Claim and execute individual tasks.
Write and modify code.
Run builds, unit tests, end-to-end tests, and lifecycle checks.
Route failures back into the issue workflow.
Fix the problems and verify the result.
Merge completed pull requests.

According to Alibaba, the repository had accumulated 265 commits, 127 pull requests, and 151 issues after approximately 16 days of autonomous operation.

The complete project history is available in the public oh-my-cli GitHub repository.

That transparency matters. Most autonomous-agent demonstrations give us a polished video and ask us to believe that nothing caught fire outside the frame. Here, developers can inspect the issues, commits, pull requests, tests, and failures.

Still, this does not prove that the model can autonomously build any production system for 16 days. It proves that Qwen3.8-Max performed this particular task inside a structured environment with automated testing and feedback loops.

That is still meaningful, just narrower than the marketing headline.

Where Qwen3.8-Max beats Claude

Alibaba published a large benchmark table comparing Qwen3.8-Max with Claude Opus 4.8, Claude Fable 5, GPT-5.6 Sol, and Qwen3.7-Max.

Here are the most relevant results:

Benchmark	Qwen3.8-Max	Claude Opus 4.8	Claude Fable 5	Result
Terminal Bench 2.1	86.6	84.6	84.6	Qwen leads
PaperBench	93.0	80.3	88.8	Qwen leads
OSWorld-Verified	86.1	83.4	85.0	Qwen leads
SWE-bench Pro	67.7	69.2	80.0	Claude leads
FrontierSWE	73.5	70.0	88.8	Mixed; Fable 5 leads

These results suggest three areas where Qwen3.8-Max looks particularly strong.

1. Terminal-based agent work

Its 86.6 score on Terminal Bench 2.1 puts it ahead of both Claude models in Alibaba's comparison.

That makes Qwen especially interesting for command-line agents, environment setup, testing, deployment workflows, and tasks that require repeated tool use rather than a single code-generation response.

2. Research reproduction

Qwen3.8-Max scored 93.0 on PaperBench, ahead of Claude Fable 5's 88.8 and Opus 4.8's 80.3.

Alibaba also demonstrated a five-day research task in which the model reproduced a paper's experimental pipeline, ran 33 rounds of GPU training, and then searched for improvements to the original method.

This is potentially more useful than another model becoming marginally better at generating React components nobody requested.

3. Computer and visual interaction

On OSWorld-Verified, which evaluates an agent's ability to operate computer environments, Qwen3.8-Max scored 86.1.

The model uses visual output as part of its feedback loop. It can inspect an interface, identify errors, revise its plan, and try again. That matters for browser agents, desktop automation, document workflows, UI testing, and multimodal development.

Where Claude still wins

The "Qwen kills Claude" headline falls apart once we examine harder repository-level engineering tasks.

On SWE-bench Pro, Qwen3.8-Max scored 67.7. Claude Fable 5 scored 80.0.

On FrontierSWE, Qwen scored 73.5 while Fable 5 reached 88.8.

That is not a rounding error. It suggests Claude remains stronger when a task requires deep repository understanding, architectural judgment, and reliable changes across a complicated codebase.

The more honest conclusion is:

Qwen looks excellent at long-running, tool-heavy work.
It performs strongly on terminal, research, multimodal, and computer-use tasks.
Claude Fable 5 remains ahead on some of the hardest software-engineering benchmarks.
Neither model "wins" every category, because reality rudely refuses to fit inside one thumbnail.

There is another caveat: these scores come from Alibaba's evaluation table. Different benchmarks used different harnesses, time limits, context settings, and judging methods. The numbers are useful, but independent testing will matter more than launch-day charts.

How to use Qwen3.8-Max with Claude Code

QwenCloud provides an Anthropic-compatible API, allowing Claude Code to use Qwen3.8-Max without replacing the Claude Code interface.

First, install Claude Code:

npm install -g @anthropic-ai/claude-code

Then configure it to use Qwen:

export ANTHROPIC_MODEL="qwen3.8-max"
export ANTHROPIC_SMALL_FAST_MODEL="qwen3.8-max"
export ANTHROPIC_BASE_URL="https://dashscope-intl.aliyuncs.com/apps/anthropic"
export ANTHROPIC_AUTH_TOKEN="YOUR_QWEN_API_KEY"

claude

You will need a QwenCloud API key. The international endpoint may differ depending on your account or deployment region, so check the current QwenCloud documentation before configuring it.

And please do not paste your real API key into a public DEV article. Becoming an involuntary cloud-compute philanthropist is rarely part of the content strategy.

Can you run it locally?

Not casually.

Although only 95 billion parameters are active during each pass, the full model contains 2.4 trillion parameters. Open weights do not magically convert that into something your laptop can run between Chrome tabs.

Once the weights are released, practical deployment will likely require substantial multi-GPU infrastructure, aggressive quantization, or a hosted inference provider.

For most individual developers, QwenCloud will be the realistic way to use the full model.

The verdict

Qwen3.8-Max does not kill Claude.

It does something more consequential: it brings frontier-scale agent capabilities closer to the open-weight ecosystem.

Its strongest argument is not a single benchmark score. It is the combination of long-horizon execution, terminal performance, multimodal feedback, research reproduction, and a public 16-day development trace.

Based on the evidence available today:

I would consider Qwen3.8-Max for long-running agents, terminal workflows, research automation, visual tasks, and jobs that benefit from repeated feedback.
I would still prefer Claude Fable 5 for the hardest repository-level engineering work, especially when first-pass reliability matters.
I would wait for independent evaluations before treating Alibaba's benchmark table as the final verdict.

Qwen3.8-Max is not the model that makes Claude irrelevant.

It is the model that makes the frontier race significantly less comfortable, and that is far more interesting.

Have you tested Qwen3.8-Max in QwenCloud or Claude Code? Share the task, harness, and result in the comments. "It felt smarter" is emotionally valid, but logs are sexier.

Sources

Claude Opus 5 Is Better at Coding and Harder to Trust

Aditi Gupta — Wed, 29 Jul 2026 13:18:53 +0000

Claude Opus 5 completed one of my coding tasks considerably faster than Opus 4.8.

There was just one problem: it confidently reported that the issue was fixed when it wasn’t.

That experience captures the trade-off with Anthropic’s latest Opus model. It is faster and more capable on difficult, multi-step work, but polished output can make its mistakes harder to notice.

After testing it on coding and agent tasks, I changed three parts of my workflow.

1. Start with medium reasoning effort

More reasoning is not automatically better. For routine coding tasks, begin with medium effort and increase it only when the problem genuinely requires deeper investigation.

Higher effort can consume more tokens, expand the scope of the task, and produce a solution far more elaborate than the one you requested.

2. Verify outcomes, not explanations

A convincing explanation is not evidence that the task was completed correctly.

Ask for or independently run the relevant tests. Review the files that changed. Confirm the original bug no longer exists.

The dangerous failure mode is not nonsense. It is an incorrect result presented like finished work.

3. Control the scope

Define what the model may change before it begins:

Which files can be modified?
What behaviour must remain unchanged?
Which tests must pass?
Can it create subagents or expand the task?

Opus 5 is strongest when the job requires investigation across multiple steps. For a small, clearly defined change, that same initiative can become unnecessary complexity.

The quick version

I condensed my findings, the confidently wrong problem, and the three changes I recommend into this 90-second video:

My broader verdict is simple: Opus 5 is a meaningful upgrade for difficult coding and agent work, but only when verification is part of the workflow.

I published the complete review, including pricing, benchmark comparisons, use cases, and switching advice, on Hashnode:

hashnode.com

Have you tested Opus 5? Did it improve your workflow, or merely become more articulate while being wrong?