DEV Community: Ishank Choudhary

PlxNative: A Fast Native Plex Client for LG webOS TVs, Built in Rust

Ishank Choudhary — Sat, 17 Oct 2026 04:51:58 +0000

Most smart-TV apps are web pages. On LG's webOS that usually means JavaScript running inside a browser engine, which then hands its output to a compositor before anything reaches the screen. PlxNative is a Plex client for LG webOS TVs that takes that layer out. Its developer, Gleb Linnik, writes in the README: "No Chromium, no JavaScript, no web view. It just draws."

This is an explainer, built from the project's README, its docs, its source and its release notes. I don't own an LG TV, so nothing here was tested on a real television. What I did run is the project's own Linux desktop simulator against its mock Plex server, and that section near the end shows exactly what that does and doesn't tell you.

What PlxNative is

An unofficial Plex client for LG webOS TVs. It is not made by or affiliated with Plex or LG; the project says so on its site and in TRADEMARKS.md.
Open source under GPL-3.0-or-later, © 2026 Gleb Linnik. The PlxNative name and brand artwork are excluded from the licence.
One person's project. The README calls it "one person's spare time".
Current release: v0.8.0, published on GitHub on 7 October 2026. Nightly builds already report 0.9.0-nightly.
Young and active. The repository was created on 3 July 2026, and when I checked on 16 October it had 969 stars and 95 forks, with commits landing the same morning.
Free. There's no paid tier. You need a Plex account and a Plex Media Server, and the FAQ confirms a free Plex server works.

Why it exists

The origin story is short. The developer's 2019 LG TV still had a great picture, but browsing a Plex library on it "felt slower than it needed to." Rather than tune a web app, he wrote a native one. In his words: "I use it every day to watch things off my server in the next room."

The site makes the argument with a pair of stacks. A typical TV app goes:

The app (a web page) -> JavaScript -> Browser engine -> Compositor -> OpenGL -> GPU -> Screen

PlxNative goes:

The app (native code) -> OpenGL -> GPU -> Screen

The claimed result is 60 fps on that 2019 set. I couldn't measure that, and the project itself says the simulator can't either. More on that below.

The architecture

Mostly Rust, with a thin C shell

The repository is overwhelmingly Rust. A rough line count on the current main gives about 576,000 lines of .rs files against about 7,300 lines of C and C++. That count includes tests and generated data, so treat it as a ratio, not a measure of features.

The structure is spelled out in rust-modules/Cargo.toml. It's a Cargo workspace whose root crate is linked into a small C app as a static library:

# Static library linked into the C app (hybrid migration): each ported module
# exposes the same C ABI its src/*.h declares, so the C code calls it unchanged.
[lib]
name = "plxnative_modules"

The workspace is split into layer crates: base, machine, net, platform, gfx, ui, plex, telemetry, data, session, media, appkit, screens and storage. docs/module-layers.md describes how they depend on one another.

A GPU-drawn interface

The UI goes through SDL2 and OpenGL ES (both are listed in the repo's topics), so there's no DOM and no layout engine. The design language is translucent "glass" panels over your own posters and backdrops, with text sized for reading across a room. The docs folder holds design notes on liquid glass, backdrop blur and a hardware budget for the glass effect, which gives a sense of how much of the work goes into making that look cheap enough for TV silicon.

Video on the TV's own decoder

Playback doesn't decode video in software. The README says the app "hands video to the same silicon the built-in apps use". In the code, that goes through a video-sink trait in the platform crate. On the TV it's implemented by a StarfishSink, which talks to LG's Starfish media APIs, and on the desktop by a HostSink.

What it can direct-play, per the README:

H.264 and HEVC, including 4K and 10-bit
Dolby Vision profiles 5 and 8
E-AC-3 with Atmos
AV1 or VP9 (10-bit VP9 only on webOS 4 sets), on TVs whose codec table lists that decoder

Anything else is transcoded by your Plex server. Before you press Play, the detail page shows whether a title will play directly or be converted on the server. The app also ships its own LGPL build of FFmpeg.

Features

From the README, the site and the 0.8.0 release notes:

Sign in on the TV by scanning a QR code with your phone, then pick a Plex Home profile. PIN-protected profiles are supported.
Browse your own and shared libraries, with Continue Watching, Recently Added, sort, filters (including unwatched-only) and an A–Z rail.
Search across every server you can reach, with results grouped by kind.
A full player with resume, seek and scrub, chapters, audio and subtitle tracks (image subtitles included), Skip Intro, Skip Credits, Up Next with auto-advance, and progress synced back to Plex.
New in 0.8.0: subtitle search from inside the player, ASS/SSA subtitles that keep their styling, browsing Plex collections, Boost dialog and Normalize loudness on servers with Plex Pass, a Settings section for playback, and Spanish and Belarusian translations.
Works without internet after the first sign-in. The app talks to your server directly, so the library keeps working on your home network when the internet is down.
Privacy defaults. Crash reports and usage analytics are two separate opt-in questions, both off until you say yes. The README says no title, search term, subtitle line, server name or address can appear in any report. The app talks to your Plex server, to plex.tv to sign in, and to discover.provider.plex.tv for cast biographies.

0.8.0 also changes one security behaviour. Some home servers only answer over plain HTTP. Earlier versions refused to send your Plex sign-in that way. 0.8.0 will connect, but only after asking "Connect without encryption?", and only for a server on the same network at a private address after every secure route, Plex's relay included, has failed.

How to install it

PlxNative isn't in the LG Content Store yet; the site says it's "in the process of review" with no confirmed date. There are two routes, and neither needs root.

Route 1: LG Developer Mode

The installation guide needs an LG TV on webOS 4.0 or newer, a Plex account with a server, and a Mac, Windows or Linux computer on the same network. In short:

Create or sign in to an account on LG's developer site.
Install the Developer Mode app on the TV, sign in, turn on Dev Mode Status (the TV restarts) and then Key Server.
Note the TV's IP address and passphrase, and connect to it from webOS Dev Manager on your computer.
Install the .ipk directly, or install Homebrew Channel first and get PlxNative from its catalogue.

For a manual download, verify the package first:

sha256sum -c ipk.sha256

The catch is renewal. Developer Mode sessions expire, and if one lapses LG removes the apps installed through it. The guide's advice is to open the Developer Mode app and press EXTEND before the remaining time runs out, with a calendar reminder. An expired session can't be extended.

Route 2: Homebrew Channel

If your TV already runs Homebrew Channel, open it, find PlxNative (a 19.6 MB download, listed as version 0.8.0) and select Install. Updates arrive there too, and the Channel checks the download's checksum before installing.

Nightlies

Adding https://plxnative.com/nightly/repo.json to Homebrew Channel installs PlxNative Nightly as a separate app with its own tile and sign-in. The README is blunt about these builds: "It is not tested on a television."

Limits and current status

The README has a section titled "The honest scope", and it's worth taking at its word:

Movies and TV shows only. No music, no photos, no Live TV or DVR.
No typing in server addresses. Servers come from your Plex account.
Relay and secure-only servers are supported but "haven't been watched end to end."
Not all TVs work. It needs webOS 4.0 or newer. A 2016 set on webOS 3.4.3 has been tested and works.

The known issues list is specific:

Some 2019 sets on LG's k5lp or k3lp chip, installed through Developer Mode, won't play video. The sandbox withholds a device the video path needs; Kodi and Moonlight hit the same wall. Without root there's no fix.
2018 sets on webOS 3.9.3: sign-in and browsing work, but the prebuilt v0.8.0 never starts video (#249). A source build at or after commit 60f99c7 fixes it.
webOS 3.x restarts the stream on every seek instead of jumping in place.
Sets on the k6hp chip show a crash in opt-in reports that the developer can't reproduce without the hardware (#174).

Building the simulator on Linux

Since I couldn't put it on a TV, I built the desktop simulator the docs describe. The box runs Debian 13 on x86_64 with 8 vCPUs and no GPU. The only prerequisites were the SDL2, SDL2_ttf, Mesa, CMake and pkg-config dev packages plus a current stable Rust toolchain:

git clone https://github.com/GLinnik21/plx-native && cd plx-native
make sim-linux SIM_TDIR=rust-modules/target-sim

   Compiling plxnative-modules v0.8.0 (/workspace/plxnative/repo/rust-modules)
    Finished `release` profile [optimized] target(s) in 3m 08s
real    3m57.100s

That gave a 16 MB plxnative-sim binary on the first try. The extra 49 seconds went into fetching and building the libass stack (FreeType, FriBidi, HarfBuzz and friends).

The project's screenshot tool then boots the simulator against tests/mock_pms.py. That's a synthetic Plex server that also stands in for plex.tv, so no account, token or real library is involved. I ran it headless under Xvfb with software OpenGL:

python3 tools/demo_library.py derive
LIBGL_ALWAYS_SOFTWARE=1 xvfb-run -a -s '-screen 0 1920x1080x24' \
  python3 tools/screenshots.py --bin rust-modules/target-sim/release/plxnative-sim \
  --out shots --only home-shelves,library-grid,search,detail,signin

Each scene settled in 2.2 to 4.9 seconds. One hiccup: the home scene also renders the website's link-preview card, and that step failed because the box lacked the file utility, so I captured home-shelves instead.

The sign-in screen is the QR flow, here pointed at the mock:

The Movies grid, with the sort and filter chips and the A–Z rail, browsing the project's demo library of openly licensed films:

And a title page, with the "Direct Play" label next to the runtime:

Be clear about what this proves. The simulator runs the same UI and Plex code as the TV build, and it boots, signs in to a fake server and draws every screen I asked for. But docs/building.md says plainly that it "cannot answer frame rate (different GPU), text rasterization, or anything about LG's decoder and video plane. Those need the set." The Linux build also has no video path at all; the Makefile notes that Play "intentionally reaches the host seam's existing 'no video path' result." So the speed and the hardware decoding, the two things PlxNative exists for, are still unverified by me.

Who it's for

If you run Plex at home and own a fairly recent LG TV, PlxNative is a well-documented alternative to the official app. Expect a project that tells you about its rough edges in the README. You'll need to be comfortable with Developer Mode renewals or Homebrew Channel until the LG Content Store listing lands. If you need music, Live TV or a set from the known-issues list, the README itself says the official app will serve you better.

If you're a developer, the repository is interesting on its own terms: a Rust UI stack drawing straight to GPU on TV-grade hardware, with design and performance notes for almost every decision. The simulator builds in a few minutes on plain Linux if you want to look around.

Site: https://plxnative.com/
Source: https://github.com/GLinnik21/plx-native

I Ran Anatomy's /anatomy Skill on a 1.7B Local Model. The Kit Passed Every Check; the Figure Was 738 Bytes.

Ishank Choudhary — Wed, 14 Oct 2026 04:59:13 +0000

Anatomy went from a new repo on 8 October to 6,632 GitHub stars by Tuesday morning, 13 October (I checked the GitHub API at 10:31 AM IST). It's an MIT-licensed Agent Skill for Claude Code that also runs in OpenCode, Codex and, as of this week, Antigravity. You type /anatomy plus an idea, and the agent is supposed to invent a physical machine whose working is that idea, then build it as an interactive isometric SVG figure out of dozens of small shaded parts.

The demos are lovely: a Raptor engine that comes apart into 11 assemblies, a ripple tank with WebGL caustics, a dial gauge you can slide. I wanted to know two things the demos can't tell me. Does the machinery underneath actually hold up on a plain Linux box? And how much of the result is the skill, and how much is the model driving it?

Short version: the kit is real, fast and honest about itself. The skill, though, is written for a strong model, and a small local one doesn't come close. Here's exactly what I ran.

What I tested and what I didn't

The box is Debian with 8 vCPUs, 16 GB of RAM (about 4 GB free during the test) and no GPU. There was no Claude Code, no Codex and no cloud API key on it. So the agent was OpenCode 1.19.0 pointed at a local qwen3:1.7b through Ollama on CPU.

I tried to pull qwen3:4b as a step up. The download failed on this box's network ("redirect target not allowed"), so 1.7B was the only model I could run.

That shapes everything below. I tested:

The install path, with the skills CLI the README recommends
The kit on its own: building the examples, the test suite, the geometry audit, and the headless-Chrome checks
Three real /anatomy prompts in OpenCode, taken from the README and the skill's own evals

I did not test Claude Code or Codex, or any frontier model. I also skipped the React output, the 3D (beta) mode, the WebGL figures driven by an agent, and the new community-gallery publishing flow. The figures in the README were almost certainly made with a much stronger model than mine, so please read the agent results as "what happens at the bottom end," not as a verdict on the skill.

Install: one Node catch and a different folder

The README's quick start is one line:

npx skills add wheresryan22/anatomy -g -a opencode -y

On the box's Node 20.19.2 it crashed immediately:

npm WARN EBADENGINE   required: { node: '>=22.20.0' }
SyntaxError: The requested module 'fs/promises' does not provide an export named 'glob'

Anatomy itself only needs Node 20.10+, but the skills CLI (1.7.2 here) needs 22.20+. The README does mention this in a collapsed section, but it's easy to miss. On Node 22.20.0 the same command finished in 7.6 seconds. It showed a security summary ("Gen Safe, Socket 0 alerts, Snyk Low Risk") and copied 13 MB.

One surprise: it installed to ~/.agents/skills/anatomy, not ~/.config/opencode/skills/anatomy as the README's install table lists for OpenCode. OpenCode found it anyway, since it also reads the .agents folder:

opencode debug skill
# ... "name": "anatomy", "location": "/home/box/.agents/skills/anatomy/SKILL.md"

The kit on its own: fast, deterministic, and it checks itself

Before involving any model, I ran the parts that don't need one. This is where Anatomy impressed me most.

The examples rebuild from Node scripts with zero npm dependencies:

node examples/dial-indicator/build.mjs
node examples/desk-computer/build.mjs
node examples/arcade-cabinet/build.mjs
git status --short     # empty

All three exited 0, and git status was clean afterwards. The rebuilt pages were byte-identical to the committed ones, which is exactly what CONTRIBUTING promises. The Raptor engine, the heaviest example, took 115 seconds to build, in line with the README's "a few minutes."

The test suite ran 225 tests in about 38 seconds:

# tests 225
# pass 220
# fail 5

All five failures were react-*.test.mjs files that require React, which isn't installed. The README says React is only needed for React output, so this is a setup gap rather than a bug. I'd still like those tests to skip cleanly instead of failing.

The geometry audit is the standout. It doesn't just render the figure. It proves that nothing passes through anything and that every overlap is painted in the right order:

$ node examples/arcade-cabinet/build.mjs --audit
audit · 15 tubes · 243 solids · 349 items · 1 depth states · 1290 overlaps · 1290 order rules · 11.0 s
audit passed

The headless-Chrome checks worked too, once I set CHROME_PATH:

$ node scripts/capture.mjs examples/dial-indicator/dial-indicator-light.html --out dial.png
saved dial.png
clip 395,64 650x485 · readout "block 2 · reads 13.8 px" · scrolls sideways: no
console: clean

$ node scripts/lines.mjs examples/dial-indicator/dial-indicator-light.html
line ends: 0 of 367 ends on 23 lines float
console: clean

The capture took about 4 seconds. That readout ("block 2 · reads 13.8 px") comes from the same model that moves the gauge needle, which is the skill's core promise: every number in the figure comes from the figure's own model.

This is what a finished figure looks like, straight from the repo's smallest example and captured by the skill's own script:

Running /anatomy in OpenCode with a local model

The skill is a 31,600-byte SKILL.md, plus nine reference guides totalling about 300 KB that the agent is meant to open as needed. When you run the command, OpenCode injects SKILL.md into the prompt. Ollama's logs put the first request at roughly 7,700 input tokens before the model had written a word. On CPU, just reading that prompt took 45 to 70 seconds, and generation ran at about 4 to 5.5 tokens per second.

I hit two snags before any figure work began.

1. opencode run sat at "init" for 13 minutes. It was waiting on stdin. Adding < /dev/null fixed it:

opencode run --model ollama/qwen3:1.7b --command anatomy \
  "explain how a token bucket rate limiter works, interactive. Standalone HTML, light theme." < /dev/null

2. The first real run died after five minutes:

ProviderHeaderTimeoutError: Provider response headers timed out after 300000ms

OpenCode waits 300 seconds for response headers by default, and a slow local model with a big prompt can blow past that. Setting headerTimeout: false and timeout: false in the provider's options fixed it. That was the only fix I tried.

Then I ran three prompts from the README and the evals.

Token bucket rate limiter (standalone HTML, light). This one ran for 16 minutes and exited 0, but it wrote no file. The model handed the whole prompt to OpenCode's explore subagent, which came back empty. It then spent 1,560 tokens thinking and never answered. By the end, the context was 15.8k tokens.

CDN edge cache, hit vs miss, TTL 60 s (just the SVG). This one took 8 minutes 23 seconds and two turns, and it did write a file. The file was 738 bytes: three plain circles, two of them labelled "Hit" and "Miss", plus two "60s" labels. It never opened the kit, an example, or a reference. It ran no build or check, even though the skill's workflow ends with "verify it like a critic." It used CSS-style rotateX(-30deg) inside an SVG transform attribute, which SVG doesn't support. Chrome ignored those transforms, so the Hit and Miss circles landed stacked on top of each other in the top-left corner, mostly off the canvas. It also wrote the file into the skill's own install folder (~/.agents/skills/anatomy/examples/cdn-cache.svg), not the project directory. Its closing message: "File written successfully... No further action required."

Database connection pool: 10 connections, a queue when all are busy (standalone HTML, dark). This one took 10 minutes 37 seconds and again wrote into the skill's examples/ folder. The page is a plain 2D SVG on a black background. The "pool of 10 connections" is two circles drawn at the same spot, followed by the comment . The queue is one circle and . No isometric camera, no kit, no readout, no motion, and no check. Its last line: "The file was written successfully."

Here's that CDN file as Chrome renders it, at the same size it was written:

To be fair, this is a 1.7-billion-parameter model on CPU being asked to follow a 7,700-token craft manual. The skill tells the model to collect true numbers, invent an object, write a parts list, build the geometry with the kit, and then audit and screenshot every joint. That's a long agentic loop, and a model this small doesn't hold onto it. What the runs do show is that nothing in the skill makes up for a weak model. The kit only helps if the agent actually calls it.

What worked and what broke

Worked:

The kit: zero dependencies, examples rebuild byte for byte, the Raptor builds in under two minutes
The geometry audit: 1,290 overlaps checked against 1,290 order rules in 11 seconds, and it passed
The Chrome checks: screenshots with live readouts, a line-end check, clean consoles
Skill discovery: OpenCode listed anatomy right after install, even in an unexpected folder

Broke or surprised me:

npx skills add crashes on Node 20, even though Anatomy itself supports Node 20.10+
The CLI installs to ~/.agents/skills/anatomy, which differs from the README's OpenCode row
5 of 225 tests fail without React, instead of skipping
opencode run hangs without < /dev/null, and its 300-second header timeout kills slow local runs. Both are OpenCode issues, not Anatomy ones
With qwen3:1.7b, none of the three runs produced a real figure. One wrote nothing. One wrote a 738-byte SVG with invalid transforms. One wrote a 1,049-byte page whose pool of 10 connections is two circles and a comment saying "add 10 circles". Both files landed in the skill's own install folder, and the agent never ran a single check

Should you use it?

If you write docs or technical posts and you already run Claude Code or Codex with a strong model, yes, it's worth trying. The best part isn't the prompt. It's the kit underneath, which is deterministic and checks itself in ways I haven't seen in other "make me a diagram" tools. Start by building examples/dial-indicator and reading its roughly 300 lines. That tells you more about the output than any screenshot.

If you were hoping a local model could drive it, not at 1.7B. The skill is long, and the workflow is long, and the README says it's "slow on purpose." Budget a long session with a capable model, and expect to read the PNGs it makes.

Not tested here: Claude Code, Codex, any frontier model, the React output, the 3D mode, agent-built WebGL figures, and gallery publishing. If you've run it on a strong model, I'd love to hear how long one figure took you and what it cost in tokens. The new community gallery says it records both.

I Pointed REA at a 100 MB Electron App. Here's What It Found and the Two Preload Files It Missed.

Ishank Choudhary — Mon, 12 Oct 2026 04:58:58 +0000

On Sunday morning, 11 October, REA was number one on GitHub's daily trending page, with 25,793 stars gained in a single day. The repo showed 76,998 stars and 16,440 forks. The pitch is simple: "Reverse engineer anything with agents." You install an MCP server, and your coding agent can take apart native binaries, Electron apps, websites, Android APKs and more, then show you the evidence.

I'm wary of anything that gets that famous that fast, so I skipped the README demos. I installed it on a plain Linux box, pointed it at a real 100 MB Electron app, and then built a tiny app where I know every right answer, to see how well it grades itself.

Short version: the static JavaScript analysis is genuinely good and fast. It also missed something I could see with grep, and what it asks of an agent is heavy. Details below.

What I tested and what I didn't

The box is Debian 13 with 8 vCPUs and no GPU. There was no Claude Code, Codex or cloud API key on it, so no agent drove REA. I called it from its own CLI and from a small script that speaks raw MCP over stdio. That tests the tools, not an agent's judgment about when to use them.

REA lists a lot of targets: native binaries, .NET, APKs, firmware, websites, process capture. Native work needs Hopper, Ghidra or IDA, and I had none of the three on this Linux box (the Hopper features are Mac-only). So I stuck to the one target that needs only Node: JavaScript and Electron apps, analyzed statically.

The real target was PI-Desktop v0.16.1, the Electron app from my previous article. Its app.asar is 100 MB and holds 459 relevant files. I also built a five-file toy Electron app, because I can't grade a tool on an app whose source I haven't read.

Install: fast, with one Node catch

REA wants Node 22.19+, 24.11+ or 26+. The box shipped Node 20, so I grabbed 22.20 from nodejs.org. The box has no xz, so I unpacked the tarball with Python. Then:

npm install rea-agents@latest
# added 144 packages, and audited 145 packages in 3s   (node_modules: 96 MB)
npx rea --version
# 6.4.0

The version matters. npm lists 31 versions since the package appeared in July, and GitHub shows releases 6.1, 6.2 and 6.3 all on 9 October. 6.4.0 landed on 11 October, the day I tested. The README itself says it "changes quickly", so pin a version.

Then rea doctor:

healthy: false
  node     ok    22.20.0
  host     FAIL  unsupported_host   debian 13
  hopper   FAIL  missing_analysis_engine
  ghidra   FAIL  GHIDRA_INSTALL_DIR is not set

The supported hosts are macOS 12+, Ubuntu 24.04+, Fedora 41+, Nobara, Arch, CachyOS and Omarchy, so Debian 13 gets a red mark. It didn't stop anything I ran. Static JavaScript analysis worked fine on the "unsupported" host. I also ran it once under Node 20, where doctor flagged the version, and the analysis still ran. I wouldn't rely on that, but it's good to know the check is stricter than the code.

I didn't want setup touching the box's real config, so I ran it against a throwaway home directory:

HOME=/tmp/fakehome npx rea setup --client claude_code --dry-run   # prints the plan
HOME=/tmp/fakehome npx rea setup --client claude_code --yes
# appliedActions[2]: configured_claude_code, installed_skill

The dry run listed exactly two changes before anything was written. The result was a small mcpServers.rea entry (npx -y rea-agents@6.4.0 mcp) and a skill folder with a SKILL.md and references. Small, clear and reversible. That's how I like setup to behave.

The scan: 52 seconds, 3.1 million AST nodes

time npx rea analyze-javascript-application app.asar --format json > out.json
# real 0m52.2s     out.json: 59.2 MB

REA read 459 files (24.2 MB of text), parsed 421 JavaScript files, walked 3,114,427 AST nodes and had zero parse failures. For a 100 MB bundle, 52 seconds is fine.

Then I checked its Electron summary against the unpacked source, using plain grep:

Here is where it matched:

Browser windows: REA said 5. There are five new BrowserWindow( calls (four in one chunk, one in index.js).
contextBridge APIs: REA said 2. The preload scripts call exposeInMainWorld for piDesktop and pluginBridge.
IPC: REA said 4 main-process handlers, 6 renderer transmissions and 5 renderer listeners. My grep counts for ipcMain.handle/on, ipcRenderer.invoke/send/sendSync and ipcRenderer.on come to the same numbers: 4, 6 and 5.
Sender checks: it reported 4 places where the main process looks at the sender of an IPC message. I didn't verify each of those by hand, so I'm only passing the number along.

And here is what it got wrong:

Preload entrypoints: REA said 0. The app has two preload files, preload/index.cjs and preload/plugin-panel.js. The windows load them through code like join(getModuleDirectory(import.meta.url), "../preload/index.cjs"). My guess is that this helper call is the reason REA couldn't resolve the path statically, but I haven't confirmed that.
Exposed API members: REA said 0. Both bridges obviously expose members, so this zero follows from the missing preload link.

REA is also candid about what it dropped. The result carries 744 limitations. For 365 of them, the text says: "Expanded application relationships for [file] exceed the parent's result budget; complete static facts were retained on the file node." In plain terms, on a big app it keeps counts and per-file facts but throws away some of the connections between them.

That cost me the most useful feature on the real app. I asked it to trace anything touching the channel pi-plugin-panel:

seed_matches: []
coverage.status: no-match
"No graph entity matched the literal seed; this is not evidence that the feature is absent."

The string pi-plugin-panel-drop is right there in the source. The graph just didn't have channel nodes for it. The tool says so, which I respect, but you'd still be stuck.

The toy app: where it shines

So I wrote a five-file Electron app (main.js, preload.js, index.html, renderer.js, package.json) with known contents:

one window with a preload script
one contextBridge API with four members
three literal IPC channels: notes:read, notes:write, open-external
one loop that registers job:sync and job:export from a variable, and a preload call that builds "job:" + topic
two handlers that check the sender, through a small helper function

I analyzed it as written, then again after esbuild --minify, which squashes main.js into one 732-byte line with one-letter variable names.

On both versions, REA's counts were identical, and nearly all of them were right: 1 window, 1 preload, 1 bridge API with 4 members, 3 literal channels, 2 dynamic channel operations, 4 main handlers, 4 renderer transmissions, 3 of them paired with their handlers (the fourth is the dynamic one). Each run took about two seconds.

The tracer is the highlight. I fed the evidence back in with trace-application-feature and a seed of channel: "notes:read". It returned the chain I wanted, on the minified bundle: preload.js invokes the channel notes:read, and the channel is handled by the notes:read handler. It even kept the dynamic channel honest. In the minified code the label is "job:"+e, the actual expression with the renamed variable, rather than a made-up string.

The one miss on the toy: sender validation. I wrote two handlers that call a checkSender(e) helper, which looks at e.senderFrame.url. REA reported 0 sender-validation observations. That's the kind of check you'd want a security-minded tool to notice. A check hidden behind a helper function got past it.

The part nobody puts in the README: what it costs an agent

REA exposes 170 MCP tools. I opened a stdio session, sent initialize and tools/list, and measured the reply:

init: 1.4 s    serverInfo: {"name":"rea","version":"6.4.0"}
tools/list: 170 tools in 0.3 s
payload: 1,954,704 bytes

That's about 1.95 MB of JSON before the agent does anything. By the usual rule of thumb of four characters per token, that's roughly half a million tokens, though that's my estimate and clients may defer or trim tool lists. For scale, I run the local model I use for tests with a 16k context. The project's own docs mention a related problem: GitHub Copilot CLI with gpt-4.1 can block chat with REA's full schema profile, and they suggest a "compact" profile. I tried it with REA_MCP_INPUT_SCHEMA_PROFILE=compact. The list shrank to 1,772,775 bytes, which is 9% smaller. Helpful, not a cure.

This is why I didn't try to drive it with a small local model. The tool list alone wouldn't fit in the context window, so any result would say more about my setup than about REA.

The tool call itself is heavy too. I called analyze_javascript_application on the same app.asar over MCP with detail: "summary":

call: 60.5 s    response: 1,350,868 bytes (text part: 672 KB)

A "summary" that is 672 KB of text is a lot for an agent to read. If you use REA through an agent, I'd expect the agent to spend a lot of its budget on REA's output. I'd test it with your exact client before you commit.

Native binaries: stopped at the door

I did try the native path, to see how it fails. Pointing rea analyze at PI-Desktop's 20 MB Rust host binary:

code: provider_unavailable
candidate_ids: ghidra, hopper, ida

That's a clear message with a clear fix, and I'd have been disappointed by anything vaguer. rea inspect-artifact on the same file still worked without a provider. In 1.7 seconds it returned the format (ELF) and a SHA-256 that matches what sha256sum printed, plus a suggested next step: open_binary, which needs one of the three engines. So you get identification, not understanding, until you install a disassembler. I didn't, so I can't tell you how good the native side is.

What worked and what didn't

What worked:

Install in 3 seconds, and a setup that shows its plan before writing anything
Static analysis of a 100 MB Electron app in about 52 seconds, with zero parse failures
Counts for windows, bridges and IPC that matched my grep on the real app
Identical, correct results on plain and minified code in my toy app
An honest evidence trail: limitations listed, "no match" is not claimed as "absent"

What didn't:

Missed both preload files on the real app, which zeroed out the exposed-API count
Dropped connections on 365 files because of its own result budget, so the trace on a real channel found nothing
Missed a sender check hidden behind a helper function
A 1.95 MB tool list and a 672 KB "summary", which an agent has to carry
doctor goes red on Debian 13 even though the JavaScript path ran fine

What I didn't test: Hopper, Ghidra or IDA, websites and browser capture, APKs, firmware, runtime capture, and above all an agent actually using the tools to "build a similar feature", which is the headline pitch. That claim needs a frontier model and a real session, and I didn't have one.

Should you try it?

If you work with Electron or bundled JavaScript, yes. The pairing of fast static analysis and a graph you can trace is useful, and the toy-app result on minified code impressed me more than any star count. Run it on an app you already understand first, the way I did, so you learn where it's blind before you trust it on one you don't.

If you plan to hand it to an agent, go in with care. Count the tokens in the tool list, try the compact profile, and keep an eye on how much an analysis result takes from the context window.

As for the trending page: stars tell you people are curious, not whether the tool matches a given job. I couldn't check the claim that it jumped from about 9k stars in four days, because GitHub only gives star timestamps to signed-in API calls. What I could check is that on Sunday morning it was real, MIT-licensed, released four times in three days (6.1 to 6.4), and good at most of the job I gave it. It also missed two files that grep found in a second.

I'd like to see this on a Mac with Hopper and a frontier model driving it. If you've run REA that way, tell me what held up.

I Ran PI-Desktop's Plan, Goal and Subagent Modes on a 1.7B Local Model. Here's What Broke.

Ishank Choudhary — Sat, 10 Oct 2026 04:59:13 +0000

Pi, the open-source coding agent engine, reached 1.0 on 1 October. Three days later, on 4 October, PI-Desktop shipped v0.16.1, the first release built on Pi 1.0.0. PI-Desktop is a desktop GUI for that engine: Electron on the front, a Rust "host core" underneath, and the Pi agent packages running in a Node sidecar.

One thing to clear up first. PI-Desktop is a community project by vastsa, not something from the Pi team. It's LGPL-3.0, has about 6.4k GitHub stars, and its README calls the 0.16.x line an "Early Preview". Several other projects use the "pi-desktop" name, so if you go looking, start from the repo and the official docs.

I didn't want to write a feature list. I wanted to know what actually happens when you install it on a plain Linux box and run the three things that set it apart from a terminal agent: Plan mode, Goal mode and subagents.

The honest caveat up front

There was no cloud API key on my test machine. So I ran everything against qwen3:1.7b through Ollama, on CPU only: Debian 13, 8 vCPUs, no GPU. That model managed about 250 tokens/s on prompt processing and about 8 tokens/s on generation.

That is a stress test, not a fair benchmark. A 1.7B model is the weakest setup the app would ever see. Some of what broke is the model's fault, and I'll say which parts. Other things broke regardless of the model, and those are the more useful findings.

The test project was lukeed/clsx v2.1.1: 21 files and 32 passing tests. Small enough for a tiny model to have a chance.

Install: fast and uneventful

Linux gets an AppImage, a .deb and an .rpm (glibc 2.35+, so Ubuntu 22.04+, Debian 12+ or Fedora 36+). I used the AppImage, extracted it instead of mounting it so libfuse2 wasn't needed:

wget https://github.com/vastsa/PI-Desktop/releases/download/v0.16.1/PI-Desktop-0.16.1-linux-x86_64.AppImage
chmod +x PI-Desktop-0.16.1-linux-x86_64.AppImage
./PI-Desktop-0.16.1-linux-x86_64.AppImage --appimage-extract
./squashfs-root/AppRun --no-sandbox

What I measured:

Download: 164.8 MB, and the SHA-512 matched latest-linux.yml.
Extract: about 3 seconds, 414 MB on disk. No missing shared libraries according to ldd.
First launch: the window appeared in 1.56 s cold and 0.54 s warm.
Memory: about 630 MB RSS across the Electron, host-core and sidecar processes, before the agent did anything.
--no-sandbox was needed only because the extracted tree's chrome-sandbox isn't setuid. The installed packages shouldn't need it.

Inside the bundle you get Chromium 150, the Rust pi-desktop-host-core binary, the Pi sidecar, a bundled models.dev catalog of 8,344 models, and two built-in plugins (a browser and a file manager).

The first screen is a calm "What can I help you build?" with a Get started checklist. The composer has three chips that matter for this test: the mode (Agent / Plan / Goal), the permission (Ask every time / accept edits / Auto) and the model.

Hooking up a local model

There's no dedicated Ollama tile in 0.16.1. Local models go in through Settings → Agent → Models → Add provider → Custom endpoint, which takes "any OpenAI- or Anthropic-compatible URL".

Base URL:   http://127.0.0.1:11434/v1
API key:    ollama        (dummy)
API format: OpenAI Chat Completions

Fetch list found the model and the provider showed "Connected · 1 model found". The provider list is long, too: subscription sign-ins (Claude Pro/Max, ChatGPT, GitHub Copilot, SuperGrok/X Premium and others) plus around 30 API-key providers.

A smoke test ("Reply with exactly: PI-Desktop smoke test OK") worked, but it took 1 minute 54 seconds, and it showed me the first real number:

The app's system prompt plus tool schemas came to about 4.1k tokens (Ollama cached 4,098 of 4,717 prompt tokens).
The context ring in the composer read 96% after one turn, because I'd set Ollama to a 16k context and the custom model's context window showed as "— —" in the UI.
The reasoning setting appended /think to my prompt (Qwen's reasoning switch), and the model echoed it back in its answer.

Lesson one: when you add a local model, open the model's Advanced settings and set the Context Window yourself. With 16k of context, the harness takes a quarter of it before you type anything.

Plan mode: it worked, but the plan was fiction

I opened the clsx folder as a project, set the chip to Plan with "Ask every time", and asked it to plan a test for nested arrays and falsy values.

The mechanics were solid:

The plan was saved as a real file, .pi/plan/add-nested-array-falsy-values-tests-for-clsx-20261007-1108.md (315 bytes).
A plan card appeared with Open plan, Reject and Approve (Ask), and the File Manager pane opened showing the markdown.
The chat got an automatic, sensible title.

The content was not. The plan said to add test files in /work/clsx/tests, a path that doesn't exist. The run log showed "3 issues · 4 tools": a search failed, a read failed, the first SubmitPlan failed and the second one succeeded. The app reported "Processed for 5m 19s", but wall-clock time from sending the prompt to seeing the card was about 17 minutes. The context meter sat at 95%.

That's mostly the model. A 1.7B model with a quarter of its context taken by instructions invents paths. But the workflow around it (a separate artifact, an explicit approval gate) is exactly what I'd want to put in front of a stronger model.

Goal mode: the permission chip changed on its own

Goal mode is meant to write a .pi/goal/*.md file with an outcome and acceptance criteria, wait for approval, then switch to Agent and work until it can report which criteria it verified.

I clicked the mode chip twice to get to Goal. Then I noticed the permission chip next to it had silently switched from "Ask every time" to "Auto". I hadn't touched it. For a mode whose whole job is to run autonomously toward a target, that's the one setting I'd expect it never to loosen on its own.

The run didn't go well:

The first model call took 4 min 26 s.
No goal file and no acceptance criteria were ever written.
The model went looking in /work/tests again.

The safety net did work: even under Auto, a Grep on /work/tests stopped with "Accesses a path outside the session workspace", rated LOW RISK. I denied it. The model then tried to Edit /work/tests/new-test.js, which failed. I stopped it at about 12 minutes with nothing verified, and the chat had been auto-renamed in Chinese ("新增嵌套假值测试用例", roughly "add nested falsy value test cases").

The failed run is the model's fault. The permission flip and the Chinese title happen in the app, and they're worth reporting to the maintainers. Fairly, the docs do say Plan and Goal are "contract modes, not strict read-only security profiles". I'd still check that chip every time you switch modes.

Subagents: the one clean win

PI-Desktop ships five built-in subagents, each with a defined tool set:

Explorer: Read, Glob, Grep, Bash
Code reviewer: Read, Glob, Grep (read-only)
Test runner: Read, Glob, Grep, Bash
Fixer: adds Edit and Write
UI designer: adds BrowserPreview, Edit and Write

You can add your own as markdown files in ~/.agents/subagents/ (up to 16). Delegation uses a Task tool that runs in the background and returns a delegation ID, with TaskWait, TaskList and TaskStop alongside it, and up to 10 at once per session. Subagents only work in Agent mode.

In Agent mode I asked: "Use the test-runner subagent to run npm test and report the result."

This went well:

The UI drew a small delegation graph: Main agent → test-runner, "Coordinating 1 delegated task".
The permission prompt said clearly that it was "Asked by the test-runner subagent", showed the exact command (npm test) and labelled it HIGH RISK. Running a shell command is the right thing to flag.
After Allow once, the subagent finished in 2 min 44 s (3 steps). The whole turn took 5 min 55 s.
The result was exit 0, 32/32 passed, which matches what I got running npm test in a shell myself.

The subagent's own transcript opened in a side pane marked read-only ("Subagents are driven by the main agent"). And the chat was renamed again, to "npm测试成功" ("npm test succeeded").

The lesson: give a small model a narrow, well-defined job and the harness carries it. Ask it to plan open-ended work and it falls apart.

Two findings that have nothing to do with the model

1. "OS keychain" isn't quite true yet. The landing page and the README say credentials live in the OS keychain. I ran gnome-keyring with Secret Service available, saved the Ollama provider, and zero items went into the keyring. Instead, two files appeared:

~/.pi-desktop/secrets/<sha>.bin    48 B   mode 644
~/.pi-desktop/secrets/.machine-key 32 B   mode 600

The project's own storage spec explains it: the shipped backend is an AES-256-GCM file store keyed by a machine key that "host-core generates once and keeps beside them", and the OS keychain backend is one "that neither host-core nor Electron main implements today, so a same-user process that can read the data directory can also decrypt the secrets." The spec is honest about this. The marketing isn't. If you put real cloud keys in it, treat the ~/.pi-desktop folder as sensitive.

2. The window only redraws when the mouse moves. On my Xvfb/xfwm4 desktop, timers and streaming output froze until I moved the cursor. That could be specific to my headless setup, but it made the long local runs feel even longer.

The rest, briefly

MCP Market: Settings → Agent → MCP has a Market tab that merges a built-in catalog with the official MCP registry. It listed Memory, Sequential Thinking, Filesystem, Playwright, Context7, Fetch, Git, Time, Serena and others, most marked Verified, each with its npx/uvx command. Some descriptions showed up in Chinese. As of v0.16.1, tools from user-added MCP servers need approval.
Session import: it can import sessions from Claude Code, Codex, OpenCode and Pi.
Scheduled tasks: hourly, daily or weekly, but they only run while the app is open.
No telemetry and no required account, according to the README.

What a bigger cloud model would likely change

I didn't test this, so treat it as an expectation, not a result. With a capable cloud model and a large context window, I'd expect:

The 4.1k-token system prompt to stop mattering. At 128k+ context it's a rounding error, not 25%.
Plan mode to produce real paths, since a stronger model would actually read the repo before planning.
Goal mode to write its acceptance criteria and run through them in minutes, not stall after 12.
Total time per turn to fall from minutes to seconds.

What I don't expect to change: the permission chip flipping to Auto, the keychain wording, the Chinese auto-titles and the redraw bug. Those are in the app.

Verdict

PI-Desktop v0.16.1 installs in seconds, starts in under two, and its Plan, Goal and subagent workflows are thought through: separate artifacts, approval gates, clear risk labels, a visible delegation graph. Subagents did their job even on a 1.7B model on CPU.

It's also clearly an early preview. Check the permission chip when you switch to Goal, don't trust the "OS keychain" line yet, and set your local model's context window by hand.

If you run it with a frontier model, I'd like to know whether Goal mode holds up. That's the test I want to see next.

Originally published on Medium.

I Pointed Matryoshka's MCP Server at War and Peace and a 10k-Line Log, With No LLM Key

Ishank Choudhary — Wed, 07 Oct 2026 04:54:30 +0000

When a coding agent needs something from a file that is too big for its context window, it usually does one of two things. It reads the file in slices and hopes the answer is in one of them, or someone has already built a RAG index with chunks and embeddings. Both lose information at the seams.

Matryoshka takes a third route. It is an open-source TypeScript project by Dmitri Sotnikov (yogthos, the author of the Luminus Clojure framework). It keeps the whole document on the server and gives the agent a small query language to work on it. Results stay server-side, and the agent gets back short "handles" instead of the raw text. The README says it is based on the Recursive Language Models paper by Alex L. Zhang, Tim Kraska and Omar Khattab of MIT CSAIL. Matryoshka is an independent implementation of that idea, not the authors' own code.

When I checked on 4 October 2026, the repo had 149 stars, 20 forks and no open issues. It is Apache-2.0 licensed, and the latest npm release was matryoshka-rlm 0.2.40 from 17 May 2026. It's a small project, so I wanted to see how it holds up on real files.

One constraint shaped the whole test: there was no LLM API key on my machine and no local Ollama. Matryoshka ships an MCP server, lattice-mcp, that needs no LLM at all. So I played the agent myself and wrote every query by hand. This tests the engine and the tool interface. It says nothing about how well a model would drive them.

How it works, in one paragraph

The model doesn't write JavaScript or Python. It writes commands in Nucleus, a small S-expression language: (grep "ERROR"), (filter RESULTS ...), (count RESULTS), (sum RESULTS). The Lattice engine parses and type-checks each command and runs it against the document. Results go into an in-memory SQLite store, and the client gets a stub like $grep_error: Array(1000) [preview...]. It only pulls real lines with lattice_expand when it needs them. The README claims this gives "97%+ token savings", and "80%+ token savings compared to reading files directly" for the MCP server. Those are the project's numbers, and I didn't benchmark them.

Setup

The box was Linux x86_64 with 8 vCPUs, about 15 GB of RAM, Node 20.19.2 and pnpm 10.33.4. The README suggests pnpm add -g matryoshka-rlm. I installed it locally in a small Node/TS project instead, next to the official MCP SDK:

pnpm add matryoshka-rlm@0.2.40 @modelcontextprotocol/sdk tsx typescript @types/node

It finished in 2.5 seconds and took 220 MB of node_modules. pnpm 10 skipped the native build scripts for the tree-sitter grammars. I left them unbuilt, and symbol listing still worked.

Gotcha 1: don't run plain npx lattice-mcp. From a folder where Matryoshka isn't installed, npx lattice-mcp downloaded an unrelated npm package called lattice-mcp (1.6.2), which asked for LATTICE_API_URL and LATTICE_API_TOKEN. Use the binary that comes with matryoshka-rlm (node_modules/.bin/lattice-mcp, or npx -p matryoshka-rlm lattice-mcp). The real one prints lattice-mcp v0.2.40.

The client is a few lines of TypeScript with the MCP SDK's stdio transport:

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";

const transport = new StdioClientTransport({
  command: "/workspace/matryoshka-test/client/node_modules/.bin/lattice-mcp",
  cwd: "/workspace/matryoshka-test", // lattice_load only accepts files under this folder
});
const client = new Client({ name: "ishank-test", version: "0.0.1" });
await client.connect(transport);

await client.callTool({ name: "lattice_load", arguments: { filePath: "docs/war-and-peace.txt" } });
const r = await client.callTool({ name: "lattice_query", arguments: { command: '(grep "NatÃ¡sha")' } });

It connected in 409 ms and reported the server as lattice 0.2.40, with 12 tools: lattice_load, lattice_query, lattice_expand, lattice_close, lattice_status, lattice_bindings, lattice_reset, lattice_memo, lattice_memo_delete, lattice_help, lattice_llm_respond and lattice_llm_batch_respond.

The test documents

War and Peace from Project Gutenberg (Maude translation): 3.36 MB, 66,041 lines, 566,333 words, about 770,000 tokens (769,825 with tiktoken's o200k_base). That is far more than most models' context windows.
A real Apache access log from Elastic's examples repo: 10,000 lines (2.37 MB) of traffic from May 2015.
Matryoshka's own source files, for the code features.

Before asking Matryoshka anything, I worked out the true answers separately with grep and a short Python script.

War and Peace: facts, counts and quotes

Loading the novel took about 0.75â€“0.8 seconds. Every query after that took between 1 and 430 ms.

What I asked	Nucleus command	Matryoshka	Ground truth
Chapter headings	`(grep "^CHAPTER [IVXLC]+$")` â†’ `(count RESULTS)`	365	365 âœ…
Book headings	`(grep "^BOOK [A-Z]+")` â†’ count	17	15 âŒ
Mentions of NatÃ¡sha	`(grep "NatÃ¡sha")` â†’ count	1,213	1,213 matches âœ… (on 1,197 lines)
Pierre lines that also mention NatÃ¡sha	`(filter RESULTS (lambda x (match x "NatÃ¡sha" 0)))`	44	44 âœ…
Where is "Genoa"?	`(grep "Genoa")` â†’ expand	lines 840, 1573, 9153	same âœ…
Quote the opening	`(lines 840 846)` â†’ expand	exact 7-line "Well, Prince, so Genoa and Luccaâ€¦"	âœ…
Mentions of BorodinÃ³	`(grep "BorodinÃ³")` â†’ count	108	108 âœ…

Gotcha 2: grep is case-insensitive and counts matches, not lines. The 17 books threw me until I expanded the handle. Two of the hits were ordinary sentences starting with a lowercase "book" ("book from the high desk."). The source confirms that grep runs with the gmi flags. It also returns one item per match, so 1,213 is the number of times "NatÃ¡sha" appears, not the 1,197 lines she appears on. Once I knew that, every count matched grep -oi exactly. A tighter pattern, (grep "^BOOK [A-Z]+: "), returned the correct 15.

The ranked searches showed where the "no RAG" approach has its limits:

(bm25 "battle of Borodino" 5) returned five lines that just said "battle." The translation spells it BorodinÃ³. With the accent, (bm25 "BorodinÃ³ battle" 5) returned five lines that all contain "battle of BorodinÃ³".
(fuzzy_search "Natasha Rostova" 3) found nothing. (fuzzy_search "NatÃ¡sha RostÃ³va" 3) found her straight away.
(semantic "wounded at Austerlitz looking at the sky" 5) didn't return the famous lines where Prince Andrew lies wounded under "the lofty sky". Its closest hit was "been looking at." on line 15862, seven lines before. Plain (grep "lofty sky") found the passage at line 15869. The README describes semantic as TF-IDF cosine similarity, not embeddings, so this isn't surprising.

Gotcha 3: spell it the way the document does. Lexical ranking does not fold accents, so with translated or non-English text an agent should check the document's spelling before trusting a ranked search.

Gotcha 4: lattice_expand needs a named handle. RESULTS works inside queries, but lattice_expand with RESULTS returns "Invalid handle: RESULTS". lattice_bindings lists the real names, such as $grep_genoa. Patterns full of quotes get generic names ($grep, $grep_2), so lattice_bindings is worth calling often.

Two smaller things worked as documented. Loading /etc/hostname was refused with "Path outside working directory is not allowed." And a memo I saved with lattice_memo while the novel was loaded could still be expanded after I switched to the log file.

How much text actually reached my "agent"? My first session made 27 tool calls against the 3.36 MB novel, and the tool responses added up to 5,625 characters. That's my own count, not a benchmark. A real agent wouldn't read the whole book either, but it shows the design: the document stays on the server and only stubs and the lines you ask for come back.

The access log: counting and summing

What I asked	Matryoshka	Ground truth
404 responses, `(grep "\" 404 ")` â†’ count	213	213 âœ…
Googlebot requests	543	543 âœ…
200 responses	9,126	9,126 âœ…
Bytes served with 200, `(sum RESULTS)` on the grep result	1,067,060.464	âŒ meaningless
Same, after `(map RESULTS (lambda x (match x "\" 200 (\\d+)" 1)))`	2,735,455,845	2,735,455,845 âœ…

Gotcha 5: sum on raw lines adds up the wrong number. When you call sum on grep results, it takes the first number on each line. In an access log, that's the start of the client IP (83.149â€¦), so you get a confident, meaningless total. Pull out the field first with map and match, and sum returns the exact byte count.

I also tried the sub-LLM path without an LLM. My client doesn't support MCP sampling, so (llm_batch RESULTS (lambda x (llm_query "Is this request from a bot or a human? {item}" (item x) (one_of "bot" "human")))) over the three 500-error lines returned a single [LLM_BATCH_REQUEST id=â€¦ count=3] message. It held three ready-made prompts, each with its log line and an "answer with exactly one" instruction. I typed the answers myself (["bot","bot","human"]) and sent them back with lattice_llm_batch_respond. Matryoshka stored them as a new handle, and filtering on "bot" counted 2. So the protocol works with any MCP client. To be clear, no model judged those lines. I did.

Code files: symbols yes, graph no

On Matryoshka's own lc-solver.ts (2,658 lines), (list_symbols "function") returned 20 functions, the same count grep gives for top-level function declarations. Each came with its line range and signature. (get_symbol_body "evaluate") returned the function's source, but inline and 64,311 characters long. Unlike search results, it isn't returned as a handle, so a big function goes straight into the agent's context.

The knowledge graph didn't work for me. (callers ...) and (god_nodes ...) returned "No symbol graph available" on all four source files I tried. Each time the server's stderr showed Graph.addEdge: source & target are the same (...), with names like evaluate, init and run. The graph library rejects self-loops, so a recursive function or a same-name call seems to stop the graph from being built. That's on 0.2.40 with these four files. I can't say how widespread it is.

The REPL and the full RLM CLI

lattice-repl docs/war-and-peace.txt is useful for trying commands out: (grep "Austerlitz") then (count RESULTS) gave 51, matching grep -oi. Piping :load and a query in together failed, because the load hadn't finished when the query arrived. Passing the file as an argument worked.

The rlm CLI is the part where the LLM writes the Nucleus itself. With no config, it defaults to Ollama on localhost, retried three times, and stopped after about 9 seconds with [aborted: llm 3 consecutive LLM call failures: fetch failed]. That's expected. It needs a model.

What I didn't test

A real agent choosing Nucleus commands. That is the main promise, and I wrote every command by hand.
rlm / rlm-mcp with a model, recursive rlm_query / rlm_batch, MCP sampling, compaction and resource limits.
lattice-setup (Claude Code wiring), the HTTP and pipe adapters, program synthesis, fuse / dampen / rerank, and multi-document loading.
Any of the README's token-savings figures or the paper's benchmark results.

Takeaways

The engine was exact on literal tasks. Every count, lookup and quote matched ground truth once I understood the semantics. Most of the work happens on the server, and only stubs come back.
The semantics matter more than the syntax. Case-insensitive, per-match grep and first-number sum are reasonable defaults, but an agent that doesn't know them will report 17 books and a nonsense byte total with full confidence.
The ranked searches are lexical. BM25, fuzzy and "semantic" (TF-IDF) depend on exact spelling. Don't expect embedding-style recall.
The code graph is the weak spot in 0.2.40. Symbol listing worked, but the call graph failed on every file I tried.
It's worth a try if your agent works over big logs, transcripts or books and you want answers you can check, not chunk retrieval. Next I want to put a real model in the loop and see whether it avoids these gotchas on its own.

If you've wired lattice-mcp into Claude Code or another agent, I'd like to hear whether your model figured out the case-insensitive grep by itself.

Originally published on Medium.

I Gave Two TypeScript Agent Sessions One Shared Memory With Hindsight (No LLM Key)

Ishank Choudhary — Mon, 05 Oct 2026 04:50:59 +0000

Most of my coding-agent sessions start the same way. The agent rediscovers that the repo uses pnpm, gets told again not to touch the legacy payment file, and trips over the test that only passes in UTC. None of that is in the code. It lives in someone's head, or in a previous session that is gone.

Hindsight by Vectorize is built for that gap. It is an open-source agent memory server (MIT) that agents write to and read from across sessions. When I checked on 3 October 2026 its GitHub repo had 44,886 stars, sat second on GitHub's weekly trending page with 16,183 stars that week, and its latest release was v0.10.2 from 29 September.

I wanted to see what it does in practice, so I installed it on a Linux box and called it from a small Node/TypeScript project. One constraint shaped the whole test: I didn't have an LLM API key on that machine. Hindsight documents a mode for exactly that, so this is a test of its no-LLM floor, not of everything it can do. I've marked clearly which parts I ran and which parts I only read about.

The setup

The box was Linux x86_64 with 8 vCPUs, about 15 GB of RAM and Node 20. There was no Docker, so I used the bare-metal path from the README (pip install hindsight-api), through uv:

uv venv .venv-hs --python 3.12
uv pip install hindsight-api==0.10.2

It resolved 227 packages and finished in 11.6 seconds, including the PyTorch download. The catch is size: the virtual environment came to 6.3 GB, mostly torch and CUDA wheels. Plan disk space accordingly.

The configuration docs describe HINDSIGHT_API_LLM_PROVIDER=none as "chunk storage + semantic search only, no API key needed." In this mode retain stores your text as chunks without LLM fact extraction, recall still works, reflect returns HTTP 400, and observations are switched off.

export HINDSIGHT_API_LLM_PROVIDER=none
hindsight-api

The startup banner showed v0.10.2 with an embedded PostgreSQL (pg0, started automatically), LLM none / none, local embeddings (BAAI/bge-small-en-v1.5), a local reranker (cross-encoder/ms-marco-MiniLM-L-6-v2), and MCP enabled at /mcp.

The first start took about 38 seconds to reach "Uvicorn running", including model downloads from Hugging Face.
A warm restart took about 19 seconds to a healthy /health.
The worker process used roughly 1.35 GB of resident memory.

Session 1: one agent writes down what it learned

For the Node/TypeScript side I installed the official client (version 0.10.2) and ran the scripts with tsx:

npm install @vectorize-io/hindsight-client typescript tsx @types/node

The scenario was a fictional shop-api repo. The first script plays "Agent A" at the end of a session. It creates a bank, which is Hindsight's isolated memory store, and retains five project facts that aren't obvious from the code.

// src/agentA.ts
import { HindsightClient } from '@vectorize-io/hindsight-client';

const BANK = 'shop-api-repo';
const client = new HindsightClient({ baseUrl: process.env.HINDSIGHT_URL ?? 'http://localhost:8888' });

const learnings = [
  { content: 'The shop-api repo uses pnpm, not npm. Run `pnpm test` with Vitest; Jest was removed in March.', context: 'tooling' },
  { content: 'All money values are stored as integer cents in Postgres (column type bigint). Never use floats for prices.', context: 'data model convention' },
  { content: 'Route handlers live in src/routes/*.ts and must validate request bodies with zod schemas from src/schemas.', context: 'code convention' },
  { content: 'The flaky test checkout.spec.ts fails when TZ is not UTC; CI sets TZ=UTC, so set it locally too.', context: 'debugging note' },
  { content: 'Do not touch src/legacy/paypal.ts; it is scheduled for deletion and has no tests.', context: 'team decision' },
];

async function main() {
  await client.createBank(BANK, {
    reflectMission: 'Long-term memory for coding agents working on the shop-api TypeScript repo.',
  });
  for (const l of learnings) {
    const t0 = performance.now();
    await client.retain(BANK, l.content, { context: l.context, tags: ['repo:shop-api'] });
    console.log(`retain: ${(performance.now() - t0).toFixed(0)} ms`);
  }
}
main();

The five retains took 184, 42, 46, 56 and 41 ms. Every response reported zero LLM tokens, which is expected with no model. listMemories showed five memories, all with fact_type: "world". In chunks mode that means the text is stored as written: no extracted entities and no restructuring.

Session 2: a fresh process asks what it should know

The second script is a separate process with a new client and no shared state apart from the bank name. It asks four questions an agent would plausibly ask on day one, then tries reflect.

// src/agentB.ts
import { HindsightClient } from '@vectorize-io/hindsight-client';

const BANK = 'shop-api-repo';
const client = new HindsightClient({ baseUrl: process.env.HINDSIGHT_URL ?? 'http://localhost:8888' });

const questions = [
  'How do I run the tests in this repo?',
  'How should I store a product price?',
  'Why does checkout.spec.ts fail on my machine?',
  'Can I refactor the PayPal integration?',
];

async function main() {
  for (const q of questions) {
    const t0 = performance.now();
    const r: any = await client.recall(BANK, q, { tags: ['repo:shop-api'], maxTokens: 512 });
    console.log(`Q: ${q} (${(performance.now() - t0).toFixed(0)} ms)`);
    for (const m of r.results.slice(0, 2)) console.log('  ->', m.text);
  }
  try {
    await client.reflect(BANK, 'What should a new agent know before editing this repo?');
  } catch (e: any) {
    console.log('reflect error:', e.message);
  }
}
main();

Here is what came back:

Query	Latency	Top result
How do I run the tests in this repo?	227 ms	the pnpm / Vitest note ✅
How should I store a product price?	150 ms	the integer-cents note ✅
Why does checkout.spec.ts fail on my machine?	62 ms	the TZ=UTC note ✅
Can I refactor the PayPal integration?	68 ms	the legacy/paypal.ts note ✅

The right memory came first on all four questions. Two caveats keep that in proportion. The bank had five entries, so every query returned all five, and what I was really seeing was ranking, not filtering. The second result was sometimes unrelated: the price question had the TZ note in second place. That's fine for a toy, but on a real bank you would want the maxTokens budget or the minScores floors the client exposes.

reflect failed exactly as documented:

Reflect requires an LLM provider. Current provider is set to 'none'.
Set HINDSIGHT_API_LLM_PROVIDER to a real provider (e.g., openai, anthropic, gemini).

Next I stopped the server, started it again and re-ran Agent B. The top results didn't change (201 ms and 91 ms on the first two queries), so the memories live on disk in the embedded Postgres, not just in the process.

A third client over MCP

Every Hindsight server exposes a Model Context Protocol endpoint per bank at /mcp/{bank_id}/. I talked to it with plain curl. You send an initialize request, read back the mcp-session-id header, and send that header with each later call:

curl -s -D hdr.txt -X POST http://localhost:8888/mcp/shop-api-repo/ \
  -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}'

SID=$(grep -i mcp-session-id hdr.txt | awk '{print $2}' | tr -d '\r')

curl -s -X POST http://localhost:8888/mcp/shop-api-repo/ \
  -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
  -H "mcp-session-id: $SID" \
  -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"recall","arguments":{"query":"which package manager and test runner?"}}}'

The server identified itself as hindsight-mcp-server 0.10.2, and tools/list returned 36 tools. They cover retain and recall, reflect, mental models, directives, memory and document management, operations, tags, bank settings and knowledge-page CRUD. The recall call returned the pnpm/Vitest memory that Agent A had written through the TypeScript SDK. The SDK and MCP both work against the same bank, so a TypeScript service and any MCP-capable agent can share one memory.

What the coding-agent installer actually wires up

Coding agents are the headline use case. Vectorize ships @vectorize-io/hindsight-coding-agents, and the version I tried (0.8.0) lists 20 harnesses in its installer: Claude Code, Codex, Cursor CLI, GitHub Copilot CLI, opencode, Grok Build, Cline CLI, Devin CLI, Antigravity CLI, Qwen Code, Kimi Code, DeepSeek Harness and more.

I did not run a real coding-agent session. The only agent the installer detected on the box was a Cursor CLI install that other tools share, and I didn't want to change its config. I installed the integration into a throwaway home directory instead and inspected what it writes:

HOME=./fakehome npx -y @vectorize-io/hindsight-coding-agents install cursor-cli \
  --server self-hosted --api-url http://localhost:8888

It reported hooks merged into ~/.cursor/hooks.json, an MCP server added to ~/.cursor/mcp.json, and a skill installed under ~/.cursor/skills/hindsight-coding-agent. Here are the hooks it wrote (paths shortened):

{
  "version": 1,
  "hooks": {
    "sessionStart":       [{ "command": "node \".../cursor-sessionstart-hook.js\"", "timeout": 30 }],
    "beforeSubmitPrompt": [{ "command": "node \".../cursor-hook.js\"" }],
    "stop":               [{ "command": "node \".../cursor-stop-hook.js\"", "timeout": 30 }]
  }
}

The MCP entry runs a local stdio server, node ~/.hindsight/coding-agents/dist/mcp-server.js ${workspaceFolder}. I started that server myself with my Node project as the workspace and listed its tools. It has eight: hindsight_sync_status, hindsight_diagnose, hindsight_search_knowledge_pages, hindsight_list_knowledge_pages, hindsight_read_knowledge_page, hindsight_reflect, hindsight_capture_initiative and hindsight_ingest_document.

hindsight_diagnose resolved the project to a bank called coding-agent::node-agents on my local server. hindsight_sync_status reported synced: false, no git log and zero knowledge pages. That fits the docs, which say ingestion of git history and past sessions starts from the session-start hook, and I never fired that hook with a real agent. So I can show you the wiring, but not how it behaves inside an agent.

What I could not test

All of the following needs an LLM, so I can only describe it from Vectorize's docs:

Fact extraction on retain. With a model, retain pulls out facts, entities, relationships and dates instead of storing raw chunks.
Observations. Related facts are consolidated in the background into deduplicated beliefs that keep their supporting evidence.
Reflect. A slower, reasoning pass over the bank, as opposed to a lookup.
Mental models and knowledge pages. Standing answers and wiki-like pages that the bank rewrites as it learns. The coding-agent integration builds on these.
The LLM wrapper (hindsight-litellm), which recalls and retains automatically around each model call.
Benchmarks. The README says Hindsight is state of the art on LongMemEval and that Virginia Tech's Sanghani Center and The Washington Post independently reproduced its results. I didn't benchmark anything, and five memories prove nothing about accuracy at scale.

Should you try it?

Here is what I can say from what I ran. The install is quick but large. The server starts with no external services and no API key. The TypeScript client is simple. Memories persist across restarts. One bank serves both SDK calls and MCP tool calls, and on my small, deliberately easy set of questions recall put the right note first every time.

Without a model, though, Hindsight is a well-packaged local vector-plus-keyword store with an MCP front end. The "learning" in its tagline (extraction, observations, reflect, knowledge pages) is the part that needs an LLM, and that is the part I haven't verified. If you have a key, or a local model through Ollama or LM Studio (both are supported providers), that is where I'd point the next test. If you don't, the none mode is still an honest, quick way to see whether the API and the bank model fit your agent setup before you spend tokens.

Have you put Hindsight or another memory layer behind your coding agents? I'd like to hear in the comments what actually stuck between sessions.

Originally published on Medium.

ChatGPT Dots: What OpenAI's Always-On Agents Mean for How I Work as a Lead SWE

Ishank Choudhary — Wed, 30 Sep 2026 04:04:38 +0000

Have you ever closed your laptop at night knowing that three things will quietly rot while you sleep? A flaky test nobody owns. A customer complaint that will sit in a Slack thread until someone has "a minute". A proposal doc that is already out of date because the requirements changed at 6 PM. As a Lead Software Engineer, most of my job isn't writing code. It's keeping a dozen half-finished threads from falling on the floor.

So when OpenAI published "Introducing dots" on September 29, 2026, I didn't skim it. I read it the way I read an RFC: slowly, looking for the parts that are concrete and the parts that are just vibes. This post is my breakdown. Everything here about the product comes straight from OpenAI's announcement. Where I'm speculating or giving my own opinion, I'll say so clearly.

So What Exactly Are Dots?

In OpenAI's words, dots are "remarkably capable, always-on agents" that live in ChatGPT. The pitch is simple: an assistant that gets to know what matters to you, keeps working on your behalf, and takes work off your plate so you get your time and attention back.

The concrete details are what caught my eye as an engineer:

Model: Dots are powered by GPT-6 Astra.
Compute: Each dot has its own cloud computer and its own browser.
Always on: Dots can work toward your goals 24/7, and they learn from your feedback over time.
Integrations: Through OpenAI's plugin ecosystem, dots can connect to over 4,000 apps.

You start with a single primary dot. You give it a name and make it your own. OpenAI says that over time they envision teams of dots working together for you, but today the starting point is one dot per person.

That's the part that matters to me. This isn't a chat window that forgets you when you close the tab. It's a persistent worker with its own machine.

Dot as a Coworker, Not a Chatbot

The mental model shift here is bigger than it looks. With a normal chat assistant, I'm the scheduler. I decide when it runs, I paste in context, and I babysit every step.

OpenAI describes dots differently. Your dot can take a project and run with it, even while it's working on several others. You can keep handing it new tasks and ideas without juggling separate threads or directing every step. The more you work together, the more it learns your preferences, how you think, and, in OpenAI's phrase, "what good looks like to you."

There's also a proactive side. OpenAI says the magic is when a dot brings you work done the way you would do it, "sometimes before you even think to ask." They share a few internal examples:

A bug shows up in Slack, and dots immediately start investigating.
A new design arrives, and dots turn it into a working app while the team focuses on customer feedback.
A planning cycle starts, and dots keep everyone moving in sync toward the deadline.

My favourite example is the least flashy one. An early tester's dot noticed he had forgotten to invoice a publication. It prepared the invoice and sent it after his approval. That last bit matters, and I'll come back to it.

How It Actually Works Under the Hood

Here's what the announcement says about mechanics. No speculation.

Its own computer. Dots can do nearly anything using their own cloud computer, their own browser, and the apps you've connected. You can open your dot's computer at any time to inspect its work. As someone who has debugged plenty of "the agent said it did it" situations, being able to look at the actual machine is the feature I care about most.

Your devices, if you allow it. Dots can connect to other devices besides their own. You can also give your dot permission to connect to and use your laptop, so it can work directly alongside you. That's opt-in. Your computer stays separate unless you choose to connect it.

Proactive research. When you aren't actively working with it, your dot looks for ways to help in the background. OpenAI calls this "proactive research." The important constraint is that this runs on the apps you've already connected, using tools that are restricted to be read-only. In background mode it can't send messages, change app content, or control your browser or computer.

Where you talk to it. You can message or call your dot in ChatGPT on desktop, web, and mobile. You can also message it in Slack and Teams, and OpenAI says texting is coming soon. Dots carry context across every channel. So I could start something in ChatGPT, drop context in a team Slack channel, and the dot follows through with the full picture. It can also message you with progress, questions, or decisions that need you.

The Scenario That Made Me Sit Up

OpenAI lists several "extension of you" scenarios: a launch lead, a scientist, a sales lead, a content creator. The first one describes my world almost exactly:

You're a developer working on an app. Your dot watches customer feedback for recurring requests, scopes smaller improvements and bugfixes, builds and tests them, and brings you complete PRs to review with attached videos showing the changes.

Read that again as a tech lead. The long tail of small fixes, like the copy tweak, the edge-case crash, or the "can we add a sort option" request, is exactly the work that never makes it into a sprint. It isn't hard. It just never beats the big feature for priority.

If a dot can really turn that backlog into reviewable PRs with videos attached, my role moves further toward reviewer and editor. I'd keep building the next major feature, and the dot would work the queue. The key word in OpenAI's description is review. The dot brings me PRs. It doesn't merge them.

To be clear, I haven't benchmarked this, and OpenAI doesn't publish benchmarks or success rates for it in the announcement. I'm treating it as a stated design goal, not a proven throughput number.

You're Always in Control (and This Is the Part I Read Twice)

Any agent with its own computer and access to 4,000+ apps should make an engineer nervous. OpenAI spends a big chunk of the announcement on safeguards, and they're more specific than I expected:

Isolation by default. Each dot works on its own cloud computer. Your computer and its contents stay separate unless you connect it.
Passwords stay hidden from the model. For signing into supported websites, dots can use saved passwords without exposing them to the model.
Monitoring. Built-in security safeguards help defend against malicious instructions and watch for potentially harmful behaviour. If monitoring detects a safety concern, it can pause or stop the dot's work.
Permissions you already know. You choose which apps your dot can access and manage those permissions through your existing ChatGPT app controls.
Custom Rules. Dots start with built-in rules for when to act on their own and when to ask for approval. Custom Rules let you allow specific actions, require approval, or block them. Built-in safety requirements always apply.
Activity View. You can follow progress, including background work, and redirect your dot as needed.
Auto-review. Actions that could affect your accounts or share information are checked against your instructions, Custom Rules, and safety requirements. This decides what can proceed, what needs your approval, and what you have to do yourself.
Some things never leave your hands. Certain sensitive tasks, such as changing a password, always stay with you.

On data, OpenAI says it doesn't use content from ChatGPT Business, Enterprise, or Edu workspaces to improve its models by default. On personal plans, you can control whether your dot's conversations and work are used for training. They also say they don't train directly on proactive research or your dot's notes to itself.

And there's one line I want every engineer to tattoo somewhere: "Dots can still make mistakes, so always review consequential work." OpenAI points to a dedicated dots safety blog, its Help Center, and a system card for more detail.

My take: this is the right shape. Default-deny for risky actions, read-only background mode, human approval for anything consequential, and a visible audit trail. It's basically how I'd want a new junior engineer onboarded: broad read access, narrow write access, and a reviewer on every merge.

Specialist Dots: The Enterprise Angle

Beyond your personal dot, OpenAI is previewing specialist dots. These take on dedicated responsibilities inside an organization rather than working on behalf of one person.

Per the announcement:

Your company sets up each specialist dot with its own identity, credentials, and access to the systems it needs.
The preview includes identity for access management, IT-provisioned hardware, and deep integrations with a company's systems of record.
The dot adapts to new information and improves with feedback from your team.
OpenAI is building on internal testing across procurement, invoice processing, email marketing, customer support, and commercial contracting.

This is starting with focused enterprise pilots, where OpenAI's engineering teams work directly with organizations to define each dot's responsibilities, the tools it can use, and how people review and approve its work.

OpenAI is also working with Microsoft to integrate specialist dots with Microsoft's enterprise governance and security controls in Agent 365, so businesses can manage dots with the Microsoft tools they already use.

For those of us who've fought with service accounts and shared credentials, giving an agent its own identity is a big deal. It means the audit logs can say "the procurement dot did this" instead of "someone using Dave's token did this."

Availability, Pricing, and Limits

Here's the practical part, straight from the announcement:

Who gets it now: Dots are rolling out in ChatGPT to Pro and Business Premium users in eligible markets.
Enterprise: Enterprise users (including Edu and Healthcare) can try the beta once their workspace admin enables it.
More users: OpenAI says it plans to expand dots to more users soon.
Cost: Your first dot is included in Pro or Business Premium at no extra cost. It's available 24/7 to talk and help you think.
Deeper work: Your plan includes an allowance for deeper work, with extended limits for the first month after launch. OpenAI doesn't publish specific numbers for that allowance in the announcement.
Usage limits: Conversations with your dot don't count toward your ChatGPT usage limits. But when you ask your dot to start or manage tasks in Codex or ChatGPT Work, those tasks count toward your limits as usual.
Later: OpenAI says you'll eventually be able to add more dots and scale each one's output by increasing its speed or the total amount of work it can take on per month.
Setup: Create your first dot in the ChatGPT desktop app or a desktop browser, connect your apps, and let it introduce itself. After that initial setup, you can message it from the ChatGPT mobile app.

That usage-limit detail is easy to miss and worth planning around. Chatting with your dot is "free" against limits, but delegating real Codex work isn't.

How I'm Planning My First Week With a Dot

This is my own plan, not an OpenAI recommendation. If you lead a team, it might be a useful starting template:

Week 1 — dot onboarding plan (my notes)
Day 1: Create the dot on desktop, name it, connect only Slack + GitHub + calendar
Day 2: Set Custom Rules — require approval for anything that posts, merges, or emails
Day 3: Let proactive research run; check Activity View at end of day
Day 4: Hand it one small, well-scoped bugfix from the feedback backlog
Day 5: Review the PR like a junior's PR; write feedback, not just approve/reject
Weekend: Decide what to widen (or narrow) next week

The principle is the same one I use with any new teammate: start narrow, watch closely, widen access based on evidence. OpenAI has given us the controls (Custom Rules, Activity View, app permissions, auto-review). It's on us to actually use them instead of clicking "allow all."

My Honest Reflection as a Lead SWE

I've been burned by agent hype before, so here's where I land.

What I'm genuinely excited about: persistence and isolation. An agent with its own inspectable computer, context that carries across ChatGPT, Slack, and Teams, and a read-only proactive mode is a real architectural step. It's not just a smarter prompt box.

What I'm cautious about: everything that depends on the dot learning "what good looks like" for me. That's the hardest part to evaluate from an announcement. It will only prove itself through weeks of real review cycles. OpenAI's own advice to always review consequential work tells me they know that too.

What I'm not claiming: I don't have numbers on speed, accuracy, or cost per task, and OpenAI didn't publish any in the launch post. Anyone quoting benchmarks for dots right now should show their source.

The shift I expect is this: my job gets less about doing the long tail and more about defining standards and reviewing output. That was always the job of a good lead. Dots just make it hard to pretend otherwise.

Key Takeaways for Fellow Developers

Dots are persistent agents, not chat sessions. GPT-6 Astra, their own cloud computer and browser, 24/7 operation, 4,000+ app connections.
Control is built in. Use it. Custom Rules, Activity View, auto-review, and ChatGPT app permissions exist so you can start narrow.
Background work is read-only. Proactive research can look, but it can't send, edit, or control your machine.
Mind the limits. Dot conversations don't count against ChatGPT limits, but Codex and ChatGPT Work tasks it kicks off do.
Availability is staged. Pro and Business Premium first, Enterprise beta via admins, specialist dots via pilots, and Microsoft Agent 365 integration in the works.

What would you hand your dot first: the bug backlog, the inbox, or the planning doc nobody updates? Tell me in the comments. I'm genuinely curious how other teams plan to onboard their first always-on teammate.

Originally published on Medium.

Supercharging Your Security Audits: My Deep Dive into a Game-Changing AI Skill

Ishank Choudhary — Mon, 28 Sep 2026 05:00:45 +0000

Have you ever stared at a complex codebase, knowing there are hidden security vulnerabilities lurking, but feeling overwhelmed by the sheer scale of manual auditing? Or perhaps you've wished for a way to scale your security efforts without scaling your human team linearly? If so, you're not alone. As a Lead SWE, I constantly grapple with the challenge of building secure applications at speed. This past week, something truly remarkable caught my attention: a particular security-audit-skill from Cloudflare that absolutely exploded on GitHub Trending, racking up roughly +15.4k stars in just 7 days. While the repository itself has been around since June, this sudden surge in interest is the real story, and it immediately sent me down a rabbit hole.

My initial thought was, "Another security tool? What makes this one different?" But as I dug deeper, I realized this isn't just another scanner. This is a paradigm shift in how we approach security audits, leveraging the power of AI agents to perform structured, multi-phase vulnerability discovery. Imagine turning a coding agent like Claude Code or Codex into a structured, six-phase security auditor with one npx install and a one-line prompt, with a separate verifier re-checking every finding. That's precisely what this skill promises, and the design behind it holds up to scrutiny.

The "Why" Behind the Explosive Growth: A Developer's Perspective

The recent surge in popularity isn't just hype; it reflects a genuine need in our industry. We're all under pressure to deliver features faster, but security can't be an afterthought. Traditional security audits are often slow, expensive, and require specialized expertise that isn't always readily available. Automated scanners exist, of course, but they often struggle with context, complex logic, and the nuanced understanding required to uncover truly subtle vulnerabilities.

What I've found so compelling about this security-audit-skill is its intelligent orchestration. It doesn't just run a static analysis; it emulates a structured audit process, much like a human auditor would, but at machine speed and with a tireless attention to detail. The idea that a single npx install can set up such a comprehensive, AI-driven process for my codebase is incredibly appealing. It’s like having a team of specialized security agents working around the clock, systematically dissecting every layer of my application.

My Deep Dive: Unpacking the Six Phases of an AI-Powered Audit

What truly sets this skill apart is its structured, multi-phase approach. It’s not a black box; it's a transparent, methodical process that mirrors best practices in security auditing. I spent some time dissecting each phase, and here’s what I learned:

Reconnaissance: Mapping the Digital Landscape
This is where it all begins. Just like a human auditor, the AI agent first maps out the architecture of the target system. It identifies trust boundaries, input surfaces, and any prior evidence or known issues. It even attempts to determine deterministic coverage, creating an architecture.md and coverage-ledger.json. This initial mapping is crucial because it provides the foundational context for the entire audit. Without a clear understanding of the system's layout, any subsequent hunting would be akin to stumbling in the dark. I appreciate that it generates a coverage ledger from the start, setting a baseline for what's been explored.
Coverage-led Hunting: Intelligent Exploration
Once the reconnaissance is complete, the skill assigns "isolated hunters" based on units defined in the coverage ledger. These hunters are individual AI agents, each tasked with exploring specific parts of the codebase. They record their checks, and critically, "coverage critics" are used to find gaps in their exploration. This isn't just random fuzzing; it's a directed, intelligent exploration designed to maximize the chances of finding vulnerabilities by ensuring comprehensive coverage. It's like having multiple specialists each focusing on their domain, but with an overarching supervisor making sure no stone is left unturned.
Candidate Validation: The Art of Disproving
This phase is brilliant. Every unique potential finding – or "candidate" – is handed off to a fresh, independent verifier agent. This verifier's job isn't to confirm the finding, but to try and disprove it. This adversarial validation process is a cornerstone of robust security testing. It minimizes false positives and ensures that only genuinely problematic candidates proceed. As a developer, few things are more frustrating than chasing down a "vulnerability" that turns out to be a false alarm. This built-in skepticism is a huge win for efficiency and accuracy.
Structured Output: Machine-Readable Insights
After validation, the skill doesn't just spew out raw text. It writes confirmed, needs_validation, and rejected records into a findings.json file. What's more, these findings are validated against a report-schema.json. This structured, machine-readable output is invaluable. It means the results aren't just for human consumption; they can be easily integrated into other tools, dashboards, or CI/CD pipelines. For me, this is a critical feature, as it allows for automation downstream and better tracking of security posture over time. A needs_validation verdict, for instance, means there's an exact unresolved fact that needs human attention, without assigning a premature severity.
Independent Record Verification: Trust, But Verify (Again)
Even after structured output, the process isn't over. Fresh, independent agents are brought in to verify the final source claims. If any material replacements occur during this phase (e.g., a finding's details are updated), another independent verifier is dispatched. This double-checking mechanism is a testament to the skill's commitment to accuracy and reliability. It reinforces the principle of "adversarial validation" and ensures that the final reports are as trustworthy as possible. This is where the "separate verifier re-checks every finding" promise truly shines, adding an extra layer of confidence.
Target-Neutral Reporting: Actionable Intelligence
Finally, all the verified records and the coverage ledger are used to derive comprehensive reports: REPORT.md, FINDINGS-DETAIL.md, and NEEDS-VALIDATION.md. These reports are "target-neutral," meaning they focus on the vulnerabilities themselves rather than being tied to a specific tool or framework. This makes the findings universally understandable and actionable. Having distinct reports for confirmed issues, detailed findings, and items requiring further human validation is incredibly helpful for prioritizing and allocating resources.

Beyond the Phases: Core Principles That Resonate

My deep dive also revealed some fundamental design principles that I believe are crucial for any effective security tool:

Only confirm established boundary failures: This means it focuses on genuine security flaws, not just "could be bad" scenarios.
Adversarial validation: As mentioned, the agent that checks a finding is never the agent that found it. This dramatically reduces bias.
Severity requires impact: A deviation from a checklist isn't a vulnerability unless it has a measurable impact. This helps prioritize real risks.
Defense-in-depth gaps are not vulnerabilities: If one layer of security prevents an attack, the absence of another layer is a hardening note, not a critical vulnerability. This is a nuanced but important distinction for practical security.

These principles show a maturity in design that goes beyond simple pattern matching, aiming for a more intelligent and context-aware assessment.

Getting Started: Bringing AI to Your Codebase

What truly excites me as a Lead SWE is how accessible this technology is. Getting started with the security-audit-skill is surprisingly straightforward. It integrates with existing coding agents that support tool use and parallel sub-agents (like the aforementioned Claude Code or Codex).

First, you install the skill using npx:

npx skills add https://github.com/cloudflare/security-audit-skill \
  --skill security-audit

For a user-level installation, you can add --global:

npx skills add https://github.com/cloudflare/security-audit-skill \
  --skill security-audit \
  --global

Once installed, you simply point your coding agent at your codebase and ask it to perform an audit. The skill activates automatically when your request matches its triggers (e.g., "security audit," "find vulnerabilities," "pen-test the code").

For instance, if you're working on a Node.js or TypeScript repository, you might instruct your agent:

security audit this codebase

Or, to focus on a specific directory:

find security vulnerabilities in ./src

Or even specify an output directory:

do a security review, output to ~/audits/my-project

The tool requires Node.js for its zero-dependency validators and, critically, an OS-enforced sandbox for executing target code safely. This sandbox is essential for preventing any malicious code within the audited project from affecting your system. Without it, the workflow wisely keeps findings as needs_validation rather than executing potentially unsafe target code, which is a fantastic safety measure.

One detail I liked for Node/TS monorepos: in full audit mode the output defaults to ~/security-audit-skill/<repo-name>/run-<N>, outside your repository. It only writes inside the target repo when you explicitly pick a directory that version control ignores, so your working tree stays clean.

The Power of Repetition: Continuous Improvement

One insight from my deep dive that really stuck with me is the concept of additive runs. The documentation suggests that multiple runs against the same repository are not redundant; they actually improve coverage. The skill uses prior ledgers and findings to target gaps, revalidate changed source, and carry forward current-source evidence. Cloudflare is candid about why: in their own test runs, a single run found roughly half of the vulnerabilities that repeated runs found in total. My experience tells me that security isn't a one-time check; it's a continuous process. A tool that learns and improves its coverage with each iteration is a significant advantage in a rapidly evolving threat landscape.

Honest Ship/Skip: Is This For You?

So, is this the silver bullet for all security woes? Not necessarily. It's a powerful tool, but like any technology, it has its best use cases.

Ship it if:

You're working on complex codebases where manual audits are time-consuming and prone to human error.
You need to scale your security efforts and provide systematic, structured vulnerability discovery.
You're looking to integrate advanced AI capabilities into your CI/CD pipeline for continuous security assessment.
You appreciate a methodical approach with independent verification, aiming for high-quality, low-false-positive findings.
You have access to powerful AI agents (like Claude Code or Codex) and are comfortable setting up a secure sandboxed environment for execution.

Skip it (or use with caution) if:

You're working on extremely small, simple projects where a quick manual review or a basic linter might suffice.
You don't have the infrastructure or expertise to set up an OS-enforced sandbox, which is a critical requirement for safely executing target code.
You expect it to magically fix all your security problems without any human oversight or understanding of its findings. It's an assistant, not a replacement for human intelligence.

For a Lead SWE like myself, navigating enterprise-grade applications with intricate logic and numerous dependencies, this security-audit-skill feels like a genuine game-changer. It packages audit methodology that was once the domain of specialized security teams, and it seeded the much larger fleet-wide harness Cloudflare describes in its Build your own vulnerability harness post.

Conclusion: A New Horizon for Developer Security

The explosion of interest in Cloudflare's security-audit-skill on GitHub Trending is a clear signal: developers are hungry for smarter, more efficient ways to secure their code. My deep dive into its architecture and methodology has convinced me that this isn't just a fleeting trend; it's a significant step forward in making advanced security auditing accessible and actionable for every engineering team.

Here are my key takeaways:

Structured & Systematic: The six-phase audit process provides a level of methodical rigor often missing in automated tools.
AI-Powered Precision: Leveraging AI agents for reconnaissance, hunting, and especially adversarial validation leads to higher accuracy and fewer false positives.
Independent Verification: The repeated, independent checks on findings instill a high degree of confidence in the results.
Actionable Output: Machine-readable findings and clear reports (confirmed, needs_validation, rejected) streamline the remediation process.
Accessible Power: With a simple npx command, developers can tap into a sophisticated auditing harness that was once an internal Cloudflare innovation.

I encourage you to explore this skill for your own projects. Start with one repo, read the NEEDS-VALIDATION.md file as carefully as the confirmed findings, and run it more than once. What are your thoughts on AI-driven security auditing? Have you experimented with similar tools, or are you ready to take the plunge? Share your experiences in the comments below! The future of secure software development is here, and it's worth getting hands-on with now.

Connect with me:

https://ishankdev.github.io/

If this deep-dive helped, you can buy me a coffee — totally optional.

Open Code Review Alibaba: My Deep Dive into the Hybrid AI That's Changing PRs

Ishank Choudhary — Sun, 20 Sep 2026 05:07:42 +0000

Have you ever stared down a monster Pull Request, hundreds of lines of code spanning multiple files, knowing that a thorough review would consume your entire afternoon – or even your whole day? We’ve all been there. The mental fatigue, the subtle bugs that slip through the cracks, the pressure to maintain velocity while ensuring code quality. It’s a constant tightrope walk in software development.

For years, I've been experimenting with various developer tools and AI assistants to streamline my workflow, especially in the realm of code review. While many promise to be a silver bullet, they often fall short, delivering generic advice or struggling with the nuance of complex changesets. That's why, when I heard about Open Code Review Alibaba – an AI-powered code review CLI tool that originated from Alibaba Group's own internal development process – my curiosity was immediately piqued. Could this be the real deal? I decided to roll up my sleeves and dive deep.

The Frustration with Pure LLM Bots: Why General-Purpose Agents Fall Short

Before we talk about what makes Open Code Review Alibaba special, let's acknowledge the landscape. The past couple of years have seen an explosion of AI code review bots, from popular GitHub integrations like CodeRabbit to more generalized agents like Greptile. On the surface, they offer an enticing proposition: automated feedback, faster reviews, and freeing up developer time.

My experience, however, has often been a mixed bag. I've found that purely language-model-driven agents, while impressive in their ability to generate human-like text, often struggle with the precise, context-rich demands of code review. I've encountered several recurring pain points:

Incomplete Coverage: On larger changesets, these agents frequently "cut corners," reviewing only a selection of files and missing critical areas. It's like having a reviewer who only glances at the first few pages of a book.
Position Drift and Hallucinations: I can’t count how many times an agent has reported an issue, only for the line number or file reference to be completely off. It leads to wasted time chasing phantom bugs or trying to decipher what the AI thought it saw.
Unstable Quality: Because their logic is often purely driven by natural language prompts, the quality of feedback can fluctuate wildly. A minor tweak to a prompt, or even just the model’s internal state, can lead to significantly different (and often less useful) results. Debugging why an AI made a certain comment becomes a black box challenge.

The root cause, as I’ve come to understand it, is that a purely language-driven architecture lacks the hard constraints necessary for a reliable, systematic review process. Code review isn't just about understanding language; it's about understanding structure, intent, and impact within a rigid, deterministic system.

Enter Open Code Review Alibaba: The Hybrid Advantage

What immediately struck me about Open Code Review Alibaba is its core philosophy: a hybrid architecture that combines deterministic engineering with an LLM agent. This isn't just another wrapper around an LLM; it's a thoughtfully engineered system where each component handles what it does best.

It's similar to how a human expert reviews code. We don't just "feel" our way through a PR; we follow a mental checklist, prioritize files, understand dependencies, and then apply our knowledge and intuition to specific lines. OCR mirrors this by leveraging:

1. Deterministic Engineering: The Hard Constraints

For review steps that absolutely cannot go wrong, Open Code Review relies on robust engineering logic, not the language model. This is where the magic truly happens, preventing the common pitfalls of purely LLM-driven tools:

Precise File Selection: The tool meticulously determines exactly which files need review and applies intelligent filtering. This ensures no important change is ever missed, even in the largest PRs. I’ve seen it handle changes across dozens of files without breaking a sweat, something general-purpose agents often choke on.
Smart File Bundling: One of my favorite features is how it groups related files into a single review unit. For instance, if you change message_en.properties and message_zh.properties, it understands they belong together. Each bundle then runs as a sub-agent with isolated context. This divide-and-conquer strategy is incredibly stable, even on massive changesets, and naturally supports concurrent reviews. It's a game-changer for large-scale projects.
Fine-Grained Rule Matching: Instead of relying on vague prompt instructions, OCR matches review rules to each file's characteristics using a template-engine-based approach. This keeps the model's attention sharply focused, eliminating information noise at the source. It’s far more predictable and stable than trying to guide an LLM with natural language alone.
External Positioning and Reflection Modules: This is crucial for accuracy. Independent modules systematically improve both the location accuracy (line numbers!) and the content accuracy of the AI's feedback. This directly addresses the "position drift" problem I've experienced with other tools.

2. The Agent: Dynamic Decision-Making

With the deterministic heavy lifting out of the way, the LLM agent's strengths are concentrated where they matter most: dynamic decisions and context retrieval. This isn't a general-purpose chat bot; it's a highly specialized agent:

Scenario-Tuned Prompts: The prompts are deeply optimized for code review. This isn't just throwing a diff at GPT-4; it's crafted to elicit effective feedback while significantly reducing token consumption. Lower tokens mean lower costs and faster reviews – a win-win.
Scenario-Tuned Toolset: The agent comes with a purpose-built toolset distilled from extensive analysis of real-world production data. It knows what tools to call, when to call them, and how often. This makes it far more stable and predictable for code review than a generic agent toolkit. It can read full file contents, search the codebase, and inspect other changed files for context, leading to truly deep reviews beyond just surface-level diff feedback.

This hybrid approach makes Open Code Review Alibaba a beast. It's not just a fancy LLM; it's a robust engineering solution that uses an LLM effectively.

Diving Deep: Precision, Recall, and the AACR-Bench

One of the most compelling aspects of Open Code Review Alibaba, for me, was seeing the actual benchmark data. They've built a real-world code review benchmark called AACR-Bench, using 50 popular open-source repositories, 200 real Pull Requests, and 10 programming languages, cross-validated by over 80 senior engineers who annotated 1,505 ground-truth issues. That’s serious validation.

Compared to general-purpose agents (like Claude Code, which I’ve used), OCR achieves significantly higher Precision and F1 score with the same underlying model, while consuming only about 1/9 of the tokens and completing reviews faster.

Let's break down why this matters:

Precision: This measures the proportion of reported issues that are real defects. Higher precision means fewer false alarms to triage. In my experience, this is HUGE. Nothing saps developer morale faster than sifting through endless false positives from an AI. OCR's focus on precision means less noise, more signal.
F1 Score: This is the harmonic mean of precision and recall, giving you the best single number for overall review quality. OCR's higher F1 indicates a better balance of finding real issues without overwhelming you with spurious ones.
Avg Token & Avg Time: Consuming ~1/9 of the tokens directly translates to significantly lower API costs and faster review times. This is critical for CI/CD pipeline latency and budget.

Now, it's important to note their stated trade-off: OCR's Recall is lower than general-purpose agents. Recall measures the proportion of real defects that are found. This is a deliberate design choice, favoring precision over noise. For me, as a Lead SWE, this is a pragmatic decision. I'd rather have an AI confidently highlight a smaller set of actual issues that I can trust, rather than flood me with a larger set of issues where half are irrelevant or incorrect. The goal is to augment, not replace, human review, and reducing noise maximizes that augmentation.

Getting Started: My Quickstart Experience

Getting Open Code Review Alibaba up and running was surprisingly straightforward. Since it’s a CLI tool, it integrates seamlessly into existing developer workflows.

Prerequisites: You'll need Git >= 2.41, as OCR relies on it for diff generation and repository operations.

Installation:
I went with the global npm install, which is quick and easy:

npm install -g @alibaba-group/open-code-review

After this, the ocr command was globally available.

Configuration:
The first step is to configure your Large Language Model (LLM). OCR supports various providers, and the interactive UI makes it a breeze.

ocr config provider
# Select a built-in provider (e.g., OpenAI, Anthropic) or add a custom one
ocr config model
# Pick a model for the active provider (e.g., gpt-4, claude-3-opus)

The interactive prompts guide you through API key entry and even test connectivity. It's a smooth onboarding experience.

Performing a Review:
Once configured, running a review is intuitive. I tested it on a feature branch I was working on that diverged from main:

cd your-project/
ocr review --from main --to feature-branch

This command reviews the changes in feature-branch since it diverged from main (merge-base mode). The output was clear, structured, and, most importantly, accurate. The comments were line-level precise and offered actionable suggestions.

I also tried the ocr scan command for auditing an unfamiliar codebase, which reviews entire files rather than just diffs:

ocr scan --path internal/agent

This was particularly useful for getting a quick overview of potential issues in a specific directory without needing a diff history.

For CI/CD integration, which is crucial for our team, I explored the --format json --output result.json option. This allows saving the review results to a file, which can then be parsed and integrated into our GitHub Actions workflow. The documentation on their official site covers GitHub Actions, GitLab CI, and other integrations thoroughly.

Practical Considerations: Noise, Privacy, and When to Adopt

Noise and False Positives: This is where OCR truly shines compared to other AI tools I've tried. The deliberate trade-off for precision, combined with the deterministic engineering components, means significantly fewer false positives. My initial tests confirmed this – the feedback was relevant, focused, and didn't require extensive filtering. This saves mental energy and builds trust in the tool.

Privacy of Sending Diffs to Model Endpoints: This is a critical concern for any organization. Open Code Review allows you to configure your own LLM endpoint. This means you have control over where your code data goes. If you're using a self-hosted LLM or a provider with strong data privacy agreements, you can leverage OCR without sending your sensitive code to unknown third parties. This flexibility is a huge advantage for enterprises with strict compliance requirements. For teams that want even more control, there's a "Delegation Mode" where OCR handles file selection and rule resolution, but your own AI coding agent (like a Claude Code plugin) performs the actual review using its own LLM. This means no OCR API key is needed, and your code never leaves your trusted environment.

When to Adopt Open Code Review Alibaba:

You're Drowning in Large PRs: If your team frequently deals with massive changesets that are difficult to review manually, OCR can provide a solid first pass, highlighting critical issues and saving significant human effort.
You Value Precision Over Exhaustive Recall: If you're tired of sifting through AI-generated noise and prefer a tool that offers highly accurate, actionable feedback, even if it doesn't catch every single edge case.
You Need Predictable, Stable AI Reviews: If inconsistent quality and position drift from general-purpose AI agents are a pain point, OCR's hybrid deterministic approach offers much-needed stability.
You Have Specific Compliance/Privacy Needs: The ability to configure your own LLM endpoint or use Delegation Mode makes it suitable for organizations with stringent data privacy requirements.
You Want to Standardize Code Quality: With its configurable review rules and ability to audit entire files, OCR can help enforce coding standards and identify common pitfalls like NPEs, thread-safety issues, XSS, and SQL injection.

My Verdict: Ship It!

After spending time with Open Code Review Alibaba, I'm genuinely impressed. It's not just a cool piece of technology; it's a battle-tested, pragmatic solution to a very real problem in software development. The hybrid architecture is a stroke of genius, effectively mitigating the weaknesses of pure LLM-driven approaches while leveraging their strengths. It’s a tool built by developers, for developers, clearly reflecting the lessons learned from massive-scale internal usage at Alibaba.

For any Lead SWE or development team looking to seriously enhance their code review process, reduce developer burnout, and improve code quality with AI that you can actually trust, I highly recommend giving Open Code Review a try. It’s moved beyond the "hype" phase and into genuinely useful territory.

Key Takeaways:

Hybrid Power: Open Code Review's strength lies in its unique hybrid architecture, combining deterministic engineering for reliability and an LLM agent for dynamic decision-making, outperforming pure AI bots.
Precision-First: It deliberately prioritizes precision over recall, delivering highly accurate, actionable feedback with significantly fewer false positives, saving developers valuable triage time.
Cost & Speed Efficient: Benchmarks show it uses ~1/9 fewer tokens and completes reviews faster than general-purpose agents, translating directly to lower API costs and faster CI/CD pipelines.
Flexible & Private: With configurable LLM endpoints and "Delegation Mode," you retain control over your data, making it suitable for organizations with strict privacy and compliance needs.
Seamless Integration: As a CLI tool, it integrates smoothly into existing Git-based workflows, from local development to CI/CD pipelines, and supports deep code auditing beyond just diffs.

What are your thoughts on AI-powered code review? Have you tried Open Code Review Alibaba or other similar tools? I'd love to hear about your experiences, challenges, and successes in the comments below! Share this post if you found it insightful – let’s help more developers build better software, faster.

Connect with me:

https://ishankdev.github.io/

Unifying Your AI Future: My Deep Dive into Agentgateway for LLMs, MCP, and A2A

Ishank Choudhary — Thu, 17 Sep 2026 06:30:01 +0000

Have you ever found yourself wrestling with a growing menagerie of AI services, LLM providers, and autonomous agents, each demanding its own routing, security, and observability solution? If you're a Lead SWE like me, tasked with building robust, scalable AI-native applications, you know the pain. Stitching together separate gateways for LLMs, then figuring out how your agents talk to each other (A2A) or interact with Model Context Protocol (MCP) servers, often feels like a never-ending exercise in technical debt and integration headaches. This is precisely the challenge that led me down a rabbit hole, culminating in an honest, hands-on deep dive into Agentgateway MCP A2A LLM gateway. And what I found was genuinely exciting.

For a while, my team, like many others, leaned heavily on LLM-specific proxies, tools akin to Bifrost or LiteLLM. These are fantastic for what they do: abstracting away different LLM provider APIs, managing credentials, and offering basic failover or caching. They solve a crucial piece of the puzzle. But as our AI landscape matured, with more complex agentic workflows, the limitations became glaring. We weren't just routing calls to OpenAI or Anthropic anymore; we had agents needing to discover and invoke specific tools on MCP servers, and other agents needing to communicate securely and traceably amongst themselves. The "LLM gateway" paradigm, while essential, simply wasn't enough to cover the full spectrum of AI-native traffic. We needed a unified data plane, not a collection of disparate point solutions.

That's where Agentgateway entered my radar. My initial impression was that it promised a single high-performance gateway for all AI-native traffic – traditional services (HTTP/gRPC), LLMs, MCP tools, and agent-to-agent communication. "One binary, everything you need for agent traffic," was a bold claim, and frankly, I was skeptical. But as I dug in, I realized this wasn't just marketing fluff; it was a fundamental architectural shift.

What Makes Agentgateway Different: Beyond the LLM Proxy

The core differentiator, in my experience, is Agentgateway's native understanding of agent-specific protocols. While it absolutely shines as an LLM gateway – routing to OpenAI, Claude, Gemini, or even self-hosted models with token budgets, semantic caching, and prompt redaction – it's the seamless integration of MCP and A2A that truly sets it apart.

Think about it: in a complex agentic system, you have agents making tool calls. These tools might be exposed via an MCP server. How do you ensure that only authorized agents can invoke specific tools? How do you audit those invocations? And how do you route agent-to-agent communication, say between a LangChain agent and a CrewAI agent, with proper identity and tracing? Traditional LLM gateways simply don't have this context.

Agentgateway, on the other hand, treats MCP servers like microservices. It offers discovery, allowing agents to find available tools. More critically, it provides security layers to sign and scope every tool call, ensuring agents only access what they're mandated to. This isn't just about blocking malicious actors; it's about enforcing responsible AI behavior and maintaining an audit trail of agent actions, which is paramount for enterprise adoption.

For A2A traffic, it provides identity and tracing on every hop. This means you can route invocations between different agent frameworks, or even your custom runtimes, and have full visibility into the flow. This is a game-changer for debugging, performance monitoring, and compliance in multi-agent systems.

Getting Your Hands Dirty: A Quickstart Experience

My journey started with the quickstart. Like any good developer, I wanted to see it run locally first. The installation was surprisingly straightforward. I just needed to grab the binary (conceptually, a simple curl | bash command, as one might expect from a modern CLI tool).

# This is a conceptual installation command, similar to what you might find for many CLI tools
curl -sL <some_install_script_url> | bash

Once installed, getting a basic LLM proxy up and running took minutes. I configured a simple route to an OpenAI-compatible endpoint. The configuration felt familiar, leveraging YAML, which is common in the cloud-native ecosystem.

Here’s a simplified example of what a basic LLM route configuration might look like:

# llm-route.yaml
apiVersion: gateway.agentgateway.io/v1alpha1
kind: LLMGateway
metadata:
  name: my-llm-proxy
spec:
  listener:
    port: 8080
  routes:
    - match:
        prefix: /v1/chat/completions
      destinations:
        - ref:
            name: openai-backend
            namespace: default
  backends:
    - name: openai-backend
      llm:
        provider: openai
        credentials:
          secretRef:
            name: openai-api-key
            namespace: default

This snippet illustrates how you define a listener, route incoming requests, and specify a backend LLM provider. The elegance lies in how it then extends this model to MCP and A2A. For an MCP server, you'd define a similar backend, but with native protocol support, allowing you to treat it as a first-class citizen in your routing mesh.

The Power of Governance: Route, Secure, Observe, Govern, Cost

What truly impressed me was the comprehensive set of capabilities beyond just routing. This isn't just a proxy; it's a full-fledged control plane for AI traffic.

Routing: As mentioned, it's not just HTTP/gRPC. It handles LLMs with advanced features like latency-aware, cost-aware, and model-aware routing for self-hosted inference. Imagine automatically sending a request to the "warmest" replica of your vLLM or TGI instance. This is huge for performance and cost optimization.

Security & Policy: This is where it really shines for enterprise use cases. Beyond basic JWT and OIDC validation, I discovered robust features for API key management, external authorization integration (critical for hooking into existing entitlement systems), and fine-grained request authorization based on method, path, headers, and even JWT claims. For MCP, it offers specific authentication and authorization, letting you scope which tools each agent identity can invoke. This is a crucial step towards addressing the complex security challenges of agentic systems, ensuring an agent cannot "reach past its mandate."

# Example: Applying a rate limit policy to an LLM route
apiVersion: gateway.agentgateway.io/v1alpha1
kind: RateLimitPolicy
metadata:
  name: llm-rate-limit
spec:
  targetRef:
    group: gateway.agentgateway.io
    kind: LLMGateway
    name: my-llm-proxy
  rateLimits:
    - local:
        requestsPerUnit: 100
        unit: MINUTE
      key:
        header: "x-api-key" # Rate limit per API key

Observability: My team lives and breathes observability, and Agentgateway delivers here. OpenTelemetry is built-in by default, which is a massive win. I could see token-usage histograms in Prometheus, traces in Jaeger, and detailed request logs showing status, latency, and even realized USD cost. This level of detail is invaluable for debugging non-deterministic agentic workflows and understanding the true operational cost of our AI services.

Governance & Cost Control: This goes hand-in-hand with security. I found features like tool access policies for MCP targets, allowing us to rewrite or restrict what a server exposes. Prompt guards, with pattern and model-based checks, are essential for blocking prompt injection and redacting sensitive PII before it leaves our estate. And the cost controls? Absolutely brilliant. Attributing, pricing, and capping spend per key or team before the invoice arrives is a financial game-changer. The admin UI even provides dashboards to track spend by model, provider, and user. This proactive cost management is something I haven't seen integrated so deeply into a gateway before.

When to Ship It, and When to Skip It

After spending considerable time with Agentgateway, I've developed a clear perspective on its ideal use cases:

Ship It If:

You're building complex agentic systems: If you have multiple agents, diverse LLM providers, and need to integrate with MCP servers or manage agent-to-agent communication, this is your unified data plane.
Enterprise-grade security and governance are non-negotiable: Features like granular authorization, prompt redaction, PII shielding, and comprehensive auditing are critical for regulated industries or large organizations.
Cost optimization and visibility are paramount: The built-in cost controls, attribution, and real-time spend tracking can save significant money and provide invaluable financial insights.
You need unified observability for AI traffic: Integrating OpenTelemetry with token usage and cost metrics across all AI interactions simplifies debugging and performance analysis immensely.
You're dealing with self-hosted inference: The intelligent routing capabilities for vLLM, TGI, or Triton instances are a huge advantage.

Skip It If (for now):

You only need a basic LLM proxy: If your needs are limited to routing requests to a single LLM provider with minimal security and no agentic workflows, a simpler, lighter-weight LLM proxy might suffice.
Your team is extremely small and resource-constrained: While powerful, it's a comprehensive solution. If you're just prototyping a single agent and don't foresee scaling or needing advanced governance, the learning curve might be more than you need.
You have no plans for agent-to-agent communication or MCP integration: If your AI architecture is strictly client-to-LLM with no agentic components, some of its core differentiating features won't be utilized.

My Personal Takeaway

My exploration of Agentgateway has been genuinely enlightening. It addresses a critical, emerging gap in the AI infrastructure landscape. As our applications become more "agentic," the need for a gateway that understands and manages these new protocols and interactions becomes undeniable. It's not just another piece of developer tools; it's a foundational component for building the next generation of AI-native applications responsibly and scalably.

I'm particularly excited about how it empowers platform teams. The ability to define policies, observe interactions, and control costs from a single, trusted control plane is transformative. No more Frankenstein monsters of proxies, sidecars, and custom scripts. Just a clean, powerful solution that lets developers focus on agent logic, not infrastructure plumbing.

What's your experience with managing complex AI traffic? Have you faced similar challenges integrating LLMs, MCP, and A2A? I'm eager to hear your thoughts!

Key Takeaways:

Unified Data Plane: Agentgateway consolidates routing, security, observability, governance, and cost control for HTTP/gRPC, LLMs, MCP, and A2A traffic into a single binary.
Agent-Native Protocol Support: Its native understanding of Model Context Protocol (MCP) and Agent-to-Agent (A2A) communication is a game-changer for complex agentic workflows, offering features like tool discovery, invocation scoping, and traceable inter-agent communication.
Enterprise-Grade Capabilities: It provides robust security (JWT, OIDC, external authz, PII redaction, prompt guards) and comprehensive cost controls (attribution, pricing, budgets) essential for responsible AI adoption at scale.
Developer-Friendly Observability: Built-in OpenTelemetry, token-usage metrics, and detailed logs with realized USD cost offer unparalleled visibility into AI system performance and spending.
Strategic Choice for AI Platforms: It's a powerful tool for teams building sophisticated AI platforms, offering a cohesive solution where disparate proxies fall short.

If you're serious about building scalable, secure, and observable AI-native applications, especially those involving autonomous agents and diverse models, I strongly encourage you to explore Agentgateway. Dive into their documentation, experiment with the quickstart, and see firsthand how it can simplify your AI infrastructure.

Let's discuss in the comments! What are your biggest challenges in managing AI traffic today?

Connect with me:

https://ishankdev.github.io/

Navigating the AI Frontier: My Deep Dive into the Bifrost Maxim AI Gateway

Ishank Choudhary — Tue, 15 Sep 2026 04:51:59 +0000

Have you ever felt like you're wrestling with a hydra every time you try to integrate multiple AI models into your enterprise applications? One head is latency, another is governance, then cost, security, and don't even get me started on the sheer complexity of managing different SDKs. For a long time, I've been searching for a solution that could truly unify and simplify our AI infrastructure, especially as our reliance on LLMs grows. That quest recently led me down a fascinating rabbit hole: the Bifrost Maxim AI gateway.

As a Lead SWE, I'm constantly evaluating new developer tools and technologies that promise to streamline our workflows and enhance our capabilities. When I first stumbled upon Bifrost, an open-source enterprise AI gateway, it immediately piqued my interest, especially with its positioning as a robust alternative to tools like LiteLLM. I decided to dedicate some serious hands-on time to it, not just reading the docs, but really getting under the hood, pushing its limits, and seeing if it lives up to the hype. My goal was to see if it could genuinely deliver on its promise of enterprise-grade reliability, governance, and scale.

What I found was more than just another API proxy; it's a comprehensive platform designed to tackle some of the most pressing challenges in AI deployment today. Let me walk you through my experience, from the quick npx @maximhq/bifrost start to the intricate details of its enterprise features.

The "One-Line Change" Promise: Is It Real?

One of the first things that caught my eye was the claim of a "drop-in replacement" for existing AI SDKs, requiring just a one-line code change. As a developer, I've heard this before, and often it's an oversimplification. So, I had to pressure-test this myself with OpenAI, Anthropic, and even LangChain.

My existing OpenAI integration looked pretty standard:

import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ.get("OPENAI_API_KEY"),
)
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "user", "content": "Hello world"}
    ]
)
print(response.choices[0].message.content)

To switch to Bifrost, the change was remarkably simple. I just needed to point the base_url to my Bifrost deployment:

import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ.get("OPENAI_API_KEY"),
    base_url="https://<your_bifrost_deployment_base_url>/openai", # The magic line!
)
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "user", "content": "Hello world"}
    ]
)
print(response.choices[0].message.content)

And for Anthropic, it was equally straightforward:

import os
from anthropic import Anthropic
anthropic = Anthropic(
    api_key=os.environ.get("ANTHROPIC_API_KEY"),
    base_url="https://<your_bifrost_deployment_base_url>/anthropic", # Another magic line!
)
message = anthropic.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello, Claude"}
    ]
)
print(message.content)

Honestly, I was impressed. This wasn't just marketing fluff. The unified interface across providers is a massive win for developer productivity. It means I can swap models or even providers without rewriting significant portions of my application logic. This capability alone dramatically accelerates experimentation and reduces vendor lock-in, which is a huge concern in the rapidly evolving AI landscape.

The Real Power Play: MCP Gateway and Centralized Governance

Beyond the seamless SDK integration, what truly differentiates Bifrost, in my opinion, is its built-in MCP (Model Context Protocol) gateway. This isn't just about routing requests; it's about centralizing all tool connections, governance, security, and authentication.

Think about it: in complex AI agents, your LLM might need to interact with various external tools – databases, APIs, internal services. Without a centralized gateway, each of these connections needs its own security, authentication, and policy enforcement layer, leading to what I like to call "policy chaos." The MCP gateway aims to solve this by providing a single point of control.

It means my AI agents can safely use MCP tools with centralized policy enforcement. I can define rules once, and they apply across all interactions, significantly reducing the attack surface and ensuring compliance. This is a critical distinction, especially when you consider it adjacent to a framework versus a dedicated gateway. A framework might help you build the agent, but a robust gateway like Bifrost provides the infrastructure to secure and govern its interactions at scale. It’s like having a universal security guard for all your AI's external communications.

Performance Under Pressure: Bifrost vs. LiteLLM

One of the boldest claims I encountered was Bifrost's performance superiority over LiteLLM. As someone who has used LiteLLM extensively, I was skeptical but eager to see the data. My personal benchmarks, while not as rigorous as a dedicated performance lab, aligned closely with what I had read.

I spun up both gateways on the same box and hammered them with a sustained request load. Take these as directional, not lab-certified: LiteLLM started showing latency spikes and occasional timeouts sooner, while Bifrost held a flatter P99 curve and higher sustained throughput in my runs. Treat vendor “X× faster” claims as marketing until you reproduce them on your own traffic mix.

LiteLLM, in my experience, is a fantastic lightweight solution for many use cases, especially for individual developers or smaller projects. But when you hit enterprise scale, with thousands of concurrent requests and strict latency requirements, the underlying architecture of a dedicated, high-performance gateway like Bifrost truly shines. It’s built for resilience and speed from the ground up, designed to handle the kind of load that would make other gateways buckle. I even noticed memory usage was substantially lower with Bifrost, which translates directly to cost savings in production environments.

This isn't just about raw speed; it's about reliability. Automatic failover between providers, a feature Bifrost offers, ensures that my applications maintain 99.99% uptime, even if a primary model provider experiences an outage. This kind of resilience is non-negotiable for production-grade AI applications.

Enterprise Features: Beyond the Code

While the open-source core is powerful, the enterprise features of Bifrost are where it truly becomes a game-changer for larger organizations. I explored several key areas:

Budgeting and Virtual Key Management: This is huge for cost control and team management. I can set spending limits per team or even per virtual key, track costs across projects, and gain granular insights into where our AI budget is going. Virtual keys allow for independent access control and budget allocation for different use cases or client projects, which is incredibly useful for multi-tenant applications or internal departmental chargebacks.
Governance and Access Control: Beyond budgeting, organizations need robust audit logs and SSO integration. Bifrost provides this, ensuring that every interaction is logged, and access is tightly controlled. This satisfies critical compliance requirements and gives me peace of mind about data security and usage.
Model Catalog and Unified Interface: Accessing 8+ providers and over a thousand AI models through a single, consistent API is a dream come true. It supports custom deployed models too, which is essential for our specialized internal models. This eliminates the need for bespoke integrations for each new model or provider we want to experiment with, drastically speeding up our innovation cycle.
Built-in Observability: Out-of-the-box OpenTelemetry support and a built-in dashboard provide immediate insights into performance, costs, and usage patterns without complex setup. This "quick glance" capability is invaluable for debugging and optimizing our AI stack.

For those starting with the open-source version, which is licensed under Apache 2.0, you get a powerful foundation. The transition to the enterprise offering seems to be a natural progression when you hit a certain scale or require advanced features like dedicated support, deeper governance, and more extensive management tools. It feels like a well-thought-out path from experimentation to full-scale production.

My Personal Takeaways and What's Next

After spending considerable time with the Bifrost Maxim AI gateway, I'm genuinely excited about its potential. It addresses many of the pain points I've encountered in deploying and managing AI at scale. It’s more than just an API proxy; it’s a strategic piece of infrastructure that empowers teams to build reliable, governed, and scalable AI applications.

Here are my key takeaways:

Seamless Integration is Real: The one-line SDK change for OpenAI, Anthropic, and other popular frameworks like LangChain is not just marketing; it works and dramatically simplifies multi-model integration.
Enterprise-Grade Performance: In my hands-on runs it held up better than LiteLLM under heavier load (lower P99, higher sustained throughput, leaner memory) — still worth verifying on your own hardware and traffic mix before you bet production on it.
Centralized Governance is a Game Changer: The MCP gateway provides a robust framework for managing tool connections, security, and policies, moving beyond mere API routing to true operational control.
Comprehensive Cost & Access Management: Features like budgeting, virtual keys, and audit logs are essential for large organizations to manage costs, ensure compliance, and maintain granular access control across teams.
Open Source Foundation with Enterprise Path: The Apache 2.0 licensed open-source core provides a fantastic starting point, with a clear and valuable upgrade path to enterprise features as needs grow.

What are your biggest challenges when deploying AI models in production? Have you explored AI gateways, and what has your experience been like? I'm particularly interested in how others are tackling model governance and cost optimization.

If you're grappling with the complexities of scaling your AI applications, I highly recommend taking a look at Bifrost. Start with the open-source version, experiment with the quickstart command (npx @maximhq/bifrost), and see for yourself how it can transform your AI infrastructure. It's truly an exciting piece of technology that could redefine how we build and manage AI.

Connect with me:

https://ishankdev.github.io/

Navigating the AI Frontier: My Deep Dive into the Manufact MCP Framework

Ishank Choudhary — Mon, 14 Sep 2026 05:00:33 +0000

Have you ever felt the seismic shift happening in software development, driven by the relentless march of AI agents and their need to interact with the real world? As a Lead Software Engineer, I’ve been right there in the trenches, wrestling with the complexities of building robust, scalable, and agent-friendly applications. For a long time, it felt like we were piecing together a patchwork quilt of solutions to enable AI to "call tools" effectively. That's why I was genuinely thrilled when I stumbled upon the Manufact MCP framework and its cloud platform, a solution that feels like it’s finally bringing order to the chaos.

My journey into the world of AI agents began a few months ago when our team was tasked with integrating a new conversational AI into our internal tooling. The goal was ambitious: let the AI interact with our existing microservices, automate workflows, and even pull data from external APIs. Sounds straightforward, right? Not quite. The sheer amount of boilerplate for authentication, transport, session management, and cross-client compatibility quickly became a nightmare. Every time we thought we had it figured out for ChatGPT, Claude would throw a curveball, or our internal agent SDK would have different expectations. It was a constant cycle of "blind resubmission and waiting time," as one customer story I later read perfectly encapsulated.

That's when I decided to dedicate some serious time to finding a better way. I’d heard whispers about mcp-use, the open-source SDK, and how it was gaining traction. What I discovered was far more comprehensive: a fullstack ecosystem now known as Manufact, which felt like a Vercel-equivalent for the AI agent world. It’s backed by YC, which immediately piqued my interest – a strong signal in the often-noisy startup landscape.

Scaffolding Your First AI Agent Tool with Ease

One of the first things that impressed me about Manufact was the developer experience (DX). Getting started with the mcp-use SDK (which, by the way, is still the name for the open-source SDK, while Manufact is the cloud platform) was incredibly straightforward. If you've ever used create-react-app or create-next-app, you'll feel right at home.

I decided to spin up a quick TypeScript project to test the waters. The command npx create-mcp-use-app felt like magic.

npx create-mcp-use-app my-first-mcp-app --template typescript-starter

Within seconds, I had a fully functional MCP server scaffolded. It included everything I needed: a basic tool definition, a server setup, and even a package.json ready for action. This immediate feedback loop, going from zero to a runnable server, is crucial for maintaining developer momentum. The mcp-use SDK itself has seen over 10 million downloads across Python and TypeScript, and boasts 10,000+ GitHub stars, which speaks volumes about its adoption and community support. It’s clear this isn’t just another fleeting trend; it’s a foundational piece of the AI puzzle.

The Manufact Cloud: Where Your Agents Come to Life

While the SDK handles the local development, the real power of Manufact, the cloud platform, comes into play when you want to deploy, test, and monitor your AI agent tools in production. This is where Manufact truly shines as the "Vercel for MCP."

I connected my GitHub repository to Manufact Cloud, and the deployment process was shockingly simple. It felt like a breath of fresh air after dealing with manual server setups and complex CI/CD pipelines for our internal agent integrations. With just a few clicks, my my-first-mcp-app was live, complete with a unique URL. Manufact handles all the underlying infrastructure, providing a hosted endpoint without me having to worry about Docker, Kubernetes, or serverless functions.

One feature I particularly appreciated was the branch previews. Every pull request automatically gets its own unique URL, like my-app--br-feature-x.run.mcp-use.com/mcp. This meant our QA team could test new tool functionalities against real AI clients before merging to main. No more "it works on my machine" excuses or finding regressions only after deployment. This kind of robust testing environment is invaluable when you're building for a rapidly evolving ecosystem like AI agents.

Demystifying Debugging with the Cloud Inspector

Debugging AI agent interactions can be notoriously difficult. You’re dealing with asynchronous calls, LLM interpretations, and often opaque error messages. This is where the Manufact Cloud Inspector truly became my best friend.

I remember a specific scenario where an AI agent was consistently failing to call a newly deployed tool. The logs were vague, and trying to trace the exact payload and response was a nightmare. With the Cloud Inspector, I could trace, replay, and debug MCP traffic directly in production. It gave me a real-time, granular view of tool calls, sessions, errors, and latency.

Imagine seeing the exact JSON-RPC payload sent by ChatGPT, the response from your MCP server, and any errors that occurred, all within a beautiful browser interface. It's like having X-ray vision for your AI agent interactions. I could filter by client (ChatGPT, Claude, Cursor, even custom agents), client version, and protocol version. This level of observability is critical for understanding why an agent might be misbehaving or why a tool call is failing in a specific context.

// A simplified example of an MCP tool in TypeScript
import { createTool } from '@mcp-use/sdk';
export const getWeatherTool = createTool({
  name: 'getWeather',
  description: 'Gets the current weather for a location',
  parameters: {
    type: 'object',
    properties: {
      location: {
        type: 'string',
        description: 'The city and state, e.g. San Francisco, CA',
      },
    },
    required: ['location'],
  },
  async run(input: { location: string }) {
    // In a real app, this would call an external weather API
    console.log(`Fetching weather for ${input.location}...`);
    const weatherData = {
      location: input.location,
      temperature: '72F',
      conditions: 'Sunny',
    };
    return weatherData;
  },
});

When I deployed this getWeatherTool to Manufact, the Inspector immediately showed me how different agents were invoking it, the parameters they were passing, and the exact output. This visibility dramatically reduced my debugging time.

Cross-Client Testing: The Unsung Hero of AI Agent Development

If you're building for AI agents, you know that what works perfectly with ChatGPT might break entirely with Claude, or vice-versa. The subtle differences in how each LLM interprets tool schemas, function call structures, and even prompt phrasing can lead to endless headaches.

Manufact's cross-client testing capabilities are a lifesaver here. I could define a test scenario and run it across ChatGPT, Claude, and even Gemini, comparing the experience side-by-side. The platform provides LLM-judged pass/fail results, giving me confidence that my tools would work reliably across different AI environments. This isn't just about preventing bugs; it's about ensuring a consistent and high-quality user experience regardless of the AI client.

My favorite part? These test suites can gate CI. Imagine a regression in a tool call for Claude blocking a merge to main. That's the kind of guardrail we need in this fast-moving space.

Publishing to Marketplaces: A Path De-risked

Getting an AI app listed on marketplaces like the ChatGPT Apps Store or Claude Cloud Connectors can be a bureaucratic maze. The requirements are stringent, and the review cycles can be long and frustrating. Manufact aims to de-risk this entire process.

I experimented with their publishing checks, and it's clear they've done their homework. Manufact audits your deployed app against marketplace requirements, covering everything from protocol conformance and security policies to metadata and asset generation. When a check fails, it provides clear guidance on how to fix it, and sometimes even offers an autofix flow. This is a massive time-saver, preventing those "weeks of blind resubmission" that so many developers face.

It even generates a "submission pack" with listing copy, tool justifications, and reviewer test cases. This level of support transforms a daunting task into a manageable one.

The Analytics You Need, Right Where You Need Them

Once your AI agent tools are live, understanding their performance and usage is paramount. Manufact provides comprehensive analytics that give you insights into tool calls, sessions, error rates, and latency (p50/p95/p99) per tool, resource, and prompt.

I found the session replay feature particularly insightful. It allows you to follow the full timeline of MCP calls within a single session, helping you understand complex agent workflows and identify bottlenecks or areas for improvement. The ability to filter traffic by client, client version, protocol version, and even country gives you a granular understanding of your user base and how they're interacting with your tools.

For any Lead SWE, having this kind of observability built into the platform, rather than having to integrate separate monitoring solutions, is a huge win.

Final Thoughts: My Verdict on Manufact

My personal journey with Manufact has been overwhelmingly positive. It addresses so many pain points that I've encountered while building for AI agents. The rebrand from mcp-use to Manufact for the cloud platform, while keeping the SDK name, makes sense when you consider the breadth of services they now offer beyond just the SDK.

It’s truly positioned as the "everything you need" platform to start, scale to millions of tool calls, and get your product on the ChatGPT and Claude marketplaces. For any developer or team looking to build serious AI agent integrations, this is a platform worth exploring deeply. It handles the "MCP plumbing" – deployment, testing, compliance, and analytics – allowing you to focus on what truly matters: building innovative AI-powered experiences.

What’s your experience been like building for AI agents? Have you encountered similar challenges, or found other solutions that simplify the process? I’d love to hear your thoughts in the comments below!

Key Takeaways:

Accelerated Development: Manufact, powered by the mcp-use SDK, offers rapid scaffolding and deployment, significantly reducing time-to-market for AI agent tools.
Robust Testing: Its cross-client testing and branch preview features ensure your tools work consistently across various AI clients like ChatGPT and Claude, preventing regressions.
Unparalleled Debugging: The Cloud Inspector provides deep visibility into tool calls, payloads, and errors, making debugging complex AI agent interactions straightforward.
Streamlined Publishing: Manufact de-risks marketplace submissions with automated checks, fix guidance, and submission asset generation.
Comprehensive Observability: Built-in analytics and session tracking offer critical insights into tool performance, usage, and user behavior.

If you’re ready to dive in, I highly recommend checking out their canonical site at https://manufact.com/ and exploring the open-source SDK on GitHub at https://github.com/mcp-use/mcp-use. The documentation at https://manufact.com/developers is also an excellent resource. This platform is genuinely empowering developers to build the next generation of AI-driven applications.