DEV Community: cuculhart

I benchmarked 14 AI coding agents on the same task — same model, wildly different results

cuculhart — Mon, 19 Oct 2026 13:19:50 +0000

In my last post, benchmarking a small local LLM exposed a bug in my own harness — the "weak model" was partly my code. That raised the obvious next question: how much does the tool around the model matter, measured across many tools?

So I ran 14 agent-type AI coding tools through the same task set, on the same ordinary PC (Windows 11, Ryzen 5 4500U — no GPU, nothing special), with two model backends:

BYOK: gemini-3.5-flash-lite via each user's own API key
Local LLM: qwen3.5:4b on Ollama

Scope: agent-type tools only — things that autonomously read files, edit files, and run commands. Chat assistants and subscription products (Copilot, Claude Code, Cursor, Devin) are out of scope.

What I measured

Real tasks: create a file, search/read and answer, perform a precise edit, diagnose+fix a failing test. Success rates count only the last 2 attempts per task, and results identical to baseline ("did nothing but claimed done") are marked failed.

BYOK results (gemini-3.5-flash-lite)

Tool	Passes
Cline (VS Code ext)	7/7 (100%)
Kilo Code, OpenHands, OpenCode, Aider	6/6 (100%)
Teaspoon IDE (standalone Electron)	6/7 (86%)
Zoo Code, Goose	5/6 (83%)
Continue	2/5 (40%)
SWE-agent, Void, Roo Code	0–1 attempts (0%)

Same model, same tasks — and the spread is 100% → 0%. The model is constant; the harness (prompt assembly, tool-call format, retry logic, context handling) is what varies.

Local LLM results (qwen3.5:4b, Ollama)

Tool	Passes
Cline	6/9 (67%)
Teaspoon IDE	6/9 (67%)
Void	2/3 (67%)
Aider	2/4 (50%)
Zoo Code, Kilo Code, Continue, OpenCode, Goose	~25%
Roo Code, OpenHands, Devika, Plandex	0%

The weak local model is a harsher judge: tools whose harnesses were tuned for strong cloud APIs fall apart (brittle tool-call formats, no retry on malformed output), while tools with tolerant parsing and explicit command protocols still complete tasks.

What the numbers actually show

"This model is good/bad" claims usually include the harness. Same backbone model, different success rate — that's tool-side design, not intelligence.
Small local models are the best harness test. A 4B on CPU has no slack; every weakness in the agent loop surfaces as a visible failure.
Dead ecosystems show up in measurements, not marketing. Roo Code's repo is archived; attempts got no response on either backend — the bench records it as "not evaluable" instead of pretending otherwise.
AI-vs-AI reviews add a second axis. I also had ChatGPT, Gemini, Claude, and Devin (SWE-2) score each tool's beginner-friendliness 0–5 by reading the measured data: Teaspoon IDE 3.8, Cline 3.5, everything else ≤2.8.

Caveats

Scores reflect the last two runs — small n by design (agent runs are slow on this hardware). Full per-attempt history is published per tool.
SWE-agent and Plandex hit environment constraints (Windows/autocrlf, no native build) — recorded as not evaluable, not failed.
I'm the author of one tool in the table (Teaspoon IDE). The measurement protocol is identical for all tools and the raw logs are public — judge for yourself.

I thought the local LLM was weak — it was a bug in my own harness

cuculhart — Sun, 18 Oct 2026 04:12:15 +0000

I've been benchmarking local LLMs as coding agents inside my own IDE (Teaspoon IDE) on a CPU-only machine — Ryzen 5 4500U, 32 GB RAM, no GPU, Ollama.

One symptom kept showing up across models: the agent would "re-read the same file over and over," or claim "I read it, but the contents aren't visible." Classic weak-model behavior, right?

It was a bug in my own harness. This is the record of how a 4B model exposed it — and how a control experiment proved it.

The failure that started it

The task (T3): "Running pytest shows one failure. Find the cause, fix it, and confirm all pass." I gave it to qwen3.5:4b — a small, fast 4B model — on my old harness.

What happened:

READ_FILE: test_calc.py — reads it
READ_FILE: calc.py — reads it
READ_FILE: test_calc.py — reads it again. My dedup logic answers: "unchanged — use the content you already read"
The model replies: "I read them, but the contents aren't visible here"
Tries GREP, gets 0 hits → re-reads again → burns all 6 turns and fails

I was telling it "unchanged," and it was insisting it couldn't see the contents. My first assumption: the 4B just isn't smart enough to follow.

The investigation: tool results survived exactly one turn

Tracing the code revealed the flaw. The agent loop accumulated history like this:

User messages → kept ✓
Assistant replies → kept ✓
Tool results (READ_FILE contents, command output) → present in context for exactly ONE turn after execution, then silently dropped

So:

Step 1 reads the file → step 2 can see the content → everything works
From step 3 on, that content has already vanished from context
My duplicate-read suppression — "unchanged, use what you have" — was pointing at data the model did not have

The model's "I can't see the contents" was an accurate status report. It knew it had read the file (its own past READ_FILE commands were in history) but the payload was gone. It was debugging my context assembly correctly.

Bonus discovery: past evaluations needed rewriting

This changed how earlier runs looked. My 9B model (qwen3.5:9b) had been rated as "redundantly re-reading files" — 2 of 7 steps wasted on the old harness.

That wasn't weak memory. It was legitimate self-repair: fetching content that had disappeared. The smarter models were working around my bug. The weak one just reported it honestly.

The fix (and a second bug it surfaced)

Persist tool results into running history — each step's tool output now stays in context for the whole run, so "unchanged" finally tells the truth
Strip outer quotes from GREP patterns — models kept emitting GREP "member.*discount" and the literal " characters ended up in the regex, matching nothing. Seven turns of re-quoting, zero hits. The parser now strips surrounding quotes, and a zero-match GREP reply carries a hint to simplify the pattern or just READ the file

The control experiment

Same model, same task, same prompts — only the harness fix differs:

Condition	Result
4b · old harness	All 6 turns burned · re-read loop · failed
4b · fixed	Completed in 5 turns (pytest → read → fix → re-verify, correct)
9b · old harness	7 steps with 2 re-reads (self-repairing)
9b · fixed	3 steps, zero re-reads, shortest path
e4b · fixed	4 turns, zero re-reads, clean

The same 4B model went from "hopeless" to "clean shortest-path completion" — just by fixing the code around it.

Lessons

"Model weakness" and "harness weakness" are inseparable until you check. Every "this model is useless" article may actually be rating the harness's constraints. If you build agent tools, suspect what's in the context before you suspect the model.
Dedup/context-saving must be designed against what the model can actually see. "It read it before" is only true while that content is physically in the prompt.
Weak models are the best harness-testers. The strong ones paper over defects with self-repair; the 4B, with no slack, faithfully visualized my flaw. If you're testing your own agent loop, run it on a marginal model.

Data

All raw logs, task definitions, and the evaluation record are public:

Full measured data + analysis: https://labo.mond.jp/articles/localllm-harness-bug.en.html
Fixtures, prompts, run artifacts: https://github.com/cuculhart/ai-editor-lab/tree/main/experiments/localllm
The harness (Teaspoon IDE, FSL-1.1-MIT, fix shipped in v1.5.1): https://github.com/cuculhart/teaspoon-ide

The environment is deliberately modest — an old 6-core laptop CPU, no GPU. If the harness had been worse, every model would have looked broken. Check your context before you check your model.

One canvas engine hosts CAD floor plans and wandering game units — position is data

cuculhart — Fri, 16 Oct 2026 11:25:44 +0000

A CAD floor plan of a building, and two game units wandering inside that same building. Two wildly different applications — running on the same engine.

I just open-sourced Kimetron Isle, a framework-agnostic 2D placement engine where position is data, not decoration. v1.0.0, MIT licensed.

Repo: https://github.com/cuculhart/kimetron
Playground (no install, runs in the browser): https://cuculhart.github.io/kimetron/

The same engine, two faces

plan — a building sketch on a 10 mm grid. Walls are thickness×length rectangles that may not overlap each other; butt joints ("|-" T junctions) are legal because edge-touching doesn't count as an overlap. A door is the declared exception that may sit on a wall. Rulers, dimension lines, wall centerlines, and architectural areas (site / building / core / inner) are all derived — move a wall and the floor area changes.

game — two 300 mm units wander inside the building via engine.tryMove. Walls and rocks (locked, concave polygons) are impassable; the gate is the only way out. Units can't occupy the same space, and when two come within range an effect link appears — talk, attack, item exchange. The pond is a driver-side SVG pattern.

Same engine in both cases. The only difference is the policy each driver declares: "a door may overlap a wall", "rocks can't move", "units can't stack" — a few lines of driver code each.

Why build it

I built two production apps (Plesfeda, Sethera Piece) on React Flow, and learned that what a canvas library does for you stops at rendering, dragging, selection, and the viewport. Layout and domain semantics were always on my side — so the coupling was thin, and a replacement was feasible.

But a React Flow clone would be pointless. Every canvas/diagram library out there assumes screen coordinates, pixels, and a downward Y axis. In CAD, the coordinate itself is the design data — yet no engine treats it that way. That's the gap Kimetron Isle fills.

The world-coordinate model

Real units — mm / m / inch / px as first-class units. Define a world where 1 unit = 1 mm; zoom is a drawing scale, not a magnification
Selectable origin — bottom-left + Y-up (CAD/math) or top-left + Y-down (screen)
Built-in snapping — grid, vertices, edge midpoints, centers; drivers inject custom snaps via snapExtra
Collision & occupancy — solid objects can't overlap while edge-touching stays legal; canOverlap declares pair exceptions; collisionMode picks none/flag/reject
Rotation & polygons — rotation honored all the way down to OBB/SAT collision; kind:'polygon' hit-tests and collides correctly even when concave
Programmatic movement — objectsIn, canPlace, tryMove so AI-driven motion goes through the same collision policy as human gestures
Undo & serialization — immutable model ops, document-level undo/redo, JSON serialize

Architecture

Three layers: @kimetron/core (vanilla TypeScript — model, geometry, selection, drag, hit-testing, no DOM), @kimetron/dom (SVG/CSS renderer + pointer input), @kimetron/react (a thin adapter).

Semantics live on the user's side: Kimetron assigns meaning only up to geometry and interaction. That's exactly why a floor-plan editor and a wandering-unit demo can share one engine.

Honest status

v1.0.0 is a stability commitment — additive-only changes since 0.6.0. There are still gaps: associated dimensions, named layers, and block references are 1.x territory; Svelte/Vue adapters are "structurally possible" rather than built; collaboration and cloud storage are deliberately out of scope.

If you're building something where placement itself carries meaning — floor plans, equipment layouts, seating charts, panel assignments, game maps — I'd love to hear what you'd put on it.

Most "WBS tools" are secretly Gantt editors — I built a board for real Work Breakdown Structures

cuculhart — Wed, 14 Oct 2026 12:37:36 +0000

I built Plesfeda — a board for building real Work Breakdown
Structures. Breakdown first, schedule later. MIT-licensed, fully
local, runs in your browser with no install.

Repo: https://github.com/cuculhart/plesfeda
Live demo: https://cuculhart.github.io/plesfeda/

The idea

Most things called "WBS tools" are actually Gantt chart editors. You
arrange rows and draw dates — useful, but it never builds the thing a
WBS actually is: work, broken down, into a structure.

Serious PM tools didn't help either. MS Project, OpenProj, LibrePlan —
they let you move rows, but never let you pick up a whole block of work
and hang it somewhere else in the structure. Their calendars don't even
ship with Japanese holidays, so estimates always ended up back in Excel.

Plesfeda is not a project management tool. It's a tool for calmly
judging whether a volume of work fits inside a deadline. The
protagonists are breakdown and hierarchy — dependencies,
scheduling, and the Gantt chart are all derived output, never the
input surface.

Breakdown input

The core feature is an input flow that mirrors the act of decomposing:

Place a [task] on the board and choose breakdown.
Two dotted slots appear — the first is an active input with a cursor, the second is a ghost previewing the next entry.
Every Enter promotes the ghost to active and sprouts a new ghost, so children stack up at the tempo of your thinking.
Walk away mid-entry and the active slot falls back to a ghost parked at the parent's tail — click it to resume.

Ghosts never persist into data until confirmed.

The structure is free

Reparent whole subtrees — drop a node onto another node and the entire branch moves. Drop on empty space to make it a root. Cycles are always rejected.
It's a forest — multiple roots are first-class. Add a parent later and adopt existing roots as children: bottom-up structuring.
Derived WBS numbers — every node shows 1.2.3-style numbering computed from position. Reparenting renumbers automatically; nothing is stored.
Deleting a parent is a choice — promote its children to roots, or delete the whole subtree.
Effort rollup — leaves take a hand-entered effort; parents show the recursive subtree total (Σ), so a root shows the cost of the whole structure. Units: days or hours.

Structure becomes schedule

Tasks can declare predecessors (finish-to-start). One simple rule —
leaf earliest start = max(parent start, predecessor finishes) — drives
a PERT/CPM forward pass that returns the structure's earliest finish.
The toolbar always shows the project's earliest completion, and the
critical path is highlighted on nodes and dependency edges.

Dependency cycles and ancestor/descendant dependencies are flagged as
errors, and Gantt output stays locked until they're resolved — so a
broken structure can't produce a plausible-looking schedule.

The Gantt is a read-only report rendered from the tree: baseline date,
weekend/holiday exclusion (click a day header to toggle a holiday),
milestones (zero-effort leaves render as ◆), and print-friendly
two-dimensional pagination with repeated headers and page numbers —
real report output.

Why the Gantt is read-only

Typical Gantt tools let you grab a bar and drag the date. That's
manual tail-end fitting — the structure loses the reason for the date.
In Plesfeda you fix the structure and look at the schedule, not
the other way around. Read-only Gantt is a deliberate design choice.

Fully local, by design

Data lives in the browser's localStorage and JSON files only
({app:'plesfeda', version:1} export/import). No account, no cloud,
nothing is sent anywhere.

Tech stack

React 19 + TypeScript + Vite
Canvas on @xyflow/react (React Flow, MIT)
English-first i18n — Japanese is a translation table keyed by the English source; untranslated strings fall back to English

Why the name

Plesfeda = Welsh pleth ("weave / braid") + Old Irish feda
("woods" — also the name of the letters of Ogham, the lost Celtic
script whose characters are named after trees). A structure woven
from trees.

Honest limitations

Single-player and local-only — deliberately. Editing is mouse-first,
so phones can look but won't enjoy it. The Gantt report caps at 400
days of horizon. v0.1.0.

License

MIT. Feedback and issues welcome:
https://github.com/cuculhart/plesfeda/issues

I built a cell-less spreadsheet where you count by enclosing — free, runs in your browser

cuculhart — Tue, 13 Oct 2026 11:11:28 +0000

I built Sethera Piece — a calculation board with no cells. You drop
text and number "pieces" anywhere on a canvas, draw a zone around them,
and it aggregates them live. MIT-licensed, fully local, and it runs in
your browser with no install.

Repo: https://github.com/cuculhart/sethera-piece
Live demo: https://cuculhart.github.io/sethera-piece/

The idea

A lot of what people do in Excel isn't actually spreadsheet work. We
use cells as graph paper: laying out forms, sticking notes, summing
things by hand. The grid is a constraint the job never asked for.

Sethera Piece removes the grid entirely. Objects live anywhere. If you
want a total, you draw a zone around the pieces — spatial layout is
the formula.

What it feels like

Pieces — text / number / sticky-note+number. Multi-line, alignment, colors, sizes, borderless display.
Σ zones — aggregate enclosed pieces with Sum / Count / Avg / Max / Min. Drag a piece in or out and every zone's total updates instantly. A piece that only partially overlaps a zone gets a red dashed warning so it can't silently escape aggregation.
Calc pieces — + − × ÷ with rounding modes. Wire a zone's output handle into a calc's inputs and you get chains like subtotal × tax rate → total with zero syntax.

The screenshot above is a US-style estimate built on an A4 paper
frame: the AMOUNT column sits inside a Σ zone (subtotal 4,800), a tax
piece and the zone feed a calc piece that shows the total (5,196).
Printing/PDF export captures the paper frame.

Color = category (SumIf / CountIf)

The 1.0 feature I'm happiest with: pick a zone, choose a piece color
in the inspector, and only pieces of that color are counted. Eight
usable colors = eight categories. To mark items, multi-select them
and batch-change the color — that's the whole workflow. No label
matching, no strings to typo.

Everything else in the box

Lines, rects, ellipses, arcs, curves, diagonals — table frames, flowchart connectors; grid snapping makes adjacent edges land exactly
Custom pieces — group any combination into a reusable unit; template library saves named selections across documents
CSV/TSV round-trip with Excel — paste a range or import a file (auto-detects UTF-8/UTF-16/Shift_JIS/EUC-JP); number columns become number pieces that aggregate immediately
Paper & print — A4/B5/A3 portrait/landscape, print/PDF capture
Fonts — bundled Noto Sans JP / Shippori Mincho / BIZ UDP Gothic / monospace, or a custom font name with a missing-font warning
Japanese / English UI — toggle in one click, auto-detects from the browser
Fully local — localStorage + JSON files only. Nothing is sent anywhere. No telemetry

Why the name

"Sethera" comes from Yan Tan Tethera, a counting rhyme shepherds
in northern England used to tally sheep — probably one of humanity's
oldest aggregation functions. Enclosing pieces to count them is the
modern version.

(The other image in my head: a general moving pieces on a sand table.
Move a piece and the situation changes — a board that recalculates
when you move it.)

Tech stack

React 19 + TypeScript + Vite, canvas on
@xyflow/react (React Flow, MIT).
I picked React Flow over tldraw because tldraw's license charges for
production use; React Flow is MIT. The aggregation core is just
rect-containment + a filter predicate — the spatial model does the
rest.

Honest limitations

Single-player. Real-time collaboration is still only an idea on the
drawing board — the local-first version comes first. Editing is
mouse-first; a phone can view but won't enjoy it.

License

MIT. Feedback and issues welcome:
https://github.com/cuculhart/sethera-piece/issues

Tokens arrived on time — the renderer didn't: measuring a 90x display lag in local-LLM streaming (and how Ollama fixed it)

cuculhart — Sun, 11 Oct 2026 01:48:08 +0000

I built a local-LLM coding IDE, and I noticed something odd: giving
the model all the CPU cores made the app feel slower. Tokens
were arriving at the same speed, but the UI lagged behind. So I
measured it — and then, mid-verification, a new Ollama release
changed the whole story.

TL;DR — On Ollama 0.21.0, running inference on all 6 cores vs. 4
produced the same token rate but 90x worse display lag (P95 181ms →
2ms) and 6x worse UI click response (929ms → 158ms). On Ollama
0.40.2 the gap nearly vanished, because the runner internals changed.
I shipped the thread-cap setting anyway — here's why.

What I actually wanted to measure

Not generation speed. API tok/s is easy to measure, but my question
was "how fast does what the API returned get onto the screen?"

Hypothesis: with all cores allocated, the LLM saturates the CPU, the
renderer is starved, and incoming tokens can't be painted in time.
With fewer cores, generation speed is nearly identical but the display
stays smooth — so it feels faster.

To quantify "feel," I split the clocks:

Wire arrival time — a thin proxy between the IDE and Ollama timestamps every NDJSON chunk. Accurate even when the renderer is starving.
DOM render time — a MutationObserver records streaming text length over time, giving per-chunk arrival→paint lag.
Event-loop delay — drift of a 100ms setTimeout.
Effective FPS — requestAnimationFrame fire count.
UI click probe — actually clicking a real UI button during generation and timing the response.
Fixed conditions — temperature 0, fixed seed, num_predict 600, think: false.

Environment: Ryzen 5 4500U (6 logical cores), CPU inference, Windows.
Compared "all cores" vs "4", median over a 60s window. The harness is
e2e/bench-threads.mjs (Playwright + measurement proxy) in the
repo.

Part 1: Ollama 0.21.0 — same speed, 90x more invisible

Metric	All 6 cores	4 cores
Wire-arrival speed (≈ generation)	4.6 chars/s	4.7 chars/s
Display lag P95 (arrival→DOM)	181 ms	2 ms
Display lag Max	553 ms	6 ms
Event-loop delay P95	70 ms	13.6 ms
Effective FPS	55.9	60.0
UI click response (real click)	929 ms	158 ms

Three takeaways:

Generation speed was identical — decode is memory-bandwidth bound, so cutting 6→4 threads didn't slow token production.
Display lag diverged ~90x at P95 — on all cores, tokens waited up to half a second to appear. Streaming output "backs up."
One UI click took 929ms vs 158ms (~6x) — a proxy for "can you still do light work while the LLM runs."

On the old engine the runner occupied ~5.7 cores continuously during
generation. "4 cores feels faster" wasn't an illusion — it existed as
real delay in the display pipeline.

Part 2: Ollama 0.40.2 — I upgraded, and the premise changed

Before publishing, I wondered "isn't 0.21.0 old?" and re-ran on
0.40.2. The gap nearly disappeared:

Metric	num_thread=6	num_thread=4
Wire-arrival speed	7.6 chars/s	7.5 chars/s
Display lag P95	13 ms	3 ms
Event-loop delay P95	14.2 ms	13.9 ms
Effective FPS	59.9	59.9
UI click response	123 ms	114 ms

Same story on a 9.6GB model (lagP95: 5ms vs 3ms). Inspecting the
runner process showed why:

	Ollama 0.21.0	Ollama 0.40.2
Runner	`ollama runner` (old engine)	`llama-server` (llama.cpp)
Threads, unspecified	all cores (=6)	n_threads=3 (measured)
Actual CPU at num_thread=6	~5.7 cores saturated	~4.3–4.6 cores, no saturation

The default changed: unspecified now means 3 threads — it never grabs all cores in the first place.
Even explicit num_thread=6 doesn't saturate — decode is memory- bound and the new engine's threads simply wait instead of spinning.
Headroom exists now: 1.5–2 spare cores at all times, so the renderer/OS/other apps can preempt. UI starvation doesn't happen.
Prefill behaves the same way — UI response stayed 90–180ms during prompt processing.

The "all cores make it stutter" experience was a real property of the
old engine's design, and the new engine fixed it internally.

So is a thread cap useless? Not quite

Two reasons to keep the setting:

Guarantee, not tuning. Engine internals change per version — the unspecified default moved 6→3 in one release. num_thread is an explicit ceiling that stays put regardless of version, model, or whatever else is running on the machine.
API reach. The option only exists on Ollama's native /api/chat.

Bonus finding: the "thinking" switch

While measuring, I noticed the model (qwen3.5:4b, a thinking model)
produced a long reasoning trace — input analysis, greeting candidates,
style polish — before answering a plain "hello." On a modest PC that
trace alone costs tens of seconds to minutes. Ollama's native API has
a think flag; think:false suppresses it (verified: zero
pre-content chunks in our runs). For practical local-LLM speed it's as
effective a switch as the thread cap.

Where this becomes a tooling difference

Here's the part that turned this into a product feature: these
options only exist on Ollama's native /api/chat.

What you want	Ollama `/api/chat`	OpenAI-compatible `/v1`
Cap CPU cores	`options.num_thread`	No way to pass it
Thinking on/off	`think`	No way to pass it
Output length / seed	`options.*`	Partially

Most VS Code-extension agents don't expose "CPU core count for local
LLM" as a setting — with an OpenAI-compatible client there's simply no
channel to send it through. To run a local LLM without killing the
host PC, you need a design that reaches the native API's options.

I shipped this in Teaspoon IDE v1.2.0 — a free, standalone
Electron AI coding IDE I maintain — as "CPU Threads" and "Thinking"
settings, plus the IDE's Ollama path moved to /api/chat. It's BYOK
(bring your own Gemini key) or fully offline via Ollama, and
source-available under FSL-1.1-MIT.
Product page: https://cuculhart.com/teaspoon.en.html

Lessons

tok/s explains only half the experience — same generation speed, but starved rendering changed perceived speed by a measured 90x.
"Use everything" isn't always optimal — and "how much to use" depends on engine internals that change between versions.
Re-verify on the latest environment — re-testing on the new release showed the phenomenon was engine-dependent, which made the article stronger, not weaker.
UI responsiveness is measurable — split wire time and DOM time, and "feels slow" becomes a number.
Local-LLM features are bounded by API reach — a design limited to the compatibility API's common denominator can't implement num_thread or think.

Methodology: harness e2e/bench-threads.mjs, raw data in
e2e/bench-threads-results.json — reproduce with
BENCH_THREADS=6,4 BENCH_RUNS=3 BENCH_MODEL=qwen3.5:4b node e2e/bench-threads.mjs.
n=2–3 on a single Ryzen 5 4500U, so treat the exact numbers as
one-machine data; the measurement method is the reusable part.

I built a free, BYOK AI coding agent IDE that runs fully offline on local LLMs

cuculhart — Wed, 23 Sep 2026 08:33:21 +0000

I built Teaspoon IDE, a standalone Electron app that bundles a file
explorer, Monaco editor, AI chat, Git, and a real TTY terminal into one
window — with an agent loop that lists, reads, searches, and edits
files across multiple steps, all behind approval dialogs.

Repo: https://github.com/cuculhart/teaspoon-ide
Setup guide: https://cuculhart.com/teaspoon-howto.en.html

Not a VS Code fork, not an extension — a single exe.

Why I built it

Doing contract work, I kept hitting the same walls:

Confidential code can't go to vendor clouds. In contract work you handle code and engineering knowledge that simply may not pass through someone else's servers.
Orgs that can't approve $20/mo subscriptions. There are still plenty of organizations where a monthly AI IDE subscription is a hard sell.
VS Code extension agents get tangled. Extensions are convenient, but they also collide with each other — Copilot interference being the usual suspect.
Unapproved rewrites are a review burden. When the AI silently rewrites code, the human reviewer pays the cognitive cost. So every write, diff edit, and command goes through an approval dialog, and checkpoint/rollback undoes only the AI's changes.
Docker-in-Docker tools are heavy. Nice, but a hard setup for beginners. I wanted a single exe that just runs.
Beginners need visible panes. Explorer, editor, console, and AI chat as separate, obvious panes is simply easier to learn.

Nothing matched all of this — so I built it, with AI, in under 3 days.

Features

Free + BYOK — bring your own Gemini API key, pay Google directly for what you use. No subscription, no markup.
Fully offline option — point it at Ollama (Gemma, Qwen, ...) and every request goes to localhost only. Your code never leaves the machine.
No telemetry — the only network calls go to the LLM endpoint you configure. Source-available, so you can verify that yourself.
Human-in-the-loop — writes are confined to the project root; file creation, diff edits, and shell commands all require approval.
LiteLLM proxy support — real API keys never sit on the client.
Token-efficient context — the AI gets the file tree and fetches contents on demand via // GREP: / // READ_FILE: commands.
Cheap by default — Gemini 3.8 Flash is at introductory pricing (half the standard rate) through the end of 2026, and even cheaper models like 3.5 Flash-Lite are selectable.

Tech stack

Electron / React / TypeScript / Monaco Editor / Gemini API & Ollama /
Vite

License

FSL-1.1-MIT (source-available). Free to use, modify, fork, and
redistribute for any non-competing purpose — including commercial work
inside your company. The one restriction: you can't sell it (or a
renamed clone) as a competing product. Each release automatically
converts to MIT two years after publication.

Honest limitations

It's chat/agent-centric (no inline completions), there's no codebase
index (tree + on-demand reads instead), editor tabs don't exist yet,
and it's Windows-first with the UI in English/Japanese.

Roadmap

Considering PDF/image recognition, making the agent loop more reliable
on small local models, and chat improvements (multiple chat tabs).

Feedback and bug reports welcome via
GitHub Issues.

DEV Community: cuculhart

I benchmarked 14 AI coding agents on the same task — same model, wildly different results

What I measured

BYOK results (gemini-3.5-flash-lite)

Local LLM results (qwen3.5:4b, Ollama)

What the numbers actually show

Caveats

Links

I thought the local LLM was weak — it was a bug in my own harness

The failure that started it

The investigation: tool results survived exactly one turn

Bonus discovery: past evaluations needed rewriting

The fix (and a second bug it surfaced)

The control experiment

Lessons

Data

One canvas engine hosts CAD floor plans and wandering game units — position is data

The same engine, two faces

Why build it

The world-coordinate model

Architecture

Honest status

Links

Most "WBS tools" are secretly Gantt editors — I built a board for real Work Breakdown Structures

The idea

Breakdown input

The structure is free

Structure becomes schedule

Why the Gantt is read-only

Fully local, by design

Tech stack

Why the name

Honest limitations

License

I built a cell-less spreadsheet where you count by enclosing — free, runs in your browser

The idea

What it feels like

Color = category (SumIf / CountIf)

Everything else in the box

Why the name

Tech stack

Honest limitations

License

Tokens arrived on time — the renderer didn't: measuring a 90x display lag in local-LLM streaming (and how Ollama fixed it)

What I actually wanted to measure

Part 1: Ollama 0.21.0 — same speed, 90x more invisible

Part 2: Ollama 0.40.2 — I upgraded, and the premise changed

So is a thread cap useless? Not quite

Bonus finding: the "thinking" switch

Where this becomes a tooling difference

Lessons

I built a free, BYOK AI coding agent IDE that runs fully offline on local LLMs

Why I built it

Features

Tech stack

License

Honest limitations

Roadmap