DEV Community: Devansh

How to Make Your Codebase Work for AI Coding Agents (Without Better Prompts)

Devansh — Wed, 03 Jun 2026 02:57:19 +0000

Your agent wrote valid code. It still missed the point.

Wrong package manager. Tests run with a flag your pipeline never uses. Business logic landed in a route handler because the model found a similar file three folders away. You pasted more context, tightened the prompt, ran again. Same failure on the next task.

That is not a model problem. It is a repo problem.

A wave of posts in early 2026 (Medeiros, Fabisevich, Marmelab, Sourcegraph, Vstorm, and others) converged on the same idea: agent productivity is architectural. Tools matter. Structure and feedback loops matter more.

This post is a practical distillation. No tool worship. What to add to your repository so Copilot, Claude Code, Cursor, Codex, or whatever you use next month can ship without you re-explaining the project every session.

Why your prompts stop working

Humans absorb tribal knowledge. Half-documented setup scripts. "Ask Priya about auth." Agents do not ask Priya. They pattern-match on what is in the tree and what they can grep.

Hélio Medeiros frames the repository as an interface. InfoWorld's "Coding for agents" goes further: context is infrastructure. Test commands, boundaries, and "do not touch" paths are part of how work runs when the worker is an agent.

The litmus test (use this before you blame the model):

Delete the chat history.
Open a fresh agent session on the same branch.
Give one real task: "Add a field to the checkout API" or "Fix the failing test in module X."
Do not paste architecture essays.

If the agent cannot finish using only committed files, you are still carrying the load. The agent is typing.

That test takes ten minutes. It tells you exactly where to invest next.

What you get when the repo carries the instructions

Teams that retrofit for agents report the same wins:

Fewer wrong commands (install, test, lint, migrate).
Fewer edits to generated files, lockfiles, or secrets.
Smaller diffs that match how your team actually layers code.
Less time re-typing "we use pnpm" or "migrations are generated" in every thread.

Vstorm's guide and community writeups on AGENTS.md put the setup time at roughly 15 minutes for a first version. The payback shows up in the first week of review loops you do not have to run.

You are not building for robots. You are writing down what a good senior engineer would need on day one. Agents just force the issue because they never attend onboarding.

Step 1: Add `AGENTS.md` at the repo root

A year ago every tool wanted its own rules file. .cursorrules, CLAUDE.md, .github/copilot-instructions.md, tool-specific Gemini configs. Same conventions copied four times, drifting within weeks.

AGENTS.md is the convention that stuck: one Markdown file at the root that multiple agents read. Plain text. No JSON schema. Works across Copilot, Codex, Claude Code, Cursor, and others (see this cross-platform test on DEV).

Keep tool-specific extras if you want (CLAUDE.md for Claude-only workflow). AGENTS.md should stand alone. If an agent reads one file, it should still know how to work here.

What to put in it (highest leverage first)

Copy this skeleton and fill in the blanks:

# AGENTS.md

## Project overview
[Name]. [One line: what it does].
Stack: [language, framework, database, package manager].

## Commands
# Install
[exact command]

# Dev
[exact command]

# Test
[exact command]

# Lint / format
[exact command]

## Structure
[key directories only, 10-15 lines max]
- `src/api/`: HTTP handlers
- `src/domain/`: business rules
- ...

## Conventions
- [Where new endpoints go]
- [How you name tests]
- [Patterns agents get wrong: e.g. flush() in repo, not commit()]

## Do not modify
- [generated migrations]
- [lockfiles]
- `.env`
- [auto-generated docs]

## More context
- `docs/architecture.md`
- `CONTRIBUTING.md`

The sections that prevent the most damage:

Section	What it stops
Commands	`pip` vs `uv`, `npm` vs `pnpm`, wrong test runner
Structure	Logic dropped in `main.ts` or the wrong package
Conventions	Architecturally "valid" code that violates your patterns
Do not modify	Ruined migrations, committed secrets, reformatted lockfiles

Do not paste your entire README. OpenAI's harness engineering notes (summarized widely in 2026) argue that one giant agent manual goes stale. Use AGENTS.md as a map, not an encyclopedia.

Step 2: Add `llms.txt` if agents need a wider map

Joe Fabisevich's Recap 2.0 writeup describes a small llms.txt that points agents at the right docs without dumping the whole repo into context.

Use it for pointers, not rules:

# llms.txt
/docs/architecture.md
/docs/api.md
/CONTRIBUTING.md

Put operational rules in AGENTS.md. Put "where to look next" in llms.txt or public/llms.txt for web projects.

Step 3: One golden path for commands (and match CI)

Medeiros recommends stable entrypoints, often wrapped in Make:

make bootstrap
make test
make lint
make run

Your implementation can be npm scripts, pnpm, mise, or a Taskfile. The agent does not care about the wrapper. It cares that one string always works on a clean clone and that CI runs the same string.

Bad state: local npm test, CI pnpm test --filter=api. The agent optimizes for whatever just ran in the terminal. You merge green locally and red in the pipeline.

Good state:

"scripts": {
  "test": "vitest run",
  "lint": "eslint ."
}

…and the workflow file calls pnpm test and pnpm lint, not a different incantation.

When verification is slow or flaky, the agent becomes a diff machine and you become the test runner. Fast unit tests on pure domain code (where you have any) shorten the loop more than swapping to a frontier model.

Step 4: Shrink where a change is allowed to live

You do not need hexagonal architecture on every side project. You do need obvious boundaries.

Medeiros and others recommend ports-and-adapters style layouts because they make violations visible: domain code cannot import the database driver, so the build fails when an agent takes a shortcut.

Transferable pattern for any stack:

Put business rules in one place (domain, core/, lib/domain/).
Keep framework glue thin (handlers, UI routes, CLI).
Wire dependencies at the edges (main, app/, composition root).

For a feature-folder Next.js app, that might mean: routes in app/, product logic in features/*/, shared MDX paths documented in AGENTS.md so "add a blog post" does not create data/blog/ and features/blog/data/posts/ on the same day.

Add a one-paragraph README in folders agents confuse often (src/billing/, packages/api/). Agents frequently read folder READMEs when they list a directory.

Step 5: Treat agent mistakes as repo tickets

Marmelab's agent experience post is long. The habit worth stealing is simple:

Every time an agent does something stupid, ask if the repository should have prevented it.

Agent mistake	Repo fix
Wrong test command	Add to `AGENTS.md` Commands
Reinvented helper	Add convention: search before creating
Same formatting nit on every PR	Pre-commit hook or agent hook
Broke auth on a "small" change	Document blast radius; list related paths in `AGENTS.md`

Tooling and MCP servers come last in their ordering. Most teams still fail on missing context, not missing plugins.

The 80% problem (and what to do at your scale)

Sourcegraph's agentic coding guide names a pattern teams recognize: the agent finishes the visible 80%. Tests pass in the files it touched. Days later, CI fails elsewhere because middleware, DTOs, audit logs, or a sibling service still expect the old contract.

That is incomplete context, not stupidity.

On a single app, blast radius is smaller. Still run this before you call a task done: grep for every symbol the agent renamed or exported. Open files it never touched. If something depends on the old shape, the task is not done.

On large or multi-repo codebases, you need deterministic cross-repo search and explicit scoping before merge. The fix scales up; the diagnosis stays the same.

Your 30-minute retrofit checklist

Do this on the repo you use agents on most:

Write AGENTS.md using the skeleton above (15 minutes).
Align local test/lint with CI (one script name, both places).
Add folder READMEs where agents keep landing wrong (5 minutes each, only where needed).
Run the litmus test with a fresh session and one real task.
After the task, add one line to AGENTS.md for anything the agent had to be told in chat.

Start on a small project if you are learning the pattern. Fabisevich's advice is to practice on something bounded, then port the habits to the big codebase.

Reading about agent-friendly repos does nothing until a file lands in git. The litmus test is the scoreboard.

OpenCode Go + Oh My OpenAgent: The Model Routing Config That Actually Saves Money

Devansh — Sun, 24 May 2026 02:53:03 +0000

Most guides on OpenCode Go start with the models. I want to start with the thing most guides get wrong: the limits are denominated in dollars, not requests.

That sounds like a minor distinction. It isn't.

The thing everyone misses

OpenCode Go costs $5 for the first month, then $10/month. Your usage cap is $12 per 5-hour window, $30/week, $60/month.

When you spend $12 in a 5-hour window on DeepSeek V4 Flash, you get approximately 31,650 requests. When you spend the same $12 on GLM-5.1, you get around 880. Same budget. 36x difference in volume.

This is why routing actually matters. If you pick one model and use it for everything, you are either burning premium requests on tasks that don't need them, or you are under-using cheap models that are surprisingly capable. The right move is assigning models to tasks based on what each task actually requires.

MiniMax M2.5 has a hard cap of 100,000 requests per month regardless of cost. It activates only ~10B parameters and is priced at 16.7x cheaper than Claude Opus 4.6 on input tokens. For high-volume low-complexity work, it is the obvious choice, and most people don't know it exists.

What you lose running on a single premium model

Say you put everything through DeepSeek V4 Pro: 10,200 requests per 5-hour window. That sounds fine for light use. But Oh My OpenAgent runs multiple agents in parallel. Prometheus decomposes your task, Metis synthesizes context, Atlas manages sequencing, Sisyphus runs execution, and the Librarian reads docs. A single complex task can fan out into 30-50 requests without you doing anything. Your 5-hour budget evaporates in a few hours of active work.

The problem isn't the quality gap. V4 Pro at 80.6% is within 7 percentage points of Claude Opus 4.7 at 87.6%, and for most routine tickets that gap is invisible. The problem is you don't need that quality for every step of a multi-agent workflow.

The tier breakdown with actual numbers

Here is what the available models score on benchmarks that matter for coding tasks, plus the API pricing that drives the routing math:

Model	SWE-Bench Verified	Input price (per M tokens)	Requests/5hrs ($12)	Context
Claude Opus 4.7	87.6%	$5.00	~480	200K tokens
DeepSeek V4 Pro	80.6%	$0.435 (promo, ends May 31)	~5,500	1M tokens
Kimi K2.6	80.2%	$0.95	~2,500	256K tokens
Claude Sonnet 4.6	79.6%	$3.00	~800	200K tokens
MiMo-V2.5-Pro	78.9%	~$0.40	~6,000	—
Qwen3.6 Plus	78.8%	$0.325	~7,400	1M tokens
DeepSeek V4 Flash	~79.0%	$0.14	~17,000	1M tokens
GLM-5.1	SWE-Bench Pro 58.4%	~$1.50	~1,600	200K tokens
Qwen3.5 Plus	—	$0.08	~30,000	—
MiniMax M2.5	—	$0.03	up to 100K/month	—

(Requests per 5-hour window calculated at roughly 2,500 average tokens per request.)

Note: Kimi K2.6 original series was discontinued on May 25, 2026. The model itself stays available but the series is no longer receiving updates. DeepSeek V4 Pro's promotional pricing ($0.435/M) ends May 31 — after that the price increases, which changes the requests-per-window math.

Claude Opus 4.7 at 87.6% is genuinely the strongest model available for coding tasks right now, 7 points above V4 Pro. But at $5/M tokens, it costs 35x more than DeepSeek V4 Flash per token. Within the $12/5hr window, you get around 480 Opus 4.7 requests vs 17,000 Flash requests.

DeepSeek V4 Flash sits within one point of V4 Pro in benchmark performance but at about 3x lower cost per token. For most routine coding tasks, that gap does not show up in practice. V4 Flash runs 284B total parameters with 13B active. V4 Pro runs 1.6T total with 49B active.

Kimi K2.6 is a 1-trillion-parameter MoE model with 32B active parameters, 80.2% SWE-Bench Verified. That puts it above Qwen3.6 Plus and close to V4 Pro, making it the right choice for genuinely hard multi-step reasoning when V4 Flash stalls.

GLM-5.1 sits at 744B total / 40B active. Its 200K context makes it suitable for deep planning tasks, and it handles the Oracle and Prometheus roles well at a mid-range cost point.

How Oh My OpenAgent is structured

Oh My OpenAgent v4.2.3 (as of May 2026, with 48K+ GitHub stars) uses a 3-layer architecture:

Planning Layer handles strategic decomposition and knowledge synthesis. Two agents: Prometheus (breaks down what needs to happen) and Metis (synthesizes context and prior knowledge).

Orchestration Layer is Atlas. It maintains a todo-list, enforces sequencing, and tracks completion. It does not do the work itself. It manages what gets done in what order.

Execution Layer is where the work happens. Sisyphus is the default orchestrator with a 32K extended thinking budget. Nine or more specialized agents handle specific task types.

v4.0.0 introduced Team Mode, which activates 7 additional hooks (61 total vs 54 in standard mode). Team Mode is worth enabling if you are running parallel workstreams. It is off by default.

The routing configuration

This is the community-recommended agent-to-model assignment. It is the result of a lot of trial and error, not theory:

Agent	Primary Model	Fallback
Sisyphus	Kimi K2.6	DeepSeek V4 Pro, then Qwen3.6 Plus
Hephaestus	DeepSeek V4 Pro	DeepSeek V4 Flash, then Kimi K2.6
Oracle	GLM-5.1	Kimi K2.6, then DeepSeek V4 Pro
Librarian	DeepSeek V4 Flash	Qwen3.5 Plus
Explore	DeepSeek V4 Flash	none
Prometheus	GLM-5.1	Qwen3.6 Plus, then DeepSeek V4 Pro
Metis	Qwen3.6 Plus	DeepSeek V4 Pro
Atlas	DeepSeek V4 Pro	DeepSeek V4 Flash
Code-reviewer	Kimi K2.6	DeepSeek V4 Pro
Multimodal-Looker	MiMo-V2.5-Pro	Qwen3.6 Plus

Sisyphus gets Kimi K2.6 because it runs extended thinking at up to 32K tokens. You want the strongest reasoning model here, even at lower volume. Kimi's 256K context window handles long execution traces.

Librarian and Explore get V4 Flash. These agents read docs, fetch context, and do lookup work. They do not need frontier-level reasoning. Wasting V4 Pro on Librarian is the single most common budget mistake I see.

Oracle and Prometheus both get GLM-5.1. Planning and deep reasoning are where GLM-5.1 earns its slot. It is not the cheapest model, but it is not the most expensive either, and it performs well on the kinds of open-ended decomposition tasks these agents handle.

Hephaestus (the primary coding agent) gets V4 Pro as primary with V4 Flash as fallback. The gap between them is small enough that on simpler coding tasks, falling back to Flash costs you nothing visible.

MiMo-V2.5-Pro on Multimodal-Looker is deliberate. It scored 78.9% on SWE-Bench Verified and is specifically designed for agentic workflows.

The routing decision rule

Route through V4 Flash first for any task that will exceed 100 requests. Escalate to Kimi K2.6 or V4 Pro only if V4 Flash gets stuck.

This works because V4 Flash at 79.0% SWE-Bench Verified handles the majority of real-world coding tasks correctly. The one-point gap to V4 Pro is real but rarely shows up unless you are hitting genuinely hard tickets. When it does, the fallback chain handles it.

Do not escalate preemptively. Let the model fail first, then escalate. Preemptive escalation is how you burn through your window in an hour.

What $10/month actually buys

At $60/month hard cap (the monthly ceiling), here is the math:

~5 active hours per day across 5 working days = 25 hours of active window time
Each 5-hour window: $12 budget
Routed correctly, a typical Oh My OpenAgent session on a medium-complexity feature might use 400-600 requests, weighted toward V4 Flash and Qwen3.5 Plus

In practice: you can run 8-12 substantial coding sessions per month before feeling the ceiling. For individual developer use, $10/month is genuinely enough. OpenCode hit 150K GitHub stars in May 2026 in part because that math works out.

The realistic comparison: Claude API at similar quality levels would cost $150-300/month for the same volume. That is where the 10-20x cost reduction claim comes from, and in my experience it holds.

The honest trade-off

The gap between this stack and Claude Opus 4.7 on real-world bug fixes is about 7 percentage points. That is real. Some tickets require multiple iterations where Claude would have gotten it right once. Budget for that.

The 7-point gap is an average across all task types. On well-scoped tickets with clear acceptance criteria, the gap narrows significantly. The routing configuration is specifically designed to escalate to Kimi K2.6 or V4 Pro on the tasks where that gap is most likely to show up.

Where this stack genuinely struggles: ambiguous requirements, complex multi-file refactors with implicit dependencies, and tasks that require understanding undocumented system behavior. On those, premium models earn their cost. The routing configuration handles this by putting Kimi K2.6 on the hardest tasks, but Kimi has a 256K context window vs Qwen3.6 Plus's 1M, so very long context tasks may require a different allocation.

The actual configuration

Two files control everything: opencode.json at your project root, and .omc/config.json for Oh My OpenAgent routing.

opencode.json

{
  "$schema": "https://opencode.ai/config.schema.json",
  "theme": "opencode",
  "autoshare": false,
  "model": "deepseek-v4-flash",
  "providers": {
    "opencode": {
      "models": [
        "deepseek-v4-pro",
        "deepseek-v4-flash",
        "kimi-k2.6",
        "glm-5.1",
        "qwen3.6-plus",
        "qwen3.5-plus",
        "mimo-v2.5-pro",
        "minimax-m2.5"
      ]
    }
  }
}

The "model" field sets your default. V4 Flash is the right default because it handles most tasks at lowest cost.

.omc/config.json

{
  "version": "4.2.3",
  "teamMode": false,
  "agents": {
    "sisyphus": {
      "model": "kimi-k2.6",
      "fallback": ["deepseek-v4-pro", "qwen3.6-plus"],
      "thinkingBudget": 32000
    },
    "hephaestus": {
      "model": "deepseek-v4-pro",
      "fallback": ["deepseek-v4-flash", "kimi-k2.6"]
    },
    "oracle": {
      "model": "glm-5.1",
      "fallback": ["kimi-k2.6", "deepseek-v4-pro"]
    },
    "prometheus": {
      "model": "glm-5.1",
      "fallback": ["qwen3.6-plus", "deepseek-v4-pro"]
    },
    "metis": {
      "model": "qwen3.6-plus",
      "fallback": ["deepseek-v4-pro"]
    },
    "atlas": {
      "model": "deepseek-v4-pro",
      "fallback": ["deepseek-v4-flash"]
    },
    "librarian": {
      "model": "deepseek-v4-flash",
      "fallback": ["qwen3.5-plus"]
    },
    "explore": {
      "model": "deepseek-v4-flash",
      "fallback": []
    },
    "code-reviewer": {
      "model": "kimi-k2.6",
      "fallback": ["deepseek-v4-pro"]
    },
    "multimodal-looker": {
      "model": "mimo-v2.5-pro",
      "fallback": ["qwen3.6-plus"]
    }
  },
  "routing": {
    "escalationPolicy": "on-failure",
    "budgetAlert": 10.00,
    "windowBudget": 12.00
  }
}

escalationPolicy: "on-failure" enforces the core rule: models escalate only when the primary fails, not preemptively. budgetAlert triggers a warning at $10 so you know you have $2 left in the window before the ceiling hits.

Quick start

# Install OpenCode Go
npm install -g opencode

# Install Oh My OpenAgent
npx omc install oh-my-openagent

# Create opencode.json and .omc/config.json from the templates above, then:
omc init --preset oh-my-openagent

# Check your current window spend
opencode usage --window current

Knowing where you are in the $12 window changes how aggressively you escalate to premium models.

For a deeper walkthrough of the original configuration approach, the guide that got me started is Jatin Malik's post: OpenCode Go + Oh My OpenAgent: The Complete Guide to SOTA Model Routing Without Hitting Limits. It covers the earlier v4.0-v4.1 configuration in detail and is worth reading alongside this.

I Built a Browser SDK That Detects LLM Agents. Here's How It Works.

Devansh — Fri, 22 May 2026 04:19:55 +0000

Every bot detection system I've seen works with two actors: human or bot. Block the bot, let the human through.

That model is wrong in 2026.

There is a third actor: AI agents acting legitimately on behalf of real users. Shopping assistants. Automated onboarding flows. Fintech integrations. These agents look like bots by every traditional signal: headless browser characteristics, scripted input patterns, no idle pauses. But blocking them means turning away real business.

I built Nyasa to handle all three.

Why existing tools fail now

CAPTCHA was built for bots that couldn't read distorted text. Those bots are dead. CAPTCHA farms solve challenges at $0.50 each. LLM vision models solve them in milliseconds.

Device fingerprinting catches webdriver flags and automation markers. Modern headless browsers patch those out. Playwright, Puppeteer, and every major automation framework have community patches specifically for passing fingerprint checks.

Behavioral analytics (typing cadence, mouse movement) catch scripted bots. But LLM agents don't use scripted input anymore. They type at 60-80 WPM with realistic keystroke intervals, move the mouse in curved paths, and pause at form fields before filling them.

The deeper problem is architectural. Existing systems ask: "Is this a bot?" Nyasa asks: "Who is this session?" The answer is one of three things.

The three-actor model

Nyasa classifies every session as exactly one of:

Human: no detection rules fired
AuthorizedAgent: holds a valid cryptographic identity claim, automatically bypasses bot rules
UnauthorizedBot: one or more detection rules fired

The AuthorizedAgent category is the real addition. An AI shopping assistant built on top of your product shouldn't have to pass a CAPTCHA. It should present a signed identity claim, and your system should recognize and respect it. Nyasa handles that handshake.

The signal stack

24 signals total, split across three layers.

Behavioral signals (13)

Signal	What it measures
Keystroke dwell and flight time	How long keys are held, time between keystrokes
Mouse path curvature	Deviation from straight-line movement
Paste vs typed ratio	Whether text was typed character by character or bulk-pasted
Click precision (center offset)	Distance from click point to element center
Session burst-pause rhythm	Alternation between fast activity and idle gaps
Backspace corrections	Correction frequency during text input
Scroll depth	How far down the page a session goes
Touch mechanics	Multi-touch patterns and pressure distribution
Field-level timing	Time spent on each form field before moving on
Input origin	Typed vs pasted vs dropped vs programmatic fill
Tab visibility	Whether the session loses and regains focus
File upload mechanics	How files are attached (drag, click, or programmatic)
Session rhythm	Overall pace and structure of the session

Fingerprint signals (8)

Signal	What it catches
Webdriver and CDP markers	Automation framework artifacts in the browser object
Iframe vs parent plugin consistency	Mismatches between iframe and parent context
Canvas fingerprint hash	Rendering environment identifier
WebGL renderer string	SwiftShader and LLVMpipe detection (headless GPU emulation)
Audio fingerprint via OfflineAudioContext	Audio processing environment
Incognito detection via storage quota probe	Private browsing mode
Timezone vs navigator.language consistency	Region mismatch between system and browser config
Persistent device UUID with isNew flag	Tracks whether this device has been seen before

Network signals (3)

Signal	What it measures
Page reaction time	Time from page load to first interaction
Connection type	Network type reported by Navigator API
Page load timing	Performance timing breakdown

Detection rules

Six rules run against the collected signals. Each rule fires independently. Any fired rule pushes the verdict toward UnauthorizedBot, except isAuthorizedAgent which short-circuits to AuthorizedAgent regardless of other rules.

isHeadless reads the fingerprint layer for automation markers: webdriver properties, CDP exposure, WebGL renderer strings like SwiftShader or LLVMpipe, iframe/parent inconsistencies. If the browser environment looks like a headless runner, this fires.

isScripted reads behavioral signals for bot-like input patterns. Scripted bots fill fields in milliseconds, never backspace, and move between fields in perfect sequence. This rule catches that pattern.

isLLMAgent is the hardest one. I'll cover it in the next section.

isAuthorizedAgent reads the agent identity claim from window.__nyasaAgentSignature or a meta tag. If a valid claim is present, the session is classified as AuthorizedAgent and no further rules are evaluated.

isUploadAutomation checks file upload mechanics. Human uploads involve a file picker dialog or drag interaction. Programmatic uploads bypass both. This rule catches the bypass pattern.

isMultimodalBot looks for cross-signal contradictions: a session that passes one rule but shows soft signals consistent with automation across several others. It reads sibling DetectionResults directly rather than re-sampling signals, which avoids divergence when two rules read the same underlying data at different times.

isLLMAgent deep dive

This is the rule I spent the most time on. LLM agents are genuinely hard to distinguish from fast, focused humans. Seven signals, evaluated together:

Machine-speed keystroke bursts under 20ms. Human dwell times cluster around 80-200ms. Sub-20ms bursts don't happen in human typing.
Mouse stillness above 70%. Humans move the mouse constantly, even when not clicking. LLM-driven sessions often keep the cursor parked.
Uniform keystroke variance near zero. Human typing has natural rhythm variation. LLM agent keystrokes have suspiciously consistent intervals.
Zero backspace rate. Humans make corrections. An agent filling a form it computed upfront doesn't backspace.
Pixel-perfect click precision. Humans click near the center of an element but not exactly on it. Agents click at the computed center coordinate.
Missing field exploration. Humans often click into a field, leave, return, re-read the label. LLM agents visit each field once in sequence and move on.
No idle micro-pauses. Humans have sub-second pauses between thoughts. Agent sessions show continuous forward progress.

No single signal is definitive. A fast typist has low keystroke variance. A focused user might not backspace. isLLMAgent requires several of these signals to align before it fires.

Feature extraction architecture

Early versions of Nyasa had each detection rule computing its own derived metrics from raw signals. That caused two problems: duplicated math across rules, and rules diverging when they read the same underlying signal at slightly different times during a session.

The feature extraction layer solves this. It runs once per session and computes 8 shared derived metrics before any detection rule evaluates. Every rule reads from the same computed values.

The metrics include things like overall typing variance, click precision distribution, and mouse activity ratio. Computing them once means a detection rule that needs "mouse stillness percentage" and a sibling rule that also needs it will always agree on the number.

isMultimodalBot benefits from this the most. It reads the DetectionResults of its sibling rules rather than re-running signal evaluation. Near-miss composition (where a session nearly triggers multiple rules without fully triggering any) gets caught without any rule having to re-sample data that may have aged out.

The verdict system

Every session gets a verdict object with three fields:

interface NyasaVerdict {
  type: 'Human' | 'AuthorizedAgent' | 'UnauthorizedBot';
  confidence: number;           // 0.0 to 1.0
  badges: DetectionBadge[];     // which rules fired or nearly fired
}

Confidence is a noisy-OR score across all active rules. If one rule fires with 0.8 confidence and a second fires with 0.6, the combined score is 1 - (1 - 0.8) * (1 - 0.6) = 0.92. Multiple weak signals compound.

Badge labels tell you which rules contributed. A session might come back as UnauthorizedBot with badges for isHeadless and isLLMAgent, which tells you this is a headless LLM agent. That gets different handling than a scripted form filler.

The verdict payload ships via navigator.sendBeacon. Non-blocking, fires after the page interaction completes, survives page unload. Your analytics pipeline or backend decision layer receives it without adding latency to the user-facing flow.

Architecture

The SDK runs entirely in the browser. Signals are collected passively as the session progresses. The feature extraction layer runs on a timer and on key events. Detection rules evaluate when a verdict is requested or automatically at session end.

Dual packaging

Nyasa ships as both ESM and IIFE from a single tsup build config.

ESM for bundlers:

import { createNyasa } from '@devanshhq/nyasa';

const nyasa = createNyasa({
  endpoint: 'https://your-backend.com/nyasa',
  agentBypass: true,
});

nyasa.start();

IIFE for script tags, no bundler required:

<script src="https://cdn.jsdelivr.net/npm/@devanshhq/nyasa/dist/nyasa.iife.js"></script>
<script>
  Nyasa.createNyasa({
    endpoint: '/api/nyasa',
    agentBypass: true,
  }).start();
</script>

Same source, two outputs. The tsup config handles the split. No separate build for CDN distribution.

Installation and quick start

npm install @devanshhq/nyasa

Minimal setup:

import { createNyasa } from '@devanshhq/nyasa';

const nyasa = createNyasa({
  endpoint: '/api/session-verdict',
});

nyasa.start();

// Get the verdict at any point
const verdict = await nyasa.getVerdict();
console.log(verdict.type);       // 'Human' | 'AuthorizedAgent' | 'UnauthorizedBot'
console.log(verdict.confidence); // 0.0 - 1.0
console.log(verdict.badges);     // ['isHeadless', 'isLLMAgent', ...]

On the backend, you receive the verdict via the beacon endpoint and decide what to do: allow, challenge, block, or route differently based on the type.

The authorized agent bypass

If you're building an AI agent that needs to interact with Nyasa-protected pages, set the signature before the SDK initializes:

// In your agent code, before navigating to the page
window.__nyasaAgentSignature = {
  token: 'signed-jwt-from-your-auth-server',
  agentId: 'shopping-assistant-v2',
  issuedAt: Date.now(),
};

Or via meta tag for server-rendered flows:

<meta name="nyasa-agent-signature" content="signed-jwt-here" />

The isAuthorizedAgent rule reads this claim, validates the signature, and short-circuits to AuthorizedAgent. No other rules run. The session is logged as a known agent, not blocked as a bot.

This is the part that most bot detection tools don't have a concept for. If you're running a fintech integration or an AI onboarding assistant, you shouldn't have to fight your own security layer.

What it catches that others miss

Traditional fingerprinting misses LLM agents because they run in real browsers with patched automation markers. Traditional behavioral analytics miss them because modern LLM agents have realistic typing cadence.

Nyasa catches them through the combination: machine-speed micro-bursts that no human produces, combined with zero backspace rate and pixel-perfect clicks. Any one signal has false positives. All three together don't.

The multimodal rule catches the edge cases: sessions that pass fingerprinting and look almost human behaviorally but have soft contradictions across signals that don't fit either profile cleanly.

Live: nyasa.devanshtiwari.com

GitHub: github.com/Devansh-365/nyasa

npm: npm install @devanshhq/nyasa

Your Google Sheets backend is silently dropping rows. Here's why.

Devansh — Sun, 26 Apr 2026 23:30:00 +0000

A signup form POSTing to Google Sheets is the most common "backend" on the indie web.

It works for your landing page demo. It works when you test it with 5 friends. It works right up until the moment you don't want it to fail.

Here's the part nobody tells you: Google's own values.append endpoint silently drops rows under concurrent writes. Two simultaneous POSTs can resolve to the same target row and one of them gets overwritten. No error in your logs. No error in your client. Just rows that silently didn't land.

Every "Sheets as a backend" wrapper you've heard of — SheetDB, Sheety, SheetBest, NoCodeAPI — forwards your request straight to values.append. They inherit the bug.

I spent the last few weeks building a fix. It's called SheetForge, it's MIT-licensed, and this post is about the actual bug and why the fix isn't as simple as "throw a mutex on it."

The bug, reproduced in 4 lines

await Promise.all([
  sheets.values.append({ ...rowA }),
  sheets.values.append({ ...rowB }),
  sheets.values.append({ ...rowC }),
  sheets.values.append({ ...rowD }),
]);

Four POSTs. You expect four rows. Under load you often get three. Sometimes two.

The reason is inside values.append. The operation reads the current last row, then writes to the position after it. When two calls race, they can read the same "current last row" and write to the same target cell range. One value wins. The other silently disappears.

Google has documented this. The official workaround is: "Don't write concurrently." That's a fine rule when your launch gets 3 signups. It's actively destructive when you hit HN's front page.

Why your form loses 40% of rows during a traffic spike

Your form looks like this:

// /api/signup
export async function POST(req: Request) {
  const { email } = await req.json()
  await sheets.values.append({ values: [[email, new Date().toISOString()]] })
  return Response.json({ ok: true })
}

This is deployed on Vercel. Each request gets its own isolated serverless invocation. They run truly in parallel. There is zero coordination between them.

Now Product Hunt puts you on the daily leaderboard. Forty people land on your page in the same 10 seconds. Twelve of them submit the form before the others.

Twelve concurrent POSTs to values.append.

You won't end up with twelve rows. You'll end up with something closer to eight, maybe nine. The exact number depends on how Google's backend serializes the writes internally (it doesn't, deterministically — that's the whole problem). The lost rows show no error. The users who submitted them see a green checkmark. Their email is gone.

This isn't theoretical. Levels.fyi wrote a long engineering post about running their entire site on Sheets until exactly this class of problem forced them to migrate.

Why a mutex doesn't fix it

The naive fix is obvious: put a lock in front of the Sheets API call. One write at a time per sheet. Problem solved.

In practice, this is where it breaks:

1. Your API runs on serverless. A lock in Node process memory doesn't work when requests are spread across 12 cold starts. You need a distributed lock.

2. A distributed lock has its own bugs. Redis SETNX with a TTL is the standard answer. But the classic failure mode: process A acquires the lock, gets paused by GC, TTL expires, process B grabs the lock, process A wakes up and releases "its" lock — which is now B's lock. Now two processes think they hold the lock. You're back to silent data loss.

3. Retries break everything. Your client retries a failed POST. The lock-protected write succeeds twice. Now you have duplicate rows. Fixing drops created duplicates.

4. Crashes strand the lock. If your process dies while holding the lock, the next writer has to wait for the TTL to expire. For TTLs long enough to be safe (30+ seconds), that stalls your whole write throughput.

Every one of these is a real bug I hit while prototyping. The fix isn't a mutex. The fix is a proper queue.

How SheetForge actually fixes it

The architecture, in one sentence: every write goes into a per-sheet queue, one worker per sheet pulls from that queue inside a Postgres transaction, and an idempotency key dedupes retries.

Here's the flow:

submitWrite(row)
  └─ INSERT INTO write_ledger (sheet_id, idempotency_key, payload)
     (partial unique index dedupes retries before we even touch Sheets)
  └─ XADD to Redis Stream for this sheet
  └─ return { writeId, status: 'pending' }

processNext()  ← runs in a loop on the worker
  └─ XREADGROUP from the sheet's stream
  └─ BEGIN transaction
     └─ SELECT pg_advisory_xact_lock(hashtextextended(streamKey, 0))
     └─ call sheets.values.append(payload)
     └─ UPDATE write_ledger SET status = 'committed'
  └─ COMMIT
  └─ XACK only after commit

Four things matter here:

The advisory lock is the fence. pg_advisory_xact_lock acquires a lock inside the transaction. If the transaction commits or aborts, the lock is released automatically by Postgres. There is no TTL. No lease clock. No split-brain. If your process dies mid-handler, the transaction rolls back, the lock releases, and the message redelivers.

The idempotency key is the deduper. Every write comes in with an Idempotency-Key header. A partial unique index on (sheet_id, idempotency_key) WHERE status IN ('pending', 'committed') means the database rejects duplicates before the worker even sees them. Retry the same request 100 times, you get 1 row.

The ledger is the truth. The write is durable in Postgres the moment you get the writeId back. Sheets is downstream. If Google's API is down, your writes queue up and flush when it recovers. Your users see a green checkmark and it means something.

XACK happens post-commit. Redis Streams' PEL (pending entries list) redelivers messages if they're not acked. If the worker crashes mid-transaction, Postgres rolls back, Redis redelivers, the idempotency key catches the replay. Exactly-once semantics, for real.

The full test for this concurrency

Every change to the write-queue slice requires a concurrency test. This is the one I do not break:

it('50 parallel writes land 50 rows in order', async () => {
  const sheet = await createTestSheet()
  const writes = Array.from({ length: 50 }, (_, i) => ({
    email: `user-${i}@test.com`,
    idempotencyKey: `key-${i}`,
  }))

  await Promise.all(
    writes.map(w => sheetforge.rows.insert(w, { idempotencyKey: w.idempotencyKey }))
  )
  await waitForQueueDrain(sheet.id)

  const rows = await readSheet(sheet.id)
  expect(rows).toHaveLength(50)
  expect(rows.map(r => r.email)).toEqual(writes.map(w => w.email))
})

50 parallel POSTs. 50 rows. In order. Retry safe.

This same test against raw values.append reliably fails. It fails under SheetDB, Sheety, and SheetBest too — I checked.

The typed SDK is the bonus

Once you have a proper queue, the rest gets interesting. Since SheetForge knows your sheet's header row, it can generate a typed TypeScript client with literal union types inferred from your sample cells.

Header row: email | plan | created_at
Sample cells: hi@example.com | free | 2026-04-15

Generated SDK:

import { createClient } from './sheetforge-client'

const sheet = createClient({
  apiKey: process.env.SHEETFORGE_API_KEY!,
  sheetId: 'sht_abc123',
})

await sheet.rows.insert(
  {
    email: 'hi@example.com',
    plan: 'free',  // 'free' | 'pro' — inferred from sample cells
    created_at: new Date().toISOString(),
  },
  { idempotencyKey: crypto.randomUUID() },
)

The compiler catches header drift. Rename a column in the sheet and regenerate the client — TypeScript tells you every call site that needs updating.

Getting started

One-click hosted:

https://getsheetforge.vercel.app

Self-host:

git clone https://github.com/Devansh-365/sheetforge.git
cd sheetforge
pnpm install
cp .env.example .env   # Google OAuth + DATABASE_URL + Redis
pnpm db:push
pnpm dev               # web :3000, api :3001

Prereqs: Node 20+, pnpm 9+, Postgres 14+, Redis 6+ (or Upstash REST).

The OSS core (packages/queue, packages/codegen, packages/sdk-ts) is MIT and stays free forever. The hosted SaaS runs the same code.

What SheetForge is not

It is not a Postgres replacement. If you need complex queries, indices, or relational integrity, use a real database.

It is not a high-throughput pipe. Google caps you at ~60 writes/minute per sheet regardless of what sits in front. SheetForge makes sure those writes land; it doesn't make them faster.

It is not a reason to keep Sheets as your backend forever. It's the right tool for landing pages, waitlists, internal forms, ops tools, and MVPs where you need rows to actually land. If your app outgrows Sheets, it outgrows SheetForge too.

The one takeaway

Every Sheets-as-backend wrapper you've been using has a silent data-loss bug that only shows up when traffic spikes. That's exactly the moment you most care about losing rows.

If you've ever shipped a form on Sheets and watched rows vanish mid-launch, give SheetForge a try. If it saves you one bug, star the repo.

GitHub: github.com/Devansh-365/sheetforge
Hosted: getsheetforge.vercel.app
License: MIT

Gemini 2.5 Flash was returning 37 tokens. Here's why.

Devansh — Sun, 19 Apr 2026 23:30:00 +0000

I set max_tokens: 1000 on a Gemini 2.5 Flash call.

The response came back with 37 tokens. finish_reason: "MAX_TOKENS". No error. No warning. Just a string that stopped mid-sentence.

I changed it to 2000. Got back 41 tokens. Then 5000. Got back 38.

That's when I knew something was actually broken, not just a config issue.

I spent a day tracing this. The root cause is surprising, the official docs don't explain it, and the fix depends on which version of which SDK you're using. Here's what I learned, and a diagnostic script at the end so you can figure out which variant of the bug you hit.

The symptom

Your Gemini 2.5 Flash or Pro call returns one of these shapes:

{
  "candidates": [{
    "content": { "parts": [{ "text": "" }] },
    "finishReason": "MAX_TOKENS"
  }],
  "usageMetadata": {
    "promptTokenCount": 120,
    "candidatesTokenCount": 0,
    "thoughtsTokenCount": 964,
    "totalTokenCount": 1084
  }
}

Or a truncated mid-sentence response with candidatesTokenCount near zero and thoughtsTokenCount close to whatever you set max_output_tokens to.

The word thoughtsTokenCount is the giveaway.

Why it happens

Gemini 2.5 Flash and Pro are reasoning models. Like OpenAI's o-series, they burn tokens on internal reasoning before writing the visible response. Unlike OpenAI's models, Google counts those thinking tokens against your max_output_tokens budget.

So when you ask for 1,000 tokens:

The model thinks. This uses some number of tokens, tracked as thoughtsTokenCount.
Once thoughtsTokenCount + candidatesTokenCount hits your budget, generation stops.
If thinking consumed most of the budget, candidatesTokenCount ends up near zero.

Gemini 2.5 Flash defaults to a dynamic thinking budget. It decides how much to think based on the task. For anything non-trivial, it will happily burn 90 to 98 percent of your budget on reasoning.

You can see this directly in the API response. If you're using the Google GenAI SDK:

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Summarize quantum computing.",
    config={"max_output_tokens": 1000}
)

print("Output tokens:", response.usage_metadata.candidates_token_count)
print("Thinking tokens:", response.usage_metadata.thoughts_token_count)
print("Total:", response.usage_metadata.total_token_count)
print("Finish reason:", response.candidates[0].finish_reason)

The thoughts_token_count field is where your budget actually went.

The three fixes, ranked

There are three ways to handle this, and they have real tradeoffs.

Fix	What it does	Latency	Cost	Quality	Best for
Disable thinking	`thinking_budget: 0` (Flash) or `reasoning_effort: "none"`	Fast	Low	Lower on complex reasoning	Chat UIs, structured extraction, high-volume endpoints
Cap thinking	`thinking_budget: 1024` + `max_output_tokens: 8192`	Medium	Medium	Good	Most production workloads
Dynamic thinking	Let Flash decide, set `max_output_tokens` to 8K+	Slowest	Highest	Best	Research queries, complex analysis, one-shot deep tasks

The third option is the default, and it's the source of the bug. It's only the right choice if you're actually okay with burning most of your tokens on reasoning and waiting 5 to 30 seconds per response.

Fix 1: Disable thinking for Flash

For Gemini 2.5 Flash, you can turn thinking off entirely:

from google import genai
from google.genai import types

client = genai.Client()

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Explain circuit breakers in 2 sentences.",
    config=types.GenerateContentConfig(
        max_output_tokens=1000,
        thinking_config=types.ThinkingConfig(thinking_budget=0)
    )
)

thinking_budget=0 is only valid for 2.5 Flash. Pro refuses to run without at least some thinking, and throws:

Thinking can't be disabled for this model.

For Pro, the minimum accepted value is 128. Using thinking_budget=128 gets you the closest thing to "off" that Pro allows.

Fix 2: The OpenAI-compat escape hatch (underdocumented)

If you're hitting Gemini through the OpenAI-compatible endpoint (either Google's own generativelanguage.googleapis.com/v1beta/openai or through a proxy like LiteLLM), you can use reasoning_effort instead of thinking_budget:

from openai import OpenAI

client = OpenAI(
    base_url="https://generativelanguage.googleapis.com/v1beta/openai",
    api_key=GEMINI_API_KEY
)

response = client.chat.completions.create(
    model="gemini-2.5-flash",
    messages=[{"role": "user", "content": "Explain circuit breakers"}],
    max_tokens=1000,
    reasoning_effort="none"  # or "low", "medium", "high"
)

This is barely documented. Google's official OpenAI-compatibility page mentions it in passing, and almost no tutorials cover it. But it works, and it's the cleanest way to control reasoning from code that uses the OpenAI SDK.

Mapping:

reasoning_effort: "none" → thinking_budget: 0 (Flash only)
reasoning_effort: "low" → thinking_budget: 1024
reasoning_effort: "medium" → thinking_budget: 8192
reasoning_effort: "high" → thinking_budget: 24576

Fix 3: The integration-specific gotchas

The bug manifests differently depending on your stack. Some quick notes from actual GitHub issues (python-genai #782, gemini-cli #23081, langchain-google #1490):

LangChain silently truncates output. Developers report setting max_tokens=16000 and still getting cut-off responses. Fix: pass thinking_budget via model_kwargs, or switch to the OpenAI-compat endpoint through ChatOpenAI.

LiteLLM accepts reasoning_effort and maps it to the Gemini parameter, but as of late 2025 it rejected the parameter for Pro with "Thinking can't be disabled." Fix: use reasoning_effort="low" instead of "none" for Pro.

ha-llmvision defaulted thinkingBudget to 35 to 50 tokens. That value gets fully consumed by thinking, leaving nothing for output. Fix: set to 1024 or higher.

Cline set thinkingBudget: 0 which works for Flash Lite but throws on Pro. Fix depends on which model you're targeting.

Vertex AI uses thinkingConfig.thinkingBudget nested inside the config object. Raw API requests that put it at the top level silently ignore it.

Diagnostic script

If you're not sure which variant of the bug you hit, run this:

from google import genai
client = genai.Client()

def diagnose(model: str, prompt: str, max_tokens: int):
    response = client.models.generate_content(
        model=model,
        contents=prompt,
        config={"max_output_tokens": max_tokens}
    )
    usage = response.usage_metadata
    text = response.candidates[0].content.parts[0].text or ""
    finish = response.candidates[0].finish_reason

    thinking_pct = (usage.thoughts_token_count / max_tokens * 100) if usage.thoughts_token_count else 0
    output_pct = (usage.candidates_token_count / max_tokens * 100) if usage.candidates_token_count else 0

    print(f"Model:          {model}")
    print(f"Budget:         {max_tokens}")
    print(f"Thinking used:  {usage.thoughts_token_count} ({thinking_pct:.0f}%)")
    print(f"Output tokens:  {usage.candidates_token_count} ({output_pct:.0f}%)")
    print(f"Finish reason:  {finish}")
    print(f"Response len:   {len(text)} chars")

    if finish == "MAX_TOKENS" and usage.candidates_token_count < max_tokens * 0.1:
        print("\nDIAGNOSIS: Thinking tokens ate your budget.")
        print("FIX: Set thinking_budget=0 (Flash) or reasoning_effort='none'.")
    elif finish == "MAX_TOKENS":
        print("\nDIAGNOSIS: Output actually hit the cap. Raise max_output_tokens.")

diagnose("gemini-2.5-flash", "Write a short poem about debugging.", 200)

The script prints a percentage breakdown showing exactly where your budget went. If thinking is over 50 percent of your budget, you need to cap it.

The gateway-level fix

All of this is fixable at the application layer, but it requires every caller to know about reasoning budgets. That doesn't scale if you have multiple services calling Gemini.

I run FreeLLM in front of my LLM calls. It's an OpenAI-compatible gateway that routes across six providers, and it sets the right reasoning budget per Gemini model automatically. Flash gets reasoning_effort: "none". Pro gets "low". Your full max_tokens budget goes to the actual answer. You can override per-request if you need reasoning back.

curl http://localhost:3000/v1/chat/completions \
  -d '{
    "model": "gemini/gemini-2.5-flash",
    "messages": [{"role": "user", "content": "Hello"}],
    "max_tokens": 1000
  }'

Before the gateway: 37 output tokens. After: 670+ tokens, finish_reason: stop. Same prompt, same budget.

The point is not "use my tool." The point is that gateway-level defaults let you fix provider quirks once instead of in every service.

What to take away

Gemini 2.5 is a reasoning model. Its thinking tokens count against your max_output_tokens.
The dynamic default will eat 90 to 98 percent of your budget on anything non-trivial.
For Flash, disable thinking with thinking_budget: 0 or reasoning_effort: "none".
For Pro, cap thinking with thinking_budget: 128 (minimum) or reasoning_effort: "low".
If you're using an OpenAI-compat endpoint, reasoning_effort is cleaner and underdocumented.
Run the diagnostic script above when in doubt.

The official docs don't make any of this obvious. Hopefully this post saves you the day I spent figuring it out.

GitHub: github.com/devansh-365/freellm (the gateway that handles this for you)

LiteLLM got hacked. I built a simpler LLM gateway you can actually audit.

Devansh — Tue, 14 Apr 2026 00:10:45 +0000

On March 24, 2026, LiteLLM versions 1.82.7 and 1.82.8 were uploaded to PyPI with a credential harvester, a Kubernetes lateral-movement toolkit, and a persistent remote code execution backdoor baked in.

The malicious package was live for about 40 minutes before PyPI quarantined it.

40 minutes doesn't sound like much. But LiteLLM gets 95 million downloads a month. It's the default multi-provider routing library for anyone building on LLMs. Teams running pip install litellm during that window got compromised automatically. No explicit import needed. The payload triggered on Python interpreter startup via a .pth file.

Google brought in Mandiant for the investigation. Snyk, Kaspersky, and Trend Micro all published breakdowns. The attack vector: a compromised Trivy security scanner leaked CircleCI credentials, including the PyPI publishing token and a GitHub PAT.

This is not a theoretical risk. This happened.

The real problem is not one attack

LiteLLM does a lot. 2,000+ models across 100+ providers. Proxy server, load balancing, spend tracking, A/B testing, caching, logging, guardrails, prompt management.

That scope is the problem.

A developer on HN described the codebase as having a 7,000+ line utils.py. A 30-year engineer called it "the worst code I have ever read in my life." Before the supply chain attack, a DEV Community post titled "5 Real Issues With LiteLLM That Are Pushing Teams Away in 2026" was already documenting the trust erosion.

The supply chain attack was the tipping point, not the root cause. The root cause is depending on a massive, opaque library for critical routing infrastructure.

What a simpler design looks like

I ran into the same multi-provider routing problem last year while building Metis, an AI stock analysis tool. Kept burning through Groq's free tier in 20 minutes, switching to Gemini manually, hitting their cap, switching again.

Built FreeLLM to stop doing that manually. It solves a narrower problem than LiteLLM, and that's the point.

FreeLLM is an OpenAI-compatible gateway that routes across Groq, Gemini, Mistral, Cerebras, NVIDIA NIM, and Ollama. When one provider rate-limits, the next one answers. That's the core of it.

What it does

curl http://localhost:3000/v1/chat/completions \
  -d '{"model": "free-fast", "messages": [{"role": "user", "content": "Hello!"}]}'

Your existing OpenAI SDK code works. Swap the base URL. Keep your code.

Three meta-models handle routing: free-fast (lowest latency, usually Groq/Cerebras), free-smart (best reasoning, usually Gemini 2.5 Pro), and free (max availability).

What it fixes that LiteLLM doesn't

Gemini 2.5 reasoning tokens eating your output. This is one of the most reported Gemini bugs right now. Gemini 2.5 Flash and Pro are reasoning models. They burn 90-98% of your max_tokens on internal thinking before producing visible text. Ask for 1,000 tokens and you get back 37. There are 15+ open GitHub issues about this across multiple SDKs.

FreeLLM fixes it at the gateway. Flash gets reasoning_effort: "none" by default. Pro gets "low". Your full token budget goes to the actual answer. Override per-request if you want the reasoning back.

Provider outages don't break your app. Claude went down for three consecutive days in early April. 8,000+ Downdetector reports. If your app depends on one provider, that's three days of broken service. FreeLLM's circuit breakers pull failing providers from rotation and test for recovery automatically.

Response caching without a separate layer. Identical prompts return in ~23ms with zero quota burn. The cache refuses to store truncated responses (another Gemini bug: reasoning models returning cut-off output that then poisons your cache for an hour).

Browser-safe tokens for static sites. Mint a short-lived HMAC-signed token from a serverless function, pass it to the browser, call the gateway directly from client-side JavaScript. No auth backend. No session store.

Key stacking: 360 free requests per minute

Every provider env var accepts a comma-separated list. FreeLLM rotates round-robin per key.

GROQ_API_KEY=gsk_key1,gsk_key2,gsk_key3
GEMINI_API_KEY=AI_key1,AI_key2,AI_key3

Stack 3 keys across 5 cloud providers: ~360 req/min. All free. Enough to prototype an entire product without spending anything.

Get it running

Docker:

docker run -d -p 3000:3000 \
  -e GROQ_API_KEY=gsk_... \
  -e GEMINI_API_KEY=AI... \
  ghcr.io/devansh-365/freellm:latest

Or one-click deploy on Railway or Render (buttons in the README).

Use it from Python:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:3000/v1", api_key="unused")
response = client.chat.completions.create(
    model="free-smart",
    messages=[{"role": "user", "content": "Explain circuit breakers"}]
)

TypeScript, Go, Ruby, anything that speaks OpenAI. Same pattern.

Why this matters beyond FreeLLM

The LiteLLM attack exposed something the community already suspected: critical AI infrastructure is running on libraries nobody audits.

The fix is not "use my tool instead." The fix is smaller dependencies, pinned versions, codebases you can read in an afternoon. FreeLLM is 262 tests across 22 files. TypeScript, not Python. Docker images with pinned deps. MIT licensed.

If you don't use FreeLLM, build something similarly scoped. The era of "install this 100-provider mega-library and trust it with your API keys" should be over.

262 tests. 6 providers. One endpoint. Zero cost.

GitHub: github.com/devansh-365/freellm

I built an OpenAI-compatible gateway that routes across 5 free LLM providers

Devansh — Mon, 06 Apr 2026 20:22:07 +0000

Every LLM provider has a free tier.

Groq gives you 30 requests per minute. Gemini gives you 15. Cerebras gives you 30. Mistral gives you 5.

Combined, that's about 80 requests per minute. Enough for prototyping, internal tools, and side projects where you don't want to pay for API access yet.

The problem: each provider has its own SDK, its own rate limits, its own auth, and its own downtime. You end up writing provider-switching logic, catching 429 errors, and managing API keys across five different dashboards.

I got tired of this while building Metis, an AI stock analysis tool. Kept hitting Groq's limits while Gemini had capacity sitting idle. So I built FreeLLM.

What FreeLLM does

One endpoint. Five providers. Twenty models. All free.

curl http://localhost:3000/v1/chat/completions \
  -d '{"model": "free-fast", "messages": [{"role": "user", "content": "Hello!"}]}'

Your existing OpenAI SDK code works. Just change the base URL. That's the whole migration.

How the routing works

When a request comes in, FreeLLM:

Checks which providers are healthy (circuit breakers track this automatically)
Picks the best available provider based on your model choice
If that provider returns a 429 or fails, it tries the next one
You get a response

Three meta-models handle routing:

free-fast   → lowest latency (usually Groq or Cerebras)
free-smart  → most capable model (usually Gemini 2.5)
free        → maximum availability across all providers

Providers and their free tiers

Provider	Models	Free Tier
Groq	Llama 3.3 70B, Llama 4 Scout, Qwen3 32B	~30 req/min
Gemini	2.5 Flash, 2.5 Pro, 2.0 Flash	~15 req/min
Cerebras	Llama 3.1 8B, Qwen3 235B, GPT-OSS 120B	~30 req/min
Mistral	Small, Medium, Nemo	~5 req/min
Ollama	Any local model	Unlimited

What's under the hood

This isn't a simple round-robin proxy. The routing layer handles real production concerns:

Sliding-window rate limiter. Each provider's limits are tracked independently. FreeLLM knows how many requests you've sent to Groq in the last 60 seconds and won't send another if you're near the cap.

Circuit breakers. If Gemini starts returning 500s, FreeLLM pulls it from rotation. Every 30 seconds, it sends a test request. When the provider recovers, it goes back in.

Per-client rate limiting. If you expose this to a team, each client gets their own limit. Admin auth protects the config endpoints.

Zod validation. Every request is validated before it hits any provider. Bad payloads fail fast with clear error messages.

Real-time dashboard. React frontend showing provider health, request logs, and latency. You can see which providers are healthy at a glance.

Get it running in 30 seconds

git clone https://github.com/devansh-365/freellm.git
cd freellm
cp .env.example .env   # add your free API keys
docker compose up

API on localhost:3000. Dashboard on localhost:3000/dashboard. Done.

Using it with the OpenAI SDK

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:3000/v1",
  apiKey: "not-needed",
});

const response = await client.chat.completions.create({
  model: "free-fast",
  messages: [{ role: "user", content: "Explain circuit breakers in 2 sentences" }],
});

No new SDK to learn. No migration effort.

Why I built this

I was building Metis and kept running into the same pattern: burn through Groq's free tier in 20 minutes of testing, switch to Gemini manually, hit their limit, switch to Mistral. Repeat.

Wrote a quick proxy to automate the switching. Added failover because providers go down randomly. Added circuit breakers because I didn't want to wait for timeouts. Added a dashboard because I wanted to see what was happening.

It grew into a proper tool. Open-sourced it because every developer prototyping with LLMs has this exact problem.

Stack

TypeScript, Express 5, React 19, Zod, Docker. MIT licensed.