I opened a PR from bed using a 4-bit local model as the orchestrator

Last night I opened a PR without touching my laptop. Six commits, eight files changed, 417 tests passing. I was in bed on my phone the whole time.

This isn't a "local model wrote everything" story. The local model is small, quantised, and deliberately not responsible for implementation. Its job is to understand what I want, hand the task to a stronger coding agent, and report back.

I want to explain how, but first I want to explain why I bothered, because the why shaped every decision in the setup.

The actual problem

I'm an international student. I work part-time jobs to fund myself.

The thing nobody tells you about part-time work is that it doesn't just take the hours you're on shift. It takes the hour before and the hour after, in chunks too small to start anything real. You can't open a project, load the whole thing back into your head, and get somewhere useful in the forty minutes before you have to leave. So those hours go nowhere. My assignments and my own projects were losing to a scheduling problem, not a skill problem.

That's what I was trying to fix. Not coding speed. I wanted the work to move forward when I'm not in front of the machine.

The bit that made it work

I kept getting stuck on the assumption that I needed a good coding model running locally. A good coding model does not fit on a student laptop.

Then it clicked that I had the requirement wrong. The local model doesn't have to write code. It has to understand what I asked for in a Discord message, pick the right command, hand it to something that can code, and tell me what happened.

Reading intent and dispatching is a much smaller job than implementing.

Once you separate orchestration from implementation, the hardware problem mostly goes away.

The stack

Hermes Agent from Nous Research is the orchestrator. It's MIT licensed, it has a messaging gateway that speaks Discord, Telegram, Slack and others, and it ships with skills for delegating to Claude Code and OpenCode already installed. That last part is why this build is much less work than it sounds. I did not write any glue code.

Running it on a student laptop:

• Orchestrator model: Qwen3.6-35B-A3B: a Mixture-of-Experts model with 35B total parameters but only ~3B active per token (that's the "A3B"). Unsloth GGUF at UD-Q4_K_XL, served by LM Studio.
• Machine: RTX 5070 Ti mobile, 12GB VRAM, 32GB RAM, Core Ultra 9 275HX, Windows
• LM Studio: context set to the model's native 262,144, FP8 KV cache quantisation, GPU offload set to all layers, with 32 expert layers forced onto CPU
• Hermes and all coding tools: inside WSL
• Implementation: Claude Code with Opus, or OpenCode with DeepSeek V4 Flash NEW (free on OpenCode Zen)

That LM Studio line looks contradictory and isn't. The offload slider is set to all layers, and separately, LM Studio can push the MoE expert weights for a given number of layers back to system RAM. The attention path stays on the GPU where it needs the bandwidth; the bulky expert weights sit in the 32GB of system memory. That split is what makes a 35B model fit alongside 12GB of VRAM. It's not fast, but dispatching a command doesn't need to be fast.

The full 262k context is less alarming than it sounds for the same architectural reason. Qwen3.6 is a hybrid: most layers use Gated DeltaNet linear attention, which carries a fixed-size state instead of a KV cache that grows with sequence length. Only the standard attention layers pay per-token, so the cache cost at long context is a fraction of what a fully dense model would demand. In practice my orchestration turns never get near the limit anyway.

On cost, I want to be precise, because "totally free local AI" posts usually lie by omission. The orchestration layer is genuinely free and runs on hardware I already own. The layer that writes code is the layer I pay for, and right now I don't: DeepSeek V4 Flash NEW is on OpenCode Zen's free tier, so the PR in this post cost $0.00 in API spend. Planning, implementation, six commits, the lot.

That's a real number, not a rounding. It's also a promotional tier, and it will not last forever. The honest version of the claim is that the free part is structural and the free part is temporary: orchestration stays free because it's my hardware, and implementation is free until Zen decides otherwise. If you're reading this some months from now, check the pricing before assuming the economics still hold.

Setting it up

1. Serve the model in LM Studio

Load the model, then go to the developer tab and turn on:

• Serve on local network (on)
• Enable CORS (on)

Note the base URL it gives you. It'll look like http://192.168.x.x:1234/v1.

2. The WSL networking gotcha

This is the step that will eat your afternoon if you don't know about it.

LM Studio runs on Windows. Hermes runs in WSL. WSL is a separate network namespace, so localhost:1234 from inside WSL does not reach LM Studio on the Windows host. That's why "serve on local network" has to be on and why you point Hermes at the machine's LAN IP rather than localhost.

Check it from WSL before going any further:

curl http://192.168.x.x:1234/v1/models

You want JSON back with your model ID in it. If it hangs, it's almost always Windows Defender Firewall blocking inbound connections on that port; allow LM Studio through for private networks. If it refuses outright, the server isn't running or the IP is wrong.

Don't move on to Discord until this works. Also note the LAN IP can change on DHCP renewal, so if the whole thing mysteriously dies in a week, check this first.

3. Install Hermes

In WSL:

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
source ~/.bashrc

Then run the wizard:

hermes setup

4. Point Hermes at LM Studio

hermes model

Choose a custom or OpenAI-compatible endpoint and give it the base URL from step 1. Confirm it works by starting the TUI with hermes and sending a message. If you get a response, the local model is wired up and the rest is downstream of something that already works.

Do not skip this verification. Debugging a broken Discord gateway is much harder when you don't yet know whether the model itself is reachable.

5. Install the coding agents

npm install -g @anthropic-ai/claude-code
npm i -g opencode-ai@latest

Authenticate both. Run claude once to log in, and opencode auth login for whichever provider you're using. Verify:

claude --version
opencode auth list

Also make sure gh is installed and logged in, since that's what actually opens the PR:

gh auth status

The Hermes skills for both agents are bundled and installed by default, so there's nothing to configure once the CLIs work.

6. Create the Discord bot

Go to the Discord Developer Portal, create an application, then create a bot under it.

Under Privileged Gateway Intents, enable:

• Message Content Intent (required, or the bot receives your messages with empty text)
• Server Members Intent (required, for resolving who's messaging)

Copy the bot token. It's shown once.

Invite it with this URL, substituting your application ID:

https://discord.com/oauth2/authorize?client_id=YOUR_APP_ID&scope=bot+applications.commands&permissions=274878286912

Then turn on Developer Mode in Discord (Settings, Advanced) and right-click your own username to copy your user ID.

7. Wire up the gateway

hermes gateway setup

Pick Discord, paste the token and your user ID. Or edit ~/.hermes/.env directly:

DISCORD_BOT_TOKEN=your-bot-token
DISCORD_ALLOWED_USERS=your-discord-user-id

Start it:

hermes gateway

DISCORD_ALLOWED_USERS is not optional. Recent Hermes versions fail closed, so a gateway with no allowlist will connect and then deny every inbound message. More importantly, anyone who can message that bot can run commands against your repos. Set it to your ID and nothing else.

While you're here, the rest of the blast radius is worth thinking about for two minutes. This setup turns a chat message into shell commands on your machine, which is the entire point and also the entire risk:

• Treat the bot token like a password. If it leaks, reset it in the Developer Portal.
• chmod 600 ~/.hermes/.env, and never commit it.
• Keep LM Studio on your LAN. It has no auth; do not port-forward it.
• Turn on branch protection and require CI, so a bad PR can't merge itself.
• Don't point this at a repo whose credentials you'd mind an agent stumbling into.

If the bot shows up online but silently ignores you, it's the Message Content Intent nine times out of ten. Go back to step 6.

8. Optional: a home channel

DISCORD_HOME_CHANNEL=your-channel-id

This is where proactive messages land, which matters once you start using cron jobs or want completion notifications somewhere predictable.

What a run looks like

The loop is three beats: pick an issue, get a plan, ship it.

Worth watching the first few seconds. Hermes guesses the owner wrong, tries Aegis-MD/Aegis-MD, finds nothing, then works out the repo is under my own username at PyaesoneP/Aegis-MD and carries on. That's the small model guessing at my intent, and this time it caught itself. It doesn't always, which is most of the next section.

Then I ask it to use OpenCode's plan mode on issue #44. It clones into a temp dir and hands the task down.

The plan came back better than I expected. It flagged three blocking design decisions before writing anything, including a naming collision between year_published and the existing publication_year, and a question about whether Chroma could store the list fields directly. Those are decisions I want to make myself. Having them surfaced as questions rather than silently resolved is the difference between a PR I can review on a bus and one I can't.

Once I've answered those, I tell it to go ahead.

CI runs, and I get the PR link with a summary of what landed. If I want a second read, I have it invoke Claude Code in print mode over the diff before I look at it.

End to end, planning to open PR, that run took about 40 minutes. I was not watching it. That's the part that matters: 40 minutes of wall-clock time costs me nothing if I'm on a shift, and it would have cost me an entire evening block if I'd sat with it.

That's the whole loop. It works on the bus, on the train, and on my break at work.

One practical note: my laptop has to stay awake for any of this. There's no magic here. If I know I'll want to work on the go, I leave it running before I leave the house. Windows sleep settings will absolutely ruin your afternoon otherwise.

Where this breaks

This is the section I'd want to read if someone else wrote this post.

The weakest model in the chain has the most authority

Qwen at 4-bit decides which issue I meant and what instruction goes downstream. If it misreads me, nothing errors out. It confidently dispatches the wrong task and I find out when a PR appears for something I never asked for. Cheap fix: have it echo its interpretation back before dispatching. "Working on #44, extending chunk metadata, go?" One extra message catches most of this. A fixed command vocabulary instead of free-form passthrough helps too, because then a misparse fails loudly instead of quietly succeeding at the wrong thing.

Review debt is deferred, not deleted

I didn't sit in front of the laptop for hours. Fine. Someone still has to read a +630 / -37 diff, and on a phone, on a bus, you are skimming. You are not reviewing. Be honest with yourself about which one you're doing.

A model reviewing a model's PR is weaker than it feels

Claude Code is good at catching inconsistencies inside a diff. It's much worse at noticing the entire approach was wrong, because it's reading the same framing that produced the code. Treat CI as the real gate. 417 green tests is a harder signal than any model saying it looks good. Branch protection on, nothing merges without me. I don't merge from the bus. I queue the review for when I'm back at a desk.

The bottleneck moves upstream to your issues

When implementation gets close to free, your ceiling becomes how well you wrote the ticket. A vague issue now produces a confident, well-tested, completely wrong PR, and you spend the review cycle discovering that. I write issues much more carefully than I used to, which was not a change I expected from this.

Keep a do-not-touch list

Mine is migrations, auth, dependency bumps, and CI config. Those get flagged for when I'm at a desk, no matter how clean the PR looks from the bus. The failure modes there are too expensive to catch on a phone screen.

Would I recommend it

It depends on your constraint, and I'd rather be specific than encouraging.

This is worth building if you have short blocks of time away from your machine, a laptop that can hold a small model in memory, a project with clear issues, and tests you actually trust. Every one of those matters. Take away the tests and you've built a machine for generating plausible-looking mistakes.

It's not worth much if you're already at your desk all day. You'd be adding a lossy translation layer between yourself and tools you could use directly. It's also not a way to stop reviewing code, and it won't rescue a project with vague issues and no CI. It makes those problems arrive faster.

What it actually does is narrow: it takes the hours that used to go nowhere, the shift breaks and the bus rides and the twenty minutes before class, and makes them hours where something moves. For me that was the whole point. I'm still working out what else to point it at, and triage seems like the obvious next one. If you've built something in this shape, I'd like to hear what broke.