Coding with My QuietBox

Quick Disclosure: same as always, I work for Tenstorrent. That said I'll call out the rough edges as readily as the wins.

I've spent the last few months getting real AI compute into my house. Big models, weird streaming tricks, a QuietBox 2 sitting in my lab humming away. Most of that has been benchmarking work. At some point you have to stop measuring the thing and start using it.

So here's the using it. I pointed OpenCode, a terminal coding agent, at my own inference server, and told it to build me a Tetris clone. The whole thing stayed on my LAN. Setup took about ninety seconds, generation took a minute nine, and it worked.

Why Coding Agents

Coding agents are a good stress test for a local setup. They need tool calling that actually works, since the agent has to read and write files on its own. They need a long context window, because it's pulling your project into the prompt. And they need to hold a plan together across several steps.

That's a lot of surface area to get right at once. When it all lines up, you get a pretty strong signal that the rest of your local tooling will work too.

On the QB2 Side

This part is a single command. It will pull the model with its weights, then serve it.

tt serve mando2222/qwen3.8-27b-dflash2-p300x2-q4kv

That's Qwen3.8-27B, quantized to 8 bit with a Q4 KV cache, sharded across the two P300 cards in the QuietBox 2. If you read my QuietBox post, this is the 1x4 fabric config doing its thing: four chips across two physical cards, pooling their memory so a 27B model with a big context window fits comfortably.

The serve command gives you an OpenAI compatible endpoint. That compatibility is the whole ballgame for what comes next. It means tools built for the OpenAI API can point at your box with a config change and no code change.

Where That Model Came From

Those two commands hide a few months of work, and a good chunk of it isn't mine.

The model string qwen3.8-27b-dflash2-p300x2-q4kv looks like a mouthful, but every segment of it is a decision somebody had to make. Which attention implementation to use. How the weights get split across the cards. What precision the KV cache runs at, which is the difference between a comfortable 256k context window and running out of memory halfway through a long file. I didn't work any of that out from a blank page.

tt-model has a growing set of published model configs, and people have been putting real effort into them. I started by pulling down what was already there and reading it carefully. Different configs were good at different things. One had the attention path sorted out. Another had a smarter approach to how the shards were laid out across chips. A third had done the legwork on quantization quality, which is the part where it's easy to save memory and quietly lose a chunk of the model's ability to follow instructions.

Most of what I did was integration work. Take the win from one config, take the win from another, get them to coexist, then measure whether the combination actually held up instead of assuming it would. That last step is where most of the time went. A lot of runs, a lot of tweaking one variable at a time, a lot of results that looked promising and didn't survive a second look.

None of that would have happened on the timeline it did if the starting points weren't already out there in public. I want to be clear about that, same as I was with Colibri in my earlier post. Credit belongs upstream. The reason I published this config back to tt-model instead of keeping it in a folder on my box is that it's how the whole thing keeps working. Somebody's going to pull this down, find the three things I got wrong, and publish something better.

On the Mac Side

With OpenCode installed and your inference server running, connecting them is four steps.

1. Get the model ID from your server

curl -fsS http://YOURIP:20000/v1/models

Replace the address with your server's IP and port. Use the exact model ID that comes back. In my case that's Qwen/Qwen3.8-27B. This matters more than it looks like it should. If this string doesn't match exactly, OpenCode will hand you an error that doesn't obviously point at the real problem.

2. Create the OpenCode configuration

mkdir -p ~/.config/opencode
vi ~/.config/opencode/opencode.jsonc

Then drop this in, adjusting the server address and model settings for your deployment:

{
  "$schema": "https://opencode.ai/config.json",
  "model": "qb2/qwen-qb2",
  "provider": {
    "qb2": {
      "name": "QB2 node6",
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://YOURIP:20000/v1"
      },
      "models": {
        "qwen-qb2": {
          "id": "Qwen/Qwen3.8-27B",
          "name": "Qwen3.8-27B",
          "tool_call": true,
          "reasoning": true,
          "modalities": {
            "input": ["text"],
            "output": ["text"]
          },
          "interleaved": {
            "field": "reasoning"
          },
          "limit": {
            "context": 262144,
            "output": 16384
          }
        }
      }
    }
  }
}

A few things worth understanding rather than just copying:
qb2 is your provider alias and qwen-qb2 is your model alias. Those are names you pick, and they're what you'll type on the command line. The id field is the one that has to match your server's model ID exactly, because it's what actually gets sent over the wire.

Keep the /v1 on the base URL. The @ai-sdk/openai-compatible package expects the OpenAI route layout underneath it, and leaving it off is the kind of mistake that costs you twenty minutes.

The tool_call and reasoning flags tell OpenCode what this model can do. tool_call is what lets the agent create files and run commands, so it needs to be on for any of this to work. The interleaved block tells OpenCode where to find the model's thinking in the response stream, which is why you see those "Thought: 338ms" lines in the output later.

The token limits should match your server. And this example assumes no authentication on the endpoint, which is fine for a box on your own LAN. Lock it down before exposing it anywhere else.

3. Verify the connection

opencode models qb2

You should see:

qb2/qwen-qb2

Then send a test prompt that isolates the connection from everything else:

opencode run -m qb2/qwen-qb2 \
  "Reply with exactly QB2_OK. Do not use any tools."

I like this test because it fails in a useful way. If QB2_OK comes back, your networking, your model ID, and your response parsing are all correct. Anything else tells you the problem sits upstream of any agent behavior, so you can stop debugging your config and go look at the server.

4. Start coding

From your project directory:

opencode

That's it. You're now running a coding agent against hardware you own.

So How Did I Test

I made a fresh directory, fired up OpenCode, and typed a standard prompt I use to get a feel of any new coding model:

make me a tetris clone

Tetris works well as a test because it's specific. There's a defined set of seven tetrominoes. Rotation has to work, including wall kicks when you rotate against the edge of the board. Collision detection has to be right in four directions. Line clears have to shift everything above them down. Scoring has a known formula. Any of those being subtly wrong gives you something that looks right in a screenshot and falls apart the second you play it.

It's also complex enough that the agent has to actually reason through the state management.

What Came Back

One minute and nine seconds later, a single index.html.

The part I didn't expect was the middle of that run. Before declaring victory, the model wrote itself a headless test harness. It stubbed out the grid, spawned a piece, dropped it to the bottom and checked it landed. It filled a row except for the gap where the piece was going, dropped the piece, and verified exactly one line cleared. It rotated a T piece and confirmed the cell layout actually changed. It shoved an I piece off the left wall and confirmed the collision check caught it.

It also ran four tests on the code it generated. Piece landed at bottom, line clear on full row, rotation changes shape, and wall collision detected all passed.

Nobody asked it to do that. I said "make me a tetris clone." It decided on its own that the way to be confident in a Tetris clone is to simulate one without a browser and check the invariants.

The game itself is genuinely good. All seven tetrominoes with distinct colors. A ghost piece showing where the current block will land. Next-piece preview. Scoring at 100/300/500/800 for one through four lines, scaled by level. Level up every ten lines with the fall speed increasing. Wall kicks on rotation. Pause and hard drop. Arrow keys to move and rotate, down for soft drop, space to slam it, P to pause.

I played it. It plays like Tetris.

There is Still Room To Grow

A 27B model at 4 bit is a capable mid-size model. It handled a self-contained problem with a clear spec beautifully. I'd expect it to struggle more on a large existing codebase where it has to infer the rules from context, and I'll find out when I try that.

Single-user token generation is also where this hardware is weakest, for the reasons I dug into in my last post. You're filling one row of a 32-row tile and paying for the whole bus either way. Concurrency is where the QuietBox actually stretches its legs, and this run used almost none of it. The same server can have thirty agents batching against it at once.

Still, a minute nine for a working Tetris clone is past the threshold where I'd reach for the tool during real work, which is the bar I care about.

There's also the part that's easy to miss from the output alone. Everything about that run stayed on my network. The prompt, the reasoning, the generated code. Nothing metered, nothing rate limited, nothing logged somewhere I can't see. I can run that loop a thousand times tonight and the marginal cost is electricity.

Bottom Line

The config file above is about thirty lines and takes a couple of minutes to write. That's the whole distance between having a local model and having your coding agent run on hardware you own.

What makes that possible is the OpenAI-compatible endpoint quietly becoming a universal adapter. The ecosystem of tools built on top of it will point at your hardware without knowing or caring where it's running. The tooling caught up, the hardware got small enough to sit on a desk, and the models got good enough to be useful at sizes that fit.

Next up I want to push this harder: multiple agents hitting the same server at once, which is the workload this box was built for, and a real project instead of a toy. I'll report back.

Thanks for reading, and stay tuned.

Update I have since learned that the tt-cli tool has been updated and can now run opencode with your model with just tt launch opencode once you have the model.