DEV Community: zhijie

MTP on an RTX 3090: Faster Tokens, but What About Coding Quality?

zhijie — Fri, 02 Oct 2026 21:22:39 +0000

Originally published on my blog.

Enabling MTP on this RTX 3090 raised generation throughput from 36.0 to 56.2 tokens/s, about 56% faster. The two cache tasks that succeeded in both modes also finished about 20% and 40% sooner.

One transfer-task pair produced a patch-quality difference, however. I am continuing to test MTP while keeping it off by default.

This experiment grew out of a reader's suggestion on the previous local-deployment post: try MTP or a quantization that includes it. The RTX 3090 left over from Ethereum mining was already running Qwen3.8-27B through OpenCode. I wanted to see whether faster generation would deliver correct patches sooner.

We ran eight paired comparisons, sixteen attempts, using the same model file. Each pair shared its task, seed, and initial environment; only the MTP setting changed.

MTP drafts tokens for the target model to verify

Ordinary generation typically produces one token per step. MTP (Multi-Token Prediction) uses an embedded prediction head to propose upcoming tokens, then asks the target model to verify them. Accepting several in sequence can reduce serial decoding steps. This is a speculative-decoding path in llama.cpp.

The target model still verifies the candidates. The speculative-decoding paper shows that a correct algorithm can preserve its output distribution. Speed need not inherently cost model quality.

Our existing Qwen3.8-27B Q4_K_M GGUF contained an MTP prediction layer, so we used it directly without switching to another Unsloth quantization. The server commands differed only in --spec-type none versus --spec-type draft-mtp.

Generation was 56% faster; successful tasks saved 20–40%

Generation throughput counts tokens produced per second, including the model's reasoning output. Task time also includes reading source, executing tools, and running tests. It is closer to how long you wait for a patch.

Measure	MTP off	MTP on
Aggregate coding-generation throughput	36.0 tokens/s	56.2 tokens/s
Predefined independent checks passed	35/80	26/80
Completed within deadline and passed predefined checks	2/8	2/8

The two cache pairs that succeeded in both modes provide the clearest evidence of time saved:

Paired run	MTP off	MTP on	Task time saved
Pair one	192.0 s	153.5 s	20.1%
Pair two	431.2 s	259.0 s	39.9%

Both delivered a patch passing the predefined checks sooner. A failed run ending earlier does not count as speeding up a successful task. Later prompts also diverge with the agent's actions, so aggregate throughput is not an identical-text speed comparison.

Short replies and generation-timing definitions

Short replies to matched first requests reached 38.9–39.8 tokens/s with MTP off and 74.0–84.2 tokens/s with it on. These were brief tool-selection responses; they do not show that whole coding tasks finish in half the time.

Aggregate throughput is total accepted target-output tokens divided by total corresponding server generation time across completed generations. It includes reasoning and tool calls, excludes prompt processing and rejected drafts, and omits interrupted generations without final timing records.

Task time includes prompt processing, tools, and the agent's own tests. Server startup and independent grading afterward are excluded.

Nine differing checks came from one transfer repair

The entire 35/80 versus 26/80 gap came from one transfer task at one seed. Cache, ledger, and build-planner scores matched within every pair.

With MTP off, the agent produced a production-code patch that subsequently passed all ten independent checks. It reached the deadline at 480.1 seconds without a final handoff.

With MTP on, the agent ended after 186.1 seconds without editing production code. It passed only the one check the initial implementation already passed.

The off candidate performed better on those checks, and neither attempt met the full delivery standard. We therefore tracked patch correctness and timely delivery separately: completed tasks remained 2/8 in both modes. Nine related checks are not nine independent task regressions.

Task scores and an early difference that did not recur

Task	MTP off, two scores	MTP on, two scores
Expiring cache	10/10, 10/10	10/10, 10/10
CSV ledger	1/10, 1/10	1/10, 1/10
Incremental build planner	1/10, 1/10	1/10, 1/10
SQLite transfers	1/10, 10/10	1/10, 1/10

Completed and passing requires no timeout, successful process exit, unchanged protected contracts/configuration, passing public and added tests, and all ten independent checks passing. It is not proof that every possible defect is covered.

An early exploratory comparison showed a build-planner score difference. It did not recur under these tighter controls; those scores are not pooled into this study.

The cache task also gave me a reason to inspect code after green tests. An MTP candidate's added TTL validation overflowed on a very large integer, while the paired off candidate passed the extra probe. This extreme API boundary is worth recording; it establishes neither everyday failure frequency nor MTP as the cause.

Extra review: cache overflow and shared omissions

All four cache attempts passed the ten predefined checks. One MTP candidate added math.isfinite(ttl). The contract permits finite positive integer TTLs. With 10**400 and an injected integer clock increasing on every call, validation raised OverflowError while converting the integer to a float. The paired off candidate passed.

Ledger probes for large exact amounts, header names, and read-time errors failed in both modes at both seeds. A wallet destination-integer-overflow probe passed off and failed on in the second pair; it failed both modes in the first pair.

Extra review ran after grading, never reached the agent, and was not added retrospectively to the eighty checks. The paired records retain details.

Faster output still exhausts the same allowance

Most unsuccessful attempts shared a problem: they spent the 8,192-token response allowance on reasoning, ended with length, and never edited production code. Many 1/10 scores belong to unchanged starting code; they do not mean a tenth of the repair was completed.

Increasing context capacity would not resolve that limit. Context determines how much input and output can fit; the response allowance caps what one generation can produce. MTP can produce tokens faster without adding room to that allowance.

This also limits the quality comparison. Both modes frequently produced no patch, leaving few completed repairs to compare. There were only two seeds per task and four small Python projects. The eighty checks are correlated, not eighty independent samples.

The transfer difference deserves investigation, but its cause remains unresolved. Numerical behavior in batched verification, state handling, and sampling are possible avenues to investigate. An agent's different action can also change later inputs and patches. These are possible explanations, not findings confirmed by this experiment.

Controls and the limits of this comparison

Each task ran at two seeds, with MTP off and on; the second seed reversed the order. Every attempt rebuilt the same initial code and fixed the working directory, prompts, tools, permissions, and generation settings. Ten independent test methods per task stayed outside the agent's workspace, with no results fed back to it.

A proxy checked the complete first request actually sent to the model: all eight pairs matched, as did initial file hashes. Each attempt restarted the server and disabled prefix-cache reuse in both modes. These results therefore cannot predict gains in ordinary cached conversations.

Full configuration, request audit, and scope

Setting	Fixed configuration
Model and GPU	Same Qwen3.8-27B Q4_K_M GGUF; RTX 3090 24GB
Software	llama.cpp b11146 / CUDA 12.8; OpenCode 2.0.20
Server	131,072-token capacity; one slot; all model layers on GPU
Cache and compute	q8_0 K/V; flash attention; batch 512 / microbatch 256
Generation	Medium thinking; temperature 1.0; top_p 0.95; top_k 20
Budgets	8,192 output tokens per generation; 480 seconds per task
Paired seeds	4242 and 8675309
Speculation	Off: `none`; on: `draft-mtp`; maximum draft length 3

Pairs one and two in the elapsed-time table correspond to seeds 4242 and 8675309.

Target and draft backend sampling were disabled in both modes; synthetic acceptance was not used. Each pair shared the initial commit, agent title, project configuration, and grader. Reported cached-prefix token counts were zero. The first seed ran off→on; the second ran on→off.

These were fresh-server attempts without prefix-cache reuse. Task timing excludes server startup but includes prompt processing, tools, and the agent's own tests. Independent grading afterward is excluded. Desktop and other background GPU activity were not fully controlled.

The largest actual input was 20,057 tokens. The 128K setting is capacity; this experiment did not measure full-128K coding performance or acceleration in ordinary cached conversations.

Keep the current default; test a larger output budget next

For this setup, I am keeping MTP off by default. Saving 20–40% on the successful cache tasks is useful; before expanding its use, I want to investigate the transfer-patch difference.

The next comparison should raise the output allowance equally in both modes, add paired seeds, and use tasks closer to everyday work. Cached conversations need a separate test. That should give a better basis for deciding whether faster generation consistently delivers correct patches sooner.

If you try MTP, record generation speed, task time, independent tests, and final delivery together, then decide using the tasks you actually do.

Data and references

Paired experiment JSON: scores, elapsed times, generation timings, request checks, and extra probes for sixteen attempts.
Method and data notes: attachments omit experiment dates, locations, personal paths, session identifiers, and raw conversations. They are result records, not a complete reproduction kit.
Previous post: local Qwen3.8-27B + OpenCode deployment: installation and integration. Its first-attempt study is a separate batch; scores should not be pooled with these.
llama.cpp b11146 speculative-decoding documentation and the speculative-decoding paper: mechanism background. Measurements on this card come from the paired records.

Run OpenCode from your phone or laptop: remote hosting, authentication, and reverse proxy

zhijie — Thu, 01 Oct 2026 20:09:27 +0000

Originally published on my blog. This is an adapted version of the deployment guide.

A coding agent is easier to use across devices when the workspace lives on one host. The phone or laptop opens the browser UI; the host keeps the repository, runs tools, and connects to the model provider.

That host can be a desktop, a homelab machine, or a VPS. You do not need a public domain to begin. The choices that matter are how clients reach the service, how access is authenticated, and what happens when the launching terminal closes.

Start with the workspace and authentication

Enter the project directory before starting OpenCode. For a service behind a reverse proxy on the same host:

cd /path/to/your/workspace
export OPENCODE_SERVER_PASSWORD='replace-with-a-strong-password'
opencode web --hostname 127.0.0.1 --port 8080

The default authentication username is opencode. opencode web includes the browser interface; opencode serve runs the headless API service. See the official Web documentation for the current flags.

Treat access to this UI as access to the host's development tools. Use a dedicated workspace and account with only the permissions the agent needs. Keep the password out of committed configuration and screenshots.

Choose a connection path

Private access: Keep the service on a private network. A LAN connection works for devices on that network; Tailscale is another way to connect authorized devices without exposing the IDE directly to the public internet. If you bind to 0.0.0.0, remember that it listens on every interface: restrict reachability with the host firewall and keep authentication enabled. Plain HTTP on a LAN is not encrypted by itself.

A VPS with a domain: Keep OpenCode on loopback and put an HTTPS reverse proxy in front. For example, a Caddy site can be as small as:

ide.example.com {
    reverse_proxy 127.0.0.1:8080
}

The domain must point to the server and the necessary ports must be reachable. Caddy can manage HTTPS certificates and proxy WebSocket connections; see its reverse-proxy documentation.

A reverse proxy does not automatically provide application authentication. Confirm that an unauthenticated visitor cannot enter the OpenCode workspace. With Nginx, check WebSocket upgrades and long-running streamed responses rather than testing only whether the initial page loads. The full article includes an Nginx example.

Keep the host process running

A remote host helps only while its service stays alive. On Linux, systemd can restart OpenCode after a process failure and start it after a reboot. My guide includes an example service with a dedicated working directory and a protected environment file for the password.

Use the actual installed binary path and service user. Check service logs and test a restart before relying on it for longer tasks. Closing a client browser and shutting down the host are different events; the host must remain awake and connected.

Select the model separately

Hosting OpenCode's workspace does not mean the model runs locally. The original guide uses Muse Spark 1.3 Contributor Free through OpenCode Zen as one example. Zen currently lists this as a limited-time free model, and its Contributor terms permit Meta to use prompts and completions for future model training. Review the current Zen pricing and privacy terms before using it, and keep confidential code and secrets out of that endpoint.

The deployment pattern also works with a provider appropriate to your data requirements. In my follow-up experiment, inference runs on an RTX 3090 with Qwen3.8-27B and llama.cpp.

Check the whole path

Before handing it a long task, verify these steps from the intended client device:

Authentication blocks unauthenticated access.
The expected project and model are selected.
A harmless file-read request succeeds.
Tool calls and live output work through the chosen network path.
The host service survives the terminal disconnects and restarts you expect.

The benefit is one workspace that I can reach from different screens. The phone becomes a way to check progress or send a follow-up; execution remains on the host.

Full guide with service and proxy configuration.

Qwen3.8-27B on RTX 3090 with OpenCode and measured coding results

zhijie — Thu, 01 Oct 2026 19:05:13 +0000

Originally published on my blog. This cross-post includes the setup and measured results; the linked records and configuration kit let you inspect the experiment.

My RTX 3090 is a leftover from my Ethereum mining days. Running a local model gives it a second life: the same card now powers a coding agent.

The previous post covered accessing one OpenCode workspace from a laptop or phone. This time, I wanted to move inference onto that host too and find out how far the hardware already on hand could take a local AI coding workflow.

We loaded Qwen3.8-27B Q4_K_M into the 3090's 24GB of VRAM, served it through llama.cpp, and connected OpenCode. After checking chat and tool calls, we gave it four small coding tasks.

The results make me want to keep using it: bounded fixes and small features with clear acceptance criteria can produce useful patches. Exhausted output budgets, a timeout, and defects missed by tests also showed why independent validation still matters.

I'll explain the model choice and its connection to OpenCode, then share the measured results and lessons from using it. Full setup instructions follow later; to get straight to them, jump to deployment.

Why Qwen3.8-27B

Qwen3.8-27B is a prominent local coding candidate, with results that make it worth trying: the official model card reports 61.7 on SWE-bench Pro and 73.0 on Terminal Bench 2.1. Software fixes and terminal-agent tasks are close to what I want OpenCode to do. These reported results gave me a reason to test it first.

The hardware fit mattered just as much. The 27B Q4_K_M weights occupy about 16.8GB, fitting entirely into the 3090's 24GB VRAM in our configuration with room for KV cache and runtime buffers. For this old card, the balance of capability, memory use, and speed matters more than chasing a larger parameter count.

Its agent focus also matches the workflow: reading code, calling tools, and revising changes after test feedback. The official introduction highlights coding, multi-step agent execution, and adjustable thinking, making this workflow a useful place to test those capabilities.

Those three considerations made it the starting point for this experiment. What its Q4 version can accomplish on this card still needs validation with our own tasks.

What the old card does now

With the model chosen, the next step is connecting it to a workflow that can operate on a repository. OpenCode owns the workspace and tools: reading files, editing code, running shell commands, and executing tests. llama-server handles inference, connected through a local API.

A laptop or phone connects to OpenCode through a LAN or private network.
OpenCode sends messages and tool definitions to llama-server at 127.0.0.1:8080/v1.
llama-server runs Qwen3.8-27B on the RTX 3090 and returns text or tool calls.
OpenCode executes the tools in its workspace and sends their results back to the model.

OpenCode executes the model's tool_calls and returns tool results for the next turn. The model API itself does not execute terminal commands.

The host has an RTX 3090 24GB, Ryzen 7 5800X, and 64GB RAM. We used llama.cpp b11146 / CUDA 12.8 and OpenCode 2.0.20, placing all model layers on the GPU with one generation slot.

Weights came from Ollama's qwen3.8:27b-q4_K_M tag and were loaded directly by llama-server, without installing an Ollama service. Only the text model was loaded; vision input was not tested.

Two settings matter throughout this post: 131,072 tokens of context capacity and an 8,192-token per-response output allowance. Input and output share context, and reasoning consumes the response allowance too. That distinction later explained one of the coding failures.

Large prompts fit, but fresh processing takes time

With short input, generation reached about 36.4 tokens/s. At roughly 120K input tokens, it fell to 20.9 tokens/s, and a fresh request took about three minutes to produce its first token.

The model processes the input before it starts generating. As input grows, that initial wait becomes a substantial part of the experience:

Actual input	Fresh prompt processing	Generation	First token, fresh	First token, cached
2,073 tokens	999 tokens/s	36.4 tokens/s	2.75 s	0.46 s
16,378 tokens	995 tokens/s	33.5 tokens/s	17.00 s	0.47 s
65,537 tokens	795 tokens/s	25.9 tokens/s	83.41 s	0.51 s
120,011 tokens	656 tokens/s	20.9 tokens/s	183.01 s	0.63 s

The last column is particularly relevant to coding sessions. Cached continuations reused almost the entire prefix and processed only 27–28 new input tokens. Even with roughly 120K context retained, first-token latency fell to 0.63 seconds. Sustained generation still had to attend to the long context, so it did not regain short-input speed.

This suggests a practical approach: keep stable session prefixes and supply source as needed. Repeatedly opening fresh requests with large inputs incurs the processing cost again.

A separate capacity check processed 119,968 input tokens, retrieved three widely separated values, and answered a follow-up. It verified capacity for that synthetic retrieval task. Coding quality at the same input length needs its own evaluation.

VRAM was also close to the card's limits. Samples during throughput testing showed peak total use of 22,162 MiB, minimum free memory of 1,943 MiB, and a maximum temperature of 85°C, including desktop GPU use. The configuration fit, with limited headroom for another model or GPU-heavy workload.

Timing methodology, sample counts, and source data

Every row used the same 131,072-token server capacity, varying only actual input length. Main runs disabled thinking and used synthetic source listings, temperature 0, and seed 1234. Fresh requests generated 512 tokens; continuations generated 256. Generated text was never executed.

Generation uses server decode timing, excluding initial prompt processing. First-token latency uses the streaming HTTP client, including tokenization and processing overhead. The 2K row is the median of three fresh requests; larger inputs had one fresh request and one continuation each. These measurements do not establish reliability intervals. GPU samples were collected every two seconds.

A separate short request with medium thinking generated about 35.7 tokens/s, including reasoning. Throughput does not directly measure usable code produced per second. The script measures single-request performance, excluding model loading and multi-user throughput.

Download the measurement records and summary.

The real test: four small coding projects

After measuring speed and capacity, we checked whether the setup could complete code changes. We prepared four small Python repositories: an expiring cache, a CSV ledger, an incremental build planner, and SQLite transfers. Each had a written acceptance contract, a fresh session, and an eight-minute deadline.

To check more than the agent's own tests, we prepared ten independent test methods per task, kept them outside its workspace, and never fed their results back to the model.

Task	Independent checks, before → after	Time	Agent-added tests	Outcome
Expiring LRU cache	0/10 → 10/10	≈3m 07s	23	Completed; strongest result
CSV ledger and refunds	1/10 → 10/10	≈5m 44s	40	Completed; review found boundary and error-handling gaps
Incremental build planner	1/10 → 1/10	3m 58s	0	No patch; exhausted one response's output allowance
Atomic SQLite transfers	1/10 → 10/10	8m deadline	11	Candidate passed checks; final test rerun and handoff unfinished

Three candidates passed all predefined independent checks; two completed editing, testing, and final handoff within the deadline. The wallet candidate passed checks, but the agent timed out before its final test rerun and handoff. Both outcomes matter.

First-attempt candidates passed 31/40 independent test methods. The build planner made no edits and already passed one method at baseline. Methods also differ in difficulty, so 31/40 should not be interpreted as a general coding success rate.

Two findings were more useful than the aggregate score.

One failure came from the response budget

The build-planner agent read the repository and never produced a patch. Its final response generated 8,192 tokens, all reasoning, and ended with length. OpenCode exited with status zero without a final answer.

That request had roughly 4,985 input tokens, far below the 128K limit. Thinking and the response allowance needed attention; increasing context capacity would not resolve an exhausted output budget. The termination record preserves the evidence.

A separate diagnostic restarted from the original repository with thinking disabled, keeping the prompt, allowance, and deadline unchanged. It completed in 5m 40s and passed 9/10 independent checks. The remaining failure was load_manifest() returning tuples where the original API returned lists; the agent's own test accepted that change too.

The diagnostic is reported separately and does not replace the first attempt. One additional sample at temperature 1 cannot establish that disabling thinking is generally better, but it identifies a setting worth testing further.

Many new tests still missed defects

The ledger added 40 tests, passed the predefined independent checks, and had a reasonably clear structure. Further review still found that Python's default 28-digit Decimal precision rounded a valid large amount when adding 0.01, violating the no-rounding contract. An I/O error during file iteration could also escape as a traceback.

The wallet exposed a different problem: its own concurrency tests submitted one future and immediately waited before submitting the next. Execution was sequential. Our separate eight-caller concurrent test passed, but the agent's tests had not demonstrated overlapping requests.

Review also found that crediting two cents to a destination near SQLite's signed-integer limit converted the balance to REAL while recording success. That extreme numerical boundary should be rejected with a rollback.

These probes ran after grading and were not added retroactively to the forty checks. They made me focus on what tests actually cover rather than how many were added. Review found no additional acceptance-contract defect in the cache, though expiration cleanup scans the whole cache and our assessment remains limited to the small task tested.

Coding protocol and additional test results

Each repository had three fixed public tests. Independent checks were prepared before the corresponding attempt; the evaluator did not repair candidate production code. The agent left protected acceptance contracts, public tests, and configurations unchanged.

First attempts ran serially with medium thinking and an 8,192-token per-response output allowance. Repository file tools and a narrow allowlist of local test commands were available; network tools, subagents, and cloud fallback were disabled. Infrastructure-only aborted launches were excluded from scores. Cache and ledger durations were recovered from event records and are approximate.

The evaluator subsequently ran public and agent-added tests: cache 26/26, ledger 43/43, and wallet 14/14 passed. Wallet's green rerun occurred after the agent timed out. The original build planner still failed its three public tests. The thinking-disabled diagnostic passed 47/47 public and agent-added tests, while retaining the independent-check failure described above.

These were four deliberately bounded Python repositories. We did not measure reliability across seeds, large production repositories, Q4 versus higher-precision weights, or coding quality at 64K / 128K. No controlled model comparison was performed. The coding summary retains scores and review probes.

How I would keep using it

I would give it bounded fixes and small features, state the acceptance criteria first, and inspect the actual diff, independent tests, and final handoff. It demonstrated the ability to edit across files, add regression coverage, and correct some mistakes found during its own checks.

For longer tasks, I would watch whether reasoning crowds out output and whether the workflow times out. Numerical boundaries, concurrency, and error handling still deserve review. The next experiment should retain the same tasks and checks, add repeated runs, and compare thinking settings and response allowances.

From Ethereum mining to local coding, this 3090 has found another useful job. If similar hardware is already on hand, a small task with clear acceptance criteria is a practical way to see what it can contribute to your workflow.

Reproduce the local coding setup

Here is the deployment path we used. The download kit contains the full configuration. Essential launch and verification steps are below, with additional settings under the headings below.

1. Start the model service

On Ubuntu, prepare an NVIDIA driver, curl, unzip, Python 3, and a working user session, then run:

mkdir -p ~/local-qwen && cd ~/local-qwen
curl -fL https://zjshen14.github.io/experiments/qwen3.8-27b-3090/reproduction-kit.zip -o kit.zip
unzip kit.zip
bash setup.sh
./run-server.sh

setup.sh downloads and verifies the GGUF and llama.cpp b11146 CUDA 12.8 release archives. Downloads total roughly 17.6GB, plus extracted runtime space; weights are not inside the ZIP. Prebuilt binaries avoid compiling a CUDA development toolchain. The script installs no system service and makes no global OpenCode configuration changes.

Once the service is ready, verify the API from another terminal:

curl -f http://127.0.0.1:8080/health
curl -f http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen3.8-27b","messages":[{"role":"user","content":"Reply with OK."}],"max_tokens":64,"chat_template_kwargs":{"enable_thinking":false}}'

The server listens on 127.0.0.1:8080, and local requests need no real API key. A client requiring a value can use a nonsecret placeholder such as local, but that is not access control. Keep this unauthenticated endpoint on loopback.

Full experiment settings, launch options, and background service

Component	Tested configuration
System	Ubuntu, Linux x86-64
GPU	NVIDIA RTX 3090, 24GB VRAM
CPU / RAM	Ryzen 7 5800X / 64GB
Weights	Qwen3.8-27B, Q4_K_M, 16,810,714,464 bytes
Serving engine	llama.cpp b11146, prebuilt CUDA 12.8 runtime
Agent	OpenCode 2.0.20
Context / per-response output limit	131,072 / 8,192 tokens; input and output share context
KV cache	q8_0 for both K and V
GPU / concurrency	All model layers on GPU; one generation slot
Other settings	Flash attention; batch 512 / microbatch 256
Thinking for coding tasks	Enabled, medium effort

Q4_K_M describes weight quantization; q8_0 describes KV-cache quantization. Beyond the roughly 16.8GB weight file, VRAM is needed for caches, compute buffers, and desktop applications. The kit's run-server.sh supplies the full command; these are its core options:

llama-server \
  --model models/Qwen3.8-27B-Q4_K_M.gguf \
  --alias qwen3.8-27b \
  --host 127.0.0.1 --port 8080 \
  --ctx-size 131072 --parallel 1 \
  --n-gpu-layers all --fit off \
  --flash-attn on \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --batch-size 512 --ubatch-size 256 \
  --jinja --reasoning-format deepseek --reasoning-preserve \
  --no-context-shift --no-mmproj --metrics

The packaged script sets library paths and invokes the actual binary. If other applications need VRAM, stop the server and use MODEL_CONTEXT=65536 ./run-server.sh for 64K capacity. All measurements in this post used 128K capacity.

To survive closing the launching terminal, stop the foreground server first and run:

mkdir -p logs
systemd-run --user --collect --unit=qwen-local \
  --property="WorkingDirectory=$PWD" \
  --property="StandardOutput=append:$PWD/logs/server.log" \
  --property="StandardError=append:$PWD/logs/server.log" \
  "$PWD/run-server.sh"
# Stop: systemctl --user stop qwen-local

This is a transient user service that must be started again after reboot. Survival after logging out of the entire user session depends on user-manager configuration and was not tested here.

2. Configure OpenCode

Place the kit's opencode.json in your project root, or merge its provider into ~/.config/opencode/opencode.json. This configuration was verified with OpenCode 2.0.20: model ID local-qwen/qwen3.8-27b, API base URL http://127.0.0.1:8080/v1.

Full OpenCode provider configuration

{
  "$schema": "https://opencode.ai/config.json",
  "model": "local-qwen/qwen3.8-27b",
  "providers": {
    "local-qwen": {
      "name": "Local Qwen",
      "package": "@opencode/ai/providers/openai-compatible",
      "settings": {
        "baseURL": "http://127.0.0.1:8080/v1"
      },
      "models": {
        "qwen3.8-27b": {
          "name": "Qwen3.8 27B — 128K",
          "capabilities": {
            "tools": true,
            "input": ["text"],
            "output": ["text"]
          },
          "limit": { "context": 131072, "output": 8192 },
          "compatibility": { "reasoningField": "reasoning_content" },
          "body": {
            "chat_template_kwargs": {
              "enable_thinking": true,
              "reasoning_effort": "medium"
            }
          }
        }
      }
    }
  }
}

The tested configuration uses providers / package / settings; the online provider guide has shown provider / npm / options. If your installed version differs, follow its actual schema and avoid combining structures.

Start the model service, then restart OpenCode. Our installation also reloaded the configuration with opencode reload. Existing sessions may retain their previous model; choose local-qwen/qwen3.8-27b through /models.

For the previous post's Web workflow, launch from your project and choose a different port:

export OPENCODE_SERVER_PASSWORD='replace-with-a-strong-password'
opencode web --hostname 127.0.0.1 --port 4096

8080 serves the model; 4096 serves OpenCode. On the same machine, the provider uses loopback while remote browsers connect to OpenCode. If OpenCode runs on another host or inside a container, 127.0.0.1 points to that environment and needs replacing with a private connection to the GPU host. See the previous post for network binding and authentication.

3. Verify tools, then try a task

Check an ordinary reply first, then create a harmless fixture and ask the agent to use its read tool and return the contents. Both checks passed here, along with synthetic tool-call round trips and streamed tool calls at the server.

To repeat throughput measurements, keep the server idle and choose a new output directory:

python3 benchmark_throughput.py --report-dir reports/throughput-new-run

Then try a bounded task of your own and validate it with independent tests and review.

Configuration, data, and reproduction materials

Configuration and measurement ZIP: verified downloads, full launcher, OpenCode configuration, throughput script, and data. It includes neither weights nor the complete coding-task fixtures.
Experiment notes, throughput measurements, and coding summary. Experiment dates, absolute timestamps, timezones, personal paths, and session identifiers were removed from attachments; measurements, scores, and review probes are retained.
Official Qwen model card, pinned llama-server documentation, and OpenCode provider documentation. Upstream sources provide selection context, reported benchmarks, and deployment guidance; this host's performance and four-task coding results come from the attached experiment records.

Measurements and review findings are supported by the linked experiment records.