Can a 27B Coding Model Be Useful on a 16GB GPU?

Repository with scripts and benchmark notes:

https://github.com/arczhi/5060ti-qwen3.8-27b

I wanted to answer a practical question:

Can a local coding agent actually finish work on a 16GB GPU, or will it spend most of its time reading code and never produce a reliable patch?

The test machine was:

• AMD Ryzen 9 9950X, 16 cores / 32 threads
• NVIDIA RTX 5060 Ti with 16,311 MiB VRAM
• 32GB DDR5-6000 system RAM
• NVIDIA driver 595.71.05
• CUDA 13.2
• Linux x86_64
• Docker and llama.cpp CUDA server

The model was Qwen3.8-27B GGUF using UD-IQ4_XS quantization.

The tested configuration used an 80K context window, MTP-1, Q4_0 KV cache, Flash Attention, one parallel slot, disabled reasoning, and four FFN layers offloaded to the CPU.

The agent test was more important than the short benchmark. I gave the agent a design document and asked it to implement a feature in a medium-sized Go repository.

It took about 14m20s to:

• Read the design document
• Modify seven files
• Add five tests
• Complete scoped build, vet, formatting, and diff checks

Two failing checks were confirmed to be pre-existing baseline failures.

Across 14 server requests, the measured results were:

• Prefill: 22.18-326.36 tok/s
• Decode: 11.06-12.95 tok/s
• Generation time: 7.89-184.60 seconds per request
• MTP acceptance: 84.9%-100%

The short smoke test was much faster, reaching 86.54 tok/s prefill and 17.77 tok/s decode. However, the longest request still took 184.6 seconds.

I also compared CPU FFN offload:

• No CPU FFN offload: 14m20s, 11.06-12.95 tok/s decode
• Four CPU FFN layers: 14m04s, 19.40 tok/s weighted decode
• Eight CPU FFN layers: stopped after 25m30s, 18.36 tok/s weighted decode

In this real coding-agent run, FFN4 was the best balance. It reduced GPU pressure without adding as much CPU/GPU transfer overhead as the larger offload configuration.

The main lesson is that tokens per second are only part of the story. For coding agents, I care more about:

• Whether the task was completed
• Tests passed
• Total wall-clock time
• Number of agent turns
• Context compactions
• Final repository state

This benchmark comes from one machine and one runtime configuration. The goal is reproducibility and useful comparison, not a universal performance claim.