Qwen 3.8 27B Topped Hacker News in a Day. Here's How to Run It Locally From Spring Boot

Yesterday morning my feed exploded with a model release again. But this one was different from the usual frontier drop. Qwen 3.8 27B hit the top of Hacker News and stayed there: at the time I checked, the thread had passed 1,194 points with 713 comments in under a day. That is the kind of heat normally reserved for a $5-per-million-token API announcement.

The twist is that this is a dense 27-billion-parameter open model, Apache 2.0 licensed, that people are running on laptops. Simon Willison ran it on an M5 Max MacBook Pro through LM Studio with a 17GB GGUF file and spent 21 minutes watching it think about an SVG (his comment). I build production AI systems with Spring Boot and Spring AI, so my first question was not "how smart is it?" It was: can I call this thing from the code I already have, without a second SDK or a cloud account?

The answer is yes, and the setup is smaller than the model's license file. Here is what shipped, what the community actually found when they ran it, and the exact Spring Boot wiring for a local Qwen 3.8 27B.

What actually shipped

Qwen 3.8 is the latest generation of Alibaba's open model family, and 27B is its compact dense member. The model card lists the headline details:

• A dense 27B vision-language model. A causal language model with a vision encoder, built on the Qwen3.5 architecture. It takes text, images, and video input.
• 262,144 tokens of native context. The card says it can be extended toward 1 million tokens with RoPE scaling (YaRN), though the card warns static YaRN can hurt performance on shorter inputs.
• FP8 quantization from the lab. The FP8 repo uses fine-grained fp8 with a block size of 128 and claims "performance metrics are nearly identical to those of the original model."
• Thinking on by default. Qwen3.8 operates in thinking mode by default, with three reasoning effort levels: xhigh, medium, and low. It also keeps reasoning context from earlier messages (preserve_thinking) for multi-step agent work.
• Multi-token prediction (MTP). Trained with a draft model, which is what several community benchmark runs below are using.

The model hit the ground running: 91,917 downloads and 9,465 likes on the base repo and 123,157 downloads on the FP8 repo within about a day of release. Apache 2.0 means you can use it, modify it, and ship it without asking permission.

On benchmarks, Qwen's own tables show big jumps over Qwen3.6-27B. These are vendor numbers, evaluated with the Claude Code harness at temperature 1.0 and a 256K context window, so treat them as directional:

• Terminal Bench 2.1: 73.0, up from 63.4 on Qwen3.6-27B.
• SWE-bench Pro: 61.7, up from 53.5. In the same table, Opus 4.6 Max scores 53.4.
• DeepSWE 1.1: 42.2, up from 13.3.
• LiveCodeBench v6: 90.3, up from 83.9.
• OSWorld-Verified: 84.3, up from 63.9. That is computer-use territory, and it beats Opus 4.6 Max's 72.7 in the same table.

Again: those are Qwen's own numbers. The community thread is where the model gets tested by people who do not care about vendor tables, and that is where the interesting stuff shows up.

What developers found when they actually ran it

The HN thread is unusually dense with hands-on reports, because a 27B open model is something most of the audience can actually pull and run the same day.

• Simon Willison ran it on a laptop. M5 Max MacBook Pro, LM Studio, the 17GB GGUF. It produced the best SVG pelican-on-a-bicycle he says he has seen from a local model, but it took 21 minutes, used 22,276 reasoning tokens, and produced 3,223 tokens of output. His quote: "Absolutely the best pelican I've seen from a model that runs on my laptop" (comment).
• One commenter tested it as a software engineer. They had it build a fully featured todo list web app in JS, then rewrite it in Rust with Tauri. "Model is strong for its size. It one-shotted the Web app, had no bug." The Rust rewrite had one bug, fixed in a single follow-up prompt (comment).
• Another gave it a private benchmark. Qwen 3.8 27B is only the second local model, after Gemma 4, to correctly reason through one of their private benchmarks. The caveat: it took 5x as many tokens and 12 minutes 30 seconds with MTP enabled. They also flagged VRAM efficiency: 32K of context alone took 2.5GB of VRAM, and they could not fit 128K even quantizing V to Q4_0 (comment).
• The overthinking problem is real. A commenter testing a WordPress plugin found that in xhigh reasoning mode the model "overthinks so badly that it writes terrible bushy code," and watched it cycle through "FINAL FINAL APPROACH" and "OK TRULY FINAL APPROACH" before finishing. In low mode it behaved better (comment).
• Chat templates are broken out of the box. A common complaint: the Jinja templates need fixing for reliable tool calling and thinking control. The community built Qwen-Fixed-Chat-Templates to reduce or turn off thinking, fix tool calling, and keep a 100% KV cache hit rate (comment).
• Speed depends heavily on the engine. One RTX 5090 owner reports about 138 tokens per second using the ninfer inference engine, roughly double their naive llama.cpp setup (comment). A 20GB VRAM card user reports about 30 tokens per second with a 30K context, and notes Muse Glimmer gives them 65-80 tokens per second at 128K on the same card (comment).
• The enthusiasm is about the category. One commenter sums it up: "open weight/source small dense models benefit the public the most because they just reach more people" (comment). Another puts the milestone in plain terms: if the benchmarks hold, this is "getting very close to Opus 4.6 capability," which was their personal turning point for when AI was good enough that not using it became hard to justify (comment).

The pattern across all of these reports: the model is genuinely capable for its size, and the friction is operational. Thinking tokens, context budgets, and VRAM math. Those are exactly the problems a Spring Boot integration should be solving for you, and it turns out the integration is trivial.

The build: Qwen 3.8 27B from Spring Boot, locally

Full disclosure up front: I wrote this the day the model dropped. I verified every API call below against the Spring AI reference docs and the Ollama library page, and I cross-checked the numbers against community run reports, but I have not yet pointed a production workload at this specific model. The wiring pattern is the same one I use daily with other local models through Ollama, and it is genuinely small.

Step 1: run the model locally

The easiest path is Ollama. The library already lists qwen3.8 with tags for 27b, 27b-q4_K_M, 27b-q8_0, 27b-bf16, 27b-mxfp8, 27b-nvfp4, and MTP variants (library page). Pull the quantized version that fits your hardware:

ollama pull qwen3.8:27b-q4_K_M

Hardware reality check from the thread: a 4-bit quant lands around 17GB, which runs on a Mac with 32GB+ unified memory or a 24GB GPU. On 20GB cards you will be trading context length for speed, and a 5090-class card gets you over 100 tokens per second with the right engine. If you prefer LM Studio, its Qwen3.8 page hosts the GGUF with the same quant options.

Step 2: add the Spring AI Ollama starter

This is the only dependency you need. Spring AI's Ollama starter speaks the OpenAI-compatible chat shape, so anything that runs behind Ollama is a drop-in:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-ollama</artifactId>
</dependency>

Step 3: configure the model in application.properties

Two properties. The base URL is Ollama's default local port, and the model name matches the tag you pulled:

spring.ai.ollama.base-url=http://localhost:11434
spring.ai.ollama.chat.options.model=qwen3.8:27b-q4_K_M
spring.ai.ollama.chat.options.temperature=0.7

Step 4: call it through ChatClient

Spring AI gives you the same ChatClient builder you already use for any other model. A minimal service:

@Service
public class QwenLocalService {

    private final ChatClient chatClient;

    public QwenLocalService(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    public String ask(String question) {
        return chatClient.prompt()
            .system("You are a senior software engineer. Think briefly, then answer.")
            .user(question)
            .call()
            .content();
    }
}

The system prompt matters here. Because Qwen3.8 thinks by default and will happily burn 20,000 reasoning tokens on a two-sentence answer, a prompt that sets the expected depth is your first line of defense against the overthinking the thread keeps reporting.

Step 5: multimodal input (images and video)

Qwen3.8 is a vision-language model, and Spring AI's multimodal support covers it. The documented pattern is a UserMessage with a Media object attached. Per the Spring AI multimodal reference:

UserMessage message = UserMessage.builder()
    .text("Describe this diagram and explain what the arrows mean.")
    .media(new Media(MimeTypeUtils.IMAGE_PNG, new ClassPathResource("architecture.png")))
    .build();

ChatResponse response = chatClient.prompt()
    .messages(message)
    .call()
    .chatResponse();

This is the part that makes local Qwen interesting for Java teams: document parsing, screenshot analysis, and UI recreation on images that never leave your machine.

Step 6: control thinking per request

Qwen3.8 exposes reasoning_effort (xhigh, medium, low) and the ability to disable thinking entirely. On Ollama you can pass these through the options map, so low-latency paths can skip the long reasoning pass:

chatClient.prompt()
    .system("Answer directly, no reasoning.")
    .user(question)
    .options(OllamaOptions.builder()
        .model("qwen3.8:27b-q4_K_M")
        .temperature(0.7)
        .build())
    .call()
    .content();

The model card's own advice for non-thinking mode is temperature=0.7, top_p=0.80, presence_penalty=1.5, which is a good starting point for direct-answer workloads.

What I would do differently, and a checklist

The thread's operational lessons, turned into a checklist for anyone wiring a local model into a real service:

• Never run xhigh by default. The model card makes it the default; the community reports say it overthinks, writes bushy code, and loops through "FINAL APPROACH" messages. Start at medium, escalate per task.
• Budget reasoning tokens like money. 22,276 reasoning tokens for one SVG is fine for a hobby run and a non-starter for a latency-sensitive endpoint. If a request needs a fast answer, disable thinking for that path.
• Do the VRAM math before you pull. 27B dense is context-hungry. A 32K context alone ate 2.5GB of VRAM in one report, and rivals like Muse Glimmer fit far more context on the same card. If you need long context on modest hardware, this may not be your model.
• Use fixed chat templates for tool calling. The stock templates have known issues; the community fix (Qwen-Fixed-Chat-Templates) is worth adopting if you build agents on this.
• Verify benchmark claims against your own workload. Qwen's numbers are real but vendor-run. The thread shows a model that tops tables and still needs careful prompt work in production.

The honest takeaway: Qwen 3.8 27B is the first dense open model in a long time that makes me double-check my cloud API bill. For a Spring Boot team, the integration cost is one dependency and two properties, the model is Apache 2.0, the data never leaves your network, and the main engineering work is taming its thinking. That is a trade worth testing this weekend.

I write about Java, Spring Boot, and AI every week. Subscribe, it's free.

Have you run Qwen 3.8 27B (or any local model) in a real service? What did you have to tame first, speed or thinking? Tell me about it in the comments.