DEV Community: sheephess9527

Hark Pro: an AI assistant that actually does the clicks, not just the chat

sheephess9527 — Wed, 04 Nov 2026 14:21:18 +0000

I've been trying Hark Pro, a free personal AI agent that launched in October 2026 from Hark Labs (founded by Brett Adcock of Figure AI).
The difference from a normal chatbot: it has its own cloud computer (called Handoff) that opens real browsers and finishes tasks the way you would — bookings, bills, inbox triage — instead of just answering questions. It also remembers your preferences and proactively suggests the next thing before you ask.
It's free to start on web, iOS and Android, with paid plans only adding more usage.
If you want to try it, here's my invite link: https://app.hark.com/r/UZPYYU

Show DEV: quantized.uk – a free "will it fit?" VRAM index for local LLMs

sheephess9527 — Wed, 04 Nov 2026 13:23:12 +0000

I kept re-doing the same math every time a new model dropped: which quant fits my card, how much context is left, will it spill to RAM. So I put it into a site: https://quantized.uk

VRAM calculator: pick model + quant (GGUF / AWQ / EXL2 / MXFP4...) + context length, get an estimate for weights + KV cache
129 models across ~60 GPUs/Macs (RTX 30/40/50, RX 7000/9000, Apple M1–M5, DGX Spark...); each GPU has its own page listing what fits
Plus a quant picker, model comparison, and a llama.cpp / Ollama / vLLM command generator
Cookbook guides (8GB GPU starter, Ollama in Docker / WSL2)

Free, no signup, in English and Chinese. It's hand-maintained, so numbers will be off somewhere — if your real-world usage differs, tell me and I'll fix it.

How much VRAM do you actually need for local LLMs in 2026? (Qwen3.8 27B, 24GB vs 32GB cards)

sheephess9527 — Wed, 04 Nov 2026 12:40:45 +0000

Every time a new open model drops, the same question comes back: will it fit on my GPU? Here is how to think about it, with concrete numbers.

1. Memory = weights + KV cache + a little buffer

The GGUF file size is not your VRAM requirement. On top of the weights you need the KV cache, which grows with context length, plus some activation buffer. That is why the same model can fit at 4K context and fail at 32K.

2. Worked example: Qwen3.8 27B (GGUF)

Quant	Approx. VRAM (4K ctx)
Q3_K_M	~14.3 GB
Q4_K_M	~18.5 GB (~20.4 GB at 32K)
Q5_K_M	~22.2 GB
Q6_K	~25.2 GB
Q8_0	~30.7 GB

So on a 24GB card (RTX 3090/4090, RX 7900 XTX), Q4_K_M leaves ~5.5 GB for longer context; Q5_K_M fits but is tight. On a 16GB card, look at Q3_K_M or partial CPU offload. Qwen3.8 uses a hybrid layout where only some layers keep a KV cache, so its long-context cost is far lower than a conventional 27B.

3. 24GB vs 32GB: what the RTX 5090 really buys

At 4K context, 97 of the 129 models in the quantized.uk index fit a 24GB card comfortably (102 load at all). On the 32GB RTX 5090 it's 103.
A 70B at Q4_K_M with 8K context is about 47.5 GB, so one 5090 does not change that: you still need two cards or offloading.
What the extra 8GB buys is context room on 27B–36B models, plus bandwidth: 1,792 GB/s vs 1,008 GB/s on the 4090.

4. Bandwidth decides speed, VRAM decides fit

Cards with the same 24GB run the same models at very different speeds: memory bandwidth ranges from 346 GB/s (Tesla P40) to 1,008 GB/s (RTX 4090). Generation speed roughly scales with bandwidth ÷ weight file size.

5. Macs

A Mac's GPU can only use part of unified memory (about three quarters on a 24GB Mac), so size against that, not the full number.

Tools

I maintain these free, no-signup references:

VRAM calculator (model + quant + context → weights + KV cache): https://quantized.uk/tools/vram-calc/
Command generator for llama.cpp / Ollama / vLLM: https://quantized.uk/tools/cli-gen/
What to run on 8GB / 24GB / RTX 5090: https://quantized.uk/cookbook/

Numbers are estimates plus community measurements; if your real usage differs, tell me and I'll fix it.