Build an AI Research Workbench on Your Own GPU

The useful part of an AI research workbench is not the chat box. It is the record that lets you rerun a result after the model, data, or environment changes.

The short answer: A local research workbench needs a fixed model file, versioned inputs, saved code, a run log, and a verifier. Claude Science supplies many of those controls while using Claude as the model. My fully local path keeps the model, prompts, data, and logs on hardware I own, while I route my hardest reasoning work to a remote model. Canonical URL: https://bmdpat.com/blog/local-first-ai-research-workbench-2026

What does Claude Science actually record?

Anthropic introduced Claude Science on June 30, 2026. It runs as a beta for Claude Pro, Max, Team, and Enterprise users. The app includes more than 60 scientific skills and connectors. It can use a laptop, a remote machine, or an HPC login node for the work.

The provenance controls are the important part. A generated figure includes the code, environment, plain-language description, and message history behind it. A reviewer agent checks citations, calculations, and whether figures match their code.

Claude Science is not a simple cloud notebook. Large or sensitive datasets can stay on systems the lab already controls. The context needed for each model step still goes to Claude. That makes it a hybrid design: local data and compute, remote model inference.

What changes when the model also stays local?

The first change is reproducibility. A GGUF file on disk does not update between runs unless I replace it. I can save the model hash, runtime version, prompt, parameters, source files, and output beside the result. That gives me a fixed model boundary as well as a fixed code boundary.

The second change is cost behavior. There is no per-token API charge, so I can repeat a sweep without watching a token bill. That does not make the run free. The card, machine, power, setup, and my time still count.

The third change is the data path. With a local model, the full prompt can stay on the box. For private source files or an embargoed result, that may decide the architecture before model quality enters the discussion.

What did the RTX 5090 runs cost?

I measured two local models on July 9, 2026. The full RTX 5090 sweep used Ollama 0.31.1, temperature 0, bounded output, one warmup request per model, and GPU samples from nvidia-smi every half second.

Model	Workload	Generation	Peak VRAM
llama3.1:8b Q4_K_M	short generation	228.87 tok/s	7,170 MiB
llama3.1:8b Q4_K_M	long-context summary	206.74 tok/s	7,826 MiB
gemma4:26b Q4_K_M	short generation	198.81 tok/s	19,932 MiB
gemma4:26b Q4_K_M	long-context summary	180.18 tok/s	20,233 MiB

The Llama agent-code run generated 227.79 tokens per second at 187 watts average draw. At that rate, one million output tokens used about 0.23 kWh. At $0.12 per kWh, the electricity was about $0.03 before hardware cost.

That number is not a cloud price comparison. It is an operator measurement. It tells me that repeated local runs are cheap enough to treat verification as part of the job instead of an optional last pass.

What breaks in a local research loop?

Context changes can hide a large load penalty. Every request that changed num_ctx in my sweep reloaded the model. Gemma 4 26B took about 140 seconds each time. Pinning context size removed that delay from the steady-state loop.

Cold measurements can also lie. Gemma's first prompt-eval reading after a load was 15 tokens per second. The next request on the loaded model reached 6,179. A warmup request is part of the measurement method, not a cosmetic step.

My Ollama num_batch test found the same lesson from another angle: change one setting, measure the real workload, and keep the run record. My Ternary Bonsai comparison showed that a model fitting in memory does not mean it finishes the task correctly.

When should the model stay remote?

On my rig, I route hard synthesis, novel inference, and very long context to a remote model. I have not measured my mid-size local models against frontier systems on those tasks, so I do not claim equivalence. Keeping a model local does not make it right for every job.

I use the local model when repetition, stable model behavior, data control, or predictable run cost matters more. My research loop uses both paths. The practical work is routing each step to the model that fits its constraint, then saving enough evidence to rerun the result.

What should the first local version contain?

Start with one job you already avoid because of data rules or repeat cost. Save the model file and hash. Pin the runtime, context size, temperature, and output limit. Store the source files, generated code, output, elapsed time, GPU memory, and verifier result in one run folder.

Then rerun it. If the second result cannot explain how it was made or why it passed, you built a chatbot, not a research workbench.

Accompanying prompt

What the prompt does: It turns a local research task into a reproducible run plan with explicit evidence and pass or fail checks.

Copy/paste this prompt:

Role:
You are a local AI research-run designer.

Context:
Paste the research question, model file, hardware, runtime, source files, data rules, and expected result.

Task:
1. Define the fixed model, prompt, parameters, and environment to record.
2. List the code, data, logs, and measurements the run must save.
3. Write a verifier that can mark the result pass or fail.

Output:
- A run-folder layout with exact artifact names.
- A command checklist and one pass or fail rule.

Constraints:
- Keep private data on the stated hardware.
- Separate measured results from vendor claims.
- Do not invent missing measurements.

Copy the block above.

I publish measured local AI runs and the failures behind them in The 5090 Reports. Join the email list for the next result.

---

Get the local AI lab notes (benchmark rows, VRAM fit, quant choices, what runs on consumer GPUs), M-F only when there is something worth sending: https://bmdpat.com/newsletter?utm_source=blog_md&utm_medium=aeo&utm_campaign=local-first-ai-research-workbench-2026

---

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.