How I Keep LLM Results Valid After a Driver Update

A fresh machine snapshot can make an old local LLM result look newer than it is.

The short answer: I bind every benchmark row to the machine state, runtime, settings, workload, and time from that run. A later GPU snapshot is useful health data, but it does not update an older result. Canonical URL: https://bmdpat.com/blog/local-llm-benchmark-provenance-gpu-snapshot-2026

What went wrong in my July 30 report?

My July 30, 2026 lab report captured a current RTX 5090 snapshot at 09:50. nvidia-smi reported driver 610.88, 32,607 MiB total memory, 1,905 MiB in use, 90.11 watts, and 42 C.

The same report also collected 11 benchmark rows from four older input files. Those rows included measurements from July 9 and June 14. One July source recorded driver 610.62.

Each number is valid inside its own record. The problem starts if I read the current driver in the report header as the driver used for every row in the table. The report generation time and the benchmark run time answer different questions.

Why does benchmark provenance matter?

A local inference result belongs to the full route that produced it. That route includes the GPU, driver, model file, quant, runtime version, context setting, batch setting, workload, and residency state.

Change one part and the old result becomes evidence from a different route. It may still help with a rough choice. It does not become a fresh measurement.

This matters most when two rows are close. My July measurements included gemma4:26b workshop runs at 215.7 and 218.5 output tokens per second with different num_batch settings. I should not explain that gap with a later driver snapshot. The saved rows already name the settings and source file that produced them.

My guide to benchmarking local LLM input and output separately applies the same rule within one request. Load time, prompt ingestion, output generation, and task checks need their own fields. Provenance applies across runs too.

What should one local LLM receipt contain?

I keep one record per completed route. At minimum, it carries:

captured_at
gpu_name
driver_version
model
quant
runtime_version
context
batch
workload
resident_or_cold
input_tokens
output_tokens
output_tokens_per_second
task_check
source_file

The source file is not paperwork. It is how I get back to the raw row when a summary table raises a question.

I also keep machine health beside the run when it matters. Memory use, temperature, and power can explain whether a route had room to operate. They should carry the run timestamp, not the timestamp from a later report build.

How do I combine old rows with a fresh snapshot?

I label the sections by purpose.

The current snapshot says whether the machine is available and what it reports now. The benchmark table says what happened during each saved run. The report generator may place both on one page, but it should not imply that they share one capture time.

For a decision, I filter to comparable rows first. I match the model, quant, runtime, workload, and key settings. Then I check whether the environment changed enough to require a rerun.

A driver update does not prove that an old result is wrong. It creates a clear boundary. If the result controls a production route, I rerun it on the new stack before I call it current.

The same discipline keeps eval claims readable. My post on why local LLM benchmark numbers can mislead covers denominators, runs that could not complete, and machine stability. A timestamped route receipt adds the missing link between the number and the environment.

When should I rerun the benchmark?

I rerun when the answer will change a routing or sizing decision and the measured route has changed.

That includes a new model file, quant, runtime, driver, context limit, batch setting, or workload shape. I also rerun after a machine fault that makes the old environment hard to trust.

I do not rerun every row just because a report date changed. That would replace provenance with churn. I keep the old row, mark its route and time, and collect a new row only when I need a current comparison.

The result is a smaller table I can defend. I know what ran, where it ran, and which decision the number can support.

Accompanying prompt

What the prompt does: It turns mixed local LLM benchmark files and machine snapshots into timestamped route receipts without assigning new metadata to old results.

Copy/paste this prompt:

Role:
You are reviewing local LLM benchmark provenance.

Context:
Paste the benchmark rows, source filenames, machine snapshots, and
the routing or sizing decision the measurements must support.

Task:
1. Bind each result to its own timestamp and environment.
2. Separate current machine health from historical benchmark data.
3. Group only rows with matching routes and workload shapes.
4. Flag changed routes that need a fresh run.

Output:
- A compact route-receipt table.
- A list of rows that remain comparable.
- A rerun list tied to the named decision.

Constraints:
- Keep measured values separate from later snapshots.
- Do not assign a current driver to an older row.
- Do not invent missing versions or timestamps.
- Preserve the original source filename.

Copy the block above.

I publish measured local AI build notes in The 5090 Reports. Join the email list for the next result.

---

Get the local AI lab notes (benchmark rows, VRAM fit, quant choices, what runs on consumer GPUs), M-F only when there is something worth sending: https://bmdpat.com/newsletter?utm_source=blog_md&utm_medium=aeo&utm_campaign=local-llm-benchmark-provenance-gpu-snapshot-2026

---

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.