Getting New Model Weights Onto an Air-Gapped Machine

#ai #llm #machinelearning #programming

The first model on an air-gapped machine goes on during commissioning, when nothing is running and nothing can break. Every model after that arrives on a system with users, a working configuration and no ability to ask the internet what went wrong. That is a different problem.

Why an update is not an install

Three things are true of an update that were not true of the commissioning. There is a working model in place that people depend on, so the update must be reversible. There is an existing runtime version that may or may not support the new file’s architecture identifier and quantization version. And there is no way to fetch a missing dependency after the fact — whatever you did not carry across, you do not have. The general case for the deployment style is in air-gapped AI deployments; this page is the recurring operation.

The consequence is that the transfer set is larger than the model. It has to include everything that must be true on the far side for the new weights to load, and a way to prove that what arrived is what left.

Building the transfer set

Acquire the weights on the connected side, under their licence. Open-weight models carry real terms, and some are gated: access is granted on the publisher’s platform after accepting a licence. Do that acceptance on the connected side, with an account belonging to the organisation that will run the model, and carry the licence text across with the weights. Do not attempt to route around a gate — both because it is a licence violation and because a file obtained outside the publisher’s distribution has no provenance you can verify.
Record what the publisher says the file is. Before you leave the connected network, capture the repository, revision, file names and the publisher’s own hashes if they publish them. This is the only moment you can check the file against its source; after this, all you can check is that it has not changed since you last looked.
Inspect the file before it travels. Read the header with a GGUF metadata dump and confirm the architecture identifier, quantization version and declared context length are ones your deployed runtime supports. Finding out on the far side costs you a second trip.
Include the runtime, if it needs to change. If the new architecture needs a newer llama.cpp, Ollama or driver, that binary is part of the transfer set with its own checksum. An update that lands weights the deployed runtime cannot read is an outage, not an update.
Build a manifest and sign it.
```
sha256sum *.gguf LICENSE.txt runtime.tar.gz > SHA256SUMS
gpg --detach-sign --armor SHA256SUMS
```
The checksums prove the files did not change. The signature proves the checksum list did not change, which is the part a plain hash file cannot do: anyone who can alter a file on the media can also alter a manifest sitting next to it. The verifying key must already be on the air-gapped machine, placed there at commissioning.
Add a tensor-level hash for the model itself. llama.cpp’s llama-gguf-hash --sha256 hashes the tensor payload rather than the file, which distinguishes “somebody edited a metadata key” from “the weights are different”. On an update that distinction is the difference between a re-tagged file and a substituted one.

The media is the trust boundary

An air gap is not a gap if you carry a writable, executable, firmware-programmable device across it twice a month. The removable media is now the highest-value target in the whole arrangement, because it is the only thing that crosses.

Dedicate the media and never let it be general purpose. A drive used for transfers and nothing else, labelled, stored controlled, and never plugged into an arbitrary machine. Media control is a standard requirement in every security framework that contemplates air gaps, and it is standard because it is the failure people actually have.
Prefer write-once where the volume allows it. Optical media cannot be silently modified after it is written, which removes an entire class of tampering. Model files are often too large for this to be practical, which is a reason to use it for the signed manifest even when the weights travel on a disk.
Scan on a staging host, not on the target. Malware scanning belongs on an intermediate machine that is neither the connected build host nor the protected system.
Mount defensively. On the receiving side, mount read-only and without execute permission: mount -o ro,noexec,nosuid,nodev /dev/sdb1 /mnt/transfer. A model file has no reason to be executable and no reason to be written to.
Firmware attacks are real and this does not stop them. Verifying file contents says nothing about the device’s controller. If the threat model includes that, the answer is a one-way data diode or a controlled optical process, not a better checksum.

Verifying on the receiving side

Verify the signature before you trust anything else: gpg --verify SHA256SUMS.asc SHA256SUMS. If this fails, stop. Nothing downstream is meaningful.
Verify the contents against the signed list: sha256sum -c SHA256SUMS. Expect every line to say OK. A single mismatch means the transfer is discarded, not investigated in place.
Copy into a staging directory that is not the model directory the running service reads from. The service must keep serving the old weights while you work.
Re-read the header on the target machine and confirm it matches what you recorded on the connected side. This catches a corrupted copy that still hashes correctly because you hashed the corrupted file.
Load the new model on a spare port and send it real work — a handful of representative prompts whose answers you kept from the old model. You are checking that it loads, that it stops, and that its output is not wildly different in shape. If the deployment uses LoRA adapters, this is where you find out that adapters trained against the previous base no longer apply cleanly.

Cutover and rollback

Keep the old weights. This is the whole difference between an update procedure and an install procedure, and it is the step people skip because disk is finite and the new model “worked in testing”.

The mechanism that makes rollback trivial is indirection: have the service read from a path that is a symlink, and cut over by moving the symlink rather than by overwriting files.

/srv/models/
  current -> ./llama-3.1-8b-instruct-q4km-2026-02/
  llama-3.1-8b-instruct-q4km-2025-11/
  llama-3.1-8b-instruct-q4km-2026-02/

# cutover
ln -sfn ./llama-3.1-8b-instruct-q4km-2026-02 /srv/models/current
systemctl restart llama-server

# rollback, same shape, seconds
ln -sfn ./llama-3.1-8b-instruct-q4km-2025-11 /srv/models/current
systemctl restart llama-server

Two habits make the next transfer easier. Keep an inventory file on the air-gapped machine recording, for each model, where it came from, which revision, which licence, which runtime version it was validated against and the date it went live — you cannot look any of that up later, and the person doing the next update may not be you. And keep the previous version until the new one has survived a full business cycle, not until the end of the maintenance window.

Tool names in this workflow drift: Hugging Face’s command-line client has been renamed, llama.cpp’s binaries have been renamed more than once, and package managers change their offline modes. Check the current invocation on the connected side before you build a transfer set around a command you cannot test on the far side.