DEV Community: Ivan Hopkins

The LLM Portability Drill: Redeploy an Open Model on a Second GPU Cloud

Ivan Hopkins — Tue, 29 Sep 2026 19:26:22 +0000

Access to open weights gives a team the right to operate a model in its own environment. I test portability by asking whether the team can reconstruct the complete service after the host, runtime, or provider changes.

During an urgent move, a repository may point to a newer commit, a container tag may have changed, the tokenizer may no longer match, or the serving engine may interpret tool calls differently. The weights remain available. The production service still cannot be recreated with confidence.

I use a portability drill to find that hidden state while the original deployment remains available. The drill asks whether another environment can reconstruct the same release from recorded artifacts and pass the same application checks.

Define the three levels of portability

I separate portability into three levels because each one fails differently.

Artifact portability

I start with the bytes. Every required artifact must be retrievable and verifiable: the model revision, weight files, tokenizer, processor, chat template, adapters, licenses, container image, and configuration. Hugging Face supports downloading a repository at a full commit hash, which provides a stronger reference than a mutable main branch.

Behavioral portability

Next comes behavior. The reconstructed endpoint must pass the same application-specific tests within defined tolerances. Golden cases should cover normal prompts, edge cases, structured output, tool calls, refusal behavior, and any model-specific reasoning controls. Hugging Face explains that chat models expect particular control tokens and chat templates; a template mismatch can materially change behavior even when the weights match.

Operational portability

Operational portability covers the service around the model. I want a clean environment to expose the expected API contract, pass readiness checks, load secrets through the approved mechanism, emit logs and metrics, and support the documented rollback procedure. These checks keep the drill focused on deployability and service readiness.

Freeze a complete LLM release

I keep a small release manifest in source control. This local engineering artifact remains independent of provider APIs.

release: support-model/2026-09-02
model:
  repo: org/model-name
  revision: <full-commit-hash>
  files_sha256: checksums.txt
  chat_template: templates/chat.jinja

runtime:
  image: registry/llm-server@sha256:<digest>
  engine_version: <pinned-version>
  launch_args: config/serve.args

contract:
  endpoint: /v1/chat/completions
  tool_schema: tests/tool-schema.json
  golden_set: tests/portability.jsonl

Image tags can move, so I pin the container by digest. Docker's build guidance recommends digest pinning when teams need the same image version and an auditable update path. The release also needs a deliberate patch process because an immutable image will not receive security fixes automatically.

Run the drill on a second environment

Use a clean account or project on another GPU provider. Hidden state from the first environment should have no path into the exercise.

Provision a supported GPU environment and attach only documented storage and network dependencies.
Pull the container by digest and download the model at the pinned commit.
Verify checksums before startup.
Launch from the recorded arguments, with secrets supplied through the approved external mechanism.
Run artifact checks, the behavioral golden set, API contract tests, readiness checks, and rollback.
Record every manual intervention, undocumented dependency, and provider-specific assumption.

An OpenAI-compatible endpoint can reduce client changes. Route compatibility does not standardize every behavior. For example, SGLang documents model-specific reasoning parsers and chat-template parameters alongside its OpenAI-compatible APIs.

Use tolerances instead of chasing identical text

I avoid using bitwise-identical text as the default target across different environments. vLLM states that it does not guarantee reproducibility by default and limits its reproducibility statement to the same hardware and vLLM version, even with the relevant controls enabled.

I define behavioral acceptance around the application: valid JSON, correct tool selection, required facts, prohibited outputs, and rubric scores. Exact-output assertions remain useful for deterministic transformations where they apply.

Repeat the drill when the release changes

Run the first drill while the original deployment is healthy. Repeat it after changing the checkpoint, tokenizer, serving engine, container, or API contract.

Fluence's guide to the best open-source and open-weight LLM models can help create an initial shortlist around workload, modality, license, and deployment fit. After selection, portability becomes a property of the complete release. Any undocumented file or manual fix exposed by the second environment belongs in the next manifest revision.

Two Million Open AI Models, but Most of the Attention Goes to Just 200

Ivan Hopkins — Thu, 20 Aug 2026 14:24:37 +0000

Every time I open Hugging Face, the model ecosystem seems to have expanded again. There are new base models, fine-tunes, quantisations, adapters and experimental releases arriving constantly.

At first glance, this looks like an incredibly diverse market. We have more models, more developers and more ways to run AI than ever before.

But when I looked at where the downloads actually go, I found a much more concentrated ecosystem hiding beneath all that abundance.

The Numbers Tell Two Different Stories

Hugging Face reported that its platform grew to 13 million users, more than two million public models and over 500,000 public datasets in 2025.

That is a huge amount of activity. It also shows that developers are doing far more than downloading foundation models. They are building specialised versions, adapters, benchmarks and applications around them.

The distribution of attention, however, is extremely uneven.

Approximately half of the models on Hugging Face have received fewer than 200 downloads. Meanwhile, the 200 most-downloaded models—roughly 0.01% of the models available—account for 49.6% of all downloads.

So we have millions of choices, but a surprisingly small group of models captures almost half the activity.

More Models Do Not Necessarily Mean More Choice

I do not think this concentration is automatically bad.

Developers tend to gather around models that work well, have clear documentation and are supported by popular inference tools. That creates useful network effects. Bugs are identified sooner, integrations improve and community knowledge becomes easier to find.

The problem begins when we confuse repository count with meaningful choice.

A model may technically be available but still be difficult to deploy because it lacks maintained serving templates, current dependencies or reliable quantisations. Another may perform well in a benchmark but have a restrictive licence. Some experimental fine-tunes may never receive another update.

In other words, the usable ecosystem is much smaller than the visible catalogue.

Community Depth Is Becoming a Selection Metric

When I evaluate an open or open-weight model, I no longer look only at its benchmark scores and parameter count. I also want to know what exists around it.

I look for signals such as:

Recent maintenance and active issue resolution
Compatibility with established inference frameworks
Community-created quantisations and deployment guides
Clear licensing and permitted commercial use
Fine-tunes or adapters for relevant domains
Evidence of adoption beyond the original developer

These signals do not prove that a model is production-ready. But they tell me whether I am selecting an isolated repository or joining an ecosystem that other developers are actively improving.

That distinction matters when a model becomes part of a product rather than a weekend experiment.

Popularity Is Useful, but It Can Also Mislead

Download volume is one useful signal, not a final verdict.

A highly specialised medical, legal or regional-language model may have modest downloads while still being extremely valuable to its intended users. Hugging Face itself describes open AI as a collection of overlapping sub-ecosystems rather than one uniform market.

The opposite is also true. A widely downloaded model is not automatically the right choice for every workload. Popularity cannot tell me whether it meets my latency target, fits the GPUs available to me or behaves reliably with my application’s prompts.

I normally begin with a manageable shortlist. A practical overview of leading open model families can help narrow the field, after which I compare the candidates using the workload they will actually handle.

That process gives me something a general leaderboard cannot: evidence that the model works inside my system.

Open AI Has a Supply-Chain Question

The concentration of downloads also creates a less obvious risk.

Thousands of applications may depend on a relatively small number of base models, maintainers and supporting tools. If one licence changes, a repository becomes inactive or a critical component develops a vulnerability, the effects can spread well beyond the original project.

That makes model selection resemble software supply-chain management.

Teams need to understand where a model came from, which components it depends on, how actively it is maintained and whether they could migrate if necessary. Downloading the weights gives teams more control than relying exclusively on a closed API, but it does not eliminate dependency.

The Real Shortage Is Confidence

We do not have a shortage of models. We have a shortage of dependable ways to determine which ones deserve long-term trust.

The next phase of the open AI ecosystem will not be defined only by how many models get published. It will depend on how well developers can evaluate, maintain, secure and deploy them.

Two million models sounds like unlimited choice. In practice, the models that matter are the ones surrounded by enough evidence, tooling and community support to survive contact with production.