Cutting the AI Bottleneck: Multicloud LLM Distribution with CNCF Dragonfly and OCI

How peer-to-peer networking fixes the 26 TB download problem when you're scaling large language models across Oracle Kubernetes Engine and multicloud clusters.

Note: This article was originally published on the Oracle University Community (Author badge). This is the public version so the CNCF Dragonfly community and others can read it without a login. You can also find it listed on my Oracle ACE profile.

---

The Problem Nobody Budgets For

Your team just fine-tuned a large parameter model. The safetensors checkpoint is well over 100 GB. You need it running on 200 GPU nodes across your OKE cluster by Monday.

Do the math: a ~130 GB checkpoint times 200 nodes is roughly 26 TB of network transfer from a single model hub. Every node pulls the same massive files from Hugging Face or ModelScope independently. Your shared internet egress turns into a chokepoint. Rate limits kick in. Nodes that started downloading at the same time finish hours apart. Your GPU fleet sits idle, burning money, waiting for weights.

This is the "thundering herd" problem for AI model distribution, and it only gets worse. Large model checkpoints can range from tens to hundreds of GB depending on precision, quantization, sharding, and repository version. These things aren't getting smaller. And as enterprises move from single-prompt interactions to multi-step Agentic AI workflows, the infrastructure needs to spin up inference pods fast, and an autonomous agent can't wait hours for a model to download before it can execute a task. For agentic AI systems that need to scale inference quickly across tools, regions, or clouds, model distribution becomes part of the reliability architecture.

I kept running into this while building multicloud Kubernetes infrastructure. The usual fixes (NFS mounts, pre-baked container images, object storage mirrors) all have tradeoffs. NFS can become a bottleneck or a single point of failure if it isn't designed for high availability. Baking models into container images bloats your registry and slows every pull. Object storage mirrors are useful, but they need good sync discipline to avoid serving stale artifacts.

There's a better approach, and it's already running in production at Alibaba-scale: peer-to-peer distribution.

What Is CNCF Dragonfly?

Dragonfly is a CNCF Graduated project that turns every downloading node into a seed for its peers. It was originally built for container image distribution at Alibaba, where it handles billions of daily requests, and it works by splitting large files into small pieces and distributing them across a P2P mesh.

The key idea is that the origin server (Hugging Face, ModelScope, your private OCI bucket, whatever) only gets hit once by a seed peer. As soon as a single piece lands, it's immediately available to other nodes in the cluster. Distribution starts in parallel with the initial fetch.

For that ~130 GB model across 200 nodes, in an ideal cache-warmed or preheated scenario, repeated origin downloads can be reduced dramatically, from roughly 26 TB toward a single origin copy of about 130 GB. Later nodes can download faster and more consistently as more peers cache pieces locally, since each node pulls from dozens of local peers at the same time.

Earlier this year, I contributed native protocol support for Hugging Face (hf://) and ModelScope (modelscope://) directly into the Dragonfly Rust client (PRs #1665 and #1673 in the dragonflyoss/client repository). These backends let Dragonfly understand model hub URLs natively, including authentication, revision pinning, and repository structure, without wrapper scripts or URL rewriting. The CNCF published a walkthrough of this work at https://www.cncf.io/blog/2026/04/06/peer-to-peer-acceleration-for-ai-model-distribution-with-dragonfly/.

Why This Matters on OCI

OCI has become a capable platform for AI workloads. OKE gives you managed Kubernetes with GPU node pools (A10, A100, H100 shapes), and OCI can offer competitive GPU pricing depending on region, shape, and commitment model. Preemptible instances can make experimentation cheaper still, so check the current OCI Compute pricing for your region and shape.

But the network bottleneck doesn't care which cloud you're on. If 50 OKE GPU nodes all pull a ~130 GB model from Hugging Face at the same time, you're still looking at multiple TB of internet egress, rate limiting, and idle GPUs.

Dragonfly on OKE fixes this with one architectural change: deploy it as a DaemonSet alongside your GPU workloads. The seed peer fetches from the origin once. Every other GPU node pulls pieces from its peers. Peer-to-peer distribution reduces repeated internet-origin downloads and keeps most of the transfer inside the cluster, on the fast intra-VCN network.

Deploying Dragonfly on OKE

There are two parts to a working setup: install Dragonfly, then wire your download path or container runtime to use it. Start with the Helm install:

# Add the Dragonfly Helm repository
helm repo add dragonfly https://dragonflyoss.github.io/helm-charts/

# Install Dragonfly into your OKE cluster
helm install dragonfly dragonfly/dragonfly \
  --namespace dragonfly-system --create-namespace

# Verify the DaemonSet is running on your GPU nodes
kubectl get pods -n dragonfly-system -o wide

The Helm chart installs the Dragonfly components, but that alone won't route your pulls through the P2P mesh. You need one more step depending on what you're accelerating:

1. Install the chart (above). This deploys the scheduler, seed peer, and a dfdaemon DaemonSet on your nodes.
2. For model downloads: use dfget, which talks to the local dfdaemon directly. No runtime changes needed.
3. For container image pulls: point containerd at dfdaemon as a registry mirror so image pulls traverse the P2P path.

For the image-pull case, the containerd mirror config on each node looks roughly like this:

# /etc/containerd/certs.d/<registry-host>/hosts.toml
server = "https://<registry-host>"

[host."http://127.0.0.1:65001"]
  capabilities = ["pull", "resolve"]

Here 127.0.0.1:65001 is the local dfdaemon proxy. Restart containerd after applying it. Plan for this step before expecting cluster-wide acceleration.

With that in place, every node joins the P2P mesh. For models, pull directly with the native protocols:

# Download DeepSeek-R1 with P2P acceleration across all nodes
dfget hf://deepseek-ai/DeepSeek-R1 -O /models/DeepSeek-R1/ -r

# Or from ModelScope for Chinese-origin models
dfget modelscope://qwen/Qwen-7B -O /models/qwen/ -r

# Pin a specific model version for reproducibility
dfget hf://meta-llama/Llama-3.1-70B --hf-revision v2.0 \
  -O /models/llama3/ -r --hf-token=$HF_TOKEN

Note: the hf:// and modelscope:// examples require a Dragonfly client (and Helm chart) version that includes these protocol backends. Check that your installed version bundles the Hugging Face and ModelScope support before relying on these commands.

The Multicloud Angle

The real payoff shows up in multicloud setups.

Here's a common pattern: training happens on AWS (that's where the data lake is), but inference runs on OKE (better GPU pricing). That 130 GB model needs to move between clouds. The usual way? Push to S3, pull from S3 into each OKE node. Because every node downloads independently, repeated pulls from each OKE node can increase cross-cloud egress and network overhead.

With Dragonfly deployed in both clusters, the seed peer in your OKE cluster pulls the model once from the origin (or from an OCI Object Storage mirror). The other 49 GPU nodes pull from each other. That keeps cross-cloud egress closer to a single copy and shifts the remaining transfer inside the cluster.

Same story when you're pulling from multiple hubs. Maybe your team uses Llama from Hugging Face and Qwen from ModelScope. Since both protocols are built into Dragonfly, the same P2P mesh and caching layer handles both:

# Same cluster, different model hubs, same P2P mesh
dfget hf://meta-llama/Llama-3.1-8B -O /models/llama/ -r
dfget modelscope://qwen/Qwen2-7B -O /models/qwen/ -r

That's one distribution layer handling multiple origins, with no extra operational overhead.

Integrating with OCI Services

For production on OCI, a few integrations make the architecture tighter:

OCI Object Storage as a mirror. In restricted-network or regulated environments, you can seed Dragonfly from a private Object Storage bucket instead of the public internet. Specify the access pattern up front: Dragonfly can pull over HTTPS using a pre-authenticated request (PAR), through a service gateway for private access without traversing the internet, or via the S3-compatible endpoint. Upload model artifacts once, and Dragonfly distributes them across the cluster via P2P.

OCIR for model-serving containers. Dragonfly can also speed up container image pulls from OCI Container Registry. Your vLLM or Triton inference server images, easily 15 GB+ with CUDA libraries and typically built as multi-stage Docker builds to keep the CVE surface small, can get the same P2P treatment. This is not automatic: you configure your container runtime to route OCIR pulls through Dragonfly's registry mirror/proxy, after which the image layers are distributed peer-to-peer.

OCI Monitoring for visibility. Dragonfly exposes Prometheus metrics for P2P transfer rates, cache hit ratios, and peer counts. Those metrics can be scraped by Prometheus/Grafana on OKE, and selected metrics can be forwarded into OCI Monitoring through a custom metric ingestion path if you want them alongside your other OCI telemetry.

Practical Lessons from Production

A few things I've picked up running this in practice:

Seed peer placement matters. Put your Dragonfly seed peer on a node with good internet bandwidth, ideally in the same availability domain as your NAT gateway. This cuts the latency on the initial origin fetch.

Pre-warm before scaling. If you know a new model version is coming, kick off a dfget on the seed peer before you scale up the GPU node pool. By the time new nodes come online, the model's already cached in the P2P mesh and distribution is basically instant.

Pin your revisions. In production inference, always pin model versions (--hf-revision or --ms-revision). Every node in the cluster needs to serve the exact same weights, and you don't want to debug a mismatch at 2 AM.

Connection to Oracle University Training

If you're working through OCI training on Oracle University, a lot of this maps directly to what you're studying:

The relevant Oracle University learning paths are OCI Architect Professional, OCI Architect Associate, OCI DevOps Professional, OCI Networking Professional, OCI Observability and Management Professional, OCI AI Foundations Associate, and OCI Generative AI Professional. The Architect and Networking paths cover the multicloud networking, VCN, and service gateway concepts behind cutting cross-cloud egress and the private access patterns above. DevOps and Observability cover the Kubernetes, Helm, and monitoring patterns that underpin running Dragonfly. AI Foundations and Generative AI cover the GPU compute shapes and AI workload management that are the target environment for all of this.

Deploying Dragonfly on OKE is a solid hands-on exercise if you're prepping for any of these certs, since it ties together networking, Kubernetes, and AI infrastructure in one project. You can browse the full catalog and search for these paths on Oracle MyLearn.

Try It Yourself

1. Spin up an OKE cluster with a GPU node pool (even a single A10 works for testing).
2. Install Dragonfly via Helm.
3. Run dfget hf://deepseek-ai/DeepSeek-R1/config.json -O /tmp/config.json and watch the P2P logs.
4. Scale to a second node, repeat the download, and watch the cache hit.

Model checkpoints and cluster sizes keep growing. Running Dragonfly on OKE keeps the distribution layer from becoming the bottleneck as they do.

---

Pavan Madduri is a Senior Cloud Platform Engineer at W.W. Grainger, a CNCF Golden Kubestronaut, an Oracle ACE Associate, and a CNCF TAG Workloads Foundation Tech Lead. He built the native Hugging Face and ModelScope protocol support in CNCF Dragonfly. Find him on GitHub.

This article was originally published on the Oracle University Community. See my Oracle ACE profile for more.