Why We’re Building Nest: Moving Beyond Docker Registry V2’s Data-Transfer Bottleneck

At PuffinNest, we are building a modern container registry for developers and teams that care about reliability, speed, and predictable costs.

As part of that work, we are creating a new client called Nest.

Nest is not being built because Docker commands are unfamiliar or because we want to replace established tooling for the sake of it. We are building it because the current Docker Registry HTTP API V2 model places fundamental constraints on how container images are transferred.

Those constraints become increasingly painful as image sizes, deployment frequency, team size, and geographic distribution grow.

The core problem is simple:

With the conventional Docker registry workflow, the registry service sits directly in the path of image data.

That means every pushed or pulled layer may need to pass through infrastructure operated by the registry provider—even when the actual layer is stored somewhere else.

For a small image pulled occasionally, this is easy to ignore.

At scale, it becomes an architectural bottleneck.

How a conventional container pull works

A container image is not usually transferred as one large file. It consists of a manifest, configuration data, and a collection of content-addressed layers.

During a typical pull, the client:

1. Authenticates with the registry.
2. Requests the image manifest.
3. Determines which layers it does not already have.
4. Requests those layers from registry endpoints.
5. Verifies each downloaded layer using its digest.
6. Unpacks the image locally.

Docker limits the number of concurrent transfers it performs. Docker Engine currently defaults to three concurrent layer downloads and five concurrent layer uploads, while BuildKit also applies per-registry connection limits. These limits exist for reasonable compatibility and reliability reasons, but they can constrain throughput for images containing many layers or when the network has substantial latency.

The client cannot simply invent a more efficient transfer protocol. It must communicate using the behaviours supported by the registry API and the Docker daemon.

This places an important boundary around what a registry provider can improve without controlling the client.

Storage and delivery are not the same thing

Most modern registries do not want application servers to hold terabytes of image layers on local disks.

Instead, layers are commonly stored in object storage systems such as:

• Amazon S3
• Cloudflare R2
• Google Cloud Storage
• Azure Blob Storage
• Other S3-compatible platforms

Object storage is excellent for immutable, content-addressed blobs. It is durable, scalable, and designed to transfer large objects efficiently.

However, the Docker-compatible registry still has to expose the API that Docker understands.

Depending on the registry architecture and storage backend, this can create two possible data paths.

A direct or redirected path looks roughly like this:

Docker client
     |
     | authenticate and request layer
     v
Registry API
     |
     | authorise or redirect
     v
Object storage

A proxied path looks like this:

Docker client
     |
     | request layer
     v
Registry server
     |
     | fetch layer
     v
Object storage
     |
     | return layer
     v
Registry server
     |
     | stream layer
     v
Docker client

In the second model, the registry server becomes a data-transfer middleman.

The layer may already exist in an object store built specifically for high-throughput delivery, but the bytes are pulled into the registry’s infrastructure and then sent back out to the user.

The server is no longer just handling authentication and metadata. It is carrying the entire payload.

Why proxying image data is expensive

Proxying large blobs through an application server creates costs at multiple layers.

1. The same data crosses more infrastructure

Consider a 4 GB image being pulled.

With direct object-storage delivery, approximately 4 GB travels from storage to the client.

With a full proxy path, the registry service may receive 4 GB from storage and then transmit another 4 GB to the client.

The precise billing depends on the cloud provider, network topology, and region, but the architecture creates additional data movement by design.

When that image is pulled by 100 CI workers, the registry is no longer handling a 4 GB event. It is participating in roughly 400 GB of user-facing transfer, plus any transfer required between the storage system and the registry servers.

The registry’s compute tier has effectively become a bandwidth tier.

2. Application servers must be sized for traffic rather than computation

Registry control-plane operations are relatively small:

• Authenticating users
• Checking repository permissions
• Returning manifests
• Creating upload sessions
• Recording metadata
• Producing signed transfer instructions

Blob transfer is different.

A server proxying image layers needs enough:

• Network throughput
• Open connections
• File descriptors
• Memory for buffering
• CPU for encryption and request processing
• Capacity to absorb sudden deployment spikes

The service may perform very little meaningful computation while moving gigabytes of data.

Nevertheless, it must be provisioned for peak throughput.

That is an inefficient use of application infrastructure.

3. Autoscaling becomes reactive

Container traffic is often bursty.

A deployment may cause hundreds of machines to pull the same image within seconds. A CI pipeline may push several large variants simultaneously. A regional recovery event may trigger a surge across an entire cluster.

Autoscaling can add more registry instances, but it is rarely instantaneous.

Before new capacity becomes available, existing servers can experience:

• Connection saturation
• Increased latency
• Lower per-client throughput
• Request queues
• Timeouts
• Retries

Those retries add even more traffic to an already overloaded path.

4. Egress pricing becomes part of the registry’s core economics

Storage can be inexpensive while delivery remains costly.

When every layer must be streamed through registry infrastructure, the provider has to price for:

• Storage egress
• Server or load-balancer egress
• Cross-zone or cross-region traffic
• Compute instances
• Load balancers
• TLS termination
• Monitoring and logging
• Capacity kept available for peaks

Customers may think they are paying for storage, but a large part of the provider’s cost can come from repeatedly moving the same stored bytes.

Ultimately, that cost has to appear somewhere: higher subscription prices, transfer limits, fair-use clauses, overage charges, or throttling.

Why this is bad for users

The impact is not limited to the registry operator’s cloud bill.

Users feel the architecture directly.

Slower pushes and pulls

The client’s speed becomes limited by the slowest part of the path:

client <-> registry server <-> storage

Even when the object store and the user both have fast connections, the registry’s proxy tier can constrain throughput.

A direct transfer could use the storage platform’s global edge, regional bandwidth, and optimised large-object delivery. A proxied transfer is limited by the registry server’s location and capacity.

Shared bottlenecks

When many customers use the same proxy tier, one customer’s traffic spike can affect another customer’s experience unless the service maintains substantial spare capacity.

The result can be inconsistent performance:

• A pull takes 20 seconds in the morning.
• The same pull takes two minutes during a deployment peak.
• CI jobs begin timing out.
• Retried jobs increase the load further.

Developers generally do not care which internal server is saturated. They experience an unreliable registry.

Additional latency for geographically distant users

A user may be close to an object-storage region or edge location but far from the registry’s application servers.

If the data has to pass through the registry region, the route may become:

storage region -> registry region -> user region

rather than:

nearest storage location -> user

That detour increases latency and can reduce achievable throughput.

More failure points

Every additional hop creates another place where a transfer can fail.

A proxied transfer can be interrupted by:

• The client connection
• The load balancer
• The registry instance
• The registry-to-storage connection
• Instance rotation
• A deployment of the registry service
• An upstream timeout
• A downstream timeout

Large layers make these failures particularly frustrating because the user may have already transferred a substantial amount of data before the interruption.

Resumable uploads can help, but they do not remove the underlying middleman.

Throttling becomes difficult to avoid

A registry provider cannot offer unlimited throughput from finite shared servers.

Without aggressive and expensive over-provisioning, it may eventually need to enforce:

• Per-user bandwidth limits
• Per-organisation concurrency limits
• Request-rate limits
• Upload limits
• Pull limits
• Fair-use policies

These mechanisms protect the service, but they are symptoms of a constrained shared transfer path.

For users, throttling appears at exactly the wrong moment: during a production deployment, a CI fan-out, or an urgent rollback.

Why can’t the existing Docker client solve this?

The Docker client and daemon implement an established registry protocol. That compatibility is extremely valuable: users can point Docker at a registry and use familiar commands such as docker pull and docker push.

But compatibility also limits how much the registry can change.

A registry cannot arbitrarily tell an existing client:

• Upload 32 chunks concurrently using this custom scheduling algorithm.
• Open a direct session with our storage network.
• Negotiate transfer routes dynamically.
• Deduplicate blocks smaller than an OCI layer.
• Select a nearby storage endpoint.
• Use a different retry and congestion-control strategy.
• Resume a transfer using client-specific local state.
• Exchange only the changed portions of a large layer.

The server has to communicate in ways the existing client understands.

The Distribution V2 API supports established workflows for manifests, blobs, uploads, and authentication. It is an interoperability protocol, not a blank canvas for an end-to-end transfer system.

Trying to add radically different behaviour while pretending it is still an ordinary Docker interaction produces complexity and brittle workarounds.

At some point, a better data path requires a client that understands it.

Introducing Nest

Nest is our answer to that constraint.

Nest will be a purpose-built PuffinNest client designed around modern content transfer rather than around the requirement that every operation look like a traditional Docker Registry V2 request.

The goal is to separate the control plane from the data plane.

The control plane remains responsible for sensitive decisions:

• Who is the user?
• Can they access this repository?
• Which image or artifact are they requesting?
• Which blobs are required?
• Is an upload allowed?
• What limits or policies apply?
• How long should a transfer authorisation remain valid?
• Has the completed content passed integrity checks?

The data plane is responsible for moving bytes.

A Nest transfer can therefore follow a model closer to:

Nest client
     |
     | authenticate and request a transfer plan
     v
PuffinNest API
     |
     | return short-lived, scoped transfer authorisation
     v
Nest client
     |
     | transfer blobs directly
     v
Object storage or delivery infrastructure

PuffinNest still controls access.

The difference is that the PuffinNest application server does not need to personally relay every byte.

Direct transfer does not mean public storage

It is important to distinguish direct delivery from unauthorised public access.

A secure direct-transfer architecture can use short-lived, narrowly scoped credentials or signed URLs.

For example, a download authorisation can be restricted by:

• Object key
• Repository
• Operation type
• Expiration time
• HTTP method
• Content digest
• Expected size
• User or session
• Network policy, where appropriate

An upload authorisation can allow a client to write one expected object without granting permission to list the bucket, overwrite unrelated data, or read other customers’ layers.

After an upload, PuffinNest can independently verify:

• The object exists.
• Its length is correct.
• Its digest matches the declared digest.
• The user was authorised to attach it to the repository.
• The resulting manifest is internally consistent.

The storage system moves the bytes, but PuffinNest remains the authority.

What Nest gives us room to improve

Owning both sides of the transfer contract creates opportunities that are difficult or impossible to introduce transparently through a generic Docker client.

Better concurrency

Nest can choose concurrency based on:

• Image composition
• Layer sizes
• Available bandwidth
• Connection quality
• Storage limits
• Local machine resources
• Server-issued policies

A fixed default is simple, but the ideal transfer strategy for a laptop on hotel Wi-Fi is not the same as the ideal strategy for a 10 Gbps CI runner.

Smarter retries

Not every error should restart the same operation in the same way.

Nest can distinguish between:

• Expired authorisation
• Temporary storage errors
• Network interruption
• Digest mismatch
• Rate limiting
• A partially completed upload
• A server-side policy rejection

It can refresh only what is necessary and resume safely where the underlying storage mechanism permits it.

More predictable performance

Removing the central proxy tier means transfer performance is less dependent on how busy PuffinNest’s API servers are.

The API can remain lightweight and responsive during large deployment events because it is coordinating transfers rather than carrying all of them.

Regional and edge-aware delivery

A purpose-built client can receive a transfer plan that sends it to an appropriate endpoint based on storage location, policy, or availability.

That creates a foundation for regional optimisation without forcing all traffic through one application region.

Improved progress reporting

Traditional progress output often exposes only a narrow view of what is happening.

Nest can report distinct phases such as:

• Resolving the image
• Checking local content
• Requesting authorisation
• Uploading or downloading layers
• Verifying digests
• Registering the manifest
• Finalising the operation

When something fails, users should receive an actionable explanation rather than an ambiguous timeout.

Protocol evolution

Most importantly, Nest gives PuffinNest a controlled path for improvement.

We can evolve the client and server together, introduce capability negotiation, and preserve compatibility across versions without waiting for every Docker installation to support a new transfer model.

That does not mean changing the protocol casually. Transfer software must be stable and conservative.

It means we are no longer permanently constrained by behaviours designed for a different generation of registry infrastructure.

Why PuffinNest will still support Docker workflows

Nest is not an argument against compatibility.

The Docker and OCI ecosystems are too important to ignore, and standard tooling should continue to work with PuffinNest.

Developers expect integrations with:

• Docker
• Docker Compose
• Kubernetes
• BuildKit
• CI systems
• Existing deployment platforms
• OCI-compatible tools

PuffinNest’s conventional registry interface remains essential for interoperability.

Nest is intended to provide an enhanced path for users who want better transfer performance and a client designed specifically for PuffinNest.

The two approaches serve different needs:

Docker-compatible interface:
Maximum ecosystem compatibility

Nest:
Optimised PuffinNest experience and transfer path

Standards provide reach. A native client gives us room to innovate.

The cost benefit should reach users

Lower infrastructure cost is not only an internal optimisation.

When the registry does not need to proxy every blob, we can build a service with better underlying economics.

That can support:

• More generous transfer allowances
• Fewer artificial throttles
• More predictable pricing
• Better performance during traffic spikes
• Less pressure to charge users for duplicated infrastructure work
• More investment in security, scanning, metadata, and developer tooling

A provider should not make users pay for an avoidable network detour.

Our aim is to spend infrastructure resources on things that create real value: secure access, integrity, vulnerability information, auditability, collaboration, and dependable artifact management.

Relaying the same immutable bytes through an application server does not create equivalent value.

The broader lesson

Protocols are trade-offs.

Docker Registry V2 succeeded because it created a widely supported way to distribute container images. Its compatibility and content-addressed model helped form the container ecosystem we rely on today.

But a successful standard can still impose architectural constraints.

The lesson is not that the existing protocol is bad. The lesson is that it was designed around a particular interface and set of assumptions. As requirements change, strict compatibility can prevent deeper optimisation.

For PuffinNest, the answer is not to abandon the ecosystem.

It is to support the standard path while creating a better native path.

That is why we are building Nest.

We want container transfers that are:

• Fast
• Direct
• Secure
• Resumable
• Observable
• Cost-efficient
• Less vulnerable to central throttling

Most of all, we want the registry server to coordinate and protect transfers—not become the pipe every byte is forced through.

You can learn more about PuffinNest and join the early-access waitlist at puffinnest.com.