Mirror Speed: What Actually Matters When You're Pulling from a Repo at 3 AM

I’ve spent more hours than I care to admit staring at a terminal, waiting for a git clone to finish. The progress bar crawls, the network indicator sits at a trickle, and I start questioning every DNS setting I’ve ever touched. Mirror speed is one of those topics that sounds simple—"just pick a faster mirror"—but in practice, it’s a tangle of protocol overhead, geographic routing, and filesystem cache behavior. Let me walk you through what I’ve learned from managing my own local mirror of a few large open-source Windows tool repositories, and where I’ve hit the wall.

The First Mistake: Assuming "Faster" Means "Closer"

The most common advice is to pick a mirror near you. That’s not wrong, but it’s incomplete. I once had a mirror in the same city that was consistently slower than one two time zones away. Why? Because the local mirror was behind a shared 1 Gbps link that was saturated by a CDN serving video streams. The distant mirror had a dedicated 10 Gbps pipe and a better TCP stack.

Here’s the thing: mirror speed is a function of end-to-end throughput, not just latency. You can measure it with a simple test. On Windows, I use PowerShell to download a known file from each candidate mirror and time it. But you need to be careful about caching—if you hit the same file twice, the OS file cache will skew the results)Skip the file cache by using a unique query string or a different file each time.

# Quick mirror throughput test on Windows
$urls = @(
    "https://mirror-a.example.com/file1.bin?cachebust=1",
    "https://mirror-b.example.com/file1.bin?cachebust=2"
)
foreach ($u in $urls) {
    $sw = [System.Diagnostics.Stopwatch]::StartNew()
    Invoke-WebRequest -Uri $u -OutFile "$env:TEMP\test.bin" -UseBasicParsing
    $sw.Stop()
    $size = (Get-Item "$env:TEMP\test.bin").Length
    $mbps = [math]::Round(($size * 8) / ($sw.Elapsed.TotalSeconds * 1MB), 2)
    Write-Host "$u -> $mbps Mbps"
    Remove-Item "$env:TEMP\test.bin" -Force
}

But even this has a flaw. A single file test doesn’t reflect the reality of a git clone or a package manager update, which opens hundreds of small connections. That’s where the real bottleneck often hides: connection setup overhead.

Parallelism and the Hidden Cost of Connection Setup

When you pull from a mirror, you’re not downloading one big file. You’re downloading thousands of small objects—blobs, trees, commit metadata. Each one may require a separate HTTP request. The TCP handshake, TLS negotiation, and HTTP headers can add 50–100 ms per request. If you have 10,000 objects, that’s 500–1000 seconds of pure overhead, even if your bandwidth is infinite.

That’s why I stopped using plain git clone from mirrors and started using shallow clones (--depth 1) for CI and quick testing. But for a full mirror, that’s not an option. So I turned to tools that support concurrent downloads. The most reliable I’ve found is aria2 (which works well on Windows via choco install aria2), but the real game-changer for me was a small utility called putyy res downloader 3 1 3. It’s designed specifically to fetch resource packs from multiple mirrors in parallel and then reassemble them, which is exactly the pattern that kills naive single-threaded downloaders.

Here’s a typical workflow I use when I need to refresh a large mirror of a Windows SDK:

# Using aria2c to pull a large manifest with 16 connections
aria2c -x 16 -s 16 -k 1M `
    -o sdk-mirror.zip `
    "https://mirror-a.example.com/sdk/latest.zip" `
    "https://mirror-b.example.com/sdk/latest.zip" `
    "https://mirror-c.example.com/sdk/latest.zip"

Notice I’m passing multiple URLs for the same file. aria2c will stripe the download across all of them, and if one mirror fails mid-transfer, it retries from another. That’s the kind of resilience you need when a mirror goes flaky at 2 AM.

Failure Modes You’ll Actually Hit (and How to Spot Them)

I’ve seen three distinct failure modes that kill mirror speed, and none of them show up in a simple ping test.

1. The "Stale Cache" Mirror. Some mirrors use a reverse proxy with a long TTL. You request a file, and the proxy serves you a cached version from yesterday—but the actual upstream file has changed. The download is fast, but you get corrupted or outdated data. The symptom: git clone succeeds, but git fsck reports missing objects. The fix: always verify checksums. I write a small script that compares the SHA256 of downloaded files against a manifest from the upstream repo.

2. The "Throttled by IP" Mirror. Some mirrors throttle you after the first 100 MB if you’re on a shared IP (like a corporate NAT). The speed starts high, then drops to a crawl. The symptom is a classic "hockey stick" throughput curve. The workaround: rotate through mirrors using a tool like putyy res downloader 3 1 3, which can be configured to switch mirrors when throughput drops below a threshold.

3. The "IPv6 Blackhole" Mirror. This one is subtle. Your machine prefers IPv6, but the mirror’s IPv6 route is broken—packets are dropped, not rejected. TCP retransmits kill your speed. The symptom: the download hangs for 10 seconds, then bursts. The fix: force IPv4 with curl -4 or set prefer_ipv4=1 in your ~/.gitconfig if you’re using git over HTTP.

Here’s a concrete PowerShell snippet I use to detect the IPv6 issue:

# Check if IPv6 to a mirror is actually working
$ipv6 = Test-NetConnection -ComputerName "mirror-a.example.com" -Port 443 -AddressFamily IPv6
if ($ipv6.TcpTestSucceeded) {
    Write-Host "IPv6 OK, but let's measure real throughput..."
    $sw = [System.Diagnostics.Stopwatch]::StartNew()
    curl.exe -6 -o NUL "https://mirror-a.example.com/test.bin"
    $sw.Stop()
    Write-Host "IPv6 download took $($sw.Elapsed.TotalSeconds) sec"
} else {
    Write-Host "IPv6 is broken, forcing IPv4"
}

Edge Case: The Local Filesystem Is the Real Bottleneck

I nearly pulled my hair out once because a mirror was "slow," but it turned out the mirror was fine—my local disk was the problem. I was downloading to a spinning HDD that was also serving a virtual machine’s pagefile. The disk I/O queue was so deep that writes were taking 200 ms per block. The download was actually arriving at the network card at full speed, but the OS couldn’t write it to disk fast enough.

The fix was simple: download to an SSD or RAM disk. On Windows, I use New-VHD to create a small RAM-backed drive, but that’s overkill. More practically, I just pointed my temp directory to a fast NVMe drive:

# Redirect temp downloads to fast storage
$env:TEMP = "D:\FastTemp"
$env:TMP = "D:\FastTemp"
New-Item -ItemType Directory -Path $env:TEMP -Force | Out-Null

Also, check your antivirus. Real-time scanning on every downloaded file can halve your throughput. I exclude my mirror directories from Windows Defender’s real-time scanning. That alone sped up my pulls by 30–40% on a large repo.

The Practical Workflow: What I Do Now

My current workflow for keeping a local mirror of a large Windows toolchain is this:

1. Maintain a list of 5–6 candidate mirrors across different ISPs and regions. I update this list monthly.
2. Use a segmented downloader (either aria2c or putyy res downloader 3 1 3) that can parallelize and failover.
3. Always verify checksums after download. I keep a SHA256SUMS file in the mirror root and run a PowerShell verification script.
4. Monitor throughput over time, not just once. I log the speed of each mirror in a CSV and look for trends. A mirror that was fast last week might be saturated this week.
5. Use shallow clones for ephemeral tasks, and only do a full mirror when I actually need the history.

The tool putyy res downloader 3 1 3 has become a regular part of this because it handles the failover logic automatically. It doesn’t just download—it tracks which mirror responded fastest for the last N requests and biases future requests toward that one. That’s the kind of adaptive behavior you don’t get from a simple wget.

Related resource

If you’re working with Windows resource packs and need a downloader that handles multi-mirror fallback gracefully, I’ve found putyy res downloader 3 1 3 to be a solid addition to the toolkit. It’s hosted on GitVaults, which is a catalog of open-source Windows software—useful for finding tools like this without wading through random forum posts.

Caveat

I’ve shared what works for me, but mirror speed is highly environment-specific. Your ISP’s peering, your local hardware, and even the time of day can flip which mirror is fastest. If you see something in my approach that’s wrong or have a better trick—especially around Windows-specific network stack tuning—I’d genuinely appreciate a correction. Drop a note in the comments or reach out; I’m always learning from people who’ve hit different failure modes than I have.