For the past month, one question has stayed with me: AI coding agents are making it faster and easier to produce code, but can the infrastructure surrounding it keep up? Every change still needs to pass through source control, testing, review, and eventually deployment.
A few days ago, Anthropic published an engineering article explaining how they migrated their monorepo from Git to an internally developed version-control system called Scion. What caught my attention was the architectural reasoning behind that decision.
Since then, I've seen discussions suggesting that Git is becoming obsolete and Scion represents the future of source control. But I think there's a more interesting lesson here. Git's architecture hasn't suddenly become ineffective. Anthropic's workload has evolved to a point where some of Git's design choices have become increasingly expensive.
In this article, I want to explore what pushed Anthropic toward this decision, how Scion addresses those challenges, and what other engineering teams can learn from the trade-offs.
Because as AI changes how quickly we produce software, understanding the infrastructure supporting that process becomes just as important.
Understanding What Git Was Designed For
Before exploring Anthropic's decision, it's worth understanding what made Git so successful in the first place.
Git was designed as a distributed version-control system. A normal full clone doesn't just download the latest source files. It also brings the repository's history, allowing developers to create commits, inspect previous versions, manage branches, and merge changes locally without constantly communicating with a remote server.
One of Git's strengths lies in how efficiently it stores different versions of a project.
How Git Stores Source Code
Git uses a content-addressed object model built around three important objects:
Blob: Stores file contents without the filename.
Tree: Represents a directory, containing filenames and references to blobs or other trees.
Commit: References a root tree representing a project snapshot, along with parent commits and metadata.
Each object has an identifier derived from its contents and object type. This allows Git to reuse unchanged objects across multiple commits.
Consider a Java project containing README.md, Main.java, and PaymentService.java. We create an initial commit, then modify only PaymentService.java and commit again.
Here's what happens internally:
Git's object model: both commits can reference the same unchanged file blobs, while modified content receives a new object.
The second commit creates a new blob for PaymentService.java, along with the necessary updated tree objects and a new commit object. Meanwhile, README.md and Main.java continue referencing their existing blobs.
Each commit represents a complete logical snapshot, but Git doesn't physically duplicate every unchanged file.
We can inspect this behavior directly:
git cat-file -p HEAD
git ls-tree -r HEAD~1
git ls-tree -r HEAD
The first command displays the latest commit's internal information. The other two show file objects referenced by the previous and current commits. Comparing their blob IDs reveals which contents changed and which were reused.
How This Connects to Git's Distributed Architecture
When we run git add, Git stores staged file contents as objects and updates its index. Running git commit creates the necessary tree and commit objects, then updates the current branch reference. These operations happen locally.
When synchronizing with a remote, Git uses its transfer protocols to exchange the required objects, often through compressed packfiles that reduce storage and network overhead.
This architecture gives developers substantial independence while keeping repository storage efficient.
However, that independence comes with a trade-off. Clients maintain repository data locally and perform many source-control operations themselves. For most teams, this works remarkably well. But when millions of short-lived automated environments repeatedly interact with an enormous, rapidly changing monorepo, those costs can accumulate.
That's where Anthropic's workload becomes interesting.
Where Git Started Struggling at Anthropic
Anthropic's scaling challenge becomes clearer when we look at the numbers.
In May 2026, Anthropic was merging around 2,500 code changes per day. By September, that had grown to over 30,000. Its monorepo contained nearly 100 GB of main-branch history, with a compressed working copy exceeding 25 GB.
But the number that caught my attention was the approximately 25 million code checkouts happening every day for CI alone, excluding developer environments, agent sandboxes, and research workloads.
Consider what happens when an AI coding agent starts a task. It needs source files, possibly some repository history, and a workspace to make and validate changes. Preparing that environment can involve transferring Git objects, processing compressed packfiles, materializing files on disk, and maintaining local repository state.
Now multiply those operations across millions of automated checkouts.
Anthropic estimated that naively fetching its repository for every CI checkout would generate over 600 PB of Git traffic daily. This was a hypothetical estimate, not actual measured network usage, but it illustrates why efficient repository access had become so important.
Why Existing Git Optimizations Weren't Enough
What makes this case interesting is that Anthropic had already used the optimizations engineers would typically consider.
Git provides several mechanisms to reduce repository overhead:
Shallow clones reduce the amount of commit history downloaded.
Sparse checkouts limit which files are materialized locally.
Partial clones allow certain objects to be fetched only when needed.
Anthropic also maintained internal Git mirrors to distribute traffic and used prebuilt repository snapshots, called seeds, stored in blob storage. New environments could start from these snapshots instead of fetching everything from scratch.
These optimizations helped, but their operational costs grew alongside the workload. At its peak, Anthropic operated several hundred Git mirrors, putting increasing pressure on storage and compute resources. Meanwhile, seeds fell further behind the latest commits between refreshes, and history-dependent operations such as diff and blame required additional care with partial clones.
The underlying challenge was the accumulated cost of preparing and maintaining repository state across an enormous number of clients.
That led to an interesting architectural question. Most of Anthropic's clients were already-connected agents and automated jobs, many requiring only a few files, a diff, or some repository history.
If these clients are already online, do they really need to maintain extensive local repository state and perform so many source-control operations independently?
That question became central to Scion's design.
How Scion Approaches the Same Problem Differently
Anthropic's approach was to reconsider where source-control operations should happen. Since most of its agents and automated jobs are already connected to shared infrastructure, maintaining extensive repository data on every client offers fewer benefits.
Scion addresses this through three architectural decisions: moving computation to servers, recording changes incrementally, and distributing repository data through shared caches.
Moving Source Control Operations to the Server
In Git, operations such as commits, merges, and history queries can run locally. Scion shifts these responsibilities to shared servers, while clients manage working files and maintain a bounded local object cache.
Consider an AI agent modifying a Java service. It may need only a few directories from an enormous monorepo. Scion supports sparse and on-demand file retrieval, avoiding the need to maintain extensive repository history locally. Some automated workflows can even request diffs or perform writes through APIs without creating a working copy.
This reduces duplicated work across clients, but also makes them more dependent on shared services.
A Different Way to Represent Changes
Scion also changes how repository updates are recorded.
While a Git commit references a root tree representing a project snapshot, Scion uses changesets, entries in a write-ahead log (WAL) that record files changed relative to their first parent, with content-addressed references to compressed file contents stored separately.
Changesets are created server-side without requiring a shared branch reference to move. References such as main are updated separately, reducing the coordination required for ordinary changeset creation.
But there's a trade-off. A changeset records what changed; it cannot efficiently answer every history-related question on its own.
To handle operations such as file history, blame, and commit ancestry, Scion builds additional derived data structures. These can be computed ahead of time or generated when needed, allowing the underlying change records and query indexes to evolve independently.
Scaling Through Distributed Caching
The next challenge is making repository objects available to millions of workloads without repeatedly accessing authoritative storage.
Scion uses a layered caching architecture:
Scion's checkout architecture: node-local and regional caches serve requested objects, reducing repeated access to authoritative servers.
When a client requests an object, the node-local scion-daemon checks its cache. Missing objects are fetched through regional scion-cache servers, which contact authoritative Scion servers when necessary.
Because objects are content-addressed and immutable, cached copies can be safely reused. Scion also requests specific trees and blobs rather than relying on Git-style, client-specific packfile transfers. Combined with cache locality and shared object reuse, this reduces repeated network and processing work.
Anthropic reports cache hit rates above 99% at each tier, a 200× smaller CPU and memory infrastructure footprint compared with its previous Git mirroring system, and CI and research checkouts that became roughly two orders of magnitude faster. These are results measured in Anthropic's environment, rather than general benchmarks against Git.
What makes the architecture interesting is how these decisions complement each other. Lightweight clients reduce local overhead, incremental changesets make writes inexpensive, and shared caches reduce repeated data retrieval.
However, moving more responsibilities to shared infrastructure introduces another challenge: what happens when those services or caching layers fail?
The Trade-offs Behind Scion's Design
Scion reduces repeated work across clients, but moving source-control operations to shared servers also introduces new dependencies.
With a complete local Git repository, developers can continue committing changes, inspecting history, and managing branches even when the remote server is unavailable. Scion trades some of that independence for lightweight clients that depend on shared services. For Anthropic's largely online workloads, this may be worthwhile, but it makes service availability and failure isolation particularly important.
What Happens When a Regional Cache Fails?
Consider thousands of CI workers retrieving source files through a regional cache. Under normal conditions, most requests are served from cached objects, with relatively few reaching the authoritative servers.
Now suppose that regional cache becomes unavailable. If every worker bypasses it and directly contacts the authoritative servers, the sudden traffic increase could overwhelm them, turning a regional failure into a system-wide outage.
Anthropic addresses this risk by strictly enforcing its caching hierarchy to isolate failure domains. However, the article doesn't describe its complete recovery strategy.
If I were designing a similar system, I'd consider admission control to limit upstream traffic, request coalescing to avoid duplicate fetches for the same object, and bounded retries with exponential backoff and jitter. Healthy local caches could continue serving available objects, while excess requests would be delayed or rejected to protect the authoritative service.
These are additional design considerations, rather than confirmed details of Scion's implementation.
There's also the broader operational cost. Anthropic had to build supporting systems for code review, search, and CI, introducing responsibilities around migration, maintenance, integrations, and long-term ownership.
This raises an architectural question worth considering: when do the performance and scalability benefits justify the complexity of operating an entirely different source-control system?
The answer becomes clearer when we compare organizations with different workloads.
Should Every Company Rethink Git? Three Engineering Scenarios
After studying Anthropic's decision, I started wondering how the same architectural choices would apply to companies operating at different scales. Would a startup benefit from Scion? What about a growing product company with increasingly expensive CI pipelines?
Let's consider three hypothetical scenarios.
Company A: A Small Engineering Team
A startup with 20 developers working across a few backend services probably has little reason to replace Git. Its local workflows, established tooling, and manageable repository sizes make Git a practical choice. Building a custom source-control system would introduce complexity without solving a significant bottleneck.
Company B: A Growing Product Company
Consider a company with 500 engineers, a large monorepo, and thousands of daily CI executions. Checkout latency and infrastructure costs are increasing.
I'd first investigate shallow and partial clones, sparse checkouts, shared caches, and improvements to CI workflows. Measuring network usage, CPU consumption, and checkout latency would help determine whether these optimizations are sufficient before considering a larger architectural change.
Company C: An Agent-Heavy Engineering Organization
Now consider an organization running thousands of coding agents and millions of repository operations against an enormous monorepo.
If most workloads are short-lived, continuously connected, and require only portions of the repository, maintaining extensive local state becomes increasingly expensive. A Scion-like architecture with shared source-control APIs and distributed object caches may be worth evaluating.
Even at this scale, the decision depends on whether the performance gains justify migration costs and long-term operational complexity.
Choosing Based on the Workload
An approach to source-control decisions: measure the bottleneck, optimize the existing system, and consider architectural changes when those improvements are no longer sufficient.
This is how I'd approach the decision: understand the workload, identify measurable bottlenecks, and choose the least disruptive architecture that meets the requirements. Anthropic's solution makes sense for its particular constraints, but similar benefits may be achievable with much smaller changes elsewhere.
What I Took Away From Anthropic's Decision
What stayed with me after reading about Scion was the reasoning behind Anthropic's decision. They explored the optimizations available within Git, identified where those approaches were becoming difficult to scale, and eventually reconsidered the architecture itself.
This also made me think about what happens after source control becomes faster. AI agents can generate thousands of changes, but those changes still need to be tested, reviewed, integrated, and deployed safely. Improving one part of the development pipeline can expose bottlenecks elsewhere. At some point, CI capacity, merge coordination, or validation may become the next constraint.
I'm also curious to see how Git and its ecosystem evolve. Google and Meta have already explored source-control architectures for enormous repositories, and Anthropic's experience highlights another demanding workload. As AI-assisted development becomes more common, I wouldn't be surprised to see improvements in server-side APIs, object retrieval, and caching within the broader Git ecosystem. Perhaps many teams will benefit from these ideas without ever needing to replace Git.
For me, the biggest lesson is to question the assumptions behind our infrastructure as workloads evolve. Sometimes the right decision is a targeted optimization. Other times, the scale of the problem justifies rethinking the architecture entirely.
And that's the question I'll carry into future system design decisions: if our workload grows tenfold, which parts of our architecture will struggle first, and are we prepared for that?
For a deeper look at Scion's implementation and Anthropic's reported performance improvements, I'd recommend their original engineering article: Redesigning Source Control at Anthropic.
📖 Blog by Naresh B. A.
👨💻 Backend & AI Systems Engineer | Distributed Systems · Production ML
🌐 Portfolio: [Naresh B A]
📫 Let's connect on [LinkedIn] | GitHub: [Naresh B A]
Thanks for reading. This is my personal engineering perspective, and I'd genuinely be interested in hearing where you agree or disagree. ❤️
Top comments (1)
Really like that you framed this as the workload changing rather than Git suddenly being bad. The detail that stuck with me is the seeds drifting behind main between refreshes, since that's the same pain most teams already feel at a much smaller scale with stale CI caches. I'm curious whether you think moving commits and merges to the server holds up for agents that need to work offline or through a flaky connection, or whether that's just a trade-off you accept at that size.