There's a DRAM shortage, memory is expensive, and hyperscalers own mountains of it. So when Meta retires a server, throwing away its perfectly good memory sticks is real money on fire. The problem: old sticks don't slot cleanly into new machines. Meta's fix is a small bridge chip plus a clever reuse of ideas the operating system has had for years. Three ideas do the heavy lifting — NUMA, hot/cold tiering, and CXL — and they stack together nicely.
NUMA: not all RAM is equally close
A modern server usually has more than one CPU chip on the motherboard. Each chip has its own bank of RAM wired directly to it — its local memory. The chips are also linked to each other, so CPU 0 can reach CPU 1's RAM if it needs to. It just has to hop across the link between them first, which costs time.
So access speed depends on where the memory physically sits:
- CPU 0 reaching its own local RAM: fast (~80ns)
- CPU 0 reaching the other chip's RAM: slower (~140ns), because the request crosses the inter-chip link
(Those numbers are ballpark and vary by hardware.) That's the "non-uniform" in NUMA — Non-Uniform Memory Access. Same request, different latency depending on distance.
Linux has long known about this and tries to help. A mechanism called AutoNUMA watches which CPU actually uses a given chunk of memory, and if a process's data ended up on the far chip, it quietly migrates that data over to the local one to cut the latency. The key pattern to hold onto: watch what gets accessed, then move it somewhere better.
Hot and cold: sort data by how often you touch it
Most workloads don't touch all their memory evenly. A small slice gets hammered constantly; the rest mostly sits idle. Call the busy stuff hot and the idle stuff cold.
Once you look at memory this way, an obvious strategy appears: keep hot data in your fast-but-small memory, and shove cold data into a slow-but-large tier. Because the slow tier only ever holds things nobody's really asking for, you rarely pay its latency penalty. This is the same idea behind CPU caches, the OS page cache, and RAM-versus-SSD — a fast small tier backed by a slow big one, with a placement policy keeping the right things in front.
CXL: a slower doorway for a lot more memory
CXL — Compute Express Link — lets memory talk to the CPU over PCIe, the same general-purpose high-speed lanes your graphics card uses, instead of the dedicated memory channels wired right next to the CPU. That extra hop makes CXL memory slower than native RAM, but far faster than an SSD or the network. And critically, it means memory no longer has to be a tightly-matched neighbor of the CPU — which is exactly why an old stick can join a new server through a bridge chip.
On its own, slower memory sounds like a downgrade. Here's where the three ideas click together.
The payoff: reuse the NUMA plumbing for tiering
CXL memory shows up to Linux as a NUMA node — just a "far" region, like the other chip's RAM. So Meta didn't need a new operating system. They took the AutoNUMA machinery that already does watch access, then migrate and pointed it at a new decision: not "which chip is closer," but "is this page hot or cold?" Hot pages stay in fast local RAM; cold pages get demoted to the slow CXL pool. Meta's version of this is called TPP — Transparent Page Placement — and it was merged into mainline Linux, so it's not a private fork.
The result: old DIMMs that would have been scrapped become a big, cheap, slower memory tier — used a bit like fast swap space — and the OS keeps its latency off the critical path by making sure only cold data lives there. Meta reports up to a 25% reduction in server count for some workloads.
A caveat worth keeping in mind: detecting "hot" isn't free. It leans on things like minor page faults and access-bit scanning, which themselves burn CPU cycles — so part of the real engineering is making the detection cheap enough that tiering doesn't eat its own savings. And the 25% figure comes from Meta's own paper, not independent benchmarks.
Top comments (0)