How does a database know where its data is ?

Ongkar Dasgupta — Sun, 09 Aug 2026 13:54:52 +0000

Recently, a quite an interesting research paper titled Virtual-Memory Assisted Buffer Management in Tiered Memory, authored by Yeasir Rayhan and Walid G. Aref came out in the database management ecosystem. Let us discuss about it in this article !

Instead of quickly summarizing what the authors built and listing their results, it would be great to understand the problem itself.

Although we won't go into every implementation detail of the paper, we will try to go deep enough to see why each idea becomes necessary. So, please bear with me in this journey.

We will start from a very ordinary database query, encounter one problem, solve it, see what new problem that solution creates, and keep following the trail. Interestingly, that trail will take us from databases to RAM, virtual memory, page tables, the TLB, and eventually even a modification of Linux's page migration mechanism itself.

Let's begin!

Every time we search for a record, update a value, or run a query on a large database, the processor eventually has to access the corresponding data from somewhere in the machine. Right?

Quite a substantial portion of this database may be stored persistently on a solid‑state drive (SSD), where it can be stored safely for the long term. However, frequent reading from the SSD would be far too slow. So the Database Management System (DBMS), the software that manages this database, keeps the most frequently accessed data in the computer's main memory (RAM), where the processor can access it much faster.

This already creates a very interesting problem.

A full fledged database contains far more data than what can fit in RAM all at once. The DBMS therefore has to keep track of what is currently in RAM, what is still on the SSD, and where the required data can be found when a database query asks for it.

Modern servers make this problem even more interesting because there could be additional kinds of memory between these two extremes of RAM and SSD.

And this is exactly what we are going to explore -
How does a DBMS keep track of data as it moves across these different kinds of memory?

What are these additional kinds of memory?
In modern servers, memory can be distributed across different hardware locations. There would be two such cases that are particularly relevant in this discussion.
Firstly, servers can contain multiple processor sockets, each holding a CPU package with directly attached main memory, typically DRAM (Dynamic Random-Access Memory). From the perspective of a processor running on one socket, its directly attached DRAM is its local RAM. It can also access DRAM attached to another socket, although that access is generally slower.

Secondly, we have another kind of memory access method called Compute Express Link (CXL). CXL is a high-speed interconnect that allows a processor to access additional memory attached through a CXL device, rather than limiting it to the RAM directly attached to the processor itself. One thing to note however is : From the processor's point of view, this additional memory can still be accessed using ordinary memory reads and writes but reaching it generally takes longer than reaching local RAM.

Memory resources of these two kinds including DRAM attached to another processor socket and CXL-attached memory are broadly referred to in the paper as remote memory (RMem).

So at this moment of study, our DBMS has quite a lot of choices. Instead of keeping data either in local RAM or on the much slower SSD, it can use RMem as an intermediate option.

How would you connect all of them ?

Simple. Data that is needed very frequently can stay in local RAM. When local RAM starts filling up, some data can be moved to RMem instead of being pushed all the way back to the SSD. And if that data becomes frequently needed again, it can be moved back into local RAM.
We have essentially gone from a two-level arrangement,' RAM - SSD ', to a three-level management ' RAM - RMem - SSD '.

But what exactly are we moving between these places?

A DBMS does not normally move an individual database record every time it needs one (quite obvious!). Instead, it organizes the database into fixed-size chunks called database pages. A page can contain multiple records, and each page is identified by a Page Identifier (PID).

So when a query eventually needs some data, the DBMS first needs the corresponding page. And because that page could now be in local RAM, somewhere in RMem, or only on the SSD, we finally arrive at a more technical version of the problem we started with in our title:
given a PID, how does the DBMS efficiently find where its page currently is?

One straightforward way is to maintain a Hash Table inside the DBMS. We can think of it as a lookup structure where the DBMS stores an entry connecting each page currently kept in memory to the address through which that page can be accessed. In simplified form, it looks something like:

PID 42 → memory address X

Now suppose a query needs Page 42. The DBMS cannot simply access Page 42 directly. It first takes PID 42, searches the Hash Table, obtains 'X', and only then accesses the page through that address. So before reaching the actual database page, we have introduced an additional lookup:

PID → Hash Table → memory address → actual page

The issue here is that page accesses happen continuously while a database is serving queries. At large scale, repeatedly searching this Hash Table introduces additional costs for the processor.
This raises our next question in this journey: if the purpose of the Hash Table is ultimately to translate a page identifier into a memory address, does the DBMS really need to perform that translation itself?

Well, it goes like our computer already has another mechanism whose entire job involves translating addresses: virtual memory !

When a program uses a memory address, that address is generally not the actual physical location of the data inside a RAM chip. Instead, the program works with a virtual address.

The processor translates this virtual address into a physical address using mappings maintained by the operating system in a structure called the Page Table. In a simplified form, we have something like:

virtual address X → Page Table → physical location A

Why introduce this extra layer at all? Because now the address visible to a program and the actual physical location of its data no longer have to be the same thing.

Suppose some data currently lives at physical location A and the program accesses it through virtual address X. If the operating system later moves that data to physical location B, it can simply update the mapping:

X → A
becomes
X → B

The program can still continue accessing X exactly as before. It does not need to know that the physical location underneath X has changed. And this gives us a very interesting possibility for our next database problem: What if a database page could similarly keep one fixed virtual address even while its actual physical location changes?

This is exactly the idea behind an earlier system called vmcache.

Instead of maintaining a Hash Table that repeatedly tells the DBMS where each database page is located, vmcache reserves a large range of virtual addresses and gives every database page its own fixed virtual address.

Suppose Page 42 is assigned virtual address X. That relationship does not change during the lifetime of the page:

PID 42 → virtual address X

If Page 42 is currently present in RAM, the OS Page Table takes care of the remaining translation:

PID 42 → virtual address X → Page Table → physical location A

The Hash Table lookup issue is solved ! The DBMS knows the fixed virtual address associated with the page, while the operating system's existing Page Table tells the processor where the page actually resides in physical memory ( there are two different scopes now !).

But what happens if Page 42 is not in RAM at all and currently exists only on the SSD? There would be no valid physical RAM location for X to point to. When the program tries to access X, the processor detects that the required page is not currently present in memory and triggers what is called a page fault. This transfers control to the operating system, allowing vmcache to bring the required database page from the SSD into RAM. Once loaded, the OS installs the corresponding virtual-to-physical mapping, and access can continue.

So vmcache already gives us an elegant solution when there are two levels RAM, followed by, SSD.
But Modern servers may also have _RMem sitting between them._

Now Page 42 does not simply have two possibilities, “in RAM” or “on SSD.” It may move like this:

local RAM → RMem → local RAM → RMem → SSD

And this leads directly to the question investigated in the paper we are discussing: Can the fixed-virtual-address idea of vmcache still work when the physical page is continuously moving across multiple memory tiers?

The authors extend vmcache for exactly this setting and call the resulting system vmcacheⁿ, where the n represents the possibility of having multiple memory tiers.

They preserve the same fundamental idea:

The virtual address of a database page remains fixed throughout its lifetime, while the physical memory backing that address is allowed to change.

Sounds like our problem is solved then?

Not quite. We have decided that the virtual address will remain fixed, but the actual page may still have to move between local RAM and RMem. If Page 42 is currently in local RAM and the DBMS decides that it should now reside in RMem, something still has to physically move that page from one memory tier to the other and change the mapping underneath its fixed virtual address.

To move Page 42 from local RAM to RMem, vmcacheⁿ needs to move the actual contents of that page to a physical location in RMem and then update the Page Table so that X points to this new location. The virtual address remains unchanged throughout this process. This movement of a page from one physical memory location to another is called page migration.

Linux already provides a system call called move_pages for doing exactly this. vmcacheⁿ can give it a collection of pages along with the memory location where each should be moved. Linux performs the migrations and updates their Page Table mappings, while their virtual addresses remain unchanged. Even better, move_pages can migrate multiple pages in one call rather than requiring a separate request for every page.

Our next problem got itself introduced here!

Remember that the processor needs the Page Table to translate virtual addresses such as X into physical locations. Looking up the Page Table for every memory access would itself be slow, so processors keep recently used translations in a small and extremely fast cache called the Translation Lookaside Buffer (TLB).

Suppose the processor has cached:

X → physical location A

vmcacheⁿ now migrates our page to RMem and the Page Table becomes:

X → physical location B

The Page Table is correct, but the processor may still have the old X → A translation sitting in its TLB. That cached entry must therefore be invalidated before the new mapping can safely be used. And if several processor cores have cached the same translation, the operating system may need to make all of them discard it. This operation is known as a TLB shootdown, and it is expensive because it can require interrupts between processor cores.

Now imagine vmcacheⁿ continuously moving large numbers of database pages between local RAM and RMem. These invalidations start adding up. Linux's move_pages already groups page migrations into batches, but by default it processes at most 512 pages in a migration batch. The authors realized that if more pages could be migrated together, the cost of TLB invalidation could be spread across a larger number of page movements.

This observation led them to modify the Linux page-migration interface itself. They introduced a new system call called move_pages2, which lets vmcacheⁿ control how many pages are grouped into a migration batch. It also handles failures more flexibly. Instead of certain errors on one page which makes the remaining pages in the request to be skipped, move_pages2 can record that failure and continue attempting the other eligible migrations.

And surprisingly, implementing move_pages2 required only about 150 lines of changes across four Linux kernel functions.

Our question now is: does all of this actually make the database faster?

To find out, the researchers used a server with two processor sockets. RAM attached to the processor running the workload acted as local RAM, RAM attached to the other socket acted as RMem, and a 960 GB NVMe SSD formed the final storage tier. So their experimental machine essentially gave them the same hierarchy we have been discussing:

Local RAM → RMem → SSD

The amount of local RAM was restricted to only 32 GB, forcing the system to decide what stays locally, what moves to RMem, and what eventually has to be obtained from the SSD.

The researchers then used two workloads. The first was TPC-C, a standard benchmark that simulates transactions performed by a real database system, such as placing orders and processing payments. The second was a random-read workload, where the system repeatedly requested data scattered across a large dataset. TPC-C shows whether vmcacheⁿ improves the system as a whole, while random reads make page migration between local RAM and RMem much more prominent, allowing move_pages2 to be examined more directly.

Let's focus on TPC-C first.

With TPC-C, vmcacheⁿ achieved 1.67× higher throughput than vmcache with 2× RMem, increasing to 3.82× with 4× RMem. More pages could remain in RMem instead of requiring slower SSD accesses.

But this improvement came from vmcacheⁿ as a whole, not specifically from move_pages2. For TPC-C, move_pages2 performed similarly to the existing move_pages because SSD accesses still accounted for a substantial part of the workload.

The difference became clearer with random reads, where movement between local RAM and RMem mattered much more. Here, move_pages2 achieved 1.42× higher query throughput and 1.32× higher page-migration throughput than move_pages. Its larger migration batches allowed the cost of TLB invalidation to be spread across more page movements.

However, the experiments revealed one important limitation:

Adding RMem does not automatically make the database faster.

Moving pages between local RAM and RMem has a cost of its own. When RMem was small, this migration overhead could outweigh the benefit of avoiding SSD accesses. As RMem capacity increased, keeping more pages in memory became increasingly worth the cost of moving them.

And that finally gives us an answer to the question we started with:

How does a DBMS keep track of data as it moves across different kinds of memory?

In vmcacheⁿ, the DBMS does not need to explicitly translate each PID into its changing physical address. Each database page is given a fixed virtual address, while the operating system's Page Table keeps track of which physical memory currently backs that address. The page may move from local RAM to RMem and later back again, but the address used by the DBMS remains unchanged.

The authors then solve the second problem created by this approach: moving those physical pages efficiently. Their move_pages2 system call gives vmcacheⁿ greater control over how migrations are batched, helping minimise costs such as TLB invalidation when page migration becomes a major part of the workload.

The interesting idea behind this paper is that the researchers did not eliminate the cost of managing data across different memory speeds. They moved part of the tracking problem to a mechanism the computer already has, and then identified page migration as the new bottleneck created by that decision.

Paper Reference: Rayhan, Yeasir, and Walid G. Aref. "Virtual-Memory Assisted Buffer Management In Tiered Memory." In Proceedings of the 22nd International Workshop on Data Management on New Hardware, pp. 1-5. 2026.

Paper Link: https://arxiv.org/abs/2603.03271

Trying to solve a frustrating problem: why is debugging AI systems still so unclear?

Ongkar Dasgupta — Wed, 18 Mar 2026 11:50:59 +0000

I’ve been thinking about something that keeps coming up when working with AI systems. When a model gives a wrong or weird output, it’s surprisingly hard to figure out what actually went wrong. Most of the time we’re digging through logs
or guessing where things broke.
As part of the AWS AI Ideas hackathon, I started building a concept called AutopsyAI. The idea is to look at the full pipeline from input to output and try to explain where things might have failed, instead of just showing the final result. It’s still early and not a full product yet, more like an attempt to explore whether this problem is worth solving in a structured way.
Curious how others here deal with this. If you’ve worked with AI systems in production, how do you debug failures today? Does this feel like a real problem, or is it already solved better than I think?

Please do checkout my article and support it!
https://builder.aws.com/content/3AeXXMtLdDuPwRL4xcEjJEBoQkB/aideas-autopsyai-the-missing-debugger-for-ai-systems

DEV Community: Ongkar Dasgupta

How does a database know where its data is ?

Trying to solve a frustrating problem: why is debugging AI systems still so unclear?