DEV Community

Cover image for The cachestat() Syscall: Reading Linux Page-Cache Stats from User Space
Raghu Bharadwaj
Raghu Bharadwaj

Posted on • Originally published at techveda.live

The cachestat() Syscall: Reading Linux Page-Cache Stats from User Space

#linux #kernel #c #embedded

cachestat() is a Linux system call added in kernel 6.5 that reports how many pages of a file are currently in the page cache, together with dirty, writeback, and eviction counts, for a byte range you choose. It works on a file descriptor, does not require you to map the file, and does not pull any pages into memory. Because no C library ships a wrapper for it yet, you invoke it through syscall(2) using system call number 451.

When an embedded system feels slow after a large read, or when a service takes a long time on its first request, the page cache is often the reason. Data that was read once sits in cache and is served from memory; data that was evicted has to come back from storage. Until recently there was no direct way to ask the kernel how much of a specific file is cached. The cachestat syscall fills that gap. This tutorial explains what it reports, walks through its interface, and gives a small C program you can compile and run to watch pages enter and leave the cache.

What the cachestat syscall does

The cachestat syscall queries the state of the page cache for one open file. For a byte range that you specify, it returns five counts: how many pages are cached, how many are dirty (modified but not yet written back), how many are currently under writeback, how many were evicted, and how many were recently evicted. The counts are in units of pages, so on a system with a 4 KiB page size a fully cached 4 MiB file reports 1024 cached pages.

Two properties make it useful. First, it only reads accounting state, so calling it does not fault the file's data into the cache the way a normal read() would. You can measure without disturbing what you are measuring. Second, it takes a file descriptor and a byte range rather than a memory mapping, so you do not have to mmap() the file to inspect it.

The interface

The call and its two structures are declared as follows:

int cachestat(unsigned int fd, struct cachestat_range *cstat_range,
              struct cachestat *cstat, unsigned int flags);

struct cachestat_range {
    __u64 off;
    __u64 len;
};

struct cachestat {
    __u64 nr_cache;
    __u64 nr_dirty;
    __u64 nr_writeback;
    __u64 nr_evicted;
    __u64 nr_recently_evicted;
};
Enter fullscreen mode Exit fullscreen mode

The fd argument is an open descriptor for a regular file. The cstat_range argument describes the byte range: if len is greater than 0, the range is [off, off+len]; if len is 0, the range runs from off to the end of the file. The flags argument is reserved for future use and must be set to 0; any other value returns EINVAL. The result is written into the cstat structure. On success the call returns 0; on error it returns -1 and sets errno.

Two of the fields need a plain explanation. nr_evicted counts pages in the range that were in the cache earlier and have since been removed. nr_recently_evicted counts pages whose return to the cache would signal that the file is being actively used while the system is under memory pressure. That second field is what tells you whether a file is being repeatedly read and dropped, which is a common cause of slow, storage-bound behaviour.

What you need

  • A Linux kernel of version 6.5 or later. Check with uname -r.
  • A C compiler and the standard headers.
  • No special library. Because glibc and musl do not yet provide a wrapper for cachestat(), the program calls it through syscall() and defines the two structures itself.

A program that calls cachestat

Save the following as cachestat_demo.c. It opens the file named on the command line, queries the whole file, and prints the five counts. It defines __NR_cachestat as 451 if the running toolchain's headers do not already provide it.

#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <linux/types.h>

#ifndef __NR_cachestat
#define __NR_cachestat 451
#endif

struct cachestat_range {
    __u64 off;
    __u64 len;
};

struct cachestat {
    __u64 nr_cache;
    __u64 nr_dirty;
    __u64 nr_writeback;
    __u64 nr_evicted;
    __u64 nr_recently_evicted;
};

static int do_cachestat(unsigned int fd, struct cachestat_range *range,
                        struct cachestat *cs, unsigned int flags)
{
    return syscall(__NR_cachestat, fd, range, cs, flags);
}

int main(int argc, char **argv)
{
    if (argc != 2) {
        fprintf(stderr, "usage: %s <file>\n", argv[0]);
        return 1;
    }

    int fd = open(argv[1], O_RDONLY);
    if (fd < 0) {
        perror("open");
        return 1;
    }

    struct cachestat_range range = { .off = 0, .len = 0 }; /* whole file */
    struct cachestat cs;

    if (do_cachestat(fd, &range, &cs, 0) < 0) {
        perror("cachestat");
        close(fd);
        return 1;
    }

    printf("cached pages     : %llu\n", (unsigned long long)cs.nr_cache);
    printf("dirty pages      : %llu\n", (unsigned long long)cs.nr_dirty);
    printf("writeback pages  : %llu\n", (unsigned long long)cs.nr_writeback);
    printf("evicted pages    : %llu\n", (unsigned long long)cs.nr_evicted);
    printf("recently evicted : %llu\n", (unsigned long long)cs.nr_recently_evicted);

    close(fd);
    return 0;
}
Enter fullscreen mode Exit fullscreen mode

Compile it with a normal build command:

raghu@techveda.org:~$ gcc -O2 -Wall -o cachestat_demo cachestat_demo.c
Enter fullscreen mode Exit fullscreen mode

Watching pages enter and leave the cache

Create a test file of a known size. A 4 MiB file is 1024 pages at the usual 4 KiB page size, which makes the numbers easy to read. Place it on a normal disk-backed filesystem such as ext4. Do not use a file under /tmp if that path is a tmpfs mount, because tmpfs pages are held in memory, are always counted as dirty, and are not released by the step below.

raghu@techveda.org:~$ dd if=/dev/zero of=~/testfile bs=1M count=4
4+0 records in
4+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.00393 s, 1.1 GB/s
Enter fullscreen mode Exit fullscreen mode

Drop the clean page cache so the file starts uncached. This affects the whole system and needs root, so use it only on a development machine.

raghu@techveda.org:~$ sync
raghu@techveda.org:~$ echo 1 | sudo tee /proc/sys/vm/drop_caches
1
Enter fullscreen mode Exit fullscreen mode

Now query the file. Because it was just dropped, no pages are cached:

raghu@techveda.org:~$ ./cachestat_demo ~/testfile
cached pages     : 0
dirty pages      : 0
writeback pages  : 0
evicted pages    : 0
recently evicted : 0
Enter fullscreen mode Exit fullscreen mode

Read the file once so its data is brought into the cache, then query again. The cached count now reflects the full file, 1024 pages:

raghu@techveda.org:~$ cat ~/testfile > /dev/null
raghu@techveda.org:~$ ./cachestat_demo ~/testfile
cached pages     : 1024
dirty pages      : 0
writeback pages  : 0
evicted pages    : 0
recently evicted : 0
Enter fullscreen mode Exit fullscreen mode

The exact counts on your machine can vary slightly with readahead and with other memory activity, but the pattern is reliable: zero after a drop, close to the full page count after a read. If you write to the file and query before the kernel flushes it, you will see a non-zero dirty pages value, which returns to zero after a sync. If you want to evict a single file without touching the rest of the system, call posix_fadvise() on it with POSIX_FADV_DONTNEED instead of writing to drop_caches.

How this differs from mincore

Before cachestat(), the closest tool was mincore(2). The two answer different questions. mincore() requires you to map the file with mmap() first, then fills a byte array with one entry per page telling you whether that page is resident. It reports residency of the mapping, one page at a time, and nothing else.

The cachestat syscall takes a file descriptor and a byte range directly, needs no mapping, and returns aggregate counts that include dirty, writeback, and eviction information that mincore() does not expose. For a large file, collecting residency through mincore() means allocating and scanning a large vector, while cachestat() returns a small fixed structure. When you want a summary of a file's cache state rather than a per-page map, cachestat() is the more direct and more scalable option.

Where it helps and where it does not

It is useful when you want to confirm that a working set is actually staying in cache, to measure whether a warm-up read had the intended effect, or to decide whether a file needs to be pre-read before a latency-sensitive operation. Understanding how the page cache behaves is a core part of reasoning about I/O on memory-constrained embedded systems, a theme we return to in our Linux systems engineering training.

The limits are worth stating directly. The call needs a kernel of 6.5 or later, so older long-term-support kernels will return ENOSYS. There is no library wrapper yet, so syscall() is the portable way to reach it. Files on hugetlbfs are not supported and return EOPNOTSUPP. Finally, the page cache can change between the moment the kernel reads the counts and the moment your program sees them, so treat the numbers as an accurate recent snapshot rather than a guarantee about the next instant.

Key takeaways

  • cachestat() reports cached, dirty, writeback, evicted, and recently evicted page counts for a byte range of a file.
  • It takes a file descriptor, needs no memory mapping, and does not fault the file's data into the cache.
  • It was added in Linux 6.5 and has no glibc or musl wrapper, so you call it through syscall(__NR_cachestat, ...) with number 451.
  • Set flags to 0, or the call returns EINVAL; hugetlbfs files return EOPNOTSUPP.
  • Compared with mincore(), it returns a compact summary instead of a per-page residency map and exposes dirty and eviction state as well.

Frequently asked questions

Which kernel version added the cachestat syscall?
It was added in Linux 6.5. On an older kernel the call returns -1 with errno set to ENOSYS.

Do I need a special library to call cachestat()?
No. glibc and musl do not yet ship a wrapper, so you invoke it through syscall() using __NR_cachestat, which is 451, and you define the two structures in your own source.

Does calling cachestat() load the file into the page cache?
No. It only reports the current accounting state for the range you asked about. Unlike a read(), it does not fault the file's data into memory.

How is cachestat() different from mincore()?
mincore() needs the file mapped with mmap() and returns per-page residency for that mapping. cachestat() takes a file descriptor and a byte range and returns aggregate counts that also include dirty, writeback, and eviction figures.

Further reading

This article was first published on techveda.live.

Top comments (0)