DEV Community: Ayush Bajpai

"MMAP" System Call for DBMS

Ayush Bajpai — Sun, 13 Apr 2025 07:49:10 +0000

MMAP, which stands for Memory Mapping, allows to map files into a process's virtual memory space. It makes it faster to load certain things, as the common data can be kept in primary memory (like RAM), which is significantly faster than, say HDD or SSD.

It is typically achieved using the mmap() system call provided by most modern operating systems.

How MMAP Works?

When a file is memory mapped, the operating system gets a say as to what it wants to load in the memory, and what it wants to keep in the disk.

The Operating System does not load the entire file into memory immediately, and instead it's associated with a range of virtual memory addresses.

As the program reads or writes to a file which is memory mapped:

If the page is already loaded in memory - Success ✅
If not, then a Page Fault occurs, and the control is transferred to the OS page fault handler as:
1. The OS determines which file and offset correspond to the faulting address.
2. It allocates a physical memory page (from RAM)
3. It reads the corresponding file block from disk into RAM
4. Updates the page table to map the virtual page to the newly loaded physical page.
5. Marks it read-only or writable based on the PROT_* flags.

It should, however be noted that memory might have Dirty Pages, i.e. pages that already have some changes in them in primary memory, which are not yet updated to the disk. If the OS needs to overwrite any such page, it also would require a flush policy to handle how the data will be flushed back to the disk, before overwriting.

C Example for mmap

Here's a C Example with detailed comments, that demonstrates:

Opening a file
Mapping it into memory
Reading from the mapped memory
Writing to it
Flushing changes to disk using msync()
Unmapping and closing the file

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>      // For open()
#include <sys/mman.h>   // For mmap()
#include <sys/stat.h>   // For fstat()
#include <unistd.h>     // For close(), lseek(), write()
#include <string.h>     // For memcpy()

#define FILENAME "example_mmap.txt"
#define FILESIZE 4096  // 4KB

int main() {
    int fd;
    char *mapped;
    struct stat st;

    // Step 1: Open (or create) the file
    fd = open(FILENAME, O_RDWR | O_CREAT, 0666);
    if (fd == -1) {
        perror("open");
        return EXIT_FAILURE;
    }

    // Step 2: Ensure file is at least FILESIZE long
    if (ftruncate(fd, FILESIZE) == -1) {
        perror("ftruncate");
        close(fd);
        return EXIT_FAILURE;
    }

    // Step 3: Memory-map the file
    mapped = mmap(NULL, FILESIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    if (mapped == MAP_FAILED) {
        perror("mmap");
        close(fd);
        return EXIT_FAILURE;
    }

    printf("File successfully memory-mapped.\n");

    // Step 4: Read the first few bytes (may be zero if file is empty)
    printf("Initial contents: \"%.32s\"\n", mapped);

    // Step 5: Write something to the mapped region
    const char *msg = "Hello via mmap!\n";
    memcpy(mapped, msg, strlen(msg));  // Equivalent to writing into memory

    printf("Modified contents in memory: \"%.32s\"\n", mapped);

    // Step 6: Flush changes to disk
    if (msync(mapped, FILESIZE, MS_SYNC) == -1) {
        perror("msync");
    } else {
        printf("Changes successfully flushed to disk.\n");
    }

    // Step 7: Unmap memory and close the file
    if (munmap(mapped, FILESIZE) == -1) {
        perror("munmap");
    }

    close(fd);
    printf("File unmapped and closed.\n");

    return EXIT_SUCCESS;
}

When to Use MMAP in a DBMS

MMAP is particularly beneficial when:

Workload is read-heavy.
System memory is sufficient to cache working set.
Simplicity and low-overhead are priorities.
You trust the OS’s page replacement and sync mechanisms.

It might be suboptimal for write-heavy, large-scale distributed DBMS systems like PostgreSQL or MySQL, which prefer explicit control over I/O and caching.

Limitations and Caveats

Less Control:

MMAP delegates caching and eviction to the OS, which might not be optimal for DBMS-specific patterns.

I/O Patterns:

Random writes can generate frequent page faults and dirty pages, leading to thrashing.

Consistency and Sync:

You must use msync() or mprotect() carefully to ensure consistency and durability (especially for WAL-based systems).

Portability:

Some platforms or file systems (e.g., networked FS) behave unpredictably with MMAP.

Address Space Limits:

On 32-bit systems, you’re constrained by virtual address space.

Performance Considerations of Using mmap() in DBMS

While mmap() can simplify file I/O and offer performance benefits in some cases, it's important to understand the performance trade-offs it introduces when used in database systems:

✅ Potential Advantages:

Zero-Copy I/O: Since mmap() maps a file directly into the process's address space, the kernel can avoid explicit copies between user and kernel space, improving I/O efficiency.

On-Demand Paging: The operating system loads only the necessary pages into memory, which can save memory if the entire file isn't accessed.

Automatic Page Caching: OS-level page caching can result in fast access times for frequently read data, without the database developer needing to implement a custom caching layer.

⚠️ Potential Pitfalls

Page Fault Overhead: Every access to a not-yet-loaded page triggers a page fault, which involves a context switch and can be expensive if not handled efficiently.

Less Control Over I/O Scheduling: Unlike pread()/pwrite() or custom read-ahead strategies, mmap() offloads much of the paging and flushing behavior to the kernel. This can lead to unpredictable latencies, especially under memory pressure.

Writeback Latency: Dirty pages in memory must be flushed to disk. If the OS's flushing (e.g., via msync() or madvise()) is not carefully managed, write operations can block for long periods, introducing latency spikes.

Interaction with Virtual Memory Limits: On 32-bit systems or environments with limited address space, mapping large files can exhaust virtual memory quickly, causing issues even if physical RAM is available.

📌 Example Case

Imagine a DBMS using mmap() to load a 10 GB file. If the system has only 8 GB of RAM and the working set is 7 GB, things may run smoothly. But if access patterns become random or the working set exceeds available memory, the OS might start swapping or aggressively evicting pages, leading to thrashing and degraded performance.

Real-World Example: LMDB

LMDB uses MMAP exclusively.

All DB operations happen via direct memory access.

No buffer pool, no write-ahead logging — the filesystem and MMAP handle durability.

Pages are copy-on-write to preserve snapshot isolation.

Summary

MMAP in DBMS is a technique where database files are accessed via memory instead of manual file I/O. It simplifies the design by letting the OS manage disk-to-memory translation but comes with trade-offs in control, predictability, and scalability. It's ideal for embedded or read-optimized systems, less so for complex transactional engines needing fine-grained I/O control.

Data Models in DBMS

Ayush Bajpai — Sun, 13 Apr 2025 06:18:23 +0000

Database Management System allows us to store tons of data in an efficient manner. A Data Model defines how data is stored, connected and accessed. Essentially, it is the backbone of DBMS.

Here are some common Data Models Explained, and how they store data:

1. Hierarchical Data Model

A hierarchical database model organizes data in a tree-like structure. Data is stored in records, each made up of one or more fields, and each field holds a single value. The type of a record is defined by the combination of its fields. One special type of field is a link, which connects a record to other related records. These links create parent-child relationships between records, forming a tree structure where each record can link to multiple child records, but only one parent.

Ex:- XML, Windows Registry

2. Network Data Model

Here, in contrast to the Hierarchical Data Model, the network model allows each record to have multiple parents and multiple children, forming a more flexible graph-like structure.

This applies on two levels:

At the schema level, record types are connected through relationship types, forming a graph of how data can be related.

At the data level, actual records (called record occurrences) are connected by relationships, also forming a graph

Ex:- Integrated Data Store (IDS)

3. Relational Data Model

In the relational data model, data is organized into tables, also known as relations. Each table consists of rows (called tuples) and columns (called attributes). Every row represents a single record, and each column represents a field containing a specific type of data.

Relationships between data are established through keys:

A primary key uniquely identifies each row in a table.

A foreign key creates a link between rows in different tables by referencing the primary key of another table.

Unlike the hierarchical or network models, the relational model does not rely on explicit pointers or predefined paths between records. Instead, relationships are inferred through matching key values, providing flexibility and ease of querying using structured query language (SQL).

At both the schema level (structure of tables and keys) and the data level (actual rows and values), the relational model forms a logical structure rather than a physical graph or tree. This abstraction makes it easier to understand, maintain, and query complex data sets without dealing with how data is physically linked.

Ex:- MySQL, PostgreSQL

4. Object Oriented Data Model

In the object-oriented data model, data is represented as objects, similar to how it's handled in object-oriented programming languages. Each object contains both data (attributes) and behavior (methods), encapsulated into a single unit.

Objects are grouped into classes, which define the structure (attributes) and behavior (methods) shared by all objects of that type. The model supports key object-oriented features such as:

Inheritance – Classes can inherit attributes and methods from other classes, promoting code reuse and hierarchy.

Encapsulation – Data and behavior are bundled together, protecting internal states and exposing only what is necessary.

Polymorphism – Objects can be accessed through references of their parent class type, allowing flexibility in method usage.

At the schema level, the structure of the database is defined through class hierarchies and object relationships.

At the data level, actual object instances are stored, each maintaining its state and behavior.

Relationships between objects can be expressed through object references, allowing complex structures like trees, graphs, and networks. The object-oriented model is especially suited for applications with complex data and behavior, such as CAD systems, multimedia databases, and object-oriented programming integrations.

Ex:- db4o, ObjectDB

5. Document Data Model (used in NoSQL)

In the document data model, data is stored in the form of documents, typically using formats like JSON, BSON, or XML. Each document is a self-contained unit that holds all the data for a single object or entity, including nested structures and arrays.

Documents are grouped into collections (similar to tables in the relational model), but unlike rows in a table, documents in a collection do not need to follow a fixed schema—each can have a different structure. This model provides flexibility and is well-suited for handling semi-structured or hierarchical data.

Key features include:

Schema flexibility – Documents can evolve over time without requiring changes to a rigid schema.

Embedded data – Related data can be stored within the same document (e.g., an order and its line items), reducing the need for joins.

Rich queries – Document databases support powerful query languages to filter, project, and manipulate nested fields.

Ex:- MongoDB, CouchDB

6. Key-Value Data Model (NoSQL)

The key-value data model is one of the simplest types of NoSQL database models. In this model, data is stored as a collection of key-value pairs, where:

The key is a unique identifier.

The value is the associated data, which can be a string, number, binary blob, JSON, or any arbitrary data.

This model is often compared to a hash table or dictionary, where you retrieve data by providing its key.

Key-value databases are best suited for scenarios like:

Caching (e.g., Redis, Memcached)

Session management

User preferences and profile storage

Real-time data and large-scale applications

Ex:- Redis, DynamoDB

7. Column-Family Data Model (NoSQL)

The column-family data model is a NoSQL model that stores data in columns rather than rows, making it especially suitable for large-scale, distributed systems that require fast read/write access across wide datasets.

In this model, data is organized into structures called column families, which are collections of related rows. Each row is identified by a row key, and contains multiple columns, grouped logically into column families.

Unlike the relational model:

Columns are not fixed across all rows—each row can have a different set of columns.

Columns can be grouped together for efficient storage and retrieval.

8. Graph Data Model

The graph data model represents data as a network of nodes and edges, making it ideal for capturing complex, interconnected relationships.

Nodes represent entities (such as people, products, or locations).

Edges represent relationships between those entities (such as "friends with", "purchased", or "located in").

Both nodes and edges can have properties—key-value pairs that store relevant information. This structure allows relationships to be first-class citizens, meaning they can be directly queried, navigated, and analyzed.

Ex:- Neo4j, Amazon Neptune

Optimizing Windows...

Ayush Bajpai — Thu, 10 Apr 2025 08:11:18 +0000

Complete Windows Optimization Guide

Windows, by itself, isn't a particularly optimized operating system. With its multitude of features, many of which are designed to generate revenue, it's difficult to achieve optimal performance straight out of the box. In contrast to Linux, which prioritizes overall performance and efficiency, we'll need to make some modifications to Windows to fully harness the potential of our machines.

Oh and yeah..... if you can, consider switching to linux :)

Picking Up The Right ISO

Now, while not necessary this guide will be starting from reinstalling Windows 11. The first step is to pick the right ISO. Now some people will tell you that you should use a pre-modified ISO: which are just Windows 11 ISO's but modified by others to remove the bloatware and useless things.

I wouldn't recommend something like this, since it's really hard to tell whether an ISO is secure or not for the most. So, my recommendation: Get the Official Microsoft Windows 11 ISO. We'll just make the changes after installing Windows itself.

So, head over to: https://www.microsoft.com/en-us/software-download/windows11 to grab the latest Windows 11 ISO.

Windows Installation

For installing Windows 11, You will need:

Windows 11 ISO Image
Rufus (for creating a bootable pen drive): https://rufus.ie/en/
A Pen Drive with at least 8 GB Storage (16GB or more recommended)

Here's the step by step guide for Windows Installation:

First backup all your data in the computer, as even though it's possible to not delete any data while reinstalling Windows, it's better to clear everything and get a fresh start.
Open Rufus, and connect your pen drive to your system. You should be able to see the pen drive's name at the top of your Rufus Window. In the boot selection option, press select and choose the Windows 11 ISO downloaded from the official Microsoft Link. Press Start. When Prompted for Windows Customization, Choose the options:
- (Optional) Remove requirements for 4GB+ RAM, Secure Boot and TPM 2.0
- Remove Requirement for an online Microsoft Account
- Create a local account with username:
- Disable data collection (skip privacy option)
After the setup is finished, boot into boot options, and select the pen drive to boot from. To find the boot options for your laptop/desktop, you can google the model, or use: this guide to find the key.
During Windows Installation, a few things must be considered:
- Choose English World as time and currency. This will remove most of the bloatware.
- If your installation media is missing during disk selection, you might need to manually install Intel Rapid Storage Technology Drivers (Intel RST). Download the drivers from here, and extract them to the installation pen drive. When prompted, choose load manually and browse to the folder containing the driver. Choose any one from the list.
- Delete all the partitions related to Windows. If you do not want to save ANY data (such as D: E:) you can delete all the present partitions before hitting next.
Once Your System wants to reboot after installation, remove the installation pen drive.

Initial Windows Setup

During the Windows 11 Initial setup screen, if you chose your region as English World, it may give an error. You can safely skip this. Most of the things it will ask shouldn't be a bother. However if it does ask something, here's the tips:

If it asks for data collection options, turn them all off.
If it asks for Microsoft Account Login, either skip it, or if the option is not given, press Shift + f10 to launch CMD, then type the command oobe\bypassnro which will restart the computer. Another option if this gives an issue is to again, press Shift + f10 and then in CMD type ipconfig /release then go back from the login screen, and you should have the option to setup a local account.

Once Windows Launches, make sure to set your region back to get a working Microsoft Store:

Go to settings app
On the left side go to the Time and Language section
Go to the language and region Tab
Update your Country or Region to the correct one
Go back and then in the Date & Time Tab
Select the timezone where you live

Once this is done, you should be able to use Microsoft Store as normal.

Now you can:

Update Windows

Firstly update your Windows System. This will get the latest drivers and system update. Windows should automatically take care of this, however to make sure,

Open the settings app
Head Over to the Windows Update tab on the left
Press check for updates, or update all if present

Update Nvidia Driver

If you have an Nvidia Graphics Card you need to make sure it's updated. There are two ways to update the driver for Nvidia Graphics Card:

First way is to update using Nvidia Control Panel:

Right Click on the windows desktop and select Nvidia Control Panel
Navigate to the help menu and select updates

OR

Second Method is to manually download the driver
- First let's figure out the Graphics Card you have. If you already know it, skip this step. Press Windows Key + R to open run dialog, and type dxdiag and press ok. Once the window open, go the tabs on the top labelled display to check the one with Nvidia Graphics Card. Note down the name.

- Navigate to https://www.nvidia.com/en-us/drivers/
- In the search field, search the name of your graphics card and open the relevant one.
- Download the Latest Nvidia Game Ready Driver
- Open the installer and follow the on screen prompts. DO NOT INSTALL Geforce Experience App unless you have a really good reason to.

Use Chris Titus Tech Tool

The YouTuber Chris Titus Tech has created a really great and simple to use app which allows to make automatic tweaks to Windows. To use the tool:

Search Powershell in start menu, right click and run as administrator
Type irm christitus.com/win | iex in the window and hit enter
Once the window launches, head over to the Tweaks tab up top, and from the Recommended Selections, select the standard option
Press Run Tweaks at the bottom
DO NOT USE THE ULTIMATE PERFORMANCE POWER PLAN. Unless you are running a really high end PC with AIO Coolers, it's probably gonna cause more issues than fix them. I'll talk in the next section about what's the best power options.
Once Finished, head over to the Updates tab at the top, and select the Security (Recommended) Settings
Exit the program and powershell

Change the power plan

Now, let's manually adjust the power plan to reduce the temperatures and improve performance.

Search for Edit Power Plan in the start menu and open it.
Make sure you are using the balanced power mode. Ultimate or High Performance will lead to more issues
Press Change Advanced Power Settings and change the following settings:

- Wireless Adapter Settings
    - Power Saving Mode
        - On Battery: Medium Power Settings
        - Plugged in: Maximum Performance

- USB Settings
    - USB Selective Suspend Settings
        - On Battery: Disabled
        - Plugged in: Disabled

- Intel ® Graphics Settings
    - Intel ® Graphics Power Plan
        - On Battery: Balanced
        - Plugged in: Maximum Performance

- PCI Express
    - Link State Power Management
        - On Battery: Maximum Power Savings
        - Plugged in: Off

- Processor Power Management
    - Minimum Processor State
        - On Battery: 5%
        - Plugged in: 5%

    - System Cooling Policy
        - On Battery: Active
        - Plugged in: Active

    - Maximum Processor State
        - On Battery: 100%
        - Plugged in: 100%

Setup Auto Defragment For Your Hard Disk

SKIP THIS IF YOU ARE USING AN SSD. In a hard drive the data can become scattered to different places. Hence the disk head has to jump a lot more between places to get data which definitely slows it down. This will only help in case you are using a hard disk. To check if you are:

Search for Defragment and Optimize Drives in the start menu
In the listed drives, check the column named Media Type. If it says Solid State Drive skip this section.

If the drive is a Hard Disk Drive, then set up auto defragment as follows:

In the Defragment Window, under the Scheduled Optimization press change settings.
In the new window, tick Run on a schedule with frequency Weekly
Leave the rest default and hit ok and close the defragment window.

Remove the unnecessary apps

This step will allow you to reduce the number of background processes, and also free up extra space. To remove unwanted or unnecessary apps:

Open the Settings app from start menu
Go to Apps in the left tab
Open the Installed Apps tab
Go through the list and remove all the apps you don't use. If there are some which you are uncertain whether or not can be uninstalled, a simple google search should help. Search something like Is <app_name> safe to uninstall

Disable Unnecessary Startup Programs

Some apps start when Windows starts, effectively slowing down the boot time. You can disable these apps by:

Press Ctrl + Shift + Esc or search for Task Manager in the start menu and open it
In the Task Manager, go to the tab on the left which says Startup Apps
Disable any apps you don't want to start automatically by right clicking on the app name, and selecting disable.

Use a custom DNS

A DNS is a service which converts the URL of websites (such as www.google.com) to it's IP Address (such as 142.251.xx.x). The default ISP DNS can be slow, and they can also track which sites you may be visiting. It's bette to use a custom DNS.

There are two DNS I recommend:

Cloudflare (Loads websites faster): 1.1.1.1 & 1.0.0.1
Adguard (Blocks ads on most sites and apps) : 94.140.14.14 & 94.140.15.15

To use one of these, or some other DNS you might have found online:

Search Control Panel in the start menu
In the windows, either select Network and Internet and then Network and Sharing Center or Network and Sharing Center if present directly.
In the left panel, select Change Adapter Settings
Choose the Wi-Fi Network to which you are connected. Right click on it and choose Properties
In the list double click the option labelled Internet Protocol Version 4 (TCP/IPv4)
In the bottom section, choose the option Use the following DNS Server Addresses:
In the preferred address write the main DNS Address (for cloudflare it's 1.1.1.1)
In the alternate address write the secondary DNS Address (for cloudflare it's 1.0.0.1)
Hit ok and exit.

(Optional) Disable/Reduce Turbo Boost

If you are on a computer without a good cooling system, such as a laptop, it tends to be better to disable turbo boost. Turbo boost is the technology which boosts the CPU to higher frequencies, however it also generates more heat. Reducing Turbo Boost should keep the heat to minimum, allowing for better performance. There shouldn't be any performance drop, though your experience may vary so make sure to test it well.

To Disable/Reduce Turbo Boost:

Press Windows Key + R to launch the run window
Type regedit.exe in the field and press ok
In the registry editor window, in the address at the top where it probably says Computer or HKEY.... write the following: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Power\PowerSettings\54533251-82be-4824-96c1-47b60b740d00\be337238-0d82-4146-a960-4f3749d470c7
On the right panel, double click on the file called Attributes and modify the value from 1 to 2. The data should read 0x00000002 (2)
Search for Edit Power Plan in the start menu and open it.
Press Change Advanced Power Settings
Adjust the settings:
- Processor Power Management
  - Processor Performance Boost Mode
    - On Battery: Disabled
    - Plugged in: Disabled / Efficient Aggressive (Test which works best for you)

(Optional) Use ISLC for low end PC

The Intelligent Standby List Cleaner is an utility which frees up memory taken by programs running in tha background. If you have a good amount of memory (RAM) then usage of this tool is discouraged. However if you have limited memory, here's the step by step guide:

Download ISLC tool from https://www.wagnardsoft.com/forums/viewtopic.php?t=1256
Open and Extract the files anywhere (it will permanently stay there so choose a permanent location for the tool)
Run Intelligent Standby List Cleaner ISLC as administrator
Tick the Start ISLC minimized and auto Start monitoring and Launch ISLC on user logon. (Task Scheduler) option. This will automatically run the app on each boot.
Check enable custom timer resolution
Adjust the value of Wanted Timer Resolution to 0.5000
Press the start button and minimize the software

(Risky) Undervolting

Undervolting is an entire process. Here's the basics - The more power something uses, the more heat it generates - The more heat in the system, the slower it has to run to combat the heat - called as Thermal Throttling.

In essence, while you may think using more power helps your system run faster, unless it can be properly cooled down, it actually slows down the pc to keep the CPU from frying itself.

This is where undervolting comes in. Undervolting is the process of limiting, or more accurately, offsetting the voltage you supply to your processor or the GPU, to allow it to run cooler, and get more performance per watt out of your machine without it thermal throttling

Since this is a lot risky, I will soon be creating a seperate guide on undervolting your GPU and CPU individually so you don't end up with a non-functioning device.

Conclusion

These steps all should allow your pc to run better, using less power, yet giving more performance. I will be creating separate guides on CPU and GPU Undervolting. And I will also try my best to keep this guide updated. However these modifications may end up messing with your system, so try them out at your own risk.

Thanks for reading.