Ig my own operating system? CottonOS

Aryan S Rao — Thu, 11 Jun 2026 10:39:22 +0000

I'm Building an OS From Scratch in Rust — And It Already Has a Desktop, Filesystem, and Networking

A deep dive into CottonOS: a hobby OS written entirely in Rust with a custom filesystem, GUI, preemptive multitasking, and experimental in-kernel TLS.

There's a specific kind of madness that strikes some programmers: the urge to not just use an operating system, but to build one. From the bootloader all the way up to the window manager. No Linux underneath. No BSD. Just you, your CPU, and a whole lot of page faults.

That's what CottonOS is.

I've been building it as a hobby project and learning experiment — a full OS kernel written in Rust for x86_64, featuring a custom filesystem I call CottonFS, a graphical desktop environment with working windows and a taskbar, preemptive multitasking, and an experimental in-kernel HTTPS stack. It's nowhere near production-ready, but it boots, it renders windows, it reads and writes files, and it can make HTTP requests — entirely from scratch.

The long-term vision is bigger: I want CottonOS to eventually run an on-device LLM, lightweight and fast, fully self-contained. That's miles away, but every subsystem I write brings it closer.

This post is a technical walkthrough of what I've built, the decisions I made, the hard parts, and where I want to take it. I'm actively looking for contributors who want to dig into low-level systems programming.

Why Rust for an OS?

The obvious answer: memory safety without a garbage collector. In kernel code, a use-after-free isn't just a bug — it's a privilege escalation or a hard crash with no recovery. Rust's ownership model catches entire classes of these mistakes at compile time.

But the real answer is more personal: I wanted to learn Rust deeply, and there's no deeper environment than bare metal. No allocator. No standard library. No OS to catch your mistakes. Just no_std and a custom target spec pointing at hardware.

The kernel is 95.4% Rust. The remaining ~3% is a small boot_stub.asm — a Multiboot2 header, minimal GDT, early page tables, and a long mode entry point. Once we're in 64-bit mode, it's Rust all the way down.

Boot Sequence: Getting to Rust as Fast as Possible

CottonOS uses GRUB with Multiboot2 as the bootloader. GRUB loads the kernel ELF at the 1MB physical mark and hands off to boot_stub.asm, which does just enough to get into long mode:

Sets up a stack
Initializes a flat 64-bit GDT
Creates identity-mapped PML4 page tables covering the first 1GB
Sets CR4.PAE, loads PML4 into CR3, sets EFER.LME, enables paging via CR0.PG
Far-jumps into the 64-bit code segment
Calls _start64 — our Rust entry point

From there, _start64 initializes serial output first (so we have debug logging even if the screen isn't up yet), parses the Multiboot2 info structure to find the framebuffer and memory map, then brings up each subsystem in order: interrupts → memory → filesystem → processes → drivers → GUI.

Memory Management: Three Layers

Physical Allocator

A bitmap allocator that tracks 4KB frames. On boot, we parse the Multiboot2 memory map, mark everything the kernel and framebuffer occupy as used, and the rest is available. Simple and predictable — no fragmentation at the physical layer.

Virtual Memory

Standard x86_64 4-level paging: PML4 → PDPT → PD → PT. The kernel sits in the lower half for now (higher-half support is ready but not yet wired up). Pages carry the standard permission flags: Present, Writable, User-accessible, No-Execute. On-demand mapping means we only create page table entries when they're actually needed.

Kernel Heap

Built on top of the linked_list_allocator crate. Initial size is 4MB at 0x02000000, expandable to 16MB. This registers as the global allocator, which unlocks alloc — meaning Vec, String, Box, Arc, and the rest of the heap-allocated collection types work normally inside the kernel.

CottonFS: A Custom Persistent Filesystem

This was one of the most involved parts to build. CottonFS is a simple but real persistent filesystem with a design influenced by ext2.

On-Disk Layout

Block Range	Content
0	Superblock
1–31	Inode bitmap
32–63	Data block bitmap
64–127	Inode table
128+	Data blocks

Block size is 4096 bytes. The magic number is 0x43544653 ("CTFS"). The superblock tracks total/free blocks and inodes, mount count, last mount time, and the root inode (always inode 1).

Inodes are 128 bytes on disk, holding file metadata and 12 direct block pointers plus one indirect pointer (giving a max file size of ~4MB per file — enough for a hobby OS).

VFS Layer

Above CottonFS sits a VFS abstraction using Rust traits:

pub trait FileSystem: Send + Sync {
    fn root(&self) -> Result<Arc<dyn Inode>, &'static str>;
    fn sync(&self) -> Result<(), &'static str>;
    fn stats(&self) -> Result<FsStats, &'static str>;
}

pub trait Inode: Send + Sync {
    fn read(&self, offset: u64, buf: &mut [u8]) -> Result<usize, &'static str>;
    fn write(&self, offset: u64, data: &[u8]) -> Result<usize, &'static str>;
    fn lookup(&self, name: &str) -> Result<Arc<dyn Inode>, &'static str>;
    fn create(&self, name: &str, file_type: FileType) -> Result<Arc<dyn Inode>, &'static str>;
    fn unlink(&self, name: &str) -> Result<(), &'static str>;
    fn readdir(&self) -> Result<Vec<DirEntry>, &'static str>;
}

This means mounting a DevFS at /dev was straightforward — it implements the same traits, presents /dev/null, /dev/zero, and /dev/random as virtual inodes, and the rest of the kernel doesn't need to know the difference.

Process Management and the Scheduler

CottonOS has a preemptive round-robin scheduler running at 1000Hz, driven by the PIT (Programmable Interval Timer).

Each process is represented by a Process struct with a PID, parent PID, name, priority (5 levels), state, and exit status. States are Ready, Running, Blocked, and Zombie.

Threads are supported at the kernel level with per-thread stacks and thread-local storage ready for use.

The scheduler sits on top of this: on each timer tick (every 1ms), the current process is preempted and the next ready process in priority order gets scheduled. There's also an idle process that runs when nothing else is ready.

The synchronization primitives — Mutex, Semaphore, and CondVar — are all custom-implemented. The mutex uses a wait queue so blocked threads yield the CPU rather than spinning, which actually matters at 1000Hz.

System Calls

The syscall interface is exposed via both int 0x80 and the syscall instruction. There are 40+ calls covering:

Process: exit, fork, exec, wait, getpid, yield, sleep
File I/O: open, close, read, write, seek, stat, fstat
Directory: mkdir, rmdir, unlink, readdir, chdir, getcwd
Memory: brk, mmap, munmap
System: uname, time, uptime

Userspace is still largely planned — the interesting parts right now live in kernel space — but the interface is there and the handlers are implemented.

Networking: From ARP to In-Kernel TLS

The networking stack is the newest and most experimental part of CottonOS. Here's what's implemented:

Driver layer: An RTL8139 NIC driver that handles PCI discovery, initialization, and RX/TX frame handling. Works in QEMU with -nic user,model=rtl8139.

L2/L3: Ethernet framing, ARP (cache + request/reply), and IPv4 parsing and dispatch.

Transport: ICMP echo (ping), UDP send/receive, and basic TCP client connections.

Application layer: DHCP for auto-configuration, DNS resolver for A records, and two shell commands — httpget for plain HTTP and httpsget for HTTPS.

The httpsget command performs TLS negotiation entirely in-kernel. This is very experimental — the current implementation uses encrypted TLS transport but does not yet validate server certificates. Trust model hardening is planned, but getting the handshake and record layer working in a bare-metal Rust kernel with no OS underneath is already a significant milestone.

Stability work included bounded poll loops to prevent livelock, a non-blocking IRQ lock path, and ARP auto-resolution with retries on normal TCP/UDP flows so you don't have to manually ARP before connecting.

The Desktop Environment

This is what surprises people most when they see CottonOS: it has a real graphical desktop.

The framebuffer comes from GRUB's video mode. A double-buffer handles rendering to eliminate flicker. The font is an 8×16 bitmap font drawn character by character. Mouse input comes from the PS/2 driver.

Window Manager: Windows have titles, z-order, focus, and dragging via the title bar. The controls are macOS-style (red/yellow/green circles). Keyboard and mouse events route to the focused window.

Applications bundled in the kernel:

Terminal — A full shell emulator with scrollable output history, backspace support, and working directory tracking. Runs all the shell commands listed below.
File Manager — Graphical CottonFS browser with folder icons, file sizes, double-click navigation, back button with history, and scrollable listings.
Text Editor — Multi-line editor with cursor positioning, arrow key navigation, undo/redo, file open/save/save-as dialogs, modified indicator, and line numbers.
System Info panel — Real-time display of kernel version, memory stats, filesystem usage, process count, and uptime.

Shell Commands

The integrated shell (available both in the terminal app and as the kernel shell before GUI loads) supports:

Category	Commands
Filesystem	`ls`, `cd`, `pwd`, `cat`, `touch`, `mkdir`, `rm`, `write`
System	`mem`, `df`, `ps`, `uptime`, `info`
Network	`net`, `netstats`, `arptable`, `arp`, `ping`, `dhcp`, `dns`, `setip`, `setmask`, `setgw`, `setdns`
TCP/UDP	`tcpconnect`, `tcpsend`, `tcprecv`, `tcpclose`, `httpget`, `httpsget`, `udpsend`, `udprecv`
Utilities	`echo`, `clear`, `help`, `sync`
Power	`reboot`, `halt`

The Architecture at a Glance

+-------------------------------------------------------------------+
|                        User Space (Planned)                       |
+-------------------------------------------------------------------+
|                       System Call Interface                       |
+-------------------------------------------------------------------+
|                          Kernel Space                             |
|  +-----------------+  +------------------+  +------------------+ |
|  |       GUI        |  |      Shell       |  |  Syscall Handler | |
|  | Window Manager   |  | Command Parser   |  |                  | |
|  | Desktop + Apps   |  | Builtins         |  |                  | |
|  +-----------------+  +------------------+  +------------------+ |
|  +-----------------+  +------------------+  +------------------+ |
|  |   Filesystem     |  |    Process       |  |     Memory       | |
|  |  VFS / CottonFS  |  | Scheduler        |  | Physical/Virtual | |
|  |  DevFS           |  | Threads          |  | Heap             | |
|  +-----------------+  +------------------+  +------------------+ |
|  +------------------------------------------------------------+   |
|  |  Drivers: Graphics | Keyboard | Mouse | ATA | Serial | PIT |   |
|  +------------------------------------------------------------+   |
|  |  x86_64: GDT | IDT | Paging | PIC | APIC | PIT | I/O Ports |  |
|  +------------------------------------------------------------+   |
+-------------------------------------------------------------------+
|                           Hardware                                |
+-------------------------------------------------------------------+

Building and Running It

You'll need rustup (nightly), nasm, qemu-system-x86_64, grub-mkrescue, xorriso, and x86_64-elf-ld.

Ubuntu/Debian:

sudo apt install qemu-system-x86 nasm grub-pc-bin xorriso mtools
rustup override set nightly
rustup component add rust-src llvm-tools-preview

Then:

git clone https://github.com/aryansrao/cottonos
cd cottonos
make iso      # Build bootable ISO
make disk     # Create 64MB persistent disk image
make run      # Boot in QEMU

Networking quick start (inside CottonOS):

net
dhcp
dns example.com
httpget example.com /
httpsget example.com /

What's Next

The roadmap for CottonOS includes several tracks:

Near term:

Higher-half kernel migration (page tables are ready, needs wiring)
Certificate validation for the TLS stack
ANSI color support and command history in the terminal
Proper userspace ELF loading and isolation

Medium term:

SMP (multi-core) support — the APIC code is stubbed in
Extended filesystem features (larger file sizes via double/triple indirect blocks)
More device drivers (USB, virtio)

Long term (the real goal):

Running an on-device LLM entirely within CottonOS — lightweight inference, no external dependencies, fully self-contained

That last one is ambitious. But the point of a hobby OS is that ambition is the whole point.

I'm Looking for Contributors

CottonOS is open source under GPL-3.0. I'm one person working on this in my spare time, and there's a lot of surface area. If any of this sounds interesting — low-level Rust, OS internals, graphics, networking, filesystem design — I'd love contributions at any level.

Where to start:

📁 GitHub: github.com/aryansrao/cottonos
Good first areas: ANSI color support in the terminal, command history, certificate validation for TLS, documentation improvements
Bigger challenges: Higher-half kernel, SMP, virtio drivers, userspace ELF loading

Even just opening issues, leaving feedback on the architecture, or asking questions helps me think through the design better. OS development can feel lonely when you're doing it from scratch — the community is the part I'm most looking forward to building.

If you've built something similar, or just have thoughts on the architecture decisions — I'd genuinely love to hear them in the comments. This is a learning project at heart, and every perspective helps.

NEX terminal, I created a terminal recording and sharing application.

Aryan S Rao — Tue, 30 Dec 2025 19:01:14 +0000

Hey everyone, I’m new to the dev.to platform and I want to share something I’ve been building for a while. I created an open-source Rust tool called NEX. It’s a terminal recorder and real-time collaboration tool. I noticed that terminal workflows are powerful but often poorly documented in software engineering. We spend hours in the CLI deploying servers, fixing production issues, running migrations, teaching juniors, pair programming, or experimenting locally. However, once a terminal session ends, most of that context is lost. Command history doesn’t tell the whole story, screenshots miss timing and interactivity, logs lose intent, and screen recordings can be heavy, noisy, and hard to analyse.

NEX was designed to address these issues by recording terminal sessions with PTY-level accuracy. It captures exactly what the terminal sees and does, including interactive programs like vim or htop, ANSI colours, cursor movements, and precise timing information. Instead of just dumping text, NEX saves an accurate representation of the entire session. You can replay it later exactly as it happened, with commands and outputs appearing at the same speed as the original run. Additionally, NEX extracts structured metadata such as command execution timelines, exit codes, and timestamps. This allows you to inspect and export terminal activity as real data instead of static logs. You can convert recorded sessions into CSV or JSON and use them in spreadsheets, dashboards, audit reports, or automation pipelines, which is something I missed when analysing or documenting complex CLI workflows.

Another major reason I built NEX is for collaboration. Explaining terminal issues over chat or screen-sharing can be frustrating and ineffective. NEX lets you share a live terminal session over the network, so others can watch or collaborate in real time. Each participant can also keep their own local recording of the session. There’s even a web mode that allows people to view the session from a browser without needing to install anything, which is great for demos, workshops, or non-technical stakeholders.

Compared to existing tools like script, asciinema, or traditional screen recording, NEX aims to combine the best features of all of them while avoiding their limitations. It’s lightweight, accurate, replayable, analysable, and fully open source and best of all it can serve it on website where they could just directly catch it in their phone or any device which has access to that network and have a web browser. I chose Rust for its performance, safety, and reliability, which are crucial when intercepting PTY streams and managing long-running sessions. NEX is designed to have minimal overhead, so you can leave it running without worrying about resource use. If you’re into DevOps, backend engineering, teaching, onboarding, compliance, or care about terminal craftsmanship, NEX could genuinely improve your workflow. You can record a session, replay it later to see what happened, share it live with teammates while debugging remotely, or export the data for further analysis, all with a straightforward CLI-first experience.

I built NEX because I believe terminal work represents knowledge, and that knowledge deserves better tools than fragile logs or unclear recordings. The project is actively developed and fully open source. I’d love feedback, ideas, or contributions from anyone interested. If you want to check it out, the site is nex-terminal.netlify.app and the repository is on GitHub at github.com/aryansrao/nex.

DEV Community: Aryan S Rao