DEV Community: mitali

I Thought Compilers Were Scary. So I Built Sauce.

mitali — Sun, 28 Dec 2025 06:31:50 +0000

I’ve been writing code for years. I type cargo run or npm start, hit enter, and meaningful things happen. But if you asked me what actually happens between hitting enter and seeing "Hello World," I’d have mumbled something about "machine code" and changed the subject.

That bothered me. I rely on these tools every day, but I didn't understand them.

So I started building Sauce, my own programming language in Rust. Not because the world needs another language, but because I needed to stop treating my compiler like a black box.

Turns out, a language isn't magic. It's just a pipeline.

Why We Think It's Hard

We usually see a compiler as this big, scary brain that judges our code. You feed it text, and it either gives you a working program or yells at you with an error.

I spent years thinking you needed to be a math genius to build one. I was wrong. You just need to break it down into small steps.

What Sauce Actually Is

Strip away the hype, and a programming language is just a way to move data around.

Sauce is a statically typed language that feels like a simple script. I wanted something that was clear and honest. The core ideas are simple:

Pipelines (|>) are default: Data flows explicitly from one step to the next, like a factory line.
Effects are explicit (toss): No hidden surprises or secret jumps in your code.

But to get there, I had to build the engine. And thanks to Rust, I learned that the engine is actually pretty cool.

The Architecture: It’s Just an Assembly Line

I used to think compilation was one giant, messy function. In reality, it’s a disciplined process. I’m following a strict architecture for Sauce that separates "understanding the code" (Frontend) from "running the code" (Backend).

Here is exactly how Sauce works under the hood:

The diagram above isn't just a sketch; it's the map I'm building against. It breaks down into two main phases.

Phase 1: Frontend (The Brain)

This phase is all about understanding what you wrote. It doesn't run anything yet; it just reads and checks.

Lexer (Logos): The Chopping Block
The Job: Computers don't read words; they read characters. The Lexer's job is to group those characters into meaningful chunks called "Tokens."
In Plain English: Imagine reading a sentence without spaces: thequickbrownfox. It's hard. The Lexer adds the spaces and labels every word. It turns grab x = 10 into a list: [Keyword(grab), Ident(x), Symbol(=), Int(10)].
The Tool: I used a Rust tool called Logos. It’s incredibly fast, but I learned a hard lesson: computers are dumb. If you don't explicitly tell them that "grab" is a special keyword, they might think it's just a variable name like "green." You have to set strict rules.
Parser (Chumsky): The Grammar Police
The Job: Now that we have a list of words (tokens), we need to check if they make a valid sentence. The Parser organizes these flat lists into a structured tree called the AST (Abstract Syntax Tree).
In Plain English: A list of words like [10, =, x, grab] contains valid words but makes no sense. The Parser ensures the order is correct (grab x = 10) and builds a hierarchy: "This is a Variable Assignment. The name is 'x'. The value is '10'."
The Tool: I used Chumsky, which lets you build logic like LEGOs. You write a tiny function to read a number, another for a variable, and glue them together.
The "Aha!" Moment: I learned how much structure matters. Instead of trying to parse everything in one giant loop, breaking the grammar into small, composable pieces made the language way easier to extend and reason about. It’s not magic; it’s just organizing data.
Type Checking: The Logic Check
The Job: Just because a sentence is grammatically correct doesn't mean it makes sense. "The sandwich ate the Tuesday" is a valid sentence, but it's nonsense. The Type Checker catches these logical errors.
In Plain English: If you write grab x = "hello" + 5, the Parser says "Looks like a valid math operation!" But the Type Checker steps in and says, "Wait. You can't add a Word to a Number. That's illegal." Sauce currently has a small, explicit system that catches these basic mistakes before you ever try to run the code.

Phase 2: Backend (The Muscle)

Once the Frontend gives the "thumbs up," we move to the Backend. This phase is about making the code actually run.

Codegen (Inkwell/LLVM): The Translator
The Job: This is where we leave the high-level world of "Variables" and "Pipelines" and enter the low-level world of CPU instructions. We translate our AST into LLVM IR (Intermediate Representation).
In Plain English: Sauce is like a high-level manager giving orders ("Calculate this pipeline"). The CPU is the worker who only understands basic tasks ("Move number to register A," "Add register A and B"). LLVM is the translator that turns the manager's orders into the worker's checklist.
Why LLVM? It's the same industrial-grade machinery that powers Rust, Swift, and C++. By using it, Sauce gets decades of optimization work for free. Once you figure out how to tell LLVM to "print a number," the rest stops feeling so scary.
Native Binary: The Final Product
The Job: The final step is bundling all those CPU instructions into a standalone file (like an .exe on Windows or a binary on Linux).
In Plain English: This is what lets you send your program to a friend. They don't need to install Sauce, Rust, or anything else. They just double-click the file, and it runs. (Currently, this works for simple, effect-free programs).

What Works Right Now (v0.1.0)

Sauce isn't just an idea anymore. The core compiler pipeline is alive.

Pipelines: You can write grab x = 10 |> _ and it understands it perfectly. The data flows left-to-right, just like reading a sentence.
Real Output: I can feed it real .sauce source code, and it parses it into a type-safe syntax tree.
Explicit Effects: You can use toss to signal a side effect. This currently works in the interpreter, while the LLVM backend intentionally rejects effects for now.

The Road Ahead

I have a clear plan for where this is going. Since the core architecture is stable, the next updates are about making it usable.

v0.1.x (UX): Right now, if you make a mistake, the error messages are a bit cryptic. I'm adding a tool called Ariadne to give pretty, helpful error reports (like Rust does).
v0.2.0 (Effects): This is the big one. I'll be finalizing how "Effects" work—defining rules for when you can resume a program after an error and when you have to abort.
v0.3.0 (Runtime): Merging the Interpreter and LLVM worlds so they behave exactly the same, plus adding a standard library so you can do more than just print numbers.

Why You Should Try This

I avoided building a language for years because I thought I wasn't smart enough.

But building Sauce taught me that there's no magic. It's just data structures. A Lexer is just regex. A Parser is just a tree builder. An Interpreter is just a function that walks through that tree.

If you want to actually understand how your code runs, don't just read a book. Build a tiny, broken compiler. Create a Lexer. Define a simple tree. Parse 1 + 1.

You'll learn more in a weekend of fighting syntax errors than you will in a year of just using cargo run.

Check out Sauce on GitHub. It's small, it's honest, and we are officially cooking.

Building Conflux - My Own Real-time Collaboration Engine in Rust

mitali — Thu, 13 Nov 2025 07:09:28 +0000

I’ve always used real-time tools without thinking too much about how they actually work. Google Docs, Figma, Replit multiplayer, VSCode Live Share… you type something, the other person sees it instantly, nothing breaks, nobody overwrites anybody else. It... just works.

And for some reason, I kept wondering what’s happening under the hood. How does one person’s change mysteriously appear everywhere? How are conflicts handled? What happens if two people change the same thing at the same time?

I didn’t want the “high-level” answer. I wanted to feel the system. The same way building Veridian helped me understand Git, I wanted to understand these real-time sync systems by building one myself.

So I built Conflux, a small real-time collaboration engine written in Rust. Not because the world needs another backend, but because I needed to understand how these real-time systems actually sync state between multiple users without blowing up.

Turns out, it’s not some mysterious technology. It’s a bunch of simple ideas stacked cleanly.

Why I Wanted to Build This

Whenever I saw people editing the same thing at once, my brain just assumed, “Okay, some mysterious library is doing something complicated.” But then I learned about CRDTs, and the whole idea clicked.

A CRDT (Conflict-free Replicated Data Type) is basically a data structure that never conflicts. Every update is mergeable. Everyone can edit freely without locking. And eventually, everything converges to the same state.

When I realized that, I wanted to see it for myself:

What does a CRDT-based collaboration system actually look like when you implement it?

The "Real Stuff" Behind It: What is a CRDT?

Okay, let's break this down. The "Conflict-free" part is what matters.

The "Normal" Way (The Problem):

Imagine you and I are editing a text file.

We both download the file. It says: "Hello".
I change my copy to: "Hello world".
You, at the exact same time, change your copy to: "Hello there".
I upload my version. The server now has "Hello world".
You upload your version. The server now has "Hello there".

My change is gone. Forever. You overwrote me. This is a conflict. This is what version control systems like Git spend all their time trying to manage with "merge conflicts."

The "CRDT" Way (The Solution):

CRDTs don't work like that. They don't send the whole file back and forth. They send instructions.

We both have the state: "Hello".
I make a change. My local CRDT doesn't say "the new file is 'Hello world'". It generates an instruction: (At position 5, add: " world").
You, at the same time, make a change. Your CRDT generates an instruction: (At position 5, add: " there").
I send my instruction to the server.
You send your instruction to the server.

The server, and eventually all clients, get both instructions. The "aura" of the CRDT is that it has a mathematical rule for merging these instructions so that everyone ends up with the exact same final state.

The final text might be "Hello world there" or "Hello there world". What's important is that no data was lost, and we both end up seeing the same thing without ever getting a "MERGE CONFLICT" error.

That's it. A CRDT is just a data structure with a "merge" algorithm so good that it never conflicts.

What Conflux Actually Does

Now that we know what a CRDT is, the whole system makes more sense.

Strip away the extra details, and Conflux does three simple things:

It maintains rooms – like “documents” or “sessions.”
Each room contains a CRDT document – this stores the shared state.
Clients send instructions (updates), the server merges them, and it broadcasts them to everyone else.

Here’s that same loop, but now it makes sense:

You type something → your local CRDT generates an instruction → you send that instruction (the "update") to the server → the server merges it into its own CRDT → the server broadcasts that instruction to all other clients → their local CRDTs merge it → everyone's UI updates.

That’s it. That’s the entire loop. No fancy algorithm, no weird transformations, no branching timelines.

Just: update → merge → broadcast.

And because CRDTs are designed to merge cleanly, nothing conflicts.

The Architecture (Simple Version)

To make it easier to understand, I drew it out:

Every box in this diagram has one job. No box is doing five things at once. And that simplicity is what made the whole system feel approachable.

How the Server is Designed

The Conflux server has four main pieces, and each one does exactly one job.

1. Room Manager

This is the part that keeps track of all active rooms.

If a room doesn’t exist, it creates it. If a room is idle for too long, it cleans it up.

Nothing fancy - just lifecycle management.

2. Room (Actor Loop)

A room is basically its own mini-server.

It runs a loop that just listens for commands like:

ApplyUpdate (This is our CRDT instruction!)
SetAwareness
Chat
Join
Leave

...and it applies those updates to its own YDoc.

This “actor” design means each room is isolated, which avoids the usual shared-state chaos.

3. WebSocket Server

This is the entry point.

It:

Authenticates users using JWT.
Extracts the room ID from the URL.
Upgrades the connection to a WebSocket.
Forwards incoming messages to the correct room.
Forwards outgoing messages from the room back to the client.

It accepts two types of messages:

Plain text: This becomes a chat message.
JSON: This becomes a structured CRDT or awareness update.

This makes debugging easy because you can literally type chat messages from a terminal.

4. JWT Authentication

Every login creates a new session ID (sid) inside the token.

This fixes the “everyone logs in with the same token” problem.
When a client connects, the server knows:

Which user
Which session
Which room

It’s simple, but it gives a clean identity model.

The Sync Flow (Easy Explanation)

Let’s say two people are editing a shared document.

Person A edits something. Their local CRDT applies the change instantly and generates an instruction.
A sends the instruction (the update) to the server. This is just a small binary chunk.
Server applies the instruction to its own CRDT. This keeps the server's copy of the document authoritative.
Server broadcasts the instruction. All other clients in that room receive the same update.
Each client applies it. Their local CRDT merges the instruction into whatever they already have.
Everyone stays in sync. Even if many people change the same thing at the same time, the CRDT handles the merges smoothly.

It sounds complicated, but seeing it work makes it feel simple.

Why Awareness and Chat Matter

Real-time systems aren’t just about document content.

Users need to know:

Who else is online
Where their cursor is
Who is typing
Who joined or left

So I added “awareness events.” These are just tiny JSON messages that propagate instantly to show who's who and who's where.

Chat was simple - if the client sends normal text instead of JSON, the server treats it as a chat message and broadcasts it.

It's a small feature, but it makes the system feel alive.

The Dashboard

I also built a small dashboard endpoint:

GET /dashboard

This returns:

Room ID
Number of connected clients
Number of document updates
Number of awareness events

It’s nothing fancy, but it’s incredibly useful to see the system working.

What I Learned From Building This

Just like Veridian made Git click for me, Conflux made real-time collaboration click.

Here are the things that stood out:

Real-time sync is mostly about message ordering and broadcasting.
CRDTs remove like 90% of the “conflict” problems.
Actor-style rooms make concurrency surprisingly clean.
WebSockets are way easier to work with than I expected.
Having a proper identity system early saves headaches later.
Most “complex systems” are just a chain of simple steps.

Before building Conflux, these systems felt like black boxes.

After building it, they feel like something I can understand - and even improve.

Closing Thoughts

Conflux isn’t perfect. It’s probably missing features. It's definitely not production-ready. But it does exactly what I built it for and it helped me understand how real-time collaboration works under the hood.

If you’re curious about this stuff, try building a small version yourself.

You don’t need to recreate Google Docs. Even a tiny CRDT-based shared counter will teach you a lot.

You can check out Conflux here:
https://github.com/Kayleexx/conflux

I’ll keep improving it, and I’ll probably write more about the deeper parts later.

I Built My Own Git in Rust to Understand Version Control

mitali — Sun, 12 Oct 2025 15:33:06 +0000

I've been using Git for years. Committing, pushing, pulling, occasionally panicking when things break. But if you'd asked me what actually happens when I run git commit, I'd have given you some vague answer about "saving changes" and hoped you wouldn't ask more questions.

That bothered me. So I built Veridian, my own version control system in Rust. Not because the world needs another Git, but because I needed to understand the one we already have.

Turns out, Git is way simpler than I thought.

Why Everyone Finds Git Confusing

We learn Git backwards. We memorize commands without understanding what they do. git add stages files. Okay, but what does staging actually mean? git commit saves your work. Cool, but saves it where and how?

I spent years just following commands. I could use Git, but I couldn't understand it. Building Veridian changed that.

Here's What Git Actually Is

Strip away all the commands and features, and Git is just a content-addressable storage system. Sounds complicated, but it's not.

You have a storage system where things are saved by their content, not their name. Put in the same content twice? Same storage location. Change one character? Different location.

That's Git. The "location" is a SHA-1 hash. The "storage" is the .git/objects folder.

Three Types of Objects

Blobs are file contents. Take your file, add a header like blob <size>\0, hash it, compress it, store it. Done. Git doesn't care about filenames here. Only content.

Trees are directory listings. They say "here's what this folder looked like" by listing files and folders with their hashes. Trees point to blobs and other trees.

Commits are snapshots with context. Each commit points to a tree (what your project looked like), points to parent commits (what came before), and has metadata like author, time, and message.

Three object types. That's the whole system.

Why the Hash System is Smart

Same file in multiple commits? Stored once. Changed one line in a big file? Only the new version gets stored. Want to check if two files are identical? Compare hashes, instant answer.

Git isn't making copies of your project over and over. It's storing unique pieces and building snapshots from them. That's why repos with hundreds of commits aren't huge.

Branches Are Just Files

A branch is literally a file with a commit hash in it.

The file .git/refs/heads/main has 40 characters in it, the hash of your latest commit. When you make a branch, Git writes a new file with the current commit hash. When you commit, Git updates the file with the new hash.

No copying. Just updating a small text file. That's why branches are "lightweight."

The Compression Part is Cool

Git uses zlib to compress everything before storing it. So your object files aren't just raw content, they're compressed. When I was building Veridian, I had to handle this compression and decompression for every read and write.

Here's what happens: Git takes your blob (with header), compresses it, then stores it in .git/objects/ab/cdef123... where ab is the first two characters of the hash and cdef123... is the rest. The two-character split is just to avoid having thousands of files in one directory, which would slow down file systems.

Reading it back means finding the file, decompressing with zlib, parsing the header to check object type and size, then giving you the content. Rust's standard library doesn't have zlib built in, so I used the flate2 crate for this. Took like 5 lines of code.

What Building Veridian Taught Me

I thought building a version control system would be hard. It wasn't.

Building in Rust was interesting because Rust makes you think about memory and ownership. When you're hashing files and building trees, you need to handle errors properly (what if the file doesn't exist?) and manage buffers carefully (you can't just load a 5GB file into memory).

But honestly, the version control logic itself is simple. Most of my code is just reading files, computing SHA-1 hashes, and writing compressed data. The hard part wasn't the algorithm, it was understanding what Git was actually doing.

Init Command

Make a .veridian folder. Add subfolders for objects and refs. Create a HEAD file. Done, you have a repo.

Hash-Object Command

Read file, add header, hash it with SHA-1, compress with zlib, write to .veridian/objects/. Return the hash. That's how files enter the system.

Write-Tree Command

Go through a directory. Hash every file (making blobs). Put all the names and hashes into a tree object. Hash that tree. Now you have a snapshot of your directory.

One thing I learned: tree entries need to be sorted by filename. If you don't sort them, the same directory structure produces different hashes depending on the order you process files. Git sorts them to keep hashes consistent. Small detail, but it matters.

Commit-Tree Command

Take a tree hash. Add parent commit hash if there is one. Add author info and time. Add message. Hash it all. Write to objects. Update branch pointer. Update HEAD. That's a commit.

The implementation is surprisingly small.It works like Git because Git is actually this simple.

Fun fact: Git stores timestamps as Unix timestamps (seconds since 1970) with timezone info. So a commit object has something like 1760211794 +0530 which is the timestamp and timezone offset. When you git commit, it grabs your system time and timezone. I used Rust's chrono crate for this, but you could do it with any language.

Things That Finally Clicked

Why Git is fast: It compares hashes, not file contents. 40 character strings. Super quick.

Why detached HEAD happens: HEAD normally points to a branch file, which points to a commit. Check out a commit directly? HEAD points at the commit, skipping the branch. You're detached because you're not on a branch, you're on a specific commit.

Why you can recover deleted commits: They're still in .git/objects. Just unreferenced. Use git reflog, find the hash, get it back. Only garbage collection deletes them for real.

Why merge conflicts exist: Two commits have the same parent but different changes to the same file. Git can't pick which one wins. It needs you to decide.

What I Learned

Git isn't hard because it's complicated. It's hard because we learn it wrong.

Once you get that Git is a key-value store where the key is a content hash, and there are three types of values (blob, tree, commit), everything makes sense. Branches are pointers. Merging combines trees. Rebasing replays commits on a different parent.

I used Git for years without getting it. Then I built Veridian in a week and suddenly Git made sense. Not because building is magic, but because it forces you to understand what's happening.

Why You Should Try This

You don't need to build something perfect. Just start.

Make a repo. Store a file as a blob. Build a tree. Make a commit. Do those four things and you'll understand Git better than most developers.

Building Veridian took maybe a week. Now when I use Git, I actually know what's happening. It's just data structures and file operations. Nothing complicated.

Veridian isn't perfect. It's missing features. Probably has bugs. But it taught me how Git works, and that was the point.

If you want to actually learn Git, not just use it, build something. Even if it's small. Even if it breaks. You'll learn more building for a week than reading docs for months.

Check out Veridian on GitHub. Break it, fix it, learn from it. That's how this works.