What is mysh?
A small POSIX-style shell. It's not a Bash replacement, and I'll be upfront about that. It's a working reference for how a shell turns raw text into running processes.
What it handles today:
Pipes and redirections: |, <, >, >>, and heredocs with <<
Logical operators: &&, ||, and ; with correct precedence
Subshells with parentheses
Background jobs with &, plus jobs, fg, and bg
Builtins: cd, pwd, echo, export, unset, env, exit, help
Environment variable expansion like $VAR
Under the hood, it's four stages: read a line, lex it into tokens, parse it into a tree, execute the tree with fork, execvp, pipe, dup2 and waitpid.
Simple on paper. The interesting part is what each stage has to get right.
How It Actually Works
- The Lexer: Turning Text Into Tokens The lexer reads your line and produces tokens: words, pipes, redirects, logical operators, parentheses, the background &. It also expands environment variables at this stage. So mkdir $TARGET becomes mkdir build before the parser ever sees it. This sounds trivial until you type echo "a | b". That pipe is not a pipe. It's a character inside a string. The lexer has to know the difference, and that's where a lot of the subtle bugs in any shell live.
- The Parser: Precedence Is Everything I used a recursive-descent parser. Each level of precedence gets its own function: Parse OR expressions Parse AND expressions Parse grouped terms in parentheses Parse pipelines and their redirections
The result is an expression tree with three kinds of nodes: Pipeline, And, and Or.
Why bother with a tree? Because of lines like this:
mkdir build && cd build || echo "failed"
Without correct precedence, this means the wrong thing. With a tree, && binds tighter than ||, parentheses override both, and evaluation order just falls out of the structure. No special cases.
- The Executor: Where Processes Get Wired Together This is the part everyone remembers from an OS class and nobody remembers how to write. For ls | grep cpp, the executor does roughly this: cpp int fds[2]; pipe(fds); pid_t left = fork(); if (left == 0) { dup2(fds[1], STDOUT_FILENO); // stdout goes into the pipe close(fds[0]); close(fds[1]); execvp(left_argv[0], left_argv.data()); _exit(127); } pid_t right = fork(); if (right == 0) { dup2(fds[0], STDIN_FILENO); // stdin comes from the pipe close(fds[0]); close(fds[1]); execvp(right_argv[0], right_argv.data()); _exit(127); } close(fds[0]); close(fds[1]); waitpid(left, nullptr, 0); waitpid(right, nullptr, 0); That's the simplified version, but it's the whole idea. A pipe is just two file descriptors. dup2 swaps them into the places the child expects. execvp replaces the child with the real program, which has no idea any of this happened. Notice all those close calls. Hold that thought.
- Builtins: Why cd Can't Be a Program Here's a question that sounds like a trick. Why is cd a builtin instead of a program like ls? Because a child process can't change its parent's working directory. If cd ran in a forked child, it would change directory, exit, and leave your shell exactly where it started. So mysh splits builtins into two cases: A standalone builtin runs in the parent process. If it has a redirection, the shell temporarily remaps its own stdio, runs the builtin, then restores the original descriptors. A builtin inside a pipeline runs in a child. Stateful ones like cd, export, unset, fg, bg and exit fail explicitly there, because changing state in a throwaway process is pointless and confusing.
Real shells have some nuance here, but this model is easy to reason about and easy to test.
- Signals and Job Control Press Ctrl+C in a shell and the running command dies, but the shell itself survives. That's not automatic. mysh handles SIGINT so the prompt loop stays alive, and ignores SIGQUIT. Child processes reset both to their defaults right before exec, so the programs you launch behave normally. For background jobs, each & pipeline gets its own process group and an entry in a job table. jobs lists them, fg brings one forward, bg resumes one.
What Broke
Building a shell is mostly a list of ways you can hang your terminal.
Bug 1: The pipe that never closed.
If you forget to close the unused ends of a pipe in any process, the reader never sees end-of-file. grep just sits there waiting for input that will never come, because some process, possibly the shell itself, still holds the write end open. This is the classic one. Every close in the snippet above exists because of it.
Bug 2: Builtins with redirections.
Running echo hi > file as a standalone builtin means the parent temporarily redirects its own stdout. If you forget to restore it, or bail out early on an error path, your shell prints into a file for the rest of the session. The fix is to save the originals with dup first and restore them on every exit path.
Bug 3: cd inside a pipeline.
cd /tmp | cat looks like it should work. It doesn't, and shouldn't. Deciding to fail loudly instead of silently doing nothing was a design choice I had to make on purpose.
Bug 4: Ctrl+C killing the shell.
Before signal handling was set up, interrupting a slow command took the whole shell down with it. Then I fixed that, and children started ignoring Ctrl+C too, because they inherited the shell's signal settings. Resetting them before exec fixed the second problem.
How I Tested It
A shell has a huge surface area, so testing had to be layered:
Unit tests for the lexer, parser, and tree logic (make unit)
End-to-end smoke tests that run the real shell and check stdout and stderr (make test)
A wider regression suite for edge cases (make regression)
AddressSanitizer and UBSan runs to catch memory bugs (make asan-check)
A single make check that runs the whole CI pipeline locally
The sanitizer runs were worth it. Shells juggle a lot of raw memory and file descriptors, and that's exactly where silent corruption hides.
Why You Should Build One
If you write backend code, you live in a shell. Building one changes how you see everything around it:
Pipes stop being magic. They're two file descriptors and a dup2.
You understand why cd and export are builtins.
You can finally explain what a zombie process is and why waitpid matters.
Shell scripts and their weird quoting rules start making sense.
Debugging "why is my command hanging" gets much faster.
If you care about systems programming, it's the best small project I know. It's big enough to be real, and small enough to finish.
Code Structure
mysh/
├── lexer/ # tokenization, env expansion
├── parser/ # recursive-descent, builds the expression tree
├── executor/ # fork, pipes, redirections, builtin semantics
├── builtins/ # cd, export, jobs, fg, bg...
├── core/ # REPL loop, prompt, shell state
├── utils/ # environment and signal setup
├── tests/
├── benchmarks/
└── docs/
Each stage does one job and hands a clean structure to the next. That separation made debugging much easier.
Try It
git clone https://github.com/bharqav/mysh.git
cd mysh
make
./mysh
Or with Docker:
docker build -t mysh .
docker run -it --rm mysh
Then poke at it:
mysh> ls -la | grep "cpp" | wc -l
mysh> sleep 10 &
mysh> jobs
mysh> export TARGET="build"
mysh> mkdir $TARGET && cd $TARGET || echo "Failed"
One warning: don't use it as a root shell or on a shared machine. It's built for learning.
Lessons Learned
- The kernel does most of the work. A shell is thinner than you'd expect. The hard parts, process isolation and byte streams, are handled by the OS. Your job is wiring.
- File descriptors are the real API. Once you see pipes, redirects and terminals as just numbers you shuffle around, Unix clicks.
- Small mistakes hang things. One missing close doesn't crash your program. It freezes it, which is worse.
- Parse into a tree, not a pile of if statements. Every attempt to handle && and || with string checks fell apart. The tree just worked.
- Decide what should fail. A lot of shell design is choosing which weird cases to reject clearly instead of half supporting.
What's Next
These are the gaps I know about:
Ctrl+Z and SIGTSTP support for proper job suspension
Arithmetic expansion
Fuller command substitution
Closer quoting and wildcard behavior to Bash
The Real Takeaway
You won't replace Bash with a weekend project, and you shouldn't try.
But you'll never look at a terminal the same way again. Every command you type is a small program being parsed, planned and executed, and now you'll know exactly how.
The code is on GitHub. Read it, break it, and try adding Ctrl+Z support.
https://github.com/bharqav/mysh
Top comments (0)