When you type python3 main.py and hit enter, what actually happens? How does text sitting on your hard drive end up executing on your CPU?
The answer is a program called an interpreter. Unlike a compiler, which translates source code into a standalone executable before running it, an interpreter reads your code directly, understands what it means, and executes it on the spot. Python, Ruby, JavaScript, Lua — the languages you use every day all run on interpreters under the hood.
We built one. LoxInterp is a complete interpreter written in C++23. It has full lexical scoping, closures, class inheritance, constructors, super, 39 token types, 13 AST node types — roughly 1600 lines of source, 1200 lines of tests, zero external dependencies.
The project is based on Robert Nystrom's classic tutorial Crafting Interpreters. The original uses Java; we hand-rolled a C++23 version. Open source at https://github.com/Tenaryo/LoxInterp.
What It Looks Like
Let's see it in action first. Here's a snippet of Lox — classes, inheritance, super, instances, all in one shot:
class Animal {
init(name) { this.name = name; }
speak() { print this.name; }
}
class Dog < Animal {
speak() {
super.speak();
print "woof!";
}
}
var d = Dog("Buddy");
d.speak();
Save it as demo.lox and run it with a single command:
$ ./build/LoxInterp run demo.lox
Buddy
woof!
These dozen lines of Lox go through a full pipeline from raw text to CPU execution. Let's unwrap that pipeline layer by layer.
The Architecture: Simpler Than You Think
An interpreter's skeleton has just four stages:
Source Code
│
▼
┌──────────┐ ┌──────────┐ ┌──────────────┐ ┌──────────┐
│ Scanner │ ───▶ │ Parser │ ───▶ │ Resolver │ ───▶ │Interpreter│
│ Lexer │ │ Parser │ │ Binder+Check │ │ Runtime │
└──────────┘ └──────────┘ └──────────────┘ └──────────┘
Token Stream AST Annotated AST Output
The run command in main.cpp is nothing more than these four steps called in sequence, maybe a dozen lines total:
auto tokens = scanner.scan_tokens(); // 1. Text → Token stream
auto statements = parser.parse_statements(); // 2. Tokens → AST
resolver.resolve(statements); // 3. Binding + semantic checks
interpret(statements); // 4. Walk the tree and execute
One stage feeds into the next. Let's start at the front door — the Scanner.
Scanner: Text → Token Stream
The Scanner's job is brutally mechanical. It doesn't care about logic, doesn't care about structure, doesn't even know whether 1 + 1 is valid syntax. Its only job is to recognize what's in this blob of characters and slap labels on it.
A Token is just four fields:
struct Token {
TokenType type; // LEFT_PAREN / NUMBER / STRING / IF / ...
std::string lexeme; // Raw text: "(" / "42" / "\"hello\"" / "if"
TokenLiteral literal; // Parsed value: null / 42.0 / "hello" / null
int line; // Line number, for error reporting
};
Feed it class A < B { fun f(){} } and it spits out 14 flat tokens:
CLASS, IDENTIFIER(A), LESS, IDENTIFIER(B), LEFT_BRACE,
FUN, IDENTIFIER(f), LEFT_PAREN, RIGHT_PAREN,
LEFT_BRACE, RIGHT_BRACE, RIGHT_BRACE, EOF
Notice < is labeled LESS. The Scanner doesn't know if this < means inheritance or comparison — that's the Parser's problem. class and fun are recognized as keywords, not generic identifiers. The entire scanning process is one giant switch statement, dispatching on the first character:
auto Scanner::scan_token() -> void {
char ch = advance();
switch (ch) {
case '(': add_token(LEFT_PAREN); break; // Single-char: produce directly
case '"': scan_string(); break; // String: while peek != '"'
case '0'...'9': scan_number(); break; // Number: greedy consume
case '!': add_token(match('=')?BANG_EQUAL:BANG); break; // Two-char
case '/': if (match('/')) skip_comment(); else add_token(SLASH); break;
default:
if (is_alpha(ch)) { // Identifier / keyword
while (is_alphanumeric(peek())) advance();
auto it = kKeywords.find(lexeme);
add_token(it != end ? it->second : IDENTIFIER);
} else error("Unexpected character."); // Illegal character
}
}
A keyword map decides which identifiers are "built-in" — sixteen entries:
const unordered_map<string, TokenType> kKeywords = {
{"print", PRINT}, {"var", VAR}, {"if", IF}, {"while", WHILE},
{"class", CLASS}, {"fun", FUN}, {"and", AND}, {"or", OR},
{"return", RETURN}, {"super", SUPER}, {"this", THIS}, ...
};
This is also why print becomes the keyword PRINT while clock remains a plain IDENTIFIER — clock isn't in the table. The Scanner doesn't know about it. It only works as a function call because the Interpreter, at startup, manually stuffs a clock function object into the global environment. That's where compile time and run time part ways.
Encounter @ or some other character Lox doesn't use? The Scanner prints an error to stderr, sets a flag, and keeps going. Unlike the Parser — which throws exceptions to unwind — lexical errors don't cascade. The next token is still valid, so keep scanning.
Parser: Token Stream → AST
The Scanner produces a flat, one-dimensional list of tokens. The Parser's job is to shape them into a nested, tree-structured AST — an Abstract Syntax Tree. The expression print 2 + 3 * 4; isn't a pile of independent tokens; it's a print statement wrapping an addition whose right-hand side is itself a multiplication.
From a black-box perspective: the Parser takes std::vector<Token> in, and produces std::vector<Stmt> out:
auto Parser::parse_statements() -> std::vector<ast::Stmt> {
std::vector<ast::Stmt> statements;
while (!is_at_end()) {
statements.push_back(declaration()); // Parse one top-level decl at a time
}
return statements;
}
Each top-level construct enters declaration(), which looks at the current token and dispatches:
declaration()
├─ Sees VAR → var_declaration() // var x = 1;
├─ Sees FUN → function_declaration() // fun foo(a,b) { ... }
├─ Sees CLASS → class_declaration() // class Dog < Animal { ... }
└─ Otherwise → statement() // print / if / while / for / return / expression
Let's trace an expression statement: print 2 + 3 * 4;. statement() sees PRINT and enters print_statement(), which calls expression() to parse the right-hand side 2 + 3 * 4.
Expression parsing is where the Parser earns its keep. The technique is called recursive descent — lower-precedence operators wrap the higher-precedence ones. The parse of 2 + 3 * 4 goes like this:
expression() descends to term(). term() handles + and -:
1. Call factor() for the left operand. factor() handles * and /.
factor() descends to primary(), gets Literal(2). No * or / in sight → returns Literal(2).
2. term() gets Literal(2) as the left operand. Check the next token: is it + or -?
Yes — it's PLUS. Gobble the PLUS, record op = "+".
3. term() calls factor() for the right operand.
factor() descends to primary(), gets Literal(3). Then sees STAR. Gobbles it.
Descends again, gets Literal(4). Returns Binary(*, 3, 4).
4. term() wraps everything: Binary(+, Literal(2), Binary(*, 3, 4)).
The resulting tree — note how * sits deeper, ensuring it evaluates first:
Binary
├── left: Literal(2)
├── op: "+"
└── right: Binary
├── left: Literal(3)
├── op: "*"
└── right: Literal(4)
Every precedence level follows the same template — recursively grab the left operand, then loop matching your own operators, recursively grab the right operand, wrap into a node:
auto term() -> Expr {
auto expr = factor(); // Left operand → delegate to lower level
while (match(PLUS) || match(MINUS)) { // Loop: my operators
auto op = previous();
auto right = factor(); // Right operand → delegate again
expr = Binary(expr, op, right); // Wrap
}
return expr; // My operators gone → return
}
Class declarations follow the same recursive pattern. class Dog < Animal { speak() { ... } } enters class_declaration(): grab the class name, check for < (superclass), then loop inside the braces matching FUN keywords, each time recursively calling function_declaration() to parse the method body:
auto class_declaration() -> Stmt {
auto name = consume(IDENTIFIER); // "Dog"
optional<Expr> superclass;
if (match(LESS)) // Has superclass?
superclass = Variable(consume(IDENTIFIER)); // "Animal"
consume(LEFT_BRACE);
vector<FunctionStmt> methods;
while (!check(RIGHT_BRACE))
methods.push_back(function_declaration()); // "speak() {...}"
consume(RIGHT_BRACE);
return ClassStmt{name, superclass, methods};
}
One elegant design choice: for loops don't get their own AST node. for_statement() desugars them directly into while + block at parse time. The Interpreter never needs to know for exists — it only handles while and block. Fewer node types, simpler backend.
Resolver: Compile-Time Binding
Between the Parser finishing and the Interpreter starting sits one more compile-time pass — the Resolver. Its core task: pre-compute, at compile time, where every variable lives, and stamp that information directly into the AST.
Why? Consider this innocent-looking code:
var x = "global";
{
fun f() { print x; }
f(); // Prints "global" ✓
var x = "local"; // A new x in the same scope
f(); // Should still print "global" (closure semantics)
}
If the Interpreter just naively walks up the scope chain at runtime, the second call to f() would find the newly-declared x = "local" and print the wrong thing. The Resolver prevents this.
It maintains a scope stack scopes_ — each frame is a map<name, bool> where true means "fully defined, ready to use" and false means "declared but not yet initialized" (the window between var x and the = sign). As it walks the AST:
struct Variable {
Token name;
int depth = -1; // -1 = unresolved
// >= 0 = "skip this many environment frames to find this variable"
};
When it enters {, it pushes an empty frame. When it exits }, it pops. For each Variable node, it scans the stack from top to bottom, counts how many frames separate the variable from its declaration, and writes that depth into the node. The Interpreter then uses env->get_at(depth, name) — no chain walking, no ambiguity, no pollution from later declarations.
The Resolver also catches a bunch of semantic errors at compile time — things that are syntactically valid but logically wrong:
return 42; // At top level? → Error
this.x = 1; // Outside a class? → Error
{ var x = x; } // Self-reference in initializer? → Error
class Foo < Foo {} // Inheriting from yourself? → Error
class Bar { super.m(); } // super without superclass? → Error
class Baz { init() { return 1; } } // Returning value from init? → Error
All caught with exit code 65 before a single line of code runs.
Interpreter: The Real Thing
The Interpreter is the backend — it takes the depth-annotated AST from the Resolver, recursively walks the tree, and executes. Three functions, that's it:
auto interpret(statements) → void // Create global env, execute each stmt
auto execute(stmt, env) → void // Execute one statement (side effects)
auto evaluate(expr, env) → LoxLiteral // Evaluate one expression (produces a value)
execute handles all statements. print x → evaluate x and cout the result. var x = 1 → evaluate the right side, then env->define("x", 1.0). if (cond) { A } else { B } → evaluate the condition, check truthiness, execute the chosen branch. while (cond) { body } → loop evaluating the condition and executing the body until false.
evaluate handles all expressions. Literal → return the value directly. Binary → evaluate left and right, then apply the operator. Variable → skip depth frames and grab it with env->get_at(depth, name). Call → evaluate the callee (must be a Callable), evaluate each argument, then invoke .call().
The Environment Chain: Heart of the Interpreter
Every time the program enters { } (a block), a fresh Environment is created:
struct Environment {
unordered_map<string, LoxLiteral> values_; // Variables in this scope
shared_ptr<Environment> enclosing_; // Pointer to the outer scope
};
These link into a scope chain:
Global env: {clock: Callable}
↑
Block env: {a: 1.0} enclosing_ → global
↑
Function body env: {x: 42.0} enclosing_ → block
Variable lookup walks the chain. Variable definition only writes in the current frame. Assignment walks the chain to find where it was defined.
Why shared_ptr? Because closures grab the environment at definition time:
fun makeCounter() {
var i = 0; // Local to makeCounter
fun count() { i = i + 1; return i; }
return count; // count escapes to the outside!
}
var c = makeCounter();
c(); // 1
c(); // 2
When count is defined, fn->closure captures makeCounter's local environment — the one holding i. After makeCounter returns, that environment would normally evaporate. But count's closure still holds a shared_ptr to it, so the reference count stays above zero and i lives on. Every call to c() enters Function::call():
auto Function::call(env, args) -> LoxLiteral {
auto func_env = make_shared<Environment>(closure); // Parent = captured env
for (i = 0; i < params.size(); i++)
func_env->define(params[i], args[i]); // Args bound in this frame
try {
for (auto& stmt : *body) execute(stmt, func_env);
} catch (const Return& ret) { return ret.value; } // return unwinds here
return monostate{};
}
Parameters are bound in func_env itself (depth 0). Outer variables reachable through enclosing_. That's closures, in their entirety.
Classes, Inheritance, and super
A class itself is a value — it can be assigned to variables, passed as an argument, printed:
// When a class is defined:
auto klass = make_shared<LoxClass>();
klass->name = "Dog";
klass->methods_ = { "speak": speak_fn, "init": init_fn };
env->define("Dog", klass); // Stored just like any other variable
Instantiation — Dog("Buddy") — calls LoxClass::call():
auto LoxClass::call(env, args) -> LoxLiteral {
auto instance = make_shared<LoxInstance>(); // 1. Blank instance
instance->klass = shared_from_this(); // 2. Tag its class
auto init = find_method("init"); // 3. Find constructor
if (init) {
auto bound = copy(init); // 4. Copy the function
bound->closure->define("this", instance); // 5. Bind this
if (superclass_) bound->closure->define("super", superclass_); // 6. Bind super
bound->call(env, args); // 7. Execute constructor
}
return instance; // 8. Always return instance
}
The constructor's return value is discarded — LoxClass::call() always returns the newly minted instance. That's the constructor guarantee.
Inheritance works through find_method, which walks the chain:
auto LoxClass::find_method(name) -> shared_ptr<Function> {
if (methods_.contains(name)) return methods_[name]; // Check self
if (superclass_) return superclass_->find_method(name); // Check parent
return nullptr; // Not found
}
Method overriding is automatic — start from the subclass and go up; the first match wins.
How super Works — and Why declarating_class_ Exists
super has a subtle semantic. Consider this inheritance chain:
class A { say() { print "A"; } }
class B < A { test() { super.say(); } say() { print "B"; } }
class C < B { say() { print "C"; } }
C().test(); // Should print "A", not "B" or "C"
C().test() → C doesn't have test(), so walk up to B → execute B's test(), which calls super.say(). The question is: which class does super refer to?
Logically, test was defined in class B, so its super should be B's parent — class A. But at runtime, this points to the C instance. If we naively compute this.klass.superclass_, C's parent is B — we'd find B's say(), print "B", and get it wrong.
So Function needs to remember which class it belongs to:
struct Function : Callable {
weak_ptr<LoxClass> declaring_class_; // ← "Which class defined me?"
};
Why weak_ptr rather than shared_ptr? Because LoxClass::methods_ already holds a shared_ptr<Function>. If Function held a shared_ptr back to LoxClass, we'd have a cycle — each keeps the other alive, reference counts never reach zero, memory leaks. A weak_ptr doesn't increment the reference count. It just asks: "are you still alive?" If the class is destroyed, the method doesn't need it anymore.
When a method is bound to an instance — that is, when instance.test() triggers LoxInstance::get("test") — the declaring class is used to inject super into the closure:
if (auto dc = method->declaring_class_.lock()) { // Get defining class
if (dc->superclass_ != nullptr)
bound->closure->define("super", dc->superclass_); // super = B's parent = A
}
No matter how deep the inheritance chain goes below it, super always refers to the superclass of the method's birthplace.
The init Constructor
An init method is mechanically identical to any other method — it just gets two special treatments. First, LoxClass::call() always returns the instance regardless of what init returns. Second, Function carries an is_init_ flag:
if (is_init_) {
return func_env->get_at(1, "this"); // Return this, not nil
}
This allows chaining: instance.init(x).someProp — after init runs, the return value is the instance itself.
Why return Throws an Exception
A return statement can be buried at any depth — inside an if, inside a while, inside a block, inside another if. If we passed the return value back through the call stack layer by layer, every single execute would need to check "did something below me return?" That's noise in every handler.
Instead, Function::call() wraps the body execution in a try-catch. The return statement throws a Return object. No matter how deep the nesting, it unwinds directly to the catch block at the top of the function. Clean, fast, no boilerplate.
Closing
From a raw string through the Scanner's 39 token types, through the Parser's recursive descent building the AST, through the Resolver's compile-time binding and semantic checks, to the Interpreter's environment chain and recursive execution — a working interpreter gets built one layer at a time. Around 1600 lines of source, 1200 lines of tests, zero external dependencies.
The biggest takeaway from building this: interpreters aren't magic. The core engine is a few hundred lines of tree walking. Everything else — expression evaluation, scope management, function calls — is layering features on top of that kernel. After writing your own Scanner/Parser/Interpreter, you can open the CPython or V8 source code and immediately recognize which module does what.
Of course, this implementation is heavily stripped down. No JIT compilation (tree-walk interpretation is the slowest execution model), no bytecode generation, no garbage collector (reference counting via shared_ptr is the entire story), no tail-call optimization, no real error recovery (just the simplest panic-mode synchronize), no standard library beyond a single clock function. Industrial interpreters — CPython, V8, LuaJIT — invest hundreds of thousands of lines in these directions.
But that's exactly the point. By peeling away all the performance optimizations and engineering complexity, what's left is the raw, unvarnished four-layer pipeline that every interpreter shares. If you want to see what that looks like, the code is at https://github.com/Tenaryo/LoxInterp. Pull requests and nitpicks welcome.
Top comments (0)