How I Made My Language 130 Faster by Transpiling to Python AST

When I started building SkyForge, a Russian-syntax programming language, I wrote a classic tree-walking interpreter in Python. Lexer, parser, AST, recursive evaluation. It worked.

It was also 30-100× slower than CPython.

fib(32) took 60 seconds. Not "a bit slow" — 60 seconds for a computation that plain Python does in 0.4. For a language that's supposed to be easier than Python, that's embarrassing.

I considered a few options:

• Rewrite in C — months of work, and I'd lose the Python bridge
• Use Cython or Nuitka — helps, but doesn't fix the architectural problem
• Write a real compiler — years
• Transpile to Python AST — I could do this in a weekend

I chose the last one. It worked better than I expected. This post is about how.

The Core Idea

Python's standard library has an ast module. You can build a Python AST by hand and pass it to compile(), which turns it into native CPython bytecode. That bytecode runs at C speed — no interpreter overhead.

So instead of walking my own AST and evaluating each node, I translate my AST into Python's AST and let CPython do the rest:

app.skf → my Lexer → my Parser → my AST
↓
my Transpiler
↓
Python AST
↓
compile()
↓
CPython bytecode

The whole transpiler is ~600 lines of Python. No external dependencies.

Here's what the pipeline looks like in practice:

def compile_skf(source: str, filename: str):
    tokens = Lexer(source).tokenize()
    program = Parser(tokens).parse()
    py_ast = transpile(program)
    return compile(py_ast, filename, "exec")

That's it. Three lines from source to bytecode.

A Concrete Example

Consider this SkyForge function:

функция сумма(a, b) {
    вернуть a + b
}

My parser produces a FunctionDef node with name="сумма", params=[("a", None, None), ("b", None, None)], and a body containing one Return with a Binary node inside.

The transpiler's func_def method converts this to:

ast.FunctionDef(
    name="сумма",
    args=ast.arguments(
        posonlyargs=[],
        args=[ast.arg(arg="a"), ast.arg(arg="b")],
        kwonlyargs=[],
        kw_defaults=[],
        defaults=[],
        vararg=None, kwarg=None,
    ),
    body=[
        ast.Return(
            value=ast.BinOp(
                left=ast.Name(id="a", ctx=ast.Load()),
                op=ast.Add(),
                right=ast.Name(id="b", ctx=ast.Load()),
            )
        )
    ],
    decorator_list=[],
    returns=None,
)

Tedious to write, but mechanical. Once you have a method per AST node type, you're done.

What I Learned the Hard Way

Python 3.12+ strictly validates AST positions

In older Pythons, you could skip lineno and end_lineno — the compiler would fill them in. Starting in 3.12, compile() refuses if end_lineno < lineno for any node, or if a child's range is outside the parent's.

I hit this constantly because I was setting lineno on some nodes manually and letting fix_missing_locations handle others. Result:

ValueError: AST node line range (2, 1) is not valid

The fix is brutal but effective — zero out all positions before compiling:

for node in ast.walk(module):
    for attr, val in (("lineno", 1), ("col_offset", 0),
                      ("end_lineno", 1), ("end_col_offset", 0)):
        try:
            setattr(node, attr, val)
        except (AttributeError, TypeError):
            pass
return ast.fix_missing_locations(module)

You lose useful tracebacks — everything points to line 1 — but the code runs. For a transpiler that doesn't need accurate error positions, it's a fine trade-off.

ast.Compare is not ast.BinOp

This one cost me an hour. My first binary() method put everything through ast.BinOp, including comparisons:

op_map = {
    "+": ast.Add(), "-": ast.Sub(),
    "==": ast.Eq(),  # ← wrong
    ">": ast.Gt(),   # ← wrong
}
return ast.BinOp(left=..., op=op_map[op], right=...)

Python's compiler rejected it with TypeError: expected some sort of operator, but got LtE(). BinOp accepts arithmetic operators only. Comparisons require ast.Compare, which has a different shape — it takes a list of ops and a list of comparators, because Python supports chained comparisons like a < b < c:

ast.Compare(
    left=...,
    ops=[ast.LtE()],
    comparators=[...],
)

Once I split arithmetic and comparison into separate branches, this worked.

ast.Lambda is a single expression, not a block

My language allows lambdas like this:

пусть удвоить = функция(x) { вернуть x * 2 }

The body has a Return statement inside. But ast.Lambda doesn't accept statements — only an expression:

ast.Lambda(
    args=...,
    body=ast.BinOp(...),  # ← just the expression
)

The fix is to unwrap the Return and extract its value. If someone writes a multi-statement lambda, only the first statement survives — which is a documented limitation of the language, not a bug.

Smart + is not +

SkyForge has "smart addition": if either operand is a string, coerce both to string. So "Age: " + 25 produces "Age: 25" instead of a TypeError.

In Python, this is just +. So when transpiling, I can't always emit plain ast.Add(). Instead:

if op == "+":
    # Fast path: both literals
    if isinstance(n.left, Number) and isinstance(n.right, Number):
        return ast.BinOp(left=..., op=ast.Add(), right=...)
    # Slow path: runtime coercion
    return ast.Call(
        func=ast.Name(id="__sf_add", ctx=ast.Load()),
        args=[..., ...],
        keywords=[],
    )

Where __sf_add is injected into the runtime globals before execution:

def _sf_add(l, r):
    if isinstance(l, str) or isinstance(r, str):
        return str(l) + str(r)
    return l + r

This is where transpilation gets tricky — you're paying runtime cost for a static decision. But the fast path covers the common case (arithmetic between numbers), and the slow path only fires when strings are involved.

Benchmarks

Real numbers from my laptop (Windows, Python 3.14):

Test	Interpreter	Transpiler	Speedup
fib(24) — 46 000 recursive calls	1072 ms	8 ms	134×
fib(32) — 2 178 309 calls	~60 s	371 ms	~160×
List comprehensions + string interpolation	~200 ms	4 ms	~50×

For reference, plain CPython running the same fib(32) takes ~350 ms. The transpiled version is 371 ms — within 6% of native Python speed. That's not a coincidence; it is native Python by the time compile() is done.

On flat code — reading a file, parsing JSON, calling a function once — the speedup is smaller, maybe 10-30×, because you spend more time in the runtime than in control flow. But for anything loop-heavy or recursive, the difference is dramatic.

What Doesn't Work (Yet)

The transpiler covers maybe 70% of the language. What's missing:

• Classes with это (self) and супер (super)
• Async / await
• Decorators — @кешировать, @тест
• Pattern matching (совпадает)
• Generators (выдать)
• Web applications — the runtime loop doesn't fit the compile model

For these, I fall back to the interpreter. The user chooses:

SkyForge run app.skf          # interpreter, all features, slow
SkyForge run app.skf --fast   # transpiler, fast, partial support

Adding classes and async is next on the roadmap. Async especially — I know how to do it (async def, await, run in an event loop) but need to handle the interaction with the interpreter's threading model.

Was It Worth It?

Absolutely. The entire transpiler is ~600 lines of Python. It took a weekend to write and a few more days to debug. In exchange, I got:

• A 130× speedup on compute-heavy tasks
• No new dependencies — everything uses stdlib
• No C code — maintains my Python bridge for free
• A learning opportunity — I now understand Python's AST module better than I ever wanted to

The trade-off is honest: users have to know which mode to use. But that's a documentation problem, not an architecture problem.

Takeaways

If you're building a language on top of Python and hitting the interpreter speed wall, try transpiling. It's less work than you think.

Three rules that saved me time:

1. Zero out AST positions before compiling. Don't try to be clever with lineno.
2. Read the Python ast module source. Half of it is docstrings, but the type signatures tell you everything you need to know.
3. Write a compile-check before every file write. compile(source, "<test>", "exec") catches 90% of transpiler bugs before you run anything.

The full transpiler is open source at github.com/skytech-alt/SkyForge-Docs. If you want to see the pattern for your own language, skyforge/transpiler.py is the file to read.

Happy to answer questions about the design or help debug your own AST-to-AST translations. Reach out in the comments or open an issue.