DEV Community

Nathan C.
Nathan C.

Posted on

I Rewrote One Function in Squirrel, Then in C. One Line of Python Was Just as Fast.

I maintain a 2D game engine in Python called ABS Engine. Last week I decided it needed C.

Not because anything was slow. That is the embarrassing part. I wanted to drop a .c file into a folder and call it from Python with no build step anyone has to know about. It took 197 lines, most of them docstrings.

The first function I moved over was clamp. Keep a number between a low and a high. Three comparisons.

It had already been rewritten once. Two weeks earlier I had moved that same function out of Python and into Squirrel.

So the real history of clamp in this engine is Python, then Squirrel, then C, for a function whose entire body is three comparisons.

The Squirrel detour

The engine already embedded Tcl, because the editor is Tk and Tk is Tcl underneath. Squirrel came next, and clamp is what I used to find out whether an embedded VM was practical. Here is engine/nut/math.nut, in full:

function clamp(value, low, high) {
    if (value < low) {
        return low
    }

    if (value > high) {
        return high
    }

    return value
}
Enter fullscreen mode Exit fullscreen mode

And here is how Python called it:

return float(nut_call_function("clamp", float(value), float(low), float(high)))
Enter fullscreen mode Exit fullscreen mode

Read that line twice, because it gives the whole game away. Three float() calls on the way in. A lookup in the Squirrel root table. A trip through an interpreter loop to run three comparisons. One more float() coming back. The docstring above it, and I am quoting my own repository, proudly said *Implemented in Squirrel*.

The header is the contract

The rule I set for C was that adding a function should cost two files and zero configuration. Write geometry.c, write geometry.h next to it, call it. No setup.py entry, no CMake, no remembering to recompile.

Here is mathutil.h, all of it:

double clamp(double value, double low, double high);
Enter fullscreen mode Exit fullscreen mode

The loader hands that text straight to cffi:

ffibuilder.cdef(header_path.read_text(encoding="utf-8"))
ffibuilder.set_source(
    module_name,
    source_path.read_text(encoding="utf-8"),
    include_dirs=[str(C_DIR)],
    libraries=[] if sys.platform == "win32" else ["m"],
)
Enter fullscreen mode Exit fullscreen mode

That is the trick, and it is also the biggest gotcha in the project, so let me be loud about it. cdef is not a compiler and not a preprocessor. It parses a narrow subset of C declarations. An #include fails. Include guards fail, because #ifndef means nothing to it.

So headers here are declarations and nothing else. The .c file includes its own header normally, because the real compiler handles that file and is fine with all of it. Two readers, two sets of rules. The header is the part Python is allowed to see.

Rebuilds are decided by mtime: if a compiled module exists and is newer than both the .c and the .h, it gets imported, otherwise it gets rebuilt. functools.cache on top means you pay that check once per file per process. The practical effect is the thing I wanted. You edit the C, you hit Run, the new C is live. You never type the word "build."

One error message worth stealing. If the compile succeeds but the import fails, it almost always means the build folder holds a binary from a different Python, so the loader says exactly that and names the fix:

raise ModuleNotFoundError(
    f"Compiled {source_name}, but {module_name} could not be imported from "
    f"{BUILD_DIR}. Delete that directory to build it again."
) from e
Enter fullscreen mode Exit fullscreen mode

The part where I look at what I did

Here is the Python side after the C rewrite:

_clamp = c_source("mathutil.c").clamp


def clamp(value: float, low: float, high: float) -> float:
    return float(_clamp(value, low, high))
Enter fullscreen mode Exit fullscreen mode

The attribute lookup now happens once at import instead of once per call, and the three input float() conversions are gone because cffi coerces doubles itself. Against Squirrel this is not close and was never going to be. I replaced an interpreter loop with a direct call into compiled code. I felt great about this for about a day.

Then I wrote the version I had skipped past twice:

def clamp(value, low, high):
    return min(max(value, low), high)
Enter fullscreen mode Exit fullscreen mode

No VM. No FFI. No build directory. No header that cannot contain includes. No .so that breaks when the user upgrades Python.

Run it yourself, because the numbers depend on your machine, but the shape is not really in doubt. Squirrel loses badly. C and that one line land close enough that the difference is noise in a game loop. Crossing a language boundary costs roughly a fixed amount, and the work waiting on the other side is the only thing that pays it back. Three comparisons do not pay anything back. They are cheaper than the trip.

I spent two weeks and two language integrations optimizing a function whose body costs less than calling it.

Why I kept it

Because the point was never clamp. The road exists now: when something here genuinely needs C, the work is write the .c, write the .h, call it. Building that while the stakes are zero beats building it during a performance emergency.

What I would do differently is stop calling it an optimization while I was doing it. I was building a pipeline and telling myself I was making the engine fast. Both are fine things to do. They are not the same thing, and a docstring that advertised *Implemented in Squirrel* like a feature is proof I had them confused.

Four things that bit me

  1. Link the math library on POSIX and not on Windows. MSVC goes looking for m.lib and fails.
  2. Never commit the build directory. A .so built against 3.11 will not import on 3.12, and a Linux build is useless to a Windows user.
  3. Ship sources, not binaries. MANIFEST.in has recursive-include engine/c *.c *.h and nothing else. Everyone compiles on their own machine.
  4. CI has to build it everywhere. This engine tests on Linux, macOS, and Windows across three Python versions. C that only compiles on your laptop is a broken feature with good local results.

What goes in next

Broad phase AABB collision. A real loop over real data, every frame, which is exactly where crossing the boundary starts paying for itself.

Squirrel did not leave, by the way. The commit that deleted math.nut added anim.nut, which computes animation frame start times from a list of delays. That runs once when an animation loads, not three times per frame per entity. Much better fit, and I only knew to put it there because of the two weeks I spent getting clamp wrong.

The C loader is in engine/loaders/c_loader.py at github.com/Natuworkguy/ABS-Engine. Steal the pattern. Just benchmark against min(max(value, low), high) first.

Top comments (0)