After introducing the idea of writing pure C and pure Python in separate blocks inside a single file, I’ve spent more time mapping out exactly how the PCL tool-chain will transform, compile, and run that file—keeping Python as the primary runtime while harnessing native-code speed. Everything below happens automatically when you feed a .pcl file to the pclc command-line tool.
The compiler’s first job is lexical extraction. It scans for %c … %endc and %py … %endpy regions, preserving line numbers so error messages later can point back to the original source. Each C block is written verbatim to a temporary .c file; each Python block is buffered in memory. The metadata on the opening tag—name=mylib, export(add, Vec), requires(mylib), import(add, Vec)—is stored in a small JSON manifest that drives the rest of the pipeline.
Next comes native compilation. Because the goal is maximum portability, PCL leans on the tool-chain every developer already has: GCC on Linux, MinGW-w64 on Windows, and Clang or Apple Clang on macOS. The compiler invokes something like
gcc -shared -fPIC -O3 -Wall -Werror -o mylib.so mylib.c
(or the platform-specific equivalent), applying -fvisibility=hidden so that only the symbols named in export() remain visible. If multiple C blocks share the same name=, they are linked together into a single shared object; if they declare different names, each becomes its own library, letting one .pcl script spin up several independent native modules.
While GCC works on the C code, PCL’s wrapper generator parses the exported declarations. For simple functions and structs, it uses pycparser to produce an AST, then emits a thin ctypes layer. If you mark a block with cffi=yes, the generator will instead emit a cffi.FFI() interface for zero-copy pointer sharing. A future advanced flag (pybind=yes) will let power users demand C++ compilation with pybind11, perfect for templated or overloaded APIs.
Once the shared objects are built and the wrappers written, the compiler performs stitching. It prepends import lines to each extracted Python block that declared requires(mylib), then concatenates all Python segments (in their original order) into a single runnable script. That script is saved as pcl_main.py right beside the generated libraries. Running it is as simple as:
python pcl_main.py
Behind the scenes the wrappers call ctypes.CDLL("./mylib.so"), so the host interpreter remains the regular CPython you already know; no new runtime is required. Errors raised deep inside the C code are mapped back into Python exceptions automatically: integer return codes of 0/-1 convert to False/True where appropriate, and a helper macro can be used in C to propagate errno or custom messages into RuntimeError objects.
Because everything is assembled at build time, PCL also supports stand-alone packaging. A pclc package hello.pcl --onefile command can bundle the byte-compiled Python, the shared libraries, and a tiny bootstrap loader into a single executable via zipapp or PyInstaller—the result launches on a target machine even if Python isn’t installed.
The design leaves room for power-user switches:
--debug adds -g to GCC and keeps intermediate sources for stepping with gdb.
--sanitize inserts AddressSanitizer flags so memory bugs surface immediately.
--cythonize-after can pass the final stitched Python through Cython for pure-Python speedups without touching the C blocks.
--watch starts a file-watcher that re-parses & recompiles whenever you save, enabling a buttery REPL-like workflow.
Beyond those power-user switches, I’m also planning a --split workflow for anyone who prefers—or needs—to treat the two languages as first-class citizens in their own repos. With that flag enabled, the compiler doesn’t just hide its temporary work in a cache; it writes the extracted sources and artifacts alongside your project:
hello/
├─ hello.pcl
├─ build/
│ ├─ mylib.c # exact C code lifted from %c … %endc
│ ├─ mylib.so # compiled shared library
│ ├─ mylib_wrapper.py # auto-generated ctypes or cffi bindings
│ └─ pcl_main.py # stitched Python entry-point
└─ dist/
└─ hello_onefile.exe # if you invoked --onefile
This makes the generated C editable, auditable, and reusable—you can open it in your favourite IDE, run static analyzers, or even drop it into an existing CMake project. The Python side is equally transparent: import the wrapper directly, test it with pytest, or feed it into other tools like mypy for type checking (automatic .pyi stubs are on the roadmap).
“What if the user doesn’t have GCC or Clang installed?”
To cover that, I’m exploring an experimental self-contained tool-chain option: the installer would ship a trimmed, redistributable GCC (or LLVM) binary tucked inside the pclc package, similar to how Rust’s cargo bundles rustc. When the --portable flag is toggled, the compiler temporarily extracts this embedded tool-chain to a sandbox, compiles your C blocks, and then cleans up—ensuring reproducible builds on CI servers, classrooms, or locked-down machines where admin rights are scarce. It’s still early research (license compatibility and package size matter!), but the goal is a true “works-out-of-the-box” experience for absolute beginners.
if you’re passionate about portable compilers, build-system UX, or just love shaving yak-hair off cross-platform tool-chains, your expertise would be invaluable. Let’s collaborate and make PCL the most approachable bridge between Python and C yet.
Top comments (0)