Textparser – High-performance C parsing engine using Python-compiled grammars

Boris Barbulovski — Fri, 03 Jul 2026 21:20:35 +0000

Hi everyone,
I want to share textparser, a high-performance, lightweight text parsing and AST generation library written in pure C.

💡 The Core Idea & Why It's DifferentTraditional parser generators (like Flex/Bison) come with a steep learning curve and rigid code generation. On the other end, hand-writing recursive descent parsers or state machines becomes an unmaintainable mess as your language grows.textparser bridges this gap using a hybrid JSON + Python + C workflow:You define your tokens, syntax patterns, and color styles in a clean, human-readable JSON grammar file.
A lightweight Python compiler tool processes the JSON, runs optimizations, and emits a dense, static C header array.The C runtime engine loads these pre-compiled arrays instantly. It processes raw text strings into an abstract token tree (textparser_token_item) using a highly optimized regular expression engine (crpe2).By offloading the grammar overhead and heavy state-machine parsing logic to the Python build step, the actual runtime C library stays incredibly lean, memory-efficient, and fast.

🚀 Features At A Glance30+ Languages Out-of-the-Box: Includes ready-to-use JSON grammars for C, C++, Rust, Python, JavaScript, HTML, SQL, and dozens more.Rich Token Metadata: Every parsed token tracks exact code coordinates, structural flags, and custom syntax styling options.Zero Bloat: Ideal for terminal text editors, syntax highlighters, custom linters, and lightweight static analysis tools where bringing in a massive compiler front-end is overkill.

DEV Community: Boris Barbulovski

Textparser – High-performance C parsing engine using Python-compiled grammars