DEV Community: Flude team

The Configuration Trap: How Enterprise Solutions Become SaaS Vulnerabilities

Flude team — Fri, 09 Oct 2026 13:15:44 +0000

After deciding to split the product into Enterprise and SaaS, we started designing the cloud architecture. During the planning phase, the task seemed straightforward: spin up an API, configure container-workers in the cloud, and simply pass the code to them for processing.

However, during a detailed security audit, we discovered that architectural patterns which worked flawlessly in a trusted corporate environment turn a public SaaS into an open hacking sandbox.

Anatomy of a Vulnerability: RCE via Configuration

In the Enterprise version, the configuration logic was maximally flexible: the engine simply read the flude_*_config.json configuration file directly from the client's repository.

The problem lay in exactly how these parameters were translated into parser settings under the hood. Our pipeline relied on Doxygen, which we extended with our own scripts to bypass C++ bugs. To make these filters work, the engine uses a standard Doxygen directive—INPUT_FILTER, which allows executing arbitrary shell commands for processed files.

In an isolated client perimeter, reading settings from the repository was absolutely safe. But when transferring this logic to a public cloud (GCP Cloud Run), a critical vector for Remote Code Execution (RCE) emerged. If a server blindly trusts a flude_*_config.json file from a third-party repository, an attacker can pass through overrides for parser parameters. By injecting a command like rm -rf / or a reverse-shell via INPUT_FILTER substitution, an attacker would force our system to legitimately execute this code on our own servers.

Zero-Trust Architecture and Breaking the Trust

Realizing this risk forced us to completely rethink our approach to configuration. In a public cloud, you cannot trust a single line coming from the outside. We implemented a strict Zero-Trust principle:

Abandoning Client Configurations: The SaaS version will never read flude_*_config.json from a user's repository.
Server Templates: Our cloud worker operates exclusively on hardcoded, versioned server templates for Doxyfiles and configurations.
Secure Parameter Passing: A user can only pass a strictly limited set of safe data-parameters (e.g., directories to include or exclude) via the API, which are rigidly validated on the Control Plane side. Control over which tools and filters run in the container has been permanently handed over to the server.

Fortifying the Perimeter: SSRF and Workload Identity

Having plugged the RCE hole, we turned our attention to network security. Cloning user repositories by a server is a classic Server-Side Request Forgery (SSRF) risk. To prevent attackers from forcing our workers to scan the internal GCP network under the guise of a git repository, we implemented a strict Allowlist: cloning is allowed only via HTTPS and strictly from the domains github.com, gitlab.com, and bitbucket.org.

Additionally, to minimize damage in the hypothetical event of a container breach, we completely abandoned static keys (Service Account Keys), transitioning all workers to dynamic authentication via GCP Workload Identity Federation.

Deferred Threats: Attack via Source Code

Having secured the configuration and network, we asked the next question: what if the attack vector becomes the very source code we are parsing?

We analyzed several highly specific scenarios:

Doxygen Tag Abuse: An attacker could write standard tags like \include, \verbinclude, or \htmlinclude into a code comment with a path like ../../../etc/passwd. These will force the parser to read a file outside the project directory and insert its contents into the final documentation, which is then returned to the client. Our strict directory validation is powerless here, as it only checks the declared input paths, not what Doxygen decides to read during the parsing process.
Malicious Symlinks: If an attacker commits a symlink pointing to system files of the container, and the parser follows it while traversing directories—this is another way to extract data outside the repository sandbox.
Denial of Service (DoS): In our system, the call to the Doxygen binary is limited to 60 seconds, but the parent flude_engine compile process had no timeout. A specially crafted source file (with infinite template recursion or explosive macro expansion) could permanently hang the cloud worker process.

After considering these vectors, we made a conscious decision to defer fixing them.

First, architecturally it was unclear at what level to implement these defenses—especially in light of the upcoming introduction of native parsers. We honestly recorded this debt in the project roadmap under the "Input Validation & Sanitization" section.

Second, the actual risk was significantly reduced thanks to the Data Plane architecture. All workers in Cloud Run are isolated via gVisor, and the containers themselves are absolutely ephemeral. Even in the event of a successful exploit of a symlink or an \include tag, an attacker would read files from a disposable, sterile environment without access to neighboring tasks or the host.

The Accumulating Dissatisfaction with Doxygen

that the incidents with potential RCE via INPUT_FILTER and the vulnerability of standard tags like \include became more than just cloud security issues. They were another wake-up call reminding us of Doxygen's fundamental architectural flaws. Its inflexibility, reliance on shell calls, and poor performance made it a burden for a scalable SaaS solution (and for the architecture as a whole).

Running ahead, we can say: we did not put up with this technical debt. In a couple of weeks, as soon as we finish the series of articles on SaaS infrastructure, we will explain in detail why we finally fired Doxygen.

But before that, we had to solve another pressing cloud problem. We had designed a secure Data Plane, protected from hackers, but now we needed to protect it from bankruptcy. How we built a Free-Tier, protected the API from abuse loads, and designed billing—read about it in the next episode.

Originally published on our blog: https://blog.flude.guide/blog/zero-trust-cloud-config-trap

Also read us:

A Tale of Two Fludes: The Architectural Split of Enterprise and SaaS Models

Flude team — Tue, 06 Oct 2026 06:06:10 +0000

Extracting specific code into an independent private submodule, as described in the previous article, allowed us to stabilize the engine's core. The legal boundaries were drawn, and the base architecture—a smart parser and two-level caching—proved its effectiveness.

The next logical step was transitioning from an internal tool to a commercial product. However, at this stage, we faced a fundamental contradiction between the corporate operational model and the demands of the mass market.

The Illusion of Universal Distribution

Initially, Flude's architecture was shaped by the needs of a large corporation. Security requirements in such environments are categorical: the tool must function within a closed perimeter, guaranteeing that the source code never leaves the company's servers.

To solve this, we integrated directly. A package with scripts was handed over to the client and run locally in their CI pipeline, reading configuration files directly from the repository. Within a corporate environment, this delivery model (whether source or bytecode) is ideal: the closed perimeter and strict legal obligations reliably guarantee the safety of intellectual property.

However, when planning to scale to the public market, it became obvious that directly copying the Enterprise distribution approach was technically and conceptually inapplicable.

Losing Control over Distribution

The closed-contract model works perfectly in the Enterprise sector, but to ensure stable product growth, we needed to enter the mass B2B/B2C market: small teams and independent developers.

The obvious problem was that handing over source Python scripts or a ready-made Docker image to users outside of corporate agreements inevitably leads to losing control over product distribution. Without strict legal barriers, open distribution technically allows bypassing licensing mechanisms and uncontrollably replicating the system's core.

Consequently, the approach that is standard for a trusted corporate environment turned out to be structurally incompatible with a public business model.

The Divergence

To protect the product's core, we had to rethink the delivery paradigm itself. Flude's architecture was physically divided into two independent models:

Enterprise Flude: An autonomous, local solution. Delivered exclusively under targeted contracts. Functions offline, processes code locally, and integrates with the configuration directly in the client's repository.
SaaS Flude (Public Market): A cloud-oriented model. The parser's core is never handed over to end users. Instead, a "thin client" (lightweight CI scripts) is provided, whose sole task is packing source files and securely transmitting them via API. All computational load and parsing logic remain encapsulated on our servers.

Moving to the Cloud and the Unexpected Benefit of Binaries

By adopting the SaaS model, we secured our algorithms, but significantly complicated the technical implementation. We had to design a distributed system from scratch capable of receiving and processing third-party code.

Interestingly, it was at this stage that the concept of a "single binary", which we had rejected for public distribution, turned out to be our own salvation. To avoid managing complex Python environments and dependencies on every cloud worker, we compiled our core. A standalone binary became the perfect solution for quick startups inside our own SaaS infrastructure (and later, a convenient delivery format for Enterprise clients).

How we designed the cloud with the Zero-Trust concept in mind, and why configuration files that worked flawlessly locally turned into a critical attack vector (RCE) in SaaS, we'll tell you in the next episode.

Originally published on our blog: https://blog.flude.guide/blog/tale-of-two-fludes

Also read us:

Clean Room: A Refactor That Answered a Legal Question, Not an Architectural One

Flude team — Fri, 02 Oct 2026 04:34:36 +0000

Code born inside the core engine rarely has to stay there forever. This specific move started with a question about legal boundaries. We needed a hard line between our own original logic and knowledge acquired from an external format. One of Flude's renderer families—the one built to precisely mirror an outside documentation tool—sat dangerously close to that line.

Two Risks a Refactor-in-Place Misses

Carving out a module inside the same repository does practically nothing here. A shared commit history leaves the biggest vulnerabilities wide open.

Authorship is the first major headache. Code mirroring an external format gets thoroughly mixed up with the rest of the engine. Proving independent development after the fact becomes incredibly messy. You build a credible evidence trail through total separation—a dedicated repository and a fresh commit history.

The logic itself carries its own risks. The way code bends to match an external format is highly specific knowledge. Tightly weaving it into the engine blurs the line between a general tool and client-specific insights.

A physical split cleanly shuts down both risks. A simple folder rename just papers over the cracks.

How It Played Out in Practice

We pulled the renderer family into a standalone git repository. It plugs into the main project as a private submodule. The engine interacts with it through the standard renderer interface. The class simply lives in a new zip code. The rest of the system barely noticed the change.

Standing up the new repository brought plenty of friction. The test harness relies heavily on the engine's core models. We had to check out the main engine inside the new CI pipeline. Shared relative paths completely broke down. Nailing the dependency paths took a few frustrating attempts. We temporarily dropped the markdownlint gate before restoring it to maintain project standards. Turning the CI pipeline green took some serious trial and error.

The quality checks moved right alongside the code. The entire infrastructure for comparing results against reference documents shifted over. Leaving it behind would have made it impossible to spot rendering drift.

The Result

The main engine feels much leaner now. The core and the remaining renderer family can evolve at their own natural pace. The extracted code operates completely independently. It boasts its own CI and a bulletproof line of provenance. We no longer have to worry about proving its origins retroactively.

Originally published on our blog: https://blog.flude.guide/blog/clean-room-private-plugin

Also read us:

Ghost in the Wire: Our Two-Day Hunt for a Network Bug That Didn't Exist

Flude team — Tue, 29 Sep 2026 03:38:53 +0000

Our self-hosted runner serves CI for five repositories on one physical VM. It started acting completely crazy. The engine job sat queued in GitHub Actions for over a day. It just hung there doing absolutely nothing. The supervisor meant to pick it up was restarting itself every two and a half to three minutes around the clock. We called this endless loop "the carousel" and wasted two days chasing ghosts.

The Symptom, and the First False Lead

We already had a documented network bug in this exact infrastructure. A classic VirtualBox NAT "black hole" used to kill TCP connections inside the guest VM silently. The guest OS would consider the connection alive forever. We tracked down and closed that issue a week and a half earlier.

When the supervisor started hanging on the host side, our reflexes kicked in. We assumed it was the exact same disease in a fresh disguise. That feeling was convincing enough to steer the whole investigation sideways for a full day. The watchdog detected the hang, killed the process, and brought it back up. Every new instance hung reliably after 150-190 seconds. That unnatural precision should have made us suspicious way earlier.

Escalation: Packets, Memory Dumps, Power Settings

A thorough and completely useless sweep through network theories followed. We disabled a third-party VirtualBox NDIS filter attached to the Wi-Fi adapter. We reset the MIMO power-saving settings. Nothing made a dent in the problem.

Running pktmon during an actual hang showed a completely healthy network. New TCP connections to GitHub established in 60 milliseconds. The "hung" process supposedly couldn't send a single byte. We wrongly concluded the hang was happening before the packet got sent.

A memory dump of the process via WinDbg showed twenty-two threads. None of them were sitting inside an HTTP call. We spotted suspicious modules like antivirus drivers and crypt32 certificate validators. Every single one of these plausible candidates turned out to be a dead end.

We did notice one genuinely interesting detail. The host machine dropped into Modern Standby right when the hangs occurred. We disabled sleep and screen timeouts entirely. The restarts kept happening on the exact same schedule. The correlation was totally real—but completely unrelated.

A Second Pair of Eyes Solves Half the Problem

Another session looked at the logs and caught what we missed. The watchdog was treating plain silence as a fatal hang. The GitHub API poll writes nothing when the queue is empty. It simply sleeps for twenty seconds and tries again. A few quiet cycles looked exactly like a dead process to the watchdog.

We verified this by pulling the I/O counters directly. The bytes were genuinely moving. The process was perfectly healthy with nothing to log. We updated the watchdog to check those exact counters before killing anything. The restarts stopped completely. The engine job still refused to dispatch.

The Real Cause

We ran a direct test using the exact token the supervisor uses. Querying runs?status=queued for engine returned zero results. Running the same query with a personal token showed the job instantly. Access rights were perfectly fine.

The answer was buried in the API response itself. The run had a "pending" status. The job inside that run was correctly marked as "queued". Our code filtered runs at the top level. GitHub had split the run's status from the job's status. One bad line of filtering made the code completely blind to a waiting job.

We dropped the top-level filter completely. The nested check handled everything perfectly on its own. The very first poll picked up the stuck jobs and ran the full suite beautifully.

A Small Postscript: Fixing Our Own Fix

Removing the filter spawned a new headache. The code started pulling dozens of recent runs and making separate job requests for each one. The call volume spiked massively. The carousel of restarts came back because I/O genuinely dropped to zero. We added client-side filtering to skip runs marked as completed. Everything stabilized immediately.

What We're Taking Away From This

Solid diagnostic tools will gladly prove the wrong theory. WinDbg and packet sniffers correctly showed a healthy network. We just kept interpreting those results to fit our initial bias. A simple API call with the right token proved infinitely more useful than all the heavy debugging gear.

Next time, we get into legal boundaries and clean room design. We ended up moving an entire renderer family into a private plugin. It was the only way to firmly separate our core engine from logic shaped by an external client format.

Originally published on our blog: https://blog.flude.guide/blog/ghost-in-the-wire

Also read us:

Growing Pains with Five Repositories: Gitlinks and Dual CI

Flude team — Fri, 25 Sep 2026 04:02:27 +0000

Usually, infrastructure stories start with the words: "Our project got too big, and we decided to split it into parts." But with us, everything was different.

From the very first minute of the project, we consciously chose an architecture consisting of a scatter of independent repositories. The Pipeline, engine, design-docs, and user-docs projects were created almost simultaneously, and the .gitmodules configuration file sat in the very first commit of the parent repository.

For the sake of code cleanliness, CI isolation, independent history, and separation of access rights, we built the system on Git Submodules from day one. The idea looked flawless on paper. But here is what we had to pay for it in practice.

The Gitlink Trap: The Ghost of a Local Commit

The first price we paid was the non-obvious mechanics of state synchronization. When you use submodules, the workflow demands strict discipline. If you edit code inside engine/ (while physically located in the folder of the parent repository Pipeline), you need to make two pushes. First, a git push from the folder of the submodule itself, and then, after moving up to the root of Pipeline—you must commit the updated pointer (gitlink) to the new engine commit and push Pipeline itself.

It is incredibly easy to forget the first push. On your local disk, the commit physically exists. Everything builds, the tests are green, and Pipeline happily goes to the cloud. But this only surfaces on a fresh checkout in CI. The git submodule update --init command fails with the text:

fatal: remote error: upload-pack: not our ref <sha>

ext

GitHub honestly replies: "This commit does not exist in the remote repository."

One day, three submodules fell into this trap simultaneously (design-docs, engine, user-docs). The cause was identical—small commits were made right inside the submodule folders, the gitlink in Pipeline was synchronized, but the local commits never left the developer's laptop.

There was also a colorful nuance: the fourth submodule, ude_promotion (which hosts the site), broke so badly that a normal push couldn't fix it—its local branch fell so far behind upstream that an interactive rebase was required before pushing.

Dual CI: One Code in Two Personalities

The second reckoning came from CI. The engine repository (our core) had two personalities. On the one hand, it was a standalone repository with its own build process, which checked out only this repository. On the other—it was a submodule inside Pipeline, which was tested in the context of the entire parent project.

These two personalities inevitably clashed, and this happened in two different stages.

The First Strike: Missing Dependencies
The simplest case. The test_integration_scripts.py test tried to reach the verify_pages/check_links modules, which existed only in the parent Pipeline repository. During a standalone checkout of engine, these files simply weren't on the disk. We fixed it roughly but effectively—we added a single line to the ci.yml configuration, strictly ignoring this test from the standalone run (pytest --ignore=...). But this was a one-off patch, not a systemic solution.

The Fatal Directory Ascent
Soon the bug returned in a new guise. One by one, tests started failing that used a classic trick to find parent artifacts:

Path(__file__).resolve().parents[2]

ext

In a nested run, this code ascended exactly to the root of the parent repository. But in the standalone build of engine, it jumped above the root of the repository, straight into the system filesystem of the CI runner, and crashed with a hard AssertionError.

It was after the second occurrence of this bug that we realized point patches no longer worked. We needed a general convention.

We introduced an elegant marker hack: tests began checking for the presence of our system marker directory (let's call it .workspace_config) at the ascended level. If the directory is there—it means we are in a nested context, and the tests run. If it's not there (standalone)—we politely use pytest.mark.skipif, and the tests are quietly skipped without failing with a red status.

Conclusion

A Multi-repo architecture does not give you magical "loose coupling" for free on day one.

It demands flawless push synchronization discipline and forces you to design code so that it is ready to work in several different execution contexts simultaneously. These lessons cost us broken pipelines, but without them, the system would not have survived further growth.

But the problems with Git and dual CI turned out to be just a prelude. The real mysticism began when the network intervened, and our pipeline started crashing because of a "ghost" in the wires. How we investigated an elusive network bug—read about it in the next episode.

Originally published on our blog: https://blog.flude.guide/blog/multi-repo-gitlinks-dual-ci

Also read us:

Escaping the Cloud: How Five Repositories Stopped Sharing One Minute Limit

Flude team — Tue, 22 Sep 2026 04:13:31 +0000

Splitting the monorepo granted each of our five components an independent continuous integration pipeline. Architecturally, we achieved perfect component isolation. The problem hid in the strict limits of free GitHub Actions minutes. A unified build line used to efficiently compile everything in a single run. Five separate pipelines began burning through our starting two thousand minutes like wildfire by launching independent jobs for every tiny commit.

One Physical Machine for Five Clients

We flatly refused to buy extra cloud minutes and deployed our own self-hosted runner on a standard Windows office laptop. Inside ran an Ubuntu virtual machine powered by VirtualBox. This single virtual machine methodically took turns servicing tasks for all five repositories.

Official GitHub documentation heavily advises against attaching local runners to public projects due to remote code execution risks via malicious pull requests. Our repositories are completely private, keeping us safe from outside attacks. We nevertheless implemented paranoid hygiene to guarantee build reproducibility. A single runner processes code from five different projects, and leftover file debris from a previous build could easily ruin the next one.

We isolated the guest system through classic NAT (nic1=nat), blocking its access to the host's local network. Before every new job, a script ruthlessly rolled the virtual machine back to a clean snapshot. The runner itself launched with the --ephemeral flag for one-time registration, while an additional Unregister-OrphanedRunner.ps1 script scrubbed hanging records via the API.

The Price of Genuine Isolation with a Custom Supervisor

Many wonder why we wrote a custom polling script when GitHub offers built-in Organization Runners. The standard mechanism certainly allows sharing runner pools across multiple repositories. It completely fails at managing the lifecycle of the virtual environments themselves.

The --ephemeral flag successfully removes a runner from the pool after task completion. It does nothing to restore the filesystem to its original pristine state. To wrap every single task in a strict cycle of snapshot rollbacks and VM launches, we still needed an external controller. The custom REST API polling script became a mandatory tax for ensuring a clean environment for every build.

The Watchdog Monitoring Itself

The single server became a critical point of failure for the entire project. Any freeze in the supervisor instantly paralyzed code checks and documentation publishing. For the first few weeks the script ran unsupervised and regularly demanded manual restarts whenever VirtualBox spawned zombie processes. We wrote a separate watchdog script tasked with automatically restarting the frozen controller.

The logic was primitive: if the supervisor.log file stopped updating and the VBoxHeadless process wasn't running, the watchdog pulled the plug. We shot ourselves in the foot on day one. The watchdog diligently wrote its own diagnostic logs into the very same supervisor.log file. It constantly updated the file modification time, reliably blocking its own check condition.

We quickly separated the logs into different files, and the system appeared to run smoothly. The following morning the watchdog lost its mind and began systematically killing a perfectly healthy supervisor for no apparent reason. This sent us down a two-day detective investigation into a network ghost in the machine, which we will cover in the next part.

Originally published on our blog: https://blog.flude.guide/blog/escaping-to-self-hosted-runners

Also read us:

The Phantom That Wasn't There: How Unreliable grep Made Us Chase Ghosts

Flude team — Fri, 18 Sep 2026 04:58:58 +0000

One of the hardest stages in developing a documentation generator is alignment with legacy tools. When you migrate from an old, decades-proven proprietary generator to your own custom pipeline, you are obligated to prove that you haven't lost a single class.

We wrote an automated system that compared thousands of entities generated by the old tool with what our new Flude-based pipeline found. And one day, we stumbled upon a terrifying anomaly.

Anomaly in the Matrix

The alignment script happily reported: Flude found and generated documentation for a struct named GraphicsAsset and its smart pointer GraphicsAssetPtr.

The problem was that in the old, reference documentation, these entities didn't exist. At all.

According to our logic, if the legacy generator didn't find the class, it's highly likely this class simply doesn't exist, and our parser made a mistake somewhere, "gluing" someone else's names together. We had to check the source code.

Zero Results

We did what any engineer would do—we opened a terminal and ran a quick global text search (grep / ripgrep) across the entire multi-gigabyte codebase of our enterprise C++ product:

rg "GraphicsAsset" ./sdk_sources/

Result: 0 matches. Absolute emptiness.

We checked with case-insensitivity, we searched for the smart pointer. Nothing.

Panic: The Hallucinating Parser

It was a moment of genuine horror. The codebase didn't contain the string GraphicsAsset. But somehow, our parser hadn't just found this name; it had built a beautiful HTML page for it, complete with a list of methods, properties, and inheritance!

The conclusion seemed obvious: our parser had gone insane. It had started fabricating C++ classes out of thin air. Perhaps the logic for expanding complex macros had broken, or Doxygen's cache had become so corrupted that it started stitching pieces of different files into a single "Frankenstein".

We were already preparing to open the debugger and rewrite the token-linking logic in the core.

Autopsy of the Cache

Fortunately, before tearing apart the core codebase, we decided to look at the raw intermediate data. We went to the directory where Doxygen dumped its low-level XML, and searched there.

Suddenly, sitting quietly on the disk, was a file named struct_graphics_asset.xml. We opened it.

Inside was perfectly formed XML. It didn't look like a hallucination. Moreover, in the <location> tag, the parser pedantically pointed out exactly where it got this class:
Core/Include/Graphics/MaterialConfig.h:3006-3061

The Phantom Takes Flesh

We opened the MaterialConfig.h file and scrolled down to line 3006.

And there, in the very heart of a massive legacy header, hid a completely real, living struct:

struct CORE_ABSTRACT GRAPHICS_EXPORT GraphicsAsset {
    // ... dozens of lines of valid C++ code
};

It had always been there. The legacy generator simply couldn't parse it because of complex macros, but our Flude could.

So why did the global rg yield zero matches? In enterprise monoliths, there are always plenty of reasons. Perhaps the Core/Include path was listed in a tricky .gitignore, and the smart ripgrep ignored it. Perhaps the file had an exotic Windows encoding that broke the search. Perhaps it was a mounted submodule.

Conclusion

A global text search offered an absence of evidence as evidence of absence.

This case taught us a crucial rule: a text search is not a C++ parser. Trying to analyze the structure of a C++ monolith with regular expressions or grep is a path to disaster and false conclusions.

If you doubt your parser, never use text search as the arbiter. Trust only the raw dumps of the compiler itself or the Abstract Syntax Tree (AST). And most importantly: do not try to "fix" a bug in the core until you prove its existence in the actual source of truth.

But parsing and cache problems paled in comparison to our infrastructural challenges. From day one, our project was built as an ecosystem of several independent repositories. How this architecture turned into Git traps and nearly broke our Continuous Integration—read about it in the next episode.

Originally published on our blog: https://blog.flude.guide/blog/the-phantom-bug-grep

Also read us:

The Leak That Won't Die: How We Broke Python's tempfile

Flude team — Tue, 15 Sep 2026 03:52:53 +0000

Production crashes always happen suddenly. This time, the alarm was raised by the core C++ SDK developers. Their scheduled nightly build failed on the CI runner with a classic diagnosis: No space left on device.

We ran out of disk space. A quick post-mortem revealed that the /tmp folder was bursting with thousands of temporary directories. Their names all started the same way: ude_xml_. It was our code.

The Architecture of the Crash

As we've mentioned before, our engine handles Doxygen in a tricky way. We feed it source code, it spits out a massive XML dump, and we parse that dump. For every run, the engine created a temporary folder using tempfile.mkdtemp(). A dynamically generated Doxyfile was placed there, and Doxygen would dump hundreds of megabytes of XML into it.

The plan was simple: at the very end of the pipeline, a cleanup() function would run, executing an honest shutil.rmtree() on that folder. But the plan had a flaw. If the parsing failed with an exception, the pipeline aborted, cleanup() was never called, and a folder filled with megabytes of garbage was left stranded in /tmp forever.

Complaints started pouring in from teams building massive projects. The error was flaky. We tried wrapping the call in a try/finally block, but the engine's logic was smeared across dozens of classes. The leak persisted.

A False Sense of Security: Trusting the GC

How do you solve the problem of forgotten resource cleanup in Python? "Delegate the cleanup to the Garbage Collector (GC)!" we thought.

Instead of mkdtemp, we rewrote the logic to use the tempfile.TemporaryDirectory object. Its main advantage is that when the object is deleted from memory, its finalizer (__del__) automatically triggers the folder's deletion on disk.

To ensure the folder lived exactly as long as the engine needed it, we saved a reference to the object in a global list inside the manager class:

temp_dir_obj = tempfile.TemporaryDirectory(prefix="ude_xml_")
self._active_temp_dirs.append(temp_dir_obj)

It seemed like elegant, "pythonic" code. The object lives in the manager's memory and is automatically deleted when the script finishes.

The Fatal Mistake

We rolled out this "fix" to production. For a few days, the complaints stopped. We patted ourselves on the back and closed the issue. And then the CI runners started crashing again.

We missed one "tiny" detail. Hoping that objects are always deleted, and that the Garbage Collector works immediately, is a ticking time bomb.

In the case of giant C++ projects like Flude, the engine parsed dozens of modules sequentially. A new directory was created for each module. Because of a complex web of circular references in our parsers (e.g., an AST node referenced its parent class, and the class held a list of its children), the engine's objects weren't deleted immediately. They hung in memory, waiting for a full garbage collection cycle. And, of course, _active_temp_dirs stayed in memory too.

TemporaryDirectory objects piled up in RAM by the hundreds. But the worst part was this: when the GC finally got to them and called cleanup(), it triggered a massive synchronous deletion of files from the disk, blocking the main execution thread for several minutes. Because of this, other services on the server began failing due to timeouts.

The Real Fix

The conclusion of our investigation was grim: using finalizers to free heavy system resources is an anti-pattern. We couldn't control when the GC would trigger, which meant we couldn't control the disk I/O load.

we had to rewrite the pipeline's architecture so that the lifecycle of temporary folders was managed by strict with context managers, not the GC.

We introduced a strict rule: creating the folder and calling cleanup() must happen in the same place. If any part of the engine needs these files, it accesses them inside a with block, guaranteeing that upon exiting the block (even via an exception), the folder is deleted immediately.

To protect against regressions, we also added a "janitor"—a mechanism that forcibly checks /tmp before a build starts and cleans up any orphaned garbage.

logger.warning(
    f"Orphaned temporary directory found: {td.name}. "
    "Caller failed to invoke cleanup(). Force-cleaning now."
)

Conclusion

We learned a harsh lesson. Shifting the responsibility for system resources to Python's garbage collector is a direct path to disaster.

But our victory over the garbage was just the beginning. While we were wrestling with Python, the real enemy was lurking in the very heart of our infrastructure. We thought we had tamed Doxygen, but it soon took its revenge, forcing us to rewrite the entire parser from scratch. How a simple C++ construct drove the documentation generator insane—read about it in the next episode.

Originally published on our blog: https://blog.flude.guide/blog/the-leak-that-wont-die

Also read us:

The Illusion of Speed: A Two-Level Cache Can't Speed Up What Doxygen Can't Skip

Flude team — Thu, 10 Sep 2026 17:17:18 +0000

Every single full documentation rebuild ruthlessly executed doxygen.exe and parsed its massive XML output. The tool simply didn't care if you fixed a single typo in the comments or rewrote half the SDK. On real projects this translated into minutes of agonizing waiting after every push, even though the vast majority of the work could have been safely skipped.

Two Fingerprints for Cache Control

We built the caching scheme around two independent fingerprints to strictly divide responsibilities. The xml_fingerprint determines whether the Doxygen run can be skipped entirely. We compute it as a content hash spanning the binary, the effective Doxyfile, and the source tree. Modification time checks were thrown out immediately to prevent fresh CI checkouts from falsely invalidating the cache. The OUTPUT_DIRECTORY parameter also had to be excluded from the hash to avoid a circular dependency between the fingerprint and the generated files.

The second fingerprint, ir_fingerprint, decides the fate of parsing the generated XML into a ProjectCatalog structure. It incorporates the first hash but adds parser configuration like exclude_swig_internals and the logic hash. Any minor tweak to the parsing rules instantly invalidates the saved internal representation (IR). We managed to catch a potential state desync bug before writing any code thanks to a couple of independent AI review sessions.

Boundaries of Compute Optimization

The compute time cache abruptly stops at the IR boundary. The actual transformation of the ProjectCatalog object into final Markdown is calculated from scratch every time. Observant readers might spot a BuildCacheManager class in storage.py that handles a rendering cache. It operates purely on file I/O to protect local Hugo livereload from triggering falsely.

We dropped the idea of caching the render computation for very pragmatic reasons. Rendering a single page is heavily tied to global state, such as the sidebar.toml navigation tree and cross-links. Altering the taxonomy structure breaks the layout of dozens of files simultaneously, making invalidation tracking incredibly complex. Furthermore, Doxygen processing takes long minutes, while traversing the ready IR tree takes mere seconds.

Why the Speed Remains an Illusion

Our two-level cache only knows how to do one thing—completely skip a build step when fingerprints match. Doxygen physically lacks an incremental parsing mode and cannot run any faster. Any altered character in the sources forces us to endure a full parser run. True incrementality, where only the modified file gets recomputed, is fundamentally incompatible with a monolithic XML dump.

In the architectural plans for the next major version we have already documented dropping Doxygen in favor of direct AST construction via libclang or tree-sitter—the choice between the two hasn't been made yet; the requirements treat them as equal candidates of the same technical direction. Incrementality becomes a natural property of the tool there, instead of something bolted on top with fingerprints—but that same shift will force us to throw away and rewrite the existing caching logic entirely. For now, we had to face the harsh limits of the free GitHub Actions cloud, which pushed us to build our own self-hosted runner. In the next episode we cover how we escaped to our own hardware.

Originally published on our blog: https://blog.flude.guide/blog/illusion-of-speed-caching

Also read us:

Flude.guide: The Name We Had to Find Between Udemy, a Debugger, and nude.com

Flude team — Tue, 08 Sep 2026 03:38:44 +0000

When the time finally arrived to launch this blog, an uncomfortable reality surfaced. The internal working title "UDE" was completely dead on arrival. We faced problems far beyond simple domain availability. Someone else already owned ude.com and ude.io, but the fatal blow came from trademark law. The abbreviation is already a registered ® trademark—Universal Debug Engine, a well-known microcontroller debugging tool from Germany's PLS. Launching another developer tool under the exact same name guarantees a swift cease-and-desist letter. We preferred shipping software over arguing with corporate lawyers.

Three filters running in parallel

We started hunting for a four- or five-letter replacement built around the "ude" substring. The evaluation process was brutal—and immediate. Sound, SEO potential, and legal safety were checked simultaneously. Any candidate could trip an alarm on any axis. One strike instantly eliminated the word from contention.

Candidates like "nude", "dude", and "jude" crashed out on the sound check right away. Several variants barely registered as real English words. The archaic Scots term "blude" would confuse anyone outside of Edinburgh. Common words like "nude" or "dude" carry bizarre baggage that a strict technical reference manual desperately wants to avoid.

The word "Etude" survived slightly longer. It sounds pleasant and bypasses direct trademark violations. The fatal flaw hides in its marketability. A massive global cosmetics brand already dominates search results for that term. Trying to build a presence under that banner means fighting through endless irrelevant noise. The original "UDE" title suffered from a similar chronic condition. Even ignoring the legal threat, the abbreviation looks far too much like the Udemy course platform.

We eventually landed on "Flude". It possessed no magical qualities or overwhelming strengths. It simply stood as the first candidate that survived all three filters without breaking apart.

Claiming the right zone

With the name locked in, we needed a home on the internet. Premium extensions usually sit trapped in squatter portfolios. Buying a catchy short domain for thousands of dollars makes zero sense for a young project. We refused to negotiate with domain brokers.

A quick search showed availability in the .dev space at standard registrar pricing. It sounds reasonable, except that suffix heavily implies writing code and libraries. We needed a space dedicated to final technical documentation. The .guide extension sat wide open, cost barely anything, and literally described our product. The choice made itself. We secured a flawless match between our brand name and its actual function.

Maturing alongside the infrastructure

We locked down the domain while the brand was still baking internally. The new identity eventually cemented itself deep inside our infrastructure. We purged the outdated "ude-" prefix from all Cloudflare Pages projects. A massive single-day rename swept through both the user documentation and the engine repository.

Next time, we will dive into the illusion of speed. We will demonstrate why a clever two-layer cache of XML and syntax trees completely fails to accelerate a bottleneck baked deep inside Doxygen itself.

Originally published on our blog: https://blog.flude.guide/blog/flude-guide

Also read us:

Breaking Down Our Pipeline: From a Symlink Hack to Five Independent Deployments

Flude team — Fri, 04 Sep 2026 08:01:20 +0000

Flude's documentation pipeline survived for years the same way most internal tooling does. It just worked. Nobody in their right mind wanted to touch it. The engine package generated its API reference using a clunky symlink hack we mentioned in the escape-to-GitHub-Actions episode. Platform constraints forced our hand here. Free GitHub Pages limits site publishing to public repositories only. The engine's source code absolutely had to stay private. We bypassed this by running ln -s ../engine ./user-docs/engine during the CI build. The public user-docs repo quietly built a site out of someone else's code. Flude processed those files on the fly to push the final static result out the door.

That duct-tape setup actually got the job done. It also created a concrete wall of architectural coupling. Publishing a single line of API documentation required the private engine to leak its source into a public neighbor. The user-docs project got stuck carrying the weight of an external build process. We couldn't build or test the engine's docs in isolation. The entire toolchain only triggered from inside the documentation repo. Every single run pulled down megabytes of fresh code. The design-docs project suffered from the exact same disease on a smaller scale. Its GitHub Pages deployment ran centrally out of the giant Pipeline umbrella repo. It relied on an SSH deploy key to punch through into a completely different repository.

Finding a host that asks fewer questions

We weighed two escape routes for the engine. One option involved shipping the parsed intermediate representation as a versioned artifact like a wheel or GitHub Release. Downstream projects could grab that package and render the docs locally. That sounds clean on paper. In reality, it introduces a nightmare of schema versioning we desperately wanted to avoid. The brute-force approach won out in the end. The engine now parses, renders, and deploys its own reference from its own CI pipeline. Zero intermediate contracts. Zero artifact publishing rituals.

We still had to bypass the old GitHub Pages restriction. The platform demanded a public repo—so we ditched the platform. Cloudflare Pages offers a free tier that deploys finished build artifacts while completely ignoring repository visibility. The engine finally publishes its own reference while staying hidden from the world. The actual migration required nothing more than copying Doxyfile, hugo-site, and sidebar.toml directly into the engine repository. The generated content/ and public/ directories went straight into .gitignore. They no longer haunt our commit history as static fixtures.

We rolled this exact logic down the rest of the chain. Both design-docs and user-docs migrated to Cloudflare Pages. Each project received a dedicated deploy.yml workflow. GitHub Actions provides native read access to the runner's repository, making deploy keys obsolete. The old cloudflare/pages-action@v1 had already been archived by the community. We skipped the legacy tooling and moved all five repositories straight to the supported wrangler-action.

One token to rule them all

Our perfectly clean plan slammed into a brick wall of reality right about here. Every deployment requires a CF_API_TOKEN. The textbook reaction is generating a narrowly scoped token with least privilege for each repository. Cloudflare violently disagrees with this approach. You cannot scope an API token to a single Pages project. They grant permissions across the entire account—or nothing at all. Any "narrow" token we issued would instantly gain full access to our entire infrastructure. Worse yet, Cloudflare displays the token value exactly once during creation. If you miss it, you have to roll the credential entirely.

The rollout process turned into a timed sprint. We roll the token once. Then we distribute the new string across five repositories within a few frantic minutes. Everything happens through an interactive prompt. Secrets never touch a script, a file on disk, or a Slack message. Five independent repositories now share a single token—and a single rotation window.

Auditing the dead code

Once the deployments turned green, we had to ask ourselves a tough question. How much of this machinery was actually doing anything useful? The audit unearthed a pile of historical garbage. A repository_dispatch trigger from every submodule used to restart integration tests in the umbrella repo. It fired before the gitlink even bumped. The CI was aggressively testing a state that hadn't changed. We gutted that nonsense and replaced it with a simple six-hour schedule on the bump job itself. We lost absolutely nothing in speed because those PRs never auto-merged anyway.

The language test matrix in the umbrella repo turned out to be a carbon copy of the engine's internal CI. The original version even enforced a stricter coverage threshold. A job designed to verify the built site was quietly testing an architecture that died months ago. It merged Hugo's output with a VitePress tree to check cross-links. We had long since split them into separate Cloudflare Pages projects on different domains. The only connection left is a standard external link. You never spot these ghosts reading a clean architecture diagram. They only surface when you dig through the raw runner logs.

Severing the final tie

One tiny cross-repository hook remained. The failure-alert step in every workflow dynamically pulled a shared Telegram notifier from Pipeline at runtime. That single step required another authentication token. A supposedly independent repository was still reaching back to daddy for help. We copy-pasted that action file into every single project. Maintaining five copies instead of one shared source is the real cost of independence.

You are reading this blog on a site built with the exact same pattern. It uses self-deployment with zero external deploy keys. It runs its own copy of the notifier. It lives on that exact same Cloudflare architecture.

Next time, we'll explain how we hunted down a public name for this documentation. The original working title turned out to be locked behind corporate lawyers, sending us down a rabbit hole of weird domain names.

Originally published on our blog: https://blog.flude.guide/blog/decomposing-the-pipeline-monorepo

Also read us:

The 'if not 0:' Easter Egg: How Our AI Audit Found Dead Code in the Orchestrator

Flude team — Tue, 01 Sep 2026 05:12:56 +0000

We keep exposing the internals of our project without embellishment. Recently, we launched an autonomous AI audit across the entire engine core. The agent was tasked with cleaning up type hints, fixing docstrings, and deleting dead code. Living humans rarely get around to this kind of mundane technical hygiene.

The Find

The agent systematically combed through the files and stumbled inside ude/orchestrator.py. Inside the load_sidebar_toml function, it unearthed a fascinating piece of logic. We stared at the screen and couldn't quite believe our eyes.

toml_path = doc_dir / "sidebar.toml"
if not 0:
    if not toml_path.exists():
        raise UdeException(
            f"Required sidebar.toml not found in {doc_dir}. "
            "Every document directory must contain a sidebar.toml file."
        )

Zero in Python is interpreted as falsy. The expression not 0 always evaluates to true. The outer condition simply forced an extra indentation level and was guaranteed to execute every single time. It was a useless wrapper that stole a line in the stack trace from anyone trying to debug the application.

How It Got There

A quick git blame revealed the author and the creation time. The line sneaked in via PR #56 ("Feature/section 2.1 orchestrator library api"), part of a massive orchestrator refactoring. The developer temporarily disabled the block using if 0: for a quick local debugging session. They flipped the logic back with a negation, committed the changes, and forgot to remove the wrapper before the merge. The test suite happily passed. The behavior of the code remained completely unchanged.

This construct didn't break any functionality or open security vulnerabilities. It simply added pure cognitive noise. Everyone reading this file would freeze for a second, trying to decipher the sacred meaning behind the check. That extra second of confusion accumulates into hours of wasted time across thousands of lines of code.

The Fix

toml_path = doc_dir / "sidebar.toml"
if not toml_path.exists():
    raise UdeException(
        f"Required sidebar.toml not found in {doc_dir}. "
        "Every document directory must contain a sidebar.toml file."
    )

We deleted the wrapper and shifted the code to the left. The full test suite ran clean—731 passed, 0 failed, coverage holding at 98.12%. The system's behavior hasn't shifted a millimeter. Regular automated auditing pays off exactly through these tiny cleanups. It sweeps out harmless garbage in bulk while you focus on complex challenges. Speaking of complex challenges. Next time, we'll break down exactly how we sliced our massive pipeline monorepo into five fully independent deployments.

Originally published on our blog: https://blog.flude.guide/blog/stale-conditional-easter-egg

Also read us: