One of the hardest stages in developing a documentation generator is alignment with legacy tools. When you migrate from an old, decades-proven proprietary generator to your own custom pipeline, you are obligated to prove that you haven't lost a single class.
We wrote an automated system that compared thousands of entities generated by the old tool with what our new Flude-based pipeline found. And one day, we stumbled upon a terrifying anomaly.
Anomaly in the Matrix
The alignment script happily reported: Flude found and generated documentation for a struct named GraphicsAsset and its smart pointer GraphicsAssetPtr.
The problem was that in the old, reference documentation, these entities didn't exist. At all.
According to our logic, if the legacy generator didn't find the class, it's highly likely this class simply doesn't exist, and our parser made a mistake somewhere, "gluing" someone else's names together. We had to check the source code.
Zero Results
We did what any engineer would do—we opened a terminal and ran a quick global text search (grep / ripgrep) across the entire multi-gigabyte codebase of our enterprise C++ product:
rg "GraphicsAsset" ./sdk_sources/
Result: 0 matches. Absolute emptiness.
We checked with case-insensitivity, we searched for the smart pointer. Nothing.
Panic: The Hallucinating Parser
It was a moment of genuine horror. The codebase didn't contain the string GraphicsAsset. But somehow, our parser hadn't just found this name; it had built a beautiful HTML page for it, complete with a list of methods, properties, and inheritance!
The conclusion seemed obvious: our parser had gone insane. It had started fabricating C++ classes out of thin air. Perhaps the logic for expanding complex macros had broken, or Doxygen's cache had become so corrupted that it started stitching pieces of different files into a single "Frankenstein".
We were already preparing to open the debugger and rewrite the token-linking logic in the core.
Autopsy of the Cache
Fortunately, before tearing apart the core codebase, we decided to look at the raw intermediate data. We went to the directory where Doxygen dumped its low-level XML, and searched there.
Suddenly, sitting quietly on the disk, was a file named struct_graphics_asset.xml. We opened it.
Inside was perfectly formed XML. It didn't look like a hallucination. Moreover, in the <location> tag, the parser pedantically pointed out exactly where it got this class:
Core/Include/Graphics/MaterialConfig.h:3006-3061
The Phantom Takes Flesh
We opened the MaterialConfig.h file and scrolled down to line 3006.
And there, in the very heart of a massive legacy header, hid a completely real, living struct:
struct CORE_ABSTRACT GRAPHICS_EXPORT GraphicsAsset {
// ... dozens of lines of valid C++ code
};
It had always been there. The legacy generator simply couldn't parse it because of complex macros, but our Flude could.
So why did the global rg yield zero matches? In enterprise monoliths, there are always plenty of reasons. Perhaps the Core/Include path was listed in a tricky .gitignore, and the smart ripgrep ignored it. Perhaps the file had an exotic Windows encoding that broke the search. Perhaps it was a mounted submodule.
Conclusion
A global text search offered an absence of evidence as evidence of absence.
This case taught us a crucial rule: a text search is not a C++ parser. Trying to analyze the structure of a C++ monolith with regular expressions or grep is a path to disaster and false conclusions.
If you doubt your parser, never use text search as the arbiter. Trust only the raw dumps of the compiler itself or the Abstract Syntax Tree (AST). And most importantly: do not try to "fix" a bug in the core until you prove its existence in the actual source of truth.
But parsing and cache problems paled in comparison to our infrastructural challenges. From day one, our project was built as an ecosystem of several independent repositories. How this architecture turned into Git traps and nearly broke our Continuous Integration—read about it in the next episode.
Originally published on our blog: https://blog.flude.guide/blog/the-phantom-bug-grep
Also read us:
Top comments (0)