We were still trying to squeeze the maximum out of good old Doxygen in our cloud pipeline when we suddenly encountered a classic problem of harsh enterprise C++: the parser started silently "eating" entire chunks of the codebase.
It all began when we noticed an anomaly: important structures (like BITMAPINFO and RGBQUAD) had vanished without a trace in the generated documentation.
The scariest part was that the pipeline passed successfully. Doxygen returned exit code 0, didn't issue a single warning, but the XML output for these files was practically empty. It simply swallowed the code and stayed silent.
Investigation by Binary Search
When a parser is silent, the only way to find the cause is the bisection method. We would take a problematic header file of several thousand lines, delete half of it, run the parser, and see if the necessary structures appeared.
Narrowing the search down to 50 lines, we found the culprit. It turned out to be a classic C++ trick—SFINAE (Substitution Failure Is Not An Error). The codebase extensively used the construct:
typename std::enable_if<CONDITION, bool>::type = VALUE
To a C++ compiler, this is absolutely valid code. But the legacy Doxygen parser went crazy at the sight of such a pile of angle brackets (< >). Its internal state machine broke, and it stopped understanding C++ syntax until the very end of the file. This is exactly why structures declared below the ill-fated SFINAE template simply fell out of reality.
A UNIX-style Solution
We couldn't go to the product developers and say: "Rewrite all your C++ code because our parser doesn't understand it." We needed a systemic solution that didn't touch the source files.
Fortunately, Doxygen has the INPUT_FILTER mechanism. This is a script through which every file is run before the parser sees it.
We wrote a small Python script (sfinae_filter.py) with a clever regular expression. The script caught all std::enable_if constructs on the fly and replaced them with a syntax equivalent in the number of arguments: bool = VALUE.
At the same time, we strictly observed an important rule: the script calculated the number of line breaks in the removed block and added exactly the same number of blank lines to the replacement. Thanks to this, the line numbers in the source code did not shift, and all parser warnings continued to point to the correct lines in the developers' IDEs.
Conclusion
A small Python filter solved the problem in dozens of files, returning the lost classes to the documentation. We won again.
But this incident became the "last straw." Crutches with regular expressions on top of C++ code is a road to nowhere. The language is too complex to parse as text, and the unsafe INPUT_FILTER opened holes in cloud security.
It was time to do what we had avoided for so long: fire Doxygen for good. How we migrated to the native libclang compiler frontend and why we didn't have to rewrite half the project—read about it in the next episode.
Originally published on our blog: https://blog.flude.guide/blog/crash-test-sfinae-doxygen
Also read us:
Top comments (0)