DEV Community

Muhammad Attayab Ashraf
Muhammad Attayab Ashraf

Posted on

Why I Stopped Letting LLMs Guess Code Structure — Building DocAgent with AST Parsing & Multi-Agent Verification

If you dump raw source files into an LLM and ask it to generate documentation, you usually get two things: vague summaries and hallucinated function signatures.

When I started building DocAgent—an automated documentation pipeline with C and C++ support—I wanted generated docs that an engineering team could actually trust without manually verifying every line.

Instead of relying on a single giant prompt, I split the problem into deterministic code parsing and a four-role multi-agent workflow.

The Architecture: Structure First, LLM Second

LLMs are great at explaining intent, but they are unreliable at tracking exact symbol boundaries across large C/C++ files. To solve this, I used libclang to parse the Abstract Syntax Tree (AST) before any model touches the code.

Breaking Down the Four Agent Roles

Rather than asking one agent to do everything, each step has a narrow job:

  1. Reader (AST Parser): Uses libclang in Python to extract functions, classes, parameter types, return values, and call relationships deterministically.
  2. Searcher: Pulls relevant surrounding context so documented functions reflect how they are actually used in the codebase, not just in isolation.
  3. Writer: Takes the structured AST metadata and retrieved context to generate clear, human-readable explanations.
  4. Verifier: Compares the generated documentation back against the extracted AST nodes to catch missing parameters, mismatched return types, or unsupported claims before output.

Key Takeaways From Building It

• Deterministic tools beat clever prompts. Letting libclang handle syntax and type extraction eliminated almost all structural hallucinations before the LLM even ran.
• Verification needs ground truth. A verifier agent only works well when you give it hard facts (like parsed AST signatures) to check against, rather than asking it to "double-check its own work."
• Small, focused pipelines are easier to debug. Separating retrieval, drafting, and validation made it obvious which stage failed whenever an edge case popped up in C++ headers.

Check Out the Code

You can explore the implementation and try it out on GitHub:

👉 https://github.com/Attayab-31/DocAgent-with-C-support-

If you're building developer tools or multi-agent workflows in Python, I’d love to hear how you handle verification and ground-truth checks in your pipelines!

Written by Muhammad Attayab Ashraf — Backend & Applied AI Engineer building Python APIs, RAG pipelines, and LLM systems at DevVerx. View more of my work at https://attayab.automivex.com or connect on GitHub.

Top comments (0)