Building a Zero-Hallucination Academic GraphRAG: 1,379 Nodes, Multi-Agent Orchestration & Local SLMs

📌 The Problem: Why Naive RAG Fails on 15+ Years of Academic Research

If you feed 450+ Master's theses and Ph.D. dissertations into a standard vector-based RAG pipeline (e.g., standard LangChain or LlamaIndex with cosine similarity over document chunks), you will quickly hit a wall.

Ask a naive vector RAG:

"How many Ph.D. theses in biomathematics were supervised by Dangbe or Kamgang between 2020 and 2025?"

The vector database retrieves isolated chunks mentioning "Dangbe", "Kamgang", "Ph.D.", and "biomathematics". Then, the LLM starts guessing:

• It hallucinates counts.
• It conflates Master's theses with Ph.D. dissertations.
• It cannot aggregate, group by date ranges, or traverse relationship trees.

Vector similarity finds topical proximity, not relational ground truth.

To preserve, query, and valorize 15+ years of scientific research at the Department of Mathematics and Computer Science (DMI) of the University of Ngaoundéré (Cameroon), we architected K1-MATHINFO (v3.2): a sovereign, zero-hallucination Multi-Agent GraphRAG system backed by Neo4j, OKF Ontologies, and local Small Language Models (SLMs) fine-tuned with GRPO.

---

System Architecture & Multi-Agent Orchestration

Instead of relying on a single monolithic LLM prompt, we built a deterministic multi-agent supervisor pipeline:

                 +--------------------------------+
                 |       User Natural Query       |
                 +---------------+----------------+
                                 |
                     [Input Guardrail & Anti-PII]
                                 |
                     +-----------v------------+
                     |  SupervisorAgent v4.0  |
                     |  Intent & Complexity   |
                     +-----------+------------+
                                 |
      +--------------------------+--------------------------+
      |                          |                          |
+-----v------------+   +---------v--------+   +-------------v----+
|  GraphRAGAgent   |   | MathFormalisme   |   |     OKFAgent     |
| (Neo4j Cypher &  |   | (KaTeX Certified |   | (Ontology Fact   |
| Multi-Hop Paths) |   | Specifications)  |   |  Store v0.2)     |
+-----+------------+   +---------+--------+   +-------------+----+
      |                          |                          |
      +--------------------------+--------------------------+
                                 |
                     +-----------v------------+
                     | CriticFactCheckerAgent | <--- Strict Tier-1 Audit Gate
                     | (Refutes Unproven Data)|      (No Hallucination Allowed)
                     +-----------+------------+
                                 |
                 +---------------v----------------+
                 | Certified Answer + KaTeX LaTeX |
                 +--------------------------------+

---

📂 Document Ingestion & Knowledge Graph Construction

How did we turn 15 years of dusty university archives into a live relational graph?

1. Multimodal Extraction & OCR: Ingestion of 451 PDFs and scanned defense monographs spanning 2008 to 2026.
2. Semantic Entity Normalization: Disambiguating variations of supervisor names, juries, laboratory tags (LAMAP, LAMEX, LARI, LASE), and research domains.
3. Graph Instantiation: Generating declarative Cypher statements into Neo4j:
   • 1,379 Nodes (Documents, Authors, Supervisors, Algorithms, Models, Datasets)
   • 3,871 Certified Relationships (SUPERVISED_BY, AUTHORED_BY, APPLIES_ALGO, FORMULATES_MODEL)
   • 19 Strict Labels ensuring type safety.

---

The OKF Ontology (v0.2) & WikiSkill Architecture

To prevent semantic drift, we deployed a dual-grounding mechanism:

• OKF (Ontology of Known Facts v0.2): A curated repository of 83 formal scientific ontology sheets mapping departments, sub-disciplines, methodologies, and institutional constraints.
• WikiSkill Layer: Modular, self-contained procedural skills loaded dynamically by agents (e.g., fractional calculus derivation skill, Post-Quantum cryptography verification skill, Cypher query builder skill).

---

Local SLMs Fine-Tuned via GRPO (Group Relative Policy Optimization)

Rather than bleeding proprietary academic data into public cloud APIs, we run 4B parameter Small Language Models (SLMs) completely locally on edge workstations.

• Why GRPO over PPO? Traditional RLHF with PPO requires a separate reward model taking up critical VRAM. GRPO (Group Relative Policy Optimization) samples multiple candidate reasoning paths for mathematical and graph queries, evaluating them comparatively against our deterministic Neo4j graph ground truth.
• Result: The local 4B model achieves near-zero hallucination on specialized academic prompts, with sub-2-second inference on standard hardware.

---

Real-World Complex Benchmarks (Stress Tests)

Here are three complex queries that consistently break standard commercial LLMs, but succeed deterministically on K1-MATHINFO:

Test Case 1: Filtered Ph.D. Counting with Multiple Supervisors & Date Window

Query: "How many Ph.D. theses in biomathematics were supervised by Dangbe or Kamgang between 2020 and 2025 at DMI?"

MATCH (s:Supervisor)<-[:SUPERVISED_BY]-(d:Document)-[:AUTHORED_BY]->(a:Author)
WHERE d.degree IN ['Doctorat', 'Ph.D']
  AND (toLower(s.name) CONTAINS 'dangbe' OR toLower(s.name) CONTAINS 'kamgang')
  AND d.year >= 2020 AND d.year <= 2025
RETURN d.title AS title, d.year AS year, s.name AS supervisor, a.name AS student
ORDER BY d.year DESC

Outcome: Returns exactly 5 certified dissertations (Fotso Christophe 2025, Laye Paul 2025, Kamgang Jean Claude 2023, Mbounja Besseme 2021, Tchappi Haman Igor 2020). Zero hallucination.

Test Case 2: Multi-Hop Student Trajectory (Supervisor Switching)

Query: "Are there students who changed supervisors between their Master's thesis and their Ph.D. dissertation?"

• Vector RAG: Fails completely (unable to correlate student IDs across multiple degree cycles).
• K1-MATHINFO: Traverses (a:Author)-[:AUTHORED_BY]->(d1:Master)-[:SUPERVISED_BY]->(s1) vs (a)-[:AUTHORED_BY]->(d2:PhD)-[:SUPERVISED_BY]->(s2) where s1 <> s2, isolating the exact trajectory.

Test Case 3: Departmental Discipline Ratios

Query: "What is the exact ratio between Computer Science and Mathematics theses defended at DMI?"

• Outcome: Resolves the exact breakdown: 345 Computer Science (76.5%) vs 106 Mathematics (23.5%) (Ratio 3.25:1, 42 vs 18 Ph.D. theses).

---

Formal Mathematical Modeling (Native KaTeX)

Academic validation requires exact symbolic notation. The MathFormalismeAgent injects mathematically certified formulas rendered directly via KaTeX:

• Caputo Fractional Differential SEIR Model:
  

  $$\mathcal{D}_t^\alpha S(t) = \Lambda - \beta \frac{S I}{N} - \mu S$$

• Artin Braid Group Definition (Post-Quantum Cryptography):
  

  $$B_n = \langle \sigma_1, \dots, \sigma_{n-1} \mid \sigma_i \sigma_{i+1} \sigma_i = \sigma_{i+1} \sigma_i \sigma_{i+1} \rangle$$

• Also includes native formalisms for Garside Normal Forms, Supersingular Isogenies (SIDH/CSIDH), and Hilbert Space-Filling Curves.

---

The Zero-Hallucination Arbiter: CriticFactCheckerAgent

In scientific research, an AI model must never output unverified text directly to the end user.

The CriticFactCheckerAgent acts as an institutional gatekeeper:

1. Source Graph Audit: Every name, year, formula, and thesis title generated by sub-agents is verified against the Neo4j graph.
2. Tier-1 Certification Seal: Responses grounded by 100% verified facts receive a TIER-1 : OFFICIELLEMENT CERTIFIÉ seal.
3. Institutional Boundary Enforcement: When queries probe out-of-scope subjects (e.g., asking for Chemistry dissertations in a Math/CS faculty, or historical external scientists like Turing or Einstein), the system issues a polite, certified institutional refutation instead of hallucinating fictitious works.

---

Replicability: Sovereign AI for Higher Education in Africa

What we built for the University of Ngaoundéré was intentionally engineered as a reusable blueprint.

Across Africa, hundreds of faculties of Engineering, Medicine, and Pure Sciences possess decades of high-value research sleeping on physical shelves. By packaging:

• A generic academic Neo4j property graph schema,
• Modular agentic supervisors & fact-checkers,
• Containerized local SLM inference (Docker + GGUF),

...any university can deploy its own Sovereign Research Memory within days, keeping its intellectual property completely safe, local, and accessible.

---

What's Next?

Our architecture is completing its final validation phase before the official public defense before the DMI academic jury.

Have you tackled multi-hop aggregation with GraphRAG in production? How do you ensure mathematical precision without hallucinations? Let's discuss below! 👇

Comments

[Ahmet Özel]

Leading with that query is the right call, because it exposes the actual gap: "how many PhD theses supervised by X between 2020 and 2025" is an aggregation over a typed relation, and no amount of chunk similarity produces a COUNT. Vector search answers what looks like this, not how many of these exist, and those are different questions.

Two things I would want to see in the writeup. First, how you validate the extraction step, since a graph is only as trustworthy as the entity resolution that built it, and two spellings of one supervisor name silently splits the count while still looking authoritative. Second, what happens on genuinely semantic questions rather than relational ones. Most real corpora need both paths plus a router that picks between them. Zero-hallucination is a strong claim: measured against which query set?

[Gervais Marie]

Thanks for the sharp feedback, and sorry for the late reply (caught up in the final stretch of our academic defense preparations)!

To answer your three points concisely:

1. Entity Resolution & Validation:
   
   To prevent split counts, we use a pre-ingestion normalizer (src/fuzzy_normalizer.py) that strips accents/honorifics and resolves names against a canonical dictionary of all 40+ DMI faculty using Levenshtein/difflib matching. In Neo4j, schema-level IS UNIQUE constraints on Supervisor.name coupled with declarative MERGE statements guarantee no duplicate nodes are created. The dataset was also cross-checked by hand against official physical defense registers (1997–2026).

2. Semantic vs. Relational (The Router):
   
   Our SupervisorAgent acts as an intent router:

   • Relational / Count queries: Routed to GraphRAGAgent for deterministic Neo4j Cypher aggregation.
   • Genuinely semantic questions: Routed to a Multi-Stage Hybrid Retrieval engine (PostgreSQL pgvector HNSW dense + BM25 sparse + Reciprocal Rank Fusion $k=60$ + Cross-Encoder re-ranking).
   • Composite queries: The supervisor executes both (graph isolates the relevant subgraph, semantic search ranks concepts within it).

3. "Zero-Hallucination" & Benchmark:
   
   "Zero-hallucination" is enforced structurally, not by prompt luck: our CriticFactCheckerAgent and OKF v0.2 store (SHA-256 hashes) strictly verify or refute any claim not corroborated by the active graph. We evaluated this on a curated 20-query golden benchmark (benchmark_queries.json) divided into 3 tiers: Exact Metadata, Semantic Thematic, and Multi-Hop GraphRAG.

Appreciate you reading the piece and raising the real engineering questions!

[Ahmet Özel]

The three benchmark tiers make the scope clearer, but I would describe the result as no observed hallucinations on those 20 queries. A uniqueness constraint prevents duplicate keys; it does not establish that the normalizer chose the correct person. Similarly, a hash establishes record identity rather than claim entailment.

For the next benchmark, deliberately include an unresolved name and a claim contradicted by another active record. Report deterministic aggregation separately from semantic synthesis, so the stronger guarantees on counts do not automatically extend to every generated statement.

[Gervais Marie]

Really appreciate the pushback here, Ahmet you're completely right, especially on the phrasing.

Claiming a universal "zero-hallucination" property on an open-ended corpus was definitely an overreach on my part. What we actually measured was simply zero observed hallucinations across those 20 curated benchmark queries, and that distinction should have been much clearer. I'm definitely adopting your wording in our final thesis writeup and documentation.

On entity resolution: you hit the nail on the head. Neo4j's IS UNIQUE constraint just stops the graph from fragmenting into duplicate nodes, but it won't save us if the fuzzy normalizer maps a name to the wrong person in the first place. Right now, we match extracted names against a canonical dictionary of our faculty (which we manually cross-checked against physical university registers), and anything with a Levenshtein score under 0.88 gets flagged as ambiguous rather than silently merged. But you're right — automated string matching can still stumble on close patronymics or edge cases.

Your point about cryptographic hashes vs. claim entailment is also spot on. A SHA-256 hash gives us tamper-proof source provenance, but it doesn't stop an autoregressive LLM from hallucinating a deduction when generating prose on top of it. True claim entailment requires a dedicated NLI/faithfulness check (like RAGAS or TrueLens), which is the logical next step for us.

I really like your suggestion to decouple the metrics. For the next iteration, we'll report deterministic Cypher/SQL aggregations (where counts are exact) completely separately from the generative synthesis, and we'll deliberately inject adversarial cases — like unresolvable names and contradictory records — to stress-test how well our CriticAgent refuses or refutes them.

Thanks again for taking the time to share this. This kind of critique genuinely helps us strengthen our defense before facing the jury!

[Ahmet Özel]

The manually checked faculty dictionary and explicit ambiguity flag make the entity-resolution boundary much clearer. For the adversarial names, I would include pairs that are both valid dictionary entries but differ only slightly; rejecting an unknown name is easier than avoiding the wrong known person.

Separating aggregation from synthesis also makes the claim checks more interpretable. An NLI score can help prioritize review, but I would still keep a small manually labeled set of supported, contradicted, and insufficient-evidence claims to check what that evaluator actually catches in this corpus.

[Gervais Marie]

Great point on the names. You're completely right: rejecting an unknown name is easy, but mixing up two real colleagues with similar names is the real danger.

We're adding tests specifically for those look-alike faculty names, and we'll use graph context (like their lab and usual co-supervisors) to catch ambiguities before anything gets merged into the graph.

Same for NLI: relying purely on an automated judge without a manual check is a classic trap. We’re going to label a small set of ~50 claims by hand (supported, contradicted, and insufficient evidence) to see what our evaluator actually catches.

Thanks again for the great feedback and for taking the time to discuss this, Ahmet! It really helps make the system stronger.

[Ahmet Özel]

That sounds like a useful next test. I would take the lab and co-supervisor evidence from independently verified records, since using edges extracted by the same uncertain name matcher can reinforce the initial mistake. When both candidates remain plausible, keeping the mention unresolved is an informative outcome.

For the hand-labelled claims, retaining the exact evidence span with each label would help distinguish an NLI error from missing retrieval context. The insufficient-evidence cases are especially valuable because they test whether the judge resists filling in a plausible relationship.