DEV Community

Cover image for What If Your LLM Could Never Make Up a Fake Source Again?
Deus-corp
Deus-corp

Posted on

What If Your LLM Could Never Make Up a Fake Source Again?

We built an open-source platform that gives AI models a "canonical knowledge backbone" — and it caught our test model fabricating citations in its very first run.

The Problem: Your LLM Is a Fantastic Liar

Let's be honest — we've all been there. You ask an AI assistant to write a report, and it produces a beautifully formatted document with a citation to "Dr. Z, Journal of Advanced Studies, 2025." The problem? Dr. Z doesn't exist. The journal doesn't exist. The AI just hallucinated an entire academic career to make its argument sound more convincing.

This isn't a bug in a specific model. It's a fundamental property of how large language models work: they generate plausible text, not verified facts. They're storytellers, not librarians.

But what if there was a way to give LLMs a "canonical knowledge backbone" — a system where every fact, every source, every relationship between concepts had to be explicitly structured, validated, and cryptographically signed before the model could claim it as true?

That's exactly what we built. It's called Canonical Knowledge Structure (CKS), and it's fully open-source.

How CKS Works: Three Layers, One Mission

CKS isn't a single tool — it's an ecosystem of three components, each with a clear responsibility.

cks-core — The Semantic Engine

The brain of the operation. An immutable, version-controlled graph database for knowledge. Every fact gets a unique ID. Every relationship is validated against formal constraints (no dangling references, no duplicate identities, no circular derivations). Once a fact is created, it can never be changed — only a new version can be created.

cks-runtime — The Operational Layer

The body that manages the lifecycle. Sessions, transactions, version history, and event streams. Want to know who changed what and when? The runtime tracks it all. Want to roll back to a previous state? One command.

cks-mcp — The MCP Server

The interface between LLMs and the knowledge backbone. Eight tools that any MCP-compatible AI can use: validate, serialize, explain, evolve, verify sources, list versions, compare versions, and three-way merge.

Live Experiments: We Tested It So You Don't Have To

We ran Haiku 4.5 through a series of experiments using CKS tools via Claude Desktop. Here's what happened.

Experiment 1: Catching a Fake Citation

We asked the model to create a knowledge graph about Artificial Intelligence, then added a reference to a non-existent source. When we validated the structure with the embedding_projection extension enabled, the system immediately flagged the phantom reference with a CKS-EXT-EMBEDDING-PROJECTION diagnostic. The model couldn't bluff its way through — the system mechanically detected the dangling reference.

Experiment 2: Time-Travel Debugging

We evolved the graph (added the concept "Transformers"), then asked the model to list all versions, compare them with a structural diff, and revert to the original state. The model successfully rolled back, and a subsequent serialization confirmed "Transformers" had completely disappeared.

Experiment 3: Three-Way Merge

We created a base graph, then independently evolved two branches — one adding "Parrot" linked to "Dog," the other adding "Hamster" linked to "Cat." The merge succeeded automatically because the changes didn't conflict. Then we tried a conflicting scenario: both branches modified the same property of the same object. The merge correctly refused and returned a structured conflict report.

Getting Started in 60 Seconds

pip install cks-mcp # pulls cks-runtime and cks-core automatically

Then add this to your claude_desktop_config.json:

{
  "mcpServers": {
    "cks-mcp": {
      "command": "cks-mcp"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Restart Claude Desktop. Done — your LLM now has a verifiable knowledge backbone.

Full documentation: deus-corp.github.io/cks-core

The Bigger Picture

We believe the next frontier for LLMs isn't bigger models or longer context windows — it's verifiable knowledge infrastructure. A layer that sits between the model and its output, ensuring that every claim can be traced to its origin, every source can be checked, and every change is auditable.

CKS is our open-source contribution to that vision. It's MIT-licensed, production-ready, and actively maintained.

We're looking for contributors, feedback, and real-world use cases. If you're working on RAG pipelines, knowledge graphs, or AI verification — we'd love to hear from you.

GitHub: github.com/Deus-corp
Documentation: deus-corp.github.io/cks-core

LLMs generate. CKS verifies.

Top comments (2)

Collapse
 
alexshev profile image
Alex Shev

The hard part is making citation a constraint, not a decoration. A model can produce source-shaped text very easily. The stronger pattern is forcing every claim that needs evidence to point back to a retrieved passage or to say “not found.”

Collapse
 
deuscorp profile image
Deus-corp

This is exactly the core insight we built CKS around — "source-shaped text" is the perfect description of the problem. An LLM can trivially produce {"source": "Dr. Z, 2025"} as decoration. What we did is make that decoration mechanically checkable.

Two mechanisms enforce the constraint you're describing:

  1. NoDanglingRelationConstraint — any reference to a source ID must resolve to an actual KnowledgeObject that exists in the structure. The model can't just invent an ID.
  2. verify_source tool — performs a real HTTP request and cryptographically signs the result. A hand‑written VerificationRecord without a valid signature is unconditionally rejected, even if the model "forgets" to request the check.

So the only path to a valid citation is: source actually exists → verification actually happened → signature matches. That's the "not found" path you mentioned — the model is forced to say "I didn't verify this" rather than fabricate a plausible‑looking result.

Curious — have you seen other projects that enforce this at the structural level, or is it mostly prompt‑based solutions in your experience?