DEV Community

Cover image for What Happens When an OAG System Goes Through a Real Enterprise Knowledge Workflow?
Le Trung Hieu
Le Trung Hieu

Posted on

What Happens When an OAG System Goes Through a Real Enterprise Knowledge Workflow?

How Gnosis applies Ontology-Augmented Generation (OAG) to transform enterprise documents into governed knowledge, evidence-backed answers, and AI outputs that can be validated instead of merely trusted.

Introduction

When we started building Gnosis during Relipa Hackathon 2026, the goal was not to build another chatbot.

The problem was more practical.

Enterprise knowledge was scattered across PDF contracts, Word specifications, Excel trackers, meeting records, and screenshots. Information existed, but answering a seemingly simple question often required searching multiple documents, understanding relationships between entities, and verifying whether the answer was actually supported by evidence.

Most Retrieval-Augmented Generation (RAG) systems improve access to information. But we wanted to explore a different question:

What happens when AI-generated outputs must be governed by ontology constraints, evidence requirements, and deterministic validation before they are allowed to influence a business decision?

That question led to Gnosis, a document intelligence project inspired by the principles of Ontology-Augmented Generation (OAG).

At Relipa, the project was not intended to become a generic AI assistant.

Instead, it was designed as an exploration of how enterprise knowledge could be transformed into a governed knowledge system where every answer remains connected to evidence, ontology constraints, and deterministic validation.

For organizations delivering AI development services, this distinction becomes increasingly important when moving from proof-of-concept demonstrations toward production systems. In many enterprise environments, trust, traceability, and governance ultimately matter as much as answer quality.

Rather than focusing only on retrieval performance, Gnosis explored how knowledge extraction, graph construction, validation, retrieval, and generation could operate within a single governed workflow.

The core principle remained simple throughout the project:

The LLM proposes; policy and deterministic code decide which proposals go forward.

This article focuses on how Gnosis applies that principle throughout the knowledge lifecycle.


Building Gnosis: From Documents to Governed Knowledge

The Starting Point

Enterprise documents rarely arrive as clean, structured data.

Contracts exist as PDFs. Requirements live in Word files. Operational data often appears in spreadsheets. Important information may only exist inside screenshots or embedded images.

Before retrieval, reasoning, or question answering can happen, the system must first transform those artifacts into a form that can be governed consistently.

Gnosis treats this as a knowledge-building problem rather than a search problem.

flowchart LR
DOC[Documents] --> EXT[Constrained Extraction]
EXT --> KG[Knowledge Graph]
KG --> RET[Retrieval]
RET --> VAL[Validation]
VAL --> GEN[Generation]
GEN --> OUT[Answer with Citations]

Each stage introduces a new control point where deterministic logic can evaluate what the model proposes.

Rather than viewing retrieval and generation as isolated capabilities, the project explores how they can operate inside a broader framework where evidence, policy, and ontology constraints influence the final outcome.


The Knowledge Layer

The ontology serves as the system's data contract.

Rather than allowing the model to invent arbitrary entity types and relationships, Gnosis defines:

  • which entity types exist;
  • which properties belong to each type;
  • which relationships are allowed;
  • which constraints apply to those relationships.

The ontology is therefore used as an active constraint rather than passive documentation.

A model may propose a new entity or relationship.

The system decides whether that proposal complies with the ontology.

Anything that violates the contract is rejected or placed into a review process instead of silently entering the graph.

This distinction proved critical when processing large collections of heterogeneous enterprise documents.

One practical challenge repeatedly emerged throughout development.

Enterprise knowledge is rarely organized according to how language models consume information.

The same business concept may appear in multiple documents under different names, formats, or levels of detail. A requirement might exist in a specification document, a meeting note, and an operational report while referring to the same underlying concept.

Without a shared semantic framework, every retrieval and reasoning step must repeatedly infer whether those references describe the same entity.

Ontology helps reduce this ambiguity.

Rather than relying solely on statistical similarity, the system evaluates extracted entities and relationships against explicit domain definitions. The result is not a replacement for language models, but a structured environment in which model-generated proposals can be interpreted consistently.

For organizations investing in AI development services, this distinction becomes increasingly important as AI systems move closer to business-critical workflows.


Provenance as a First-Class Requirement

One design decision influenced nearly every other component in the system.

Every piece of knowledge must retain a path back to its source.

For Gnosis, retrieval quality alone was not sufficient.

An answer needed to be traceable.

The system therefore maintains provenance information that links extracted knowledge back to the original document location.

Conceptually:

Answer
  ↓
Citation
  ↓
Knowledge Node
  ↓
Source Reference
  ↓
Original Document
Enter fullscreen mode Exit fullscreen mode

This principle enables users to move from an answer to the supporting evidence rather than treating the answer as an independent source of truth.

Provenance also plays an important role in governance. When knowledge can be traced back to its origin, validation becomes possible. When provenance is missing, confidence becomes much more difficult to justify.


Constrained Extraction

A common failure mode in graph construction occurs when ontology is applied only after extraction.

By that point, the graph may already contain inconsistent entity types and relationships.

Gnosis instead applies ontology constraints during extraction.

flowchart TD
RAW[LLM Output] --> PARSE{Valid Structure?}
PARSE -->|No| RETRY[Retry]
PARSE -->|Yes| TYPE{Known Type?}
TYPE -->|No| REVIEW[Review Queue]
TYPE -->|Yes| ENTITY[Valid Entity]
ENTITY --> REL{Valid Relationship?}
REL -->|Yes| GRAPH[Knowledge Graph]
REL -->|No| REJECT[Reject Relationship]

The important architectural decision is that the prompt guides extraction, but the parser enforces the rules.

The model can suggest.

The system validates.

This approach helps prevent invalid entities and relationships from propagating through later stages of the pipeline.


Building Trustworthy Retrieval

Enterprise questions frequently require more than one retrieval strategy.

A contract number may require keyword matching.

A business requirement may require semantic search.

An ownership question may require traversing relationships.

For that reason, Gnosis combines multiple retrieval approaches before answer generation begins.

flowchart LR
Q[Question] --> HS[Hybrid Search]
HS --> RR[Rerank]
RR --> RG[Relevance Gate]
RG --> GX[Graph Expansion]
GX --> GV[Guardrails & Validation]
GV --> LLM[LLM]

The purpose of retrieval is not to provide the model with as much context as possible.

Its purpose is to identify context that is sufficiently relevant to qualify as evidence.

The retrieval stage therefore becomes more than a search operation.

Enterprise queries often contain multiple information needs simultaneously. A user may reference an entity, imply a relationship, and request supporting evidence in a single question.

No individual retrieval strategy is sufficient for every situation.

Keyword search helps identify explicit identifiers. Semantic retrieval helps discover conceptually related information. Knowledge graph traversal helps expand context through relationships.

The objective is not to maximize context volume.

The objective is to identify candidate evidence that deserves further validation before generation begins.


No Evidence, No Answer

One of the most important behaviors implemented in Gnosis is the ability to refuse an answer.

A retrieval system will often return something.

That does not mean it has found evidence.

The relevance gate exists to distinguish between those two situations.

If no candidate satisfies the required criteria, the system returns a “no information” response rather than asking the model to improvise.

This mechanism directly reflects the central architecture principle:

The responsibility of the model is to generate answers.

The responsibility of the system is to determine whether an answer should be generated at all.

In practice, refusing to answer can be just as important as answering correctly.


Citation Validation

Even grounded answers can drift away from their sources.

For that reason, generation is followed by a citation validation stage.

flowchart LR
A[Generated Answer] --> S[Split Sentences]
S --> C[Check Citation Tags]
C --> V{Valid Source?}
V -->|Yes| KEEP[Keep Sentence]
V -->|No| DROP[Drop Sentence]

The rule is intentionally strict.

A sentence without a valid source does not remain in the final answer.

Rather than asking the model to self-evaluate, deterministic validation determines whether the sentence is eligible to survive.

This stage reinforces the separation between generation and approval that appears throughout the system.


Why Gnosis Extends Basic OAG

Standard OAG discussions often begin with structured objects that already exist.

Gnosis starts much earlier in the lifecycle.

The system must first create trustworthy knowledge objects from unstructured documents.

That requirement introduces several additional responsibilities:

  • ontology-constrained extraction;
  • provenance tracking;
  • entity resolution;
  • conflict handling;
  • retrieval validation;
  • citation validation.

These layers are not separate from OAG.

They apply the same principle to stages that occur before object-based reasoning becomes possible.

Another important observation is that many OAG discussions assume the existence of structured knowledge objects.

In practice, enterprise environments often begin with unstructured documents rather than governed objects.

Documents may contain duplicate entities, conflicting references, inconsistent terminology, or incomplete context.

As a result, governance cannot start at answer generation.

Governance must be applied throughout the knowledge lifecycle, beginning with extraction and continuing through graph construction, retrieval, relevance validation, reasoning, and final answer generation.

This perspective influenced many of the architectural choices explored in Gnosis.

Ontology, provenance, validation, retrieval, and citation checking were not treated as isolated components. Instead, they functioned as connected control layers responsible for determining which model-generated proposals were permitted to move forward.

Seen from this perspective, Gnosis extends the philosophy of OAG beyond reasoning and into the broader process of enterprise knowledge construction.


Closing Thoughts

Building Gnosis changed how we think about enterprise AI.

The challenge was rarely generating text.

The challenge was deciding whether generated outputs deserved to move forward.

Ontology constraints, provenance data, retrieval validation, relevance gates, guardrails, and citation checks all serve the same purpose.

They separate proposal from approval.

That separation ultimately became the most important lesson from the project.

The LLM proposes; policy and deterministic code decide which proposals go forward.

For enterprise knowledge systems and organizations delivering AI development services, that distinction may matter more than any individual model, retrieval strategy, or prompting technique.

Top comments (0)