Stop Guessing: Extracting Bulletproof Knowledge Graphs from Unstructured Text with LLMs & Zod

The bridge between unstructured human language and the deterministic structures required by enterprise software is one of the most delicate interfaces in modern systems architecture. To build systems that can reason reliably over vast quantities of text without drifting into fantasy, we must understand how Large Language Models ingest, interpret, and project linguistic information—and why naive approaches to this problem inevitably fail.

Consider what happens every time you ingest unstructured documents—such as messy customer support tickets, sprawling corporate annual reports, or complex research papers—into an AI-powered pipeline. You want clean, structured data. You want nodes and edges ready for a graph database. Instead, you get unconstrained markdown strings, missing fields, and hallucinated relationships that break your downstream analytics.

If you are treating LLM outputs like trusted API responses, your system is already walking a tightrope. In this deep dive, we will explore the epistemic gap between probabilistic language and deterministic schemas, draw parallels to untrusted web inputs, and build a fully typed, production-ready TypeScript pipeline using OpenAI and Zod that guarantees zero-hallucination graph persistence.

---

The Epistemic Gap: Probabilistic Language versus Deterministic Schemas

At their core, Large Language Models are sophisticated statistical engines. As established in vector representations and embedding spaces, an LLM processes text by mapping tokens into high-dimensional continuous manifolds, predicting subsequent tokens based on learned probabilistic distributions. Every output generated by an unconstrained LLM is a continuous gradient descent across probability surfaces—a creative autocomplete operating at planetary scale.

Conversely, Graph Databases, relational stores, and typed enterprise architectures demand absolute determinism. A node label must be precisely Person, not Individual, Human, or Person_Entity. An edge must be explicitly defined as WORKS_FOR, with a directional semantic that admits no ambiguity.

When we ask an unconstrained language model to extract a Knowledge Graph from a text document, we are crossing an epistemic chasm. We are forcing a continuous, probabilistic, hallucinatory engine to produce discrete, binary, invariant structural artifacts.

Without rigorous boundaries, the model will naturally exercise its generative freedom. It might return JSON missing required fields, rename keys dynamically based on conversational context, output markdown blocks wrapped in conversational pleasantries ("Sure, here is your graph!"), or—worst of all—invent entities and relationships that do not exist in the source text. In an enterprise system, this lack of structural integrity causes silent failures, cascading data corruption, and broken downstream analytics.

---

The Web Development Analogy: Unvalidated API Inputs

To understand the danger of unconstrained LLM extractions, we can look to a familiar web development parallel: Accepting Raw JSON Payloads from an Untrusted Client via an HTTP POST Request.

Imagine building a public-facing API endpoint in a Node.js web server. A client sends a raw JSON payload containing user registration data. If the backend developer writes code that blindly trusts the incoming request body—assuming that req.body.age is always a number, req.body.email is always a valid string containing an @ symbol, and req.body.roles is an array—the application is fundamentally insecure and fragile.

A malicious or malformed request can send strings where numbers are expected, inject unexpected properties, or omit mandatory fields entirely. In a web application, this leads to runtime exceptions, database constraint violations, or security vulnerabilities like mass assignment attacks.

To solve this, modern TypeScript developers employ Runtime Validation. Runtime Validation is the process of checking data integrity and structure during program execution, typically applied to external inputs (API requests, file reads) that cannot be trusted. Zod enforces runtime validation, which is critical because compile-time TypeScript checks are stripped away during compilation and cannot verify data originating from outside the application boundary. A Zod schema acts as an unyielding bouncer at the door of your application, inspecting every incoming data packet against a rigid type contract, throwing structured errors if a single byte violates the contract, and casting or stripping unknown properties if configured to do so.

Querying an LLM for structured data is identical to accepting a raw, untrusted HTTP POST request from a malicious external client. The LLM is the untrusted client. It speaks human language fluently, but it does not inherently respect the TypeScript interfaces or JSON schemas you conceptualized in your codebase. Treating an LLM response as a guaranteed typed object without a runtime validation layer is the architectural equivalent of writing const user: User = req.body in an Express application and hoping for the best.

---

The Mechanics of LLM Extraction Pipelines

An extraction pipeline is the end-to-end socio-technical process of converting raw, unstructured textual data into actionable graph nodes and edges. This pipeline consists of several distinct theoretical phases: ingestion, semantic chunking, prompt-guided extraction, structural constraint enforcement, semantic mapping, and graph persistence.

1. Semantic Chunking and Token Management

Enterprise texts—such as a 100-page corporate financial disclosure or a multi-volume medical journal—exceed the context windows of even modern models, or at least degrade in extraction fidelity when processed in a single monolithic pass. Attention mechanisms within transformer architectures suffer from the "lost in the middle" phenomenon, where information located in the center of long contexts is weighed less effectively than information at the edges.

Therefore, texts must be segmented into semantically coherent chunks. Unlike naive chunking (which splits text blindly every $N$ characters), semantic chunking relies on understanding structural boundaries—paragraphs, sections, or thematic transitions—ensuring that an entity and its contextual modifiers are not severed across two separate extraction requests.

2. The Extraction Prompt and Schema Injection

Once text is chunked, it is passed to the LLM alongside an extraction prompt. This prompt establishes the persona of a deterministic knowledge engineer and defines the ontology—the taxonomy of allowed entity types and relationship types.

However, natural language prompts alone are insufficient. To bridge the gap, modern architectures inject formal structural definitions directly into the model's generation loop. This can be achieved through various paradigms, such as JSON mode, function calling (tool use), or constrained grammar decoding (where the sampling logits of the model are dynamically masked at every token step so that it can only output tokens that conform to a specific Context-Free Grammar or JSON Schema).

3. The Validation and Remediation Loop

Even with constrained decoding, LLMs can occasionally produce structural anomalies—such as returning an empty array when entities are clearly present, or formatting a date string in a non-standard format that breaks downstream graph ingestion. This is where the runtime validation layer becomes critical.

When the raw output string returns from the LLM, it is parsed and passed through a strict runtime validation schema. If the data passes, it moves forward. If it fails, the validation engine captures the precise Zod error stack trace—detailing exactly which field failed validation (e.g., Expected string, received null at entities[2].properties.foundedYear)—and feeds this error message back to the LLM in a self-correction loop. This conversational healing process allows the model to inspect its own structural mistake and correct it before it ever pollutes the enterprise database.

---

The Microservice Analogy: Orchestrating LLMs and Graph Stores

To further cement our theoretical understanding of this pipeline, consider a second architectural analogy: A Microservices Architecture communicating via Message Queues and API Gateways.

In a distributed microservices system, you might have a data ingestion service written in Python, a billing service written in Go, and a user notification service written in TypeScript. Each service operates in its own isolated memory space, using its own internal data structures. They cannot directly access each other's objects. Instead, they communicate by serializing data into a strict contract—such as Protocol Buffers or JSON schemas enforced by an API Gateway.

If the Python data ingestion service sends a message to the TypeScript user service, it cannot simply dump raw, unformatted bytes over the wire. It must serialize the payload into a strictly defined contract. The API Gateway acts as an invariant enforcer, rejecting any payload that does not match the expected protobuf definition.

In our Knowledge Graph extraction pipeline:

• The Unstructured Text is the legacy raw data source.
• The LLM is an autonomous, probabilistic microservice that is brilliant at natural language understanding but notoriously sloppy with data types.
• The Zod Validation Layer is the API Gateway / Contract Enforcer sitting between the LLM microservice and the core database.
• The GraphDB is the ACID-compliant relational/graph database at the center of the enterprise infrastructure.

Without the API Gateway (Zod), the microservice (LLM) would corrupt the database with malformed data. By interposing a strict runtime validation layer, we create a fault-tolerant boundary that isolates the probabilistic nature of the AI from the deterministic requirements of the persistence layer.

---

The Problem of Quantization and Extraction Fidelity

When deploying LLMs to power these extraction pipelines in production environments, developers frequently encounter infrastructure constraints that lead them to adopt model Quantization.

Quantization (Model) is a technique that reduces the precision of the weights and activations in an AI model (e.g., from 32-bit floating point to 8-bit integers) to decrease model size and improve inference speed, often with minimal performance loss. By compressing a 70-billion parameter model down to 4-bit or 8-bit representations (using techniques like GPTQ, AWQ, or GGUF), organizations can run high-capability models on cost-effective hardware accelerators without relying entirely on expensive, latency-prone commercial cloud APIs.

However, quantization introduces a fascinating theoretical tension in Knowledge Graph extraction pipelines. Because quantization reduces the numerical precision of the model's internal weight matrices, it slightly compresses the continuous latent space. For creative tasks like poetry generation or casual chat, this compression is virtually imperceptible. But for structural extraction tasks—where the model must meticulously track entity references across long passages of text, maintain strict ontological consistency, and format its output into nested JSON structures—heavy quantization (such as extreme 2-bit or 3-bit quantization) can lead to a subtle degradation in instruction-following capabilities.

A heavily quantized model is more prone to "schema drift"—where it forgets a required field halfway through generating a large JSON object, or hallucinates an invalid relationship type because the compressed weights lack the precise resolution needed to recall the exact boundaries of the system prompt.

This reinforces why runtime validation is not merely a nice-to-have safeguard, but an absolute operational necessity. When working with quantized, locally-hosted models, the probability of structural output drift increases. Therefore, the strictness of your validation schemas must scale inversely with the precision of your model weights. The more quantized and resource-constrained your LLM infrastructure is, the more aggressive and robust your runtime validation and auto-correction loops must be to guarantee zero-hallucination persistence into your GraphDB.

---

The Anatomy of Semantic Nodes and Edges

To prepare for mapping extractions into a graph database, we must define what an extracted Knowledge Graph actually represents in graph theory. A Knowledge Graph is a directed, labeled multigraph $\mathcal{G} = (\mathcal{V}, \mathcal{E})$ , where:

• $\mathcal{V}$ is the set of vertices (nodes), representing distinct real-world entities (e.g., People, Organizations, Concepts, Locations). Each vertex $v \in \mathcal{V}$ possesses a type label $\tau(v)$ and a set of key-value properties $\pi(v)$ .
• $\mathcal{E}$ is the set of directed edges (relationships), representing semantic connections between pairs of vertices $(u, v)$ where $u, v \in \mathcal{V}$ . Each edge $e = (u, v)$ possesses a relationship type $\rho(e)$ and a set of properties $\pi(e)$ .

When an LLM processes text, it is tasked with performing Named Entity Recognition (NER) to populate $\mathcal{V}$ , and Relation Extraction (RE) to populate $\mathcal{E}$ .

In a naive pipeline, the LLM is asked to output a simple list of triplets: [Entity A, Relationship, Entity B]. While triplets are easy to conceptualize, real-world enterprise data is rarely so clean. Entities have aliases, attributes, confidence scores, and provenance pointers back to the exact byte offsets of the source text. Relationships have temporal validity bounds (e.g., CEO_OF from 2018 to 2022) and provenance metadata.

This is why mapping unstructured text to a graph requires rich, hierarchical schemas rather than flat strings. A robust extraction schema must capture not only the primary entities and relationships, but also the metadata that proves why the extraction was made. By binding this data to strict TypeScript types at runtime, we ensure that every node written to our GraphDB is fully typed, structurally verified, and completely traceable back to its source text.

---

Building the Production TypeScript Pipeline

Let's bridge the gap between unstructured text and a deterministic Knowledge Graph. Imagine a Customer Support SaaS application where incoming feedback tickets must be systematically parsed into a localized Knowledge Graph. This graph captures entities such as users, features, and core issues, along with the precise relationships between them (e.g., (User)-[EXPERIENCES]->(Issue)).

Below is a fully self-contained TypeScript implementation that defines a schema using Zod, leverages the OpenAI API with structured outputs via zod-to-json-schema, and processes unstructured text into typed Knowledge Graph nodes and edges.

import { z } from "zod";
import OpenAI from "openai";
import { zodToJsonSchema } from "zod-to-json-schema";

/**
 * Initialize the OpenAI client for our SaaS backend infrastructure.
 * This assumes process.env.OPENAI_API_KEY is configured correctly.
 */
const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY || "dummy-key-for-compilation",
});

/**
 * 1. Define the Zod Schema for Knowledge Graph Extraction
 * 
 * We enforce a strict structure consisting of Nodes and Edges. 
 * This ensures the LLM cannot return arbitrary free-form text or 
 * unstructured JSON objects that would break our graph database ingest pipeline.
 */
const KnowledgeGraphSchema = z.object({
  nodes: z.array(
    z.object({
      id: z.string().describe("Unique identifier or slug for the entity (e.g., 'user_123', 'feature_auth')"),
      label: z.string().describe("The semantic category of the node (e.g., 'User', 'Feature', 'Bug', 'Product')"),
      properties: z.record(z.string(), z.any()).describe("Key-value attributes associated with this entity"),
    })
  ).describe("The collection of discrete entities extracted from the text."),

  edges: z.array(
    z.object({
      source: z.string().describe("The ID of the source node where the relationship originates"),
      target: z.string().describe("The ID of the target node where the relationship points"),
      relation: z.string().describe("The uppercase verb phrase defining the edge type (e.g., 'EXPERIENCES', 'AFFECTS', 'OWNS')"),
      properties: z.record(z.string(), z.any()).optional().describe("Optional metadata about the relationship"),
    })
  ).describe("The directed connections representing relationships between the extracted nodes."),
});

// TypeScript type inference derived directly from our runtime schema
type KnowledgeGraphExtraction = z.infer<typeof KnowledgeGraphSchema>;

/**
 * 2. Define the SaaS Ingestion Pipeline Function
 * 
 * Takes raw, unstructured customer feedback and returns a guaranteed 
 * typesafe Knowledge Graph payload ready for GraphDB insertion.
 * 
 * @param unstructuredText - The raw string input from user feedback forms.
 */
async function extractKnowledgeGraph(unstructuredText: string): Promise<KnowledgeGraphExtraction> {
  // Convert our Zod schema into a JSON Schema format supported by OpenAI function calling or response formatting
  const jsonSchema = zodToJsonSchema(KnowledgeGraphSchema, { name: "KnowledgeGraph" });

  try {
    // Call the OpenAI chat completion endpoint with structured JSON outputs enforced
    const response = await openai.chat.completions.create({
      model: "gpt-4o",
      messages: [
        {
          role: "system",
          content: "You are an expert enterprise knowledge engineer. Extract entities and relationships from the provided text to form a deterministic Knowledge Graph. You must strictly adhere to the provided JSON schema."
        },
        {
          role: "user",
          content: unstructuredText
        }
      ],
      response_format: {
        type: "json_schema",
        json_schema: {
          name: "KnowledgeGraph",
          strict: true,
          schema: jsonSchema.definitions?.KnowledgeGraph || jsonSchema,
        }
      },
      temperature: 0.0 // Zero temperature ensures deterministic, non-creative extractions
    });

    const rawContent = response.choices[0]?.message?.content;

    if (!rawContent) {
      throw new Error("Failed to extract knowledge graph: Received empty response from LLM.");
    }

    // Parse the raw JSON string returned by the model
    const parsedJson = JSON.parse(rawContent);

    /**
     * 3. Runtime Validation via Zod
     * 
     * Even though OpenAI's strict JSON mode guarantees schema compliance, 
     * running the data through Zod's .parse() guarantees complete type safety 
     * and triggers automatic TypeScript type narrowing.
     */
    const validatedGraph = KnowledgeGraphSchema.parse(parsedJson);

    return validatedGraph;

  } catch (error) {
    console.error("Error during Knowledge Graph extraction pipeline:", error);
    throw error;
  }
}

/**
 * 4. Execution Example for Our SaaS Workflow
 */
async function runDemo() {
  const sampleCustomerFeedback = `
    Customer alice_99 reported that the OAuth login module crashed 
    frequently when attempting single sign-on with Google Workspace. 
    Developer bob_dev investigated and noted that the OAuth token 
    service needs an immediate patch in version 2.1.4.
  `;

  console.log("Analyzing unstructured feedback and generating Knowledge Graph...");

  const graphResult = await extractKnowledgeGraph(sampleCustomerFeedback);

  console.log("\n--- Successfully Extracted Knowledge Graph ---");
  console.log(JSON.stringify(graphResult, null, 2));
}

// Execute the demo if run directly
if (require.main === module) {
  runDemo();
}

---

Detailed Line-by-Line Architectural Breakdown

To truly master zero-hallucination knowledge graph extraction, let's dissect the implementation choices, TypeScript features, and runtime behaviors powering our code.

1. Module Imports and Dependency Initialization

import { z } from "zod";
import OpenAI from "openai";
import { zodToJsonSchema } from "zod-to-json-schema";

• import { z } from "zod": Imports Zod, a TypeScript-first schema declaration and validation library. Zod allows us to declare schemas once; it automatically infers TypeScript static types and executes runtime validations. This eliminates the drift that typically occurs between static interface definitions and dynamic API payloads.
• import OpenAI from "openai": Imports the official Node.js SDK for interacting with OpenAI endpoints. This SDK provides strongly typed request and response objects, reducing boilerplate when handling network calls.
• import { zodToJsonSchema } from "zod-to-json-schema": Imports a utility package designed to convert complex Zod schemas into standard JSON Schema drafts. This conversion is necessary because modern LLM APIs (such as OpenAI's structured outputs) accept JSON Schema definitions to constrain the model's token generation space.

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY || "dummy-key-for-compilation",
});

• const openai = new OpenAI(...): Instantiates the OpenAI client singleton. We retrieve the API key from environment variables (process.env.OPENAI_API_KEY), providing a fallback string for compile-time safety checks in offline environments.

2. Defining the Zod Schema (KnowledgeGraphSchema)

const KnowledgeGraphSchema = z.object({
  nodes: z.array(
    z.object({
      id: z.string().describe("Unique identifier or slug for the entity (e.g., 'user_123', 'feature_auth')"),
      label: z.string().describe("The semantic category of the node (e.g., 'User', 'Feature', 'Bug', 'Product')"),
      properties: z.record(z.string(), z.any()).describe("Key-value attributes associated with this entity"),
    })
  ).describe("The collection of discrete entities extracted from the text."),

• z.object({ ... }): Defines an expected JSON object structure.
• nodes: z.array(...): Specifies that the root object must contain an array of node entities.
• .describe(...): This Zod modifier is critical. The string passed to .describe() is included in the generated JSON schema that gets passed to the LLM. It acts as embedded documentation for the model, giving it precise semantic context regarding what each property should contain.

---

The Role of Deterministic Solvers in Hybrid Architectures

As we look toward the broader scope of Neuro-Symbolic AI, extracting Knowledge Graphs from unstructured text is the foundational ingestion phase. Neuro-symbolic architectures combine the pattern-recognition strengths of neural networks with the logical rigor of symbolic solvers (such as description logic reasoners, constraint satisfaction solvers, and graph query engines).

If our neural extraction phase is leaky—allowing hallucinations, invalid types, or malformed topological structures to slip into the database—the downstream symbolic solvers will fail catastrophically. Symbolic reasoners are strictly logical; they cannot reason over contradictory or ill-formed graph structures. A single hallucinated edge asserting that an inanimate object "MANAGES" a human being can trigger cascading logical contradictions in automated reasoning engines.

Therefore, achieving zero-hallucination architectures in TypeScript is not just about writing clean validation code; it is about establishing a rigorous epistemological firewall between the chaotic world of human language and the pristine, absolute logic of graph databases and deterministic solvers. By mastering the integration of strict runtime validation schemas with LLM extraction pipelines, developers can build enterprise-grade knowledge systems that are simultaneously capable of understanding unstructured human nuance and operating with absolute mathematical and logical certainty.

The concepts and code demonstrated here are drawn directly from the comprehensive roadmap laid out in the book Neuro-Symbolic AI & Knowledge Graphs, you can find it here. Check also the many other ebooks.