DEV Community: Lakhan Malviya

Turn Database Docs into Agent-Ready Knowledge (OKF Series, Part 3)

Lakhan Malviya — Sat, 03 Oct 2026 04:41:06 +0000

An AI agent that writes SQL for your database needs more than the schema. It also needs to know what your business terms mean: which columns make up a "Resource Utilization Ratio", or which three conditions turn an operation "high-risk". Most teams already have that knowledge written down somewhere, in a schema dump, a column dictionary or a page of metric definitions. The trouble is that it's not in a form an agent can find its way through.

Over this part and the next, we build the pieces that close that gap for one real database, and run them end to end:

A compiler. A small Python package that reads a database's raw documentation and writes it as an OKF bundle: one concept file per table and per business rule, linked to each other, with an index file in each folder.
An agent. A simple SQL agent on an open model. It reads the bundle the way you'd browse a wiki: the index first, then only the files the question needs. Then it writes the SQL and runs it.
A side-by-side run. We ask the same questions twice: once with the bundle, and once with all the raw documentation pasted into the prompt. Then we compare what the agent read, how many tokens it used, and the SQL it wrote.

This part builds the first piece, the compiler, and runs it on all 18 databases in the benchmark. The next part builds the agent and runs the comparison. By the end of both, you'll have a working system on your laptop: one database, its bundle, and an agent you can ask your own questions.

Our database is disaster, from LiveSQLBench, a public text-to-SQL benchmark: 10 tables and 54 business rules about disaster-response operations. If you followed the previous part, you wrote three of its concept files by hand. Now we generate all of them. If you didn't, everything you need is in this post.

Set up the project and the database

We need two things before writing any code: a place for the compiler to live, and a real database for the agent to run its SQL against.

The project layout

We add a Python package, okf_compiler, next to the data/ and bundles/ folders. (If you're starting here, set up the project and download the data as described in the tutorial guide.)

Here's where we're heading. Each module is written, in full, in the step that needs it:

okf-sql-knowledge/
├── data/
│   └── livesqlbench-base-lite/
├── bundles/
├── okf_compiler/
│   ├── __init__.py
│   ├── readers.py      read the three raw files into Python
│   ├── concept.py      write one concept file: frontmatter + body
│   ├── tables.py       build one concept file per table
│   ├── rules.py        build one concept file per business rule, with links
│   ├── indexes.py      write the index files and the log
│   └── cli.py          one command to compile a database
├── docker-compose.yml
├── requirements.txt
└── README.md

Create the okf_compiler/ folder with an empty __init__.py inside it.

The database

The agent we build later runs real SQL, so it needs the real disaster database. The LiveSQLBench team publishes a PostgreSQL image with all 18 Base-Lite databases already loaded. You need Docker Desktop to run it; see the tutorial guide.

Create docker-compose.yml in the project folder:

services:
  postgres:
    image: docker.io/shawnxxh/bird-interact-postgresql:latest
    container_name: livesqlbench_postgresql
    environment:
      POSTGRES_USER: root
      POSTGRES_PASSWORD: "123123"
    ports:
      - "5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

Start it, and follow the log:

docker compose up -d
docker logs -f livesqlbench_postgresql

Wait until the log stops printing import messages, then press Ctrl+C. The container keeps running. You don't need the database until we build the agent, so you can move on to the next step while it loads.

Each benchmark database has _template added to its name, so ours is called disaster_template. Check that its tables are there:

docker exec -it livesqlbench_postgresql psql -U root -d disaster_template -c "\dt"

You should see the ten tables:

               List of relations
 Schema |            Name             | Type  | Owner
--------+-----------------------------+-------+-------
 public | beneficiariesandassessments | table | root
 public | coordinationandevaluation   | table | root
 public | disasterevents              | table | root
 public | distributionhubs            | table | root
 public | environmentandhealth        | table | root
 public | financials                  | table | root
 public | humanresources              | table | root
 public | operations                  | table | root
 public | supplies                    | table | root
 public | transportation              | table | root

The ten tables match the ten CREATE TABLE statements in disaster_schema.txt. The database holds the data, but nothing in it says how to turn hubutilpct into a Resource Utilization Ratio. That's what the bundle is for.

Port already in use? If PostgreSQL already runs on your laptop, change the port line to "5433:5432" and use port 5433 wherever this post connects to the database.

Read the raw files into Python

The compiler's first job is to turn three different file formats (a SQL text file, a JSON object and a JSON Lines file) into one set of Python objects. Every later step works with those objects and never touches the raw files again.

Create okf_compiler/readers.py:

"""Read the three raw LiveSQLBench files for one database into Python objects."""

import json
import re
from dataclasses import dataclass, field
from pathlib import Path


@dataclass
class Column:
    name: str
    sql_type: str
    nullable: bool
    meaning: str = ""
    json_fields: dict = field(default_factory=dict)  # only for JSONB columns


@dataclass
class ForeignKey:
    column: str
    ref_table: str
    ref_column: str


@dataclass
class Table:
    name: str
    columns: list[Column]
    primary_key: list[str]
    foreign_keys: list[ForeignKey]


@dataclass
class Rule:
    id: int
    name: str
    description: str
    definition: str
    kind: str
    depends_on: list[int]


COLUMN_LINE = re.compile(r"^(\w+)\s+(.+?)\s+(NOT NULL|NULL),?$")
PRIMARY_KEY = re.compile(r"PRIMARY KEY \(([^)]+)\)")
FOREIGN_KEY = re.compile(r"FOREIGN KEY \((\w+)\) REFERENCES (\w+)\((\w+)\)")


def read_schema(path: Path) -> list[Table]:
    tables = []
    for block in path.read_text(encoding="utf-8").split("CREATE TABLE ")[1:]:
        name = block.split('"')[1]
        body = block.split(");")[0]
        columns, primary_key, foreign_keys = [], [], []
        for line in body.splitlines()[1:]:
            line = line.strip()
            if m := COLUMN_LINE.match(line):
                columns.append(Column(m[1], m[2], m[3] == "NULL"))
            elif m := PRIMARY_KEY.search(line):
                primary_key = [c.strip() for c in m[1].split(",")]
            elif m := FOREIGN_KEY.search(line):
                foreign_keys.append(ForeignKey(m[1], m[2], m[3]))
        tables.append(Table(name, columns, primary_key, foreign_keys))
    return tables


def add_column_meanings(tables: list[Table], path: Path) -> list[str]:
    """Attach each column's meaning. Returns the columns that have none."""
    meanings = {}
    for key, value in json.loads(path.read_text(encoding="utf-8")).items():
        _, table, column = key.lower().split("|")
        meanings[(table, column)] = value

    missing = []
    for table in tables:
        for column in table.columns:
            value = meanings.get((table.name.lower(), column.name.lower()))
            if value is None:
                missing.append(f"{table.name}.{column.name}")
            elif isinstance(value, dict):  # JSONB column
                column.meaning = value.get("column_meaning", "")
                column.json_fields = value.get("fields_meaning", {})
            else:
                column.meaning = value
    return missing


def read_rules(path: Path) -> list[Rule]:
    rules = []
    for line in path.read_text(encoding="utf-8").splitlines():
        if not line.strip():
            continue
        raw = json.loads(line)
        children = raw["children_knowledge"]
        rules.append(Rule(
            id=raw["id"],
            name=raw["knowledge"],
            description=raw["description"],
            definition=raw["definition"],
            kind=raw["type"],
            depends_on=children if isinstance(children, list) else [],
        ))
    return rules

The four dataclasses are the shape every later module relies on: a Table with its Columns and ForeignKeys, and a Rule.

Notice four things the raw data forced on us:

The schema is split on CREATE TABLE, not on blank lines. Each table is followed by sample rows, and a table with no sample rows would otherwise run into the next one.
Names are matched in lowercase. The files mix cases: the column impactmetrics in the schema is impactMetrics in the descriptions, and some schemas have capitalized column names.
JSONB columns have a description for every nested field, not just one sentence. We keep those field descriptions in json_fields, because an agent can't query a JSON field it doesn't know exists.
children_knowledge is either a list or -1. We turn it into depends_on, which is always a list, so later code never has to check for -1.

add_column_meanings also returns the columns it couldn't find a description for, instead of failing quietly. It's our first report of what the source documentation is missing.

Check what was read

Create inspect_db.py in the project root, next to docker-compose.yml (not inside okf_compiler/). It reads one database and prints a summary:

import sys
from pathlib import Path

from okf_compiler.readers import add_column_meanings, read_rules, read_schema

name = sys.argv[1]
folder = Path("data/livesqlbench-base-lite") / name

tables = read_schema(folder / f"{name}_schema.txt")
missing = add_column_meanings(tables, folder / f"{name}_column_meaning_base.json")
rules = read_rules(folder / f"{name}_kb.jsonl")

print(f"tables:  {len(tables)}")
print(f"columns: {sum(len(t.columns) for t in tables)}")
print(f"columns without a meaning: {', '.join(missing) or 'none'}")
print(f"rules:   {len(rules)}")
print()
for table in tables:
    joins = ", ".join(fk.ref_table for fk in table.foreign_keys) or "-"
    print(f"{table.name:<28} {len(table.columns):>3} columns   joins: {joins}")

Run it for disaster:

python inspect_db.py disaster

The output:

tables:  10
columns: 123
columns without a meaning: none
rules:   54

beneficiariesandassessments   10 columns   joins: disasterevents, operations
disasterevents                 9 columns   joins: -
distributionhubs              11 columns   joins: disasterevents
environmentandhealth          13 columns   joins: disasterevents
operations                    12 columns   joins: disasterevents, distributionhubs
coordinationandevaluation     29 columns   joins: disasterevents, operations
humanresources                 4 columns   joins: disasterevents, operations
supplies                       4 columns   joins: disasterevents, distributionhubs
transportation                18 columns   joins: disasterevents, distributionhubs, supplies
financials                    13 columns   joins: disasterevents, operations

Ten tables, the same as the ten CREATE TABLE statements, and 54 rules, one per line of disaster_kb.jsonl. Every column has a description.

That isn't true for every database. Run it for mental:

python inspect_db.py mental

The output (the table list is cut):

tables:  9
columns: 105
columns without a meaning: patients.clinleadref
rules:   63
...

One column, patients.clinleadref, has no description in the source. The compiler can't invent one, so that column will reach the bundle with its name and type only. It's the first gap we've found in the source documentation, and the compiler should report gaps like this rather than hide them.

Your project now looks like this:

okf-sql-knowledge/
├── data/
│   └── livesqlbench-base-lite/
├── bundles/
├── okf_compiler/
│   ├── __init__.py
│   └── readers.py
├── docker-compose.yml
├── inspect_db.py
├── requirements.txt
└── README.md

Generate the table concepts

Each table becomes one concept file: the file an agent opens when it needs a table's columns, their meanings and its joins.

A helper for every concept file

Every concept file the compiler writes has the same shape: YAML frontmatter, then a markdown body. One small module writes that shape, so the table and rule modules don't repeat it.

Create okf_compiler/concept.py:

"""Write one concept file: YAML frontmatter, then a markdown body."""

from datetime import datetime
from pathlib import Path

import yaml

COMPILER = "okf_compiler/0.1"
SOURCE_BASE = "https://huggingface.co/datasets/birdsql/livesqlbench-base-lite/blob/main"


def generated() -> dict:
    now = datetime.now().astimezone().isoformat(timespec="seconds")
    return {"by": COMPILER, "at": now}


def source(database: str, filename: str, title: str) -> dict:
    return {
        "id": filename.split(".")[0].removeprefix(f"{database}_"),
        "resource": f"{SOURCE_BASE}/{database}/{filename}",
        "title": title,
    }


def write_concept(path: Path, frontmatter: dict, body: str) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    header = yaml.safe_dump(frontmatter, sort_keys=False, allow_unicode=True, width=1000)
    path.write_text(f"---\n{header}---\n\n{body.strip()}\n", encoding="utf-8")

Notice two things about the frontmatter it produces. (What each field means is covered in the introduction to OKF.)

generated.by is the compiler and its version, not a person. A reader of the bundle can tell these files were written by okf_compiler/0.1, and when.
There is no verified field. Nobody has reviewed these files, and the bundle says so. If someone checks a concept later, they add verified to that file.

The table module

Create okf_compiler/tables.py:

"""Build one concept file per table."""

from pathlib import Path

from okf_compiler.concept import generated, source, write_concept
from okf_compiler.readers import Table


def describe(table: Table) -> str:
    keys = set(table.primary_key) | {fk.column for fk in table.foreign_keys}
    names = [c.name for c in table.columns if c.name not in keys]
    text = f"{len(table.columns)} columns: {', '.join(names)}."
    joins = sorted({fk.ref_table for fk in table.foreign_keys})
    if joins:
        text += f" Joins to {', '.join(joins)}."
    return text


def cell(text: str) -> str:
    return text.replace("|", "\\|").replace("\n", " ")


def json_field_lines(prefix: str, fields: dict) -> list[str]:
    lines = []
    for name, value in fields.items():
        path = f"{prefix}.{name}"
        if isinstance(value, dict):
            lines += json_field_lines(path, value)
        else:
            lines.append(f"* `{path}`: {value}")
    return lines


def table_body(table: Table, related: list[tuple[str, str]]) -> str:
    parts = ["# Schema", "", "| Column | Type | Meaning |", "|---|---|---|"]
    for c in table.columns:
        sql_type = c.sql_type + (", primary key" if c.name in table.primary_key else "")
        parts.append(f"| `{c.name}` | {sql_type} | {cell(c.meaning)} |")

    json_lines = []
    for c in table.columns:
        json_lines += json_field_lines(c.name, c.json_fields)
    if json_lines:
        parts += ["", "# JSON fields", "", *json_lines]

    if table.foreign_keys:
        parts += ["", "# Joins", ""]
        for fk in table.foreign_keys:
            parts.append(
                f"* `{fk.column}` references `{fk.ref_column}` in "
                f"[{fk.ref_table}](/tables/{fk.ref_table}.md)."
            )

    if related:
        parts += ["", "# Related knowledge", ""]
        for title, slug in related:
            parts.append(f"* [{title}](/knowledge/{slug}.md)")
    return "\n".join(parts)


def write_table(table: Table, database: str, bundle: Path, related: list[tuple[str, str]]) -> None:
    frontmatter = {
        "type": "PostgreSQL Table",
        "title": table.name,
        "description": describe(table),
        "tags": [database],
        "generated": generated(),
        "sources": [
            source(database, f"{database}_schema.txt", f"{database} schema (LiveSQLBench)"),
            source(database, f"{database}_column_meaning_base.json",
                   f"{database} column descriptions (LiveSQLBench)"),
        ],
    }
    write_concept(bundle / "tables" / f"{table.name}.md", frontmatter, table_body(table, related))

The body keeps every column's meaning word for word, because that's where details like enum values live (warehousestate is Fair, Excellent, Good or Poor). JSONB columns get an extra section that lists every nested field as a path, such as impactmetrics.population.affected, ready to use in a query.

Design choice: table descriptions come from column names, not an LLM.
description is the one line an agent sees in the index before deciding whether to open the file. A person would write "one row per distribution hub, with its capacity and storage". A compiler can't write that sentence without an LLM, and an LLM would add wording that isn't in the source docs, which would give the bundle knowledge the other setups don't get. So describe lists the table's column names and the tables it joins to: plain, but taken straight from the source. If the agent struggles to pick the right tables, this is the first thing to improve.

write_table also takes a list of related business rules. It's empty for now; the next step fills it, so each table links to the rules that use its columns.

The command

Create okf_compiler/cli.py. For now it only writes the table concepts; we extend it in the next steps:

"""Compile one LiveSQLBench database into an OKF bundle."""

import shutil
import sys
from pathlib import Path

from okf_compiler.readers import add_column_meanings, read_rules, read_schema
from okf_compiler.tables import write_table

DATA = Path("data/livesqlbench-base-lite")
OUT = Path("bundles/compiled")


def compile_database(name: str) -> None:
    folder = DATA / name
    tables = read_schema(folder / f"{name}_schema.txt")
    missing = add_column_meanings(tables, folder / f"{name}_column_meaning_base.json")
    rules = read_rules(folder / f"{name}_kb.jsonl")

    bundle = OUT / name
    shutil.rmtree(bundle, ignore_errors=True)
    for table in tables:
        write_table(table, name, bundle, related=[])

    print(f"{name}: {len(tables)} table concepts written to {bundle}")
    if missing:
        print(f"  columns without a description: {', '.join(missing)}")


if __name__ == "__main__":
    compile_database(sys.argv[1])

The compiler writes to bundles/compiled/<database>, so it never overwrites a bundle you wrote by hand, and it clears that folder first, so the bundle always matches the current source files.

Run it:

python -m okf_compiler.cli disaster

The output:

disaster: 10 table concepts written to bundles\compiled\disaster

Open bundles/compiled/disaster/tables/distributionhubs.md. It should look like this (most columns cut):

---
type: PostgreSQL Table
title: distributionhubs
description: '11 columns: hubcaptons, hubutilpct, storecapm3, storeavailm3, coldstorecapm3, coldstoretempc, warehousestate, invaccpct, stockturnrate. Joins to disasterevents.'
tags:
- disaster
generated:
  by: okf_compiler/0.1
  at: '2026-10-03T00:03:25+05:30'
sources:
- id: schema
  resource: https://huggingface.co/datasets/birdsql/livesqlbench-base-lite/blob/main/disaster/disaster_schema.txt
  title: disaster schema (LiveSQLBench)
- id: column_meaning_base
  resource: https://huggingface.co/datasets/birdsql/livesqlbench-base-lite/blob/main/disaster/disaster_column_meaning_base.json
  title: disaster column descriptions (LiveSQLBench)
---

# Schema

| Column | Type | Meaning |
|---|---|---|
| `hubregistry` | character varying, primary key | A VARCHAR(20) primary key identifying each distribution hub record (e.g., 'HUB0001'). |
| `disteventref` | character varying | A VARCHAR(20) referencing DisasterEvents(DistRegistry), linking this hub to a specific disaster event (e.g., 'DIST0001'). |
... 9 more columns

# Joins

* `disteventref` references `distregistry` in [disasterevents](/tables/disasterevents.md).

Your project now looks like this:

okf-sql-knowledge/
├── data/
│   └── livesqlbench-base-lite/
├── bundles/
│   └── compiled/
│       └── disaster/
│           └── tables/          10 concept files
├── okf_compiler/
│   ├── __init__.py
│   ├── cli.py
│   ├── concept.py
│   ├── readers.py
│   └── tables.py
├── docker-compose.yml
├── inspect_db.py
├── requirements.txt
└── README.md

Generate the rule concepts and their links

The business rules are what the schema can't tell an agent: what "Resource Utilization Ratio" means, or which conditions make an operation "high-risk". Each rule becomes one concept file, linked to the rules it depends on and to the tables whose columns it uses.

Create okf_compiler/rules.py:

"""Build one concept file per business rule, and link rules to tables."""

import re
from pathlib import Path

from okf_compiler.concept import generated, source, write_concept
from okf_compiler.readers import Rule, Table

TYPE_NAMES = {
    "calculation_knowledge": "Calculation",
    "domain_knowledge": "Business Rule",
    "value_illustration": "Value Illustration",
}
WORD = re.compile(r"[A-Za-z_][A-Za-z0-9_]*")


def make_slugs(rules: list[Rule]) -> dict[int, str]:
    """A file name for every rule, e.g. 'resource-utilization-ratio'."""
    slugs, used = {}, set()
    for rule in rules:
        name = re.sub(r"\([^)]*\)", "", rule.name)
        slug = re.sub(r"[^a-z0-9]+", "-", name.lower()).strip("-")
        if slug in used:
            slug = f"{slug}-{rule.id}"
        used.add(slug)
        slugs[rule.id] = slug
    return slugs


def find_tables(rule: Rule, tables: list[Table]) -> dict[str, list[str]]:
    """Tables whose column names appear in the rule's name or definition."""
    words = {w.lower() for w in WORD.findall(f"{rule.name} {rule.definition}")}
    found = {}
    for table in tables:
        columns = [c.name for c in table.columns if c.name.lower() in words]
        if columns:
            found[table.name] = columns
    return found


def rule_body(rule: Rule, rules_by_id: dict[int, Rule], used_by: list[int],
              slugs: dict[int, str], tables: dict[str, list[str]]) -> str:
    parts = ["# Definition", "", rule.definition]

    if tables:
        parts += ["", "# Columns used", ""]
        for table, columns in tables.items():
            names = ", ".join(f"`{c}`" for c in columns)
            parts.append(f"* [{table}](/tables/{table}.md): {names}")

    for heading, ids in (("Depends on", rule.depends_on), ("Used by", used_by)):
        links = [f"* [{rules_by_id[i].name}](/knowledge/{slugs[i]}.md)"
                 for i in ids if i in rules_by_id]
        if links:
            parts += ["", f"# {heading}", "", *links]
    return "\n".join(parts)


def write_rules(rules: list[Rule], tables: list[Table], database: str,
                bundle: Path) -> dict[str, list[tuple[str, str]]]:
    """Write every rule concept. Returns, per table, the rules that use it."""
    slugs = make_slugs(rules)
    rules_by_id = {r.id: r for r in rules}
    used_by = {r.id: [] for r in rules}
    for rule in rules:
        for parent in rule.depends_on:
            if parent in used_by:
                used_by[parent].append(rule.id)

    related = {t.name: [] for t in tables}
    for rule in rules:
        found = find_tables(rule, tables)
        for table in found:
            related[table].append((rule.name, slugs[rule.id]))

        frontmatter = {
            "type": TYPE_NAMES.get(rule.kind, rule.kind),
            "title": rule.name,
            "description": rule.description,
            "tags": [database],
            "generated": generated(),
            "sources": [source(database, f"{database}_kb.jsonl",
                               f"{database} business rules (LiveSQLBench), rule {rule.id}")],
        }
        body = rule_body(rule, rules_by_id, used_by[rule.id], slugs, found)
        write_concept(bundle / "knowledge" / f"{slugs[rule.id]}.md", frontmatter, body)
    return related

Notice three things:

Each rule type gets a readable OKF type. calculation_knowledge becomes Calculation, domain_knowledge becomes Business Rule, and value_illustration becomes Value Illustration.
Dependencies become links in both directions. depends_on comes straight from the source. Used by is the reverse, built by the compiler, so an agent that lands on either rule can reach the other.
The definition is copied as it is, LaTeX included. Converting formulas to plain text would risk changing a formula, and LLMs read LaTeX well.

find_tables is the only guess the compiler makes. The source never says which table a rule belongs to. So we look for column names inside the rule's name and definition: rule 10 mentions hubutilpct, storecapm3 and storeavailm3, and all three are columns of distributionhubs. write_rules returns those matches per table, so each table concept can list the rules that use it.

One more detail: two rules can have the same name once the brackets are removed. make_slugs adds the rule's id to the second one, so no file overwrites another.

Update the command

Replace okf_compiler/cli.py:

"""Compile one LiveSQLBench database into an OKF bundle."""

import shutil
import sys
from pathlib import Path

from okf_compiler.readers import add_column_meanings, read_rules, read_schema
from okf_compiler.rules import find_tables, write_rules
from okf_compiler.tables import write_table

DATA = Path("data/livesqlbench-base-lite")
OUT = Path("bundles/compiled")


def compile_database(name: str) -> None:
    folder = DATA / name
    tables = read_schema(folder / f"{name}_schema.txt")
    missing = add_column_meanings(tables, folder / f"{name}_column_meaning_base.json")
    rules = read_rules(folder / f"{name}_kb.jsonl")

    bundle = OUT / name
    shutil.rmtree(bundle, ignore_errors=True)
    related = write_rules(rules, tables, name, bundle)
    for table in tables:
        write_table(table, name, bundle, related[table.name])

    unlinked = [r for r in rules if not find_tables(r, tables)]
    print(f"{name}: {len(tables)} tables, {len(rules)} rules written to {bundle}")
    if missing:
        print(f"  columns without a description: {', '.join(missing)}")
    if unlinked:
        print(f"  rules not linked to any table ({len(unlinked)}):")
        for rule in unlinked:
            print(f"    {rule.id}: {rule.name}")


if __name__ == "__main__":
    compile_database(sys.argv[1])

It now writes the rules first, because the table concepts need to know which rules link to them. It also prints a report of every rule it couldn't link to a table.

Run it:

python -m okf_compiler.cli disaster

The output:

disaster: 10 tables, 54 rules written to bundles\compiled\disaster
  rules not linked to any table (4):
    42: Sustainable Operation Excellence
    50: Resource Utilization Classification
    51: Environmental Impact Classification
    52: Community Resilience Classification

Four of the 54 rules are not linked to a table. Look at their names: three are classifications, such as rule 50, Resource Utilization Classification. Rule 50 never mentions a column. It says "RUR > 5", and RUR is rule 10. So it isn't lost: an agent reaches its table in two hops, through the Depends on link to rule 10, then rule 10's link to distributionhubs.

That's why the compiler reports these rules instead of guessing. A missing link costs the agent an extra hop. A wrong link sends it to the wrong table.

How many rules end up unlinked depends on how the source docs are written. Run the compiler on news:

python -m okf_compiler.cli news

The output (most of the list is cut):

news: 8 tables, 61 rules written to bundles\compiled\news
  rules not linked to any table (48):
    6: statusWeight
    7: genderFactor
    8: occupationFactor
    10: Premium Content Rule (PCR)
    11: Personalization Priority (PP)
    12: AB Testing Cohort Analysis (ABTCA)
    ... 42 more

In news, most rules are written in business language and never name a column. Rule 6, statusWeight, says 'Premium' = 2.0, 'Enterprise' = 3.0, but not which column holds the subscription tier. Others, such as rule 31, are built only from other rules' abbreviations. Name matching can't link these, so for those rules the agent has to find the right column itself, exactly as it would with the raw docs. The report tells you where your own documentation would need a human to add the missing link.

Open bundles/compiled/disaster/knowledge/resource-utilization-ratio.md. It should look like this:

---
type: Calculation
title: Resource Utilization Ratio (RUR)
description: Measures how effectively hub capacity is being used relative to available resources
tags:
- disaster
generated:
  by: okf_compiler/0.1
  at: '2026-10-03T00:11:39+05:30'
sources:
- id: kb
  resource: https://huggingface.co/datasets/birdsql/livesqlbench-base-lite/blob/main/disaster/disaster_kb.jsonl
  title: disaster business rules (LiveSQLBench), rule 10
---

# Definition

RUR = \frac{hubutilpct}{100} \times \frac{storecapm3}{storeavailm3 + 1}

# Columns used

* [distributionhubs](/tables/distributionhubs.md): `hubutilpct`, `storecapm3`, `storeavailm3`

# Used by

* [Financial Vulnerability Zone](/knowledge/financial-vulnerability-zone.md)
* [Resource Utilization Classification](/knowledge/resource-utilization-classification.md)

Generate the index files and validate

The bundle now has 64 concept files. An agent can't open all of them for every question, so it needs a menu first: an index.md in each folder that lists every concept with its one-line description. The agent reads the menu, then opens only what it needs.

Every line in an index comes from a concept file's frontmatter: its title, its file name and its description. So the index module doesn't need the raw data at all. It reads the concept files the compiler just wrote. That also means it works on any bundle folder, including one you edit by hand.

Create okf_compiler/indexes.py:

"""Write the index files and the log for a bundle."""

from datetime import date
from pathlib import Path

import yaml

from okf_compiler.concept import COMPILER

FOLDERS = {
    "tables": "PostgreSQL tables, with column meanings, JSON fields and joins.",
    "knowledge": "Business rules: calculations, definitions and value illustrations.",
}


def read_frontmatter(path: Path) -> dict:
    text = path.read_text(encoding="utf-8")
    return yaml.safe_load(text.split("---", 2)[1])


def write_folder_index(folder: Path) -> int:
    groups: dict[str, list[str]] = {}
    for path in sorted(folder.glob("*.md")):
        if path.name == "index.md":
            continue
        meta = read_frontmatter(path)
        line = f"* [{meta['title']}]({path.name}) - {meta['description']}"
        groups.setdefault(meta["type"], []).append(line)

    parts = []
    for concept_type, lines in sorted(groups.items()):
        parts += [f"# {concept_type}", "", *lines, ""]
    (folder / "index.md").write_text("\n".join(parts), encoding="utf-8")
    return sum(len(lines) for lines in groups.values())


def write_indexes(bundle: Path, database: str) -> None:
    counts = {name: write_folder_index(bundle / name) for name in FOLDERS}

    root = ["---", 'okf_version: "0.2"', "---", "", f"# {database} database", ""]
    for name, text in FOLDERS.items():
        root.append(f"* [{name.capitalize()}]({name}/index.md) - {text}")
    (bundle / "index.md").write_text("\n".join(root) + "\n", encoding="utf-8")

    log = [
        "# Update Log",
        "",
        f"## {date.today().isoformat()}",
        f"* **Creation**: Compiled {counts['tables']} tables and {counts['knowledge']} "
        f"business rules from LiveSQLBench `{database}` with `{COMPILER}`.",
    ]
    (bundle / "log.md").write_text("\n".join(log) + "\n", encoding="utf-8")

Notice three things:

Each folder index is grouped by type. In knowledge/index.md, calculations, business rules and value illustrations each get their own heading, so an agent looking for a formula can skip the definitions.
Only the root index has frontmatter, and only okf_version. OKF allows no other frontmatter in index files.
The log has a single entry. The compiler rebuilds the bundle from scratch on every run, so the log records one thing: which compiler version built it from which source, and when.

The final command

Replace okf_compiler/cli.py one last time. It now writes the indexes, and it accepts all to compile every database:

"""Compile one LiveSQLBench database into an OKF bundle."""

import shutil
import sys
from pathlib import Path

from okf_compiler.indexes import write_indexes
from okf_compiler.readers import add_column_meanings, read_rules, read_schema
from okf_compiler.rules import find_tables, write_rules
from okf_compiler.tables import write_table

DATA = Path("data/livesqlbench-base-lite")
OUT = Path("bundles/compiled")


def compile_database(name: str) -> None:
    folder = DATA / name
    tables = read_schema(folder / f"{name}_schema.txt")
    missing = add_column_meanings(tables, folder / f"{name}_column_meaning_base.json")
    rules = read_rules(folder / f"{name}_kb.jsonl")

    bundle = OUT / name
    shutil.rmtree(bundle, ignore_errors=True)
    related = write_rules(rules, tables, name, bundle)
    for table in tables:
        write_table(table, name, bundle, related[table.name])
    write_indexes(bundle, name)

    unlinked = [r for r in rules if not find_tables(r, tables)]
    print(f"{name}: {len(tables)} tables, {len(rules)} rules written to {bundle}")
    if missing:
        print(f"  columns without a description: {', '.join(missing)}")
    if unlinked:
        print(f"  rules not linked to any table ({len(unlinked)}):")
        for rule in unlinked:
            print(f"    {rule.id}: {rule.name}")


if __name__ == "__main__":
    if sys.argv[1] == "all":
        for folder in sorted(DATA.iterdir()):
            if (folder / f"{folder.name}_schema.txt").exists():
                compile_database(folder.name)
    else:
        compile_database(sys.argv[1])

Compile disaster again:

python -m okf_compiler.cli disaster

The output is the same report as before. The difference is in the bundle: it now has an index in each folder. Open bundles/compiled/disaster/knowledge/index.md. It should look like this (each group cut short):

# Business Rule

* [Community Resilience Builder](community-resilience-builder.md) - Identifies operations that strengthen local community capacity
...
* [Vulnerable Population Hotspot](vulnerable-population-hotspot.md) - Identifies areas with highly vulnerable populations requiring priority attention

# Calculation

* [available storage percentage](available-storage-percentage.md) - Calculates what proportion of total storage capacity is currently available
...
* [Supply Chain Sustainability Index (SCSI)](supply-chain-sustainability-index.md) - Assesses the environmental sustainability of the disaster supply chain

# Value Illustration

* [coordeffectlvl](coordeffectlvl.md) - Illustrates the quality of coordination between responding agencies
...
* [staffingProfile.readiness.ppe_status](staffingprofile-readiness-ppe-status.md) - Illustrates the availability of Personal Protective Equipment

Validate

Now check the bundle against the OKF rules with the validator from okf-skills, pinned to commit 8e31878 (download steps in the tutorial guide):

python tools/okf_validate.py bundles/compiled/disaster

The output:

OKF v0.2 conformance — bundles\compiled\disaster
  concepts: 64   index.md: 3   log.md: 1
  ✓ conformant — no issues

All 64 concepts, the 3 index files and the log are conformant, with no warnings. Every link points to a file that exists, because the compiler only writes links to tables and rules it has just generated.

All 18 databases

The same command works for every database in Base-Lite:

python -m okf_compiler.cli all

It prints one report per database (most of it cut):

alien: 11 tables, 56 rules written to bundles\compiled\alien
  rules not linked to any table (10):
    40: High-Confidence Technosignature
    41: Habitable Zone Transmission
    ... 8 more
... 16 more databases
virtual: 14 tables, 55 rules written to bundles\compiled\virtual
  rules not linked to any table (15):
    12: Content Creation Impact Score (CCIS)
    ... 14 more

Then validate them all. On macOS or Linux:

for db in bundles/compiled/*/; do python tools/okf_validate.py "$db" | tail -1; done

On Windows (PowerShell):

Get-ChildItem bundles/compiled -Directory | ForEach-Object { python tools/okf_validate.py $_.FullName | Select-Object -Last 1 }

Each bundle prints its last line, so you should see 18 of these:

  ✓ conformant — no issues
  ✓ conformant — no issues
  ✓ conformant — no issues
  ...

Seeing Γ£ô instead of ✓ in PowerShell? That's only the console's text encoding, not a problem with the bundles. Run [Console]::OutputEncoding = [Text.Encoding]::UTF8 once, then run the loop again.

Eighteen databases, eighteen conformant bundles.

What we built, and what we learned

We now have a compiler that turns a database's raw documentation into an OKF bundle with one command:

One concept file per table, with every column's meaning, its JSON fields and its joins.
One concept file per business rule, linked to the rules it depends on, the rules that use it, and the tables whose columns it names.
An index in every folder, so an agent can read a short menu before opening anything.
A report of what's missing: columns without a description, and rules the compiler couldn't link to a table.

Three lessons from running it on real documentation:

A compiler should report gaps, not fill them. It can't invent a description for patients.clinleadref, and it shouldn't guess which table a rule belongs to. A missing link costs the agent one extra hop; a wrong link sends it to the wrong table.
How well the links come out depends on how the docs are written. In disaster, 50 of 54 rules name their columns, so they link on their own. In news, 48 of 61 don't. The compiler can only link what the source spells out, and its report shows exactly where a human would need to add the missing link.
Keep generated knowledge honest. Every file says which compiler wrote it, from which source file and when, and none of them claims to be verified. Anyone reading the bundle can tell it was generated, not reviewed.

What's next

The bundle is ready, but nothing reads it yet. In the next part, we build the SQL agent on an open model, point it at the disaster bundle, and ask it questions twice: once browsing the bundle, and once with all the raw documentation pasted into its prompt. Then we compare what it read, how many tokens it used, and whether its SQL was right.

The full code for this part is in the okf-sql-knowledge repository.

Sources

LiveSQLBench: the text-to-SQL benchmark the disaster database comes from.
LiveSQLBench Base-Lite dataset on Hugging Face: the raw schema, column descriptions and business rules for all 18 databases.
LiveSQLBench PostgreSQL image: the database the agent will query.
okf-skills: the OKF validator used in this part (commit 8e31878).
Your agent's knowledge needs a file format: meet OKF: the introduction to OKF and its frontmatter fields.
The previous part: writing three disaster concept files by hand.
Tutorial guide: project setup, Docker and validator download steps.

Build a Knowledge Layer for SQL Agents with OKF (Part 2)

Lakhan Malviya — Sat, 03 Oct 2026 04:22:07 +0000

You connect an AI agent to your shop's database and ask a simple question: "How many active customers do we have?"

The agent looks at the tables:

customers (customer_id, name, status, created_at)
orders    (order_id, customer_id, amount, paid, ordered_at)

It spots a status column with the value 'active', writes a query, and gives you a number in two seconds:

SELECT COUNT(*) FROM customers WHERE status = 'active';

The number is wrong.

In your company, status = 'active' only means the account hasn't been closed. When finance says "active customer", they mean someone who placed a paid order in the last 90 days. Everyone on the data team knows this. It's written in a wiki page and built into the revenue dashboard. But it's not in the database, so the agent never saw it.

SELECT COUNT(DISTINCT customer_id)
FROM orders
WHERE paid = TRUE
  AND ordered_at >= CURRENT_DATE - INTERVAL '90 days';

This is the real problem with SQL agents. The database stores your data, but not the rules for reading it. Those rules live in wikis, dashboards, dbt models and people's heads. A new analyst picks them up by asking around. An agent can only read what you give it, and when a rule is missing, it guesses. The guess runs without errors, so nobody notices.

Is this a real problem?

It's fair to ask whether the shop example is realistic or just made up to prove a point. A public benchmark puts agents in exactly this situation.

LiveSQLBench tests how well AI agents turn questions into SQL. It contains 18 databases on different topics, from disaster response to crypto trading. Each database comes with three files:

the schema: tables and columns,
a description of every column,
a separate file of business rules.

The questions use business terms, and many of those terms are defined only in the rules file. It's the same situation as our shop: the database holds the data, and the definitions live somewhere else.

Here's one case from the disaster-response database. Two of its tables look like this (only the relevant columns are shown):

operations                (opsregistry, emerglevel, ...)
coordinationandevaluation (coordopsref, safetyranking, secincidentcount, ...)

Ask an agent for "high-risk operations", and it will spot the safetyranking column, which has a value called High Risk, and filter on it, just as our shop agent filtered on status. But the rules file defines a high-risk operation as one where emerglevel is Red or Black, safetyranking is High Risk, and secincidentcount is more than 50. That's three conditions across two tables. The obvious guess catches one.

It's a hard test. At the time of writing, the best system on the LiveSQLBench leaderboard answers 48% of its tasks correctly.

What we'll build

So how do you give an agent the rules it's missing? Teams usually try one of two things.

The first is to paste everything into the prompt: every table, every column description, every rule. That works for a small database. But you pay for all those tokens on every question, and for a large database the documents don't fit in the model's context window at all.

The second is to let the agent guess from the schema alone. It's cheap, and it gives you the wrong answers we just saw.

This series builds a third option: a knowledge layer. We take the schema, the column descriptions and the business rules, and store them as an OKF bundle: a folder of small markdown files called concept files, one per table and one per business rule, with an index file in each folder.

An agent can read an OKF bundle in more than one way. It can search the files, follow the links between them like a graph, or walk through the index files. We start with the simplest, called progressive disclosure: the agent reads the top index first, opens only the concept files the question needs, and then writes SQL. OKF's index files exist for exactly this, although they're optional and other readers can work without them.

Raw docs: schema, column descriptions, business rules
        │   compiler
        ▼
OKF bundle: concept files + index files
        │   agent tools (plain Python, then MCP)
        ▼
Agent opens what it needs → writes SQL
        │   evaluation harness
        ▼
Score on LiveSQLBench

The series has four parts. Each one ends with something you can run on your laptop:

The problem and your first bundle (this post). Set up the project, explore one LiveSQLBench database, write a small OKF bundle by hand, and validate it.
The compiler. A Python script that turns a database's raw files into a full OKF bundle automatically, for all 18 databases.
The agent. Run the benchmark database locally in Docker, then build an agent that reads the bundle through progressive disclosure and writes SQL: first in plain Python, then over MCP (Model Context Protocol), so any MCP-compatible agent can use the same knowledge.
The honest test. Run the same benchmark questions four ways with open models: schema only, everything in the prompt, progressive disclosure over the bundle, and search over the same bundle. Then compare accuracy, tokens and cost, whatever the results show.

Set up the project

You need Python 3.11 or newer (the OKF validator we use later in this post requires it) and Git. Step-by-step setup for macOS and Windows is in the tutorial guide in the project repo. (If you'd rather run the finished code than build it yourself, follow the README instead.)

Following the guide, you create a project folder with this structure:

okf-sql-knowledge/
├── data/       raw LiveSQLBench files
└── bundles/    the OKF bundles we write

Both folders start empty. We fill data/ next and bundles/ later in this post.

Get the data

LiveSQLBench comes in several releases. We use Base-Lite, the smallest one: 18 databases and 270 questions. Download it into data/ as shown in the tutorial guide. It's only a few megabytes.

Here's what you get:

data/livesqlbench-base-lite/
├── README.md
├── livesqlbench_data.jsonl
├── alien/
├── archeology/
├── credit/
├── disaster/
├── ...              (18 database folders in total)
└── virtual/

There are two kinds of things here:

One folder per database (alien, credit, disaster, …). Each folder describes one database: its tables, its columns and its business rules.
livesqlbench_data.jsonl: the questions, one per line, 15 for each database.

Here's the first question for the disaster database, shortened to the fields that matter now:

{
  "instance_id": "disaster_1",
  "selected_database": "disaster",
  "query": "I need to analyze all distribution hubs based on their Resource Utilization Ratio. Please show the hub registry ID, the calculated RUR value, and their Resource Utilization Classification. Sort the results by RUR from highest to lowest.",
  "sol_sql": []
}

Notice two things.

First, the question asks for a "Resource Utilization Ratio". No column has that name. Its formula lives in the business rules, which is exactly the problem from the start of this post.

Second, sol_sql, the correct answer, is empty. The benchmark's authors hold back the answers so automated web crawlers can't collect them. You can request them by email, as the dataset page explains. We won't need them until we test the agent.

From here on, we work with the disaster database. Its folder has three files, each around 20 KB:

data/livesqlbench-base-lite/disaster/
├── disaster_schema.txt
├── disaster_column_meaning_base.json
└── disaster_kb.jsonl

Read the three raw files

We'll follow the question we just saw. To answer it, an agent needs the distribution hubs table, the formula for the Resource Utilization Ratio, and the rule that turns that ratio into a classification. Each piece lives in a different file. (For a full tour of every file and field, see data/README.md in the repo.)

The schema: `disaster_schema.txt`

Plain text, one CREATE TABLE statement per table, each followed by three sample rows. The table we need has 11 columns:

CREATE TABLE "distributionhubs" (
hubregistry character varying NOT NULL,
disteventref character varying NULL,
hubcaptons numeric NULL,
hubutilpct numeric NULL,
storecapm3 numeric NULL,
storeavailm3 numeric NULL,
coldstorecapm3 numeric NULL,
coldstoretempc numeric NULL,
warehousestate USER-DEFINED NULL,
invaccpct numeric NULL,
stockturnrate numeric NULL,
    PRIMARY KEY (hubregistry),
    FOREIGN KEY (disteventref) REFERENCES disasterevents(distregistry)
);

The schema gives names, types and joins. It doesn't say what hubutilpct or storeavailm3 mean.

The column descriptions: `disaster_column_meaning_base.json`

One sentence per column, keyed by database|table|column:

"disaster|distributionhubs|hubutilpct": "A DECIMAL(7,3) showing the percentage of hub capacity currently utilized (e.g., 85.300)."

Now hubutilpct makes sense: it's the percentage of the hub's capacity in use.

The business rules: `disaster_kb.jsonl`

This is the file our agent was missing in the opening. It holds 54 rules, one JSON object per line. Here are the two our question needs, formatted for reading:

{
  "id": 10,
  "knowledge": "Resource Utilization Ratio (RUR)",
  "description": "Measures how effectively hub capacity is being used relative to available resources",
  "definition": "RUR = \\frac{hubutilpct}{100} \\times \\frac{storecapm3}{storeavailm3 + 1}",
  "type": "calculation_knowledge",
  "children_knowledge": -1
}

{
  "id": 50,
  "knowledge": "Resource Utilization Classification",
  "description": "Categorizes distribution hubs based on their Resource Utilization Ratio (RUR) values",
  "definition": "High Utilization (RUR > 5) indicates potentially overloaded hubs that may need resource expansion; Moderate Utilization (2 ≤ RUR ≤ 5) represents optimal resource usage balance; Low Utilization (RUR < 2) indicates underutilized hubs with potential efficiency gains through resource reallocation",
  "type": "domain_knowledge",
  "children_knowledge": [10]
}

Three fields matter most for what we build next:

type says what kind of rule it is: a formula (calculation_knowledge), a business definition (domain_knowledge), or an explanation of a column's values (value_illustration).
definition is the rule itself. Rule 10 is a LaTeX formula: RUR = (hubutilpct / 100) × (storecapm3 / (storeavailm3 + 1)).
children_knowledge lists the rules this one depends on. Rule 50 lists [10], because you can't classify a hub without first computing its RUR.

Notice what's missing. Rule 10 names three columns but not their table. To learn that they belong to distributionhubs, you have to go back to the schema.

What we learned

No file is enough on its own:

File	Answers	Example
Schema	What tables and columns exist, and how they join	`distributionhubs` has `hubutilpct`
Column descriptions	What each column means	`hubutilpct` is the percentage of capacity in use
Business rules	What the business terms mean	RUR is a formula over three of those columns

Write your first concept files by hand

Now we turn those pieces into OKF. A concept file is a markdown file with a block of YAML at the top, called frontmatter, and a normal markdown body below it. (What each frontmatter field means is covered in the introduction to OKF; here we focus on using them.)

We'll write three concept files: one for the table, and one for each rule. Your project will look like this:

okf-sql-knowledge/
├── data/
│   └── livesqlbench-base-lite/
├── bundles/
│   └── disaster/
│       ├── tables/
│       │   └── distributionhubs.md
│       └── knowledge/
│           ├── resource-utilization-ratio.md
│           └── resource-utilization-classification.md
├── requirements.txt
└── README.md

Tables go in tables/, rules go in knowledge/. OKF doesn't require any particular folder names; this split simply makes the bundle easy to browse.

The table concept

Create bundles/disaster/tables/distributionhubs.md:

---
type: PostgreSQL Table
title: distributionhubs
description: One row per disaster-relief distribution hub, with its capacity, storage space, cold storage, warehouse condition and inventory measures.
tags: [disaster, logistics]
generated: { by: human:your-name, at: 2026-10-02T17:00:00+05:30 }
sources:
  - id: schema
    resource: https://huggingface.co/datasets/birdsql/livesqlbench-base-lite/blob/main/disaster/disaster_schema.txt
    title: disaster schema (LiveSQLBench)
  - id: column-meanings
    resource: https://huggingface.co/datasets/birdsql/livesqlbench-base-lite/blob/main/disaster/disaster_column_meaning_base.json
    title: disaster column descriptions (LiveSQLBench)
---

# Schema

| Column | Type | Meaning |
|---|---|---|
| `hubregistry` | varchar, primary key | Hub ID, for example `HUB_HS0I`. |
| `disteventref` | varchar | The disaster event this hub serves. |
| `hubcaptons` | numeric | Maximum capacity of the hub, in tons. |
| `hubutilpct` | numeric | Percentage of the hub's capacity currently in use. |
| `storecapm3` | numeric | Total storage capacity, in cubic meters. |
| `storeavailm3` | numeric | Storage still available, in cubic meters. |
| `coldstorecapm3` | numeric | Cold-storage capacity, in cubic meters. |
| `coldstoretempc` | numeric | Temperature of the cold storage, in °C. |
| `warehousestate` | enum | Warehouse condition: `Fair`, `Excellent`, `Good` or `Poor`. |
| `invaccpct` | numeric | Inventory accuracy, as a percentage. |
| `stockturnrate` | numeric | How often the inventory is turned over in a period. |

# Joins

`disteventref` references `distregistry` in [disasterevents](/tables/disasterevents.md).

# Related knowledge

* [Resource Utilization Ratio (RUR)](/knowledge/resource-utilization-ratio.md) is computed from `hubutilpct`, `storecapm3` and `storeavailm3`.

Replace your-name with your own name, and the date with the current time.

This one file now holds what used to be split across two raw files: the columns and types from the schema, and their meanings from the column descriptions. A few things to notice:

type is the only required field. OKF has no fixed list of types, so we pick a name that says what the concept is.
description is the line an agent sees before it opens the file. We'll use it in the index files in the next step, so it should say clearly what's in the table.
generated records who wrote the file and when, and sources points back to the raw files it came from. human: marks a person as the author.
Links use paths that start with /. They point from the root of the bundle, so they stay correct wherever the file sits.
The link to disasterevents points to a file we haven't written. That's on purpose. OKF allows links to knowledge that doesn't exist yet, and we'll see how the validator treats it.

The rule concepts

Create bundles/disaster/knowledge/resource-utilization-ratio.md for rule 10:

---
type: Calculation
title: Resource Utilization Ratio (RUR)
description: Measures how effectively hub capacity is being used relative to available resources.
tags: [disaster, logistics]
generated: { by: human:your-name, at: 2026-10-02T17:00:00+05:30 }
sources:
  - id: kb
    resource: https://huggingface.co/datasets/birdsql/livesqlbench-base-lite/blob/main/disaster/disaster_kb.jsonl
    title: disaster business rules (LiveSQLBench), rule 10
---

# Definition

RUR = (hubutilpct / 100) × (storecapm3 / (storeavailm3 + 1))

# Columns used

All three columns are in [distributionhubs](/tables/distributionhubs.md): `hubutilpct`, `storecapm3` and `storeavailm3`.

# Used by

* [Resource Utilization Classification](/knowledge/resource-utilization-classification.md)

And bundles/disaster/knowledge/resource-utilization-classification.md for rule 50:

---
type: Business Rule
title: Resource Utilization Classification
description: Categorizes distribution hubs based on their Resource Utilization Ratio (RUR) values.
tags: [disaster, logistics]
generated: { by: human:your-name, at: 2026-10-02T17:00:00+05:30 }
sources:
  - id: kb
    resource: https://huggingface.co/datasets/birdsql/livesqlbench-base-lite/blob/main/disaster/disaster_kb.jsonl
    title: disaster business rules (LiveSQLBench), rule 50
---

# Definition

| Class | Condition | Meaning |
|---|---|---|
| High Utilization | RUR > 5 | Possibly overloaded; may need more resources. |
| Moderate Utilization | 2 ≤ RUR ≤ 5 | Balanced use of resources. |
| Low Utilization | RUR < 2 | Underused; resources could be moved elsewhere. |

# Depends on

* [Resource Utilization Ratio (RUR)](/knowledge/resource-utilization-ratio.md), which must be computed first.

What changed compared with the raw rules:

Each rule type got a readable name. calculation_knowledge became Calculation, and domain_knowledge became Business Rule.
The formula is plain math instead of LaTeX, and the classification is a table instead of one long sentence. The content is the same; it's just easier to read.
The rule now says where its columns live. The raw rule 10 named three columns but not their table. The "Columns used" section links to distributionhubs, so an agent no longer has to search the schema.
The dependency is a link in both directions. "Depends on" in rule 50 replaces children_knowledge: [10]. "Used by" in rule 10 is the reverse link, so an agent that starts from either rule can find the other.

Together, these three files hold everything the disaster_1 question needs. But an agent still has to find them. That's the job of index files, which we add next.

Add index files and a log

Our bundle has three concept files. The full disaster database will have 64: 10 tables and 54 rules. An agent can't open all of them for every question; that would be no better than pasting everything into the prompt.

It needs a menu first. In OKF, that menu is a file called index.md. Each folder can have one, and it lists what's in that folder, one line per item, with a short description. The agent reads the menu, then opens only what it needs.

The root index

Create bundles/disaster/index.md:

---
okf_version: "0.2"
---

# disaster database

* [Tables](tables/index.md) - PostgreSQL tables of the disaster-response database, with column meanings and joins.
* [Knowledge](knowledge/index.md) - Business rules: formulas and definitions used in questions about this database.

This is the agent's front door. It points to the index of each folder and says what that folder holds.

The block at the top is optional. It declares which OKF version the bundle follows, which is useful because the spec can change between versions. It's the only frontmatter an index file may have, and only in the index at the root of the bundle. The other index files have no frontmatter at all.

One index per folder

Create bundles/disaster/tables/index.md:

# Tables

* [distributionhubs](distributionhubs.md) - One row per disaster-relief distribution hub, with its capacity, storage space, cold storage, warehouse condition and inventory measures.

And bundles/disaster/knowledge/index.md, grouped by rule type:

# Calculations

* [Resource Utilization Ratio (RUR)](resource-utilization-ratio.md) - Measures how effectively hub capacity is being used relative to available resources.

# Business rules

* [Resource Utilization Classification](resource-utilization-classification.md) - Categorizes distribution hubs based on their Resource Utilization Ratio (RUR) values.

Each line is the concept's title, a link to its file, and its description, copied from the frontmatter. This is why the description matters so much: in the index, it's the only thing the agent sees before deciding whether to open a file.

What the indexes make possible

With these files in place, everything the disaster_1 question needs can be reached from the root index in a few hops: from the knowledge index to the two rules, and from the rules to the table they use. Whether a real agent actually takes that path is something we'll check later in the series, when we build the agent and log what it opens.

The three index files together are under 1 KB. For comparison, the three raw files for this database total about 63 KB.

There's a catch. Every time you add a concept, you also have to add its line to the right index, by hand. With three files that's easy. With 64, it's tedious and easy to get wrong. We'll automate it in the next part.

The log

Finally, create bundles/disaster/log.md:

# Update Log

## 2026-10-02
* **Creation**: Added the [distributionhubs](/tables/distributionhubs.md) table, the [Resource Utilization Ratio](/knowledge/resource-utilization-ratio.md) and the [Resource Utilization Classification](/knowledge/resource-utilization-classification.md), written by hand from LiveSQLBench `disaster`.

The log is a short history of changes to the bundle, newest first, grouped under dates. It has no frontmatter. Humans read it to see what changed and when; an agent can read it too, but it doesn't need it to answer questions.

Your project now looks like this:

okf-sql-knowledge/
├── data/
│   └── livesqlbench-base-lite/
├── bundles/
│   └── disaster/
│       ├── index.md
│       ├── log.md
│       ├── tables/
│       │   ├── index.md
│       │   └── distributionhubs.md
│       └── knowledge/
│           ├── index.md
│           ├── resource-utilization-ratio.md
│           └── resource-utilization-classification.md
├── requirements.txt
└── README.md

Validate the bundle, then break it on purpose

Before any agent reads a bundle, whoever wrote it should check that it follows the OKF rules. A typo in the frontmatter is easy to miss by eye, and an agent won't tell you about it.

We'll use the validator from okf-skills, an open-source (MIT) toolkit for OKF. It's a single Python file. Download it as shown in the tutorial guide; the link is pinned to a fixed version, so you see the same output as below. Then run it from the project folder, with your virtual environment active:

python tools/okf_validate.py bundles/disaster

OKF v0.2 conformance — bundles/disaster
  concepts: 3   index.md: 3   log.md: 1
  ! warn   tables/distributionhubs.md: cross-link target not found: `/tables/disasterevents.md` (tolerated under §6.1)
  ✓ conformant (1 warning(s))

The bundle is conformant: it follows the OKF v0.2 rules. There's one warning, for the link to disasterevents that we left in on purpose.

That's the key difference between the two kinds of findings:

An error breaks a rule the spec requires. The bundle is not valid until you fix it.
A warning points at something worth a look, but the spec tells readers of a bundle to cope with it. A link to a file that doesn't exist yet is fine: it may simply be knowledge nobody has written.

The best way to learn the rules is to break them, so let's do that twice.

Break 1: remove the `type`

Open knowledge/resource-utilization-ratio.md and delete the line type: Calculation. Run the validator again:

OKF v0.2 conformance — bundles/disaster
  concepts: 3   index.md: 3   log.md: 1
  ✗ ERROR  knowledge/resource-utilization-ratio.md: §11.2 missing or empty required `type` field
  ! warn   tables/distributionhubs.md: cross-link target not found: `/tables/disasterevents.md` (tolerated under §6.1)
  ✗ non-conformant (1 error(s))

Now it's an error, and the whole bundle is non-conformant. type is the one field every concept must have; everything else in the frontmatter is optional. Put the line back before moving on.

Break 2: add frontmatter to a folder index

Earlier we said that only the root index.md may have frontmatter. Let's test that. Add this block to the very top of tables/index.md:

---
title: Tables
---

Run the validator:

OKF v0.2 conformance — bundles/disaster
  concepts: 3   index.md: 3   log.md: 1
  ! warn   tables/index.md: §8 index.md should contain no frontmatter
  ! warn   tables/distributionhubs.md: cross-link target not found: `/tables/disasterevents.md` (tolerated under §6.1)
  ✓ conformant (2 warning(s))

It's only a warning, so the bundle still counts as conformant. But the OKF spec is clear that folder index files have no frontmatter, so remove the block again. If you're ever unsure about a format rule, check the official OKF spec.

Run the validator once more. When you see just the one expected warning, your first OKF bundle is done.

What's next

You now have a small, valid OKF bundle: one table and two rules, linked to each other and listed in index files. Along the way, we saw three things:

The schema isn't enough. Business definitions live elsewhere, and an agent without them guesses.
Links and indexes make knowledge findable. An agent can start from one small file and reach what a question needs.
A clear description matters most. It's what an agent sees before opening a file.

Three files by hand was easy. The disaster database alone needs 64, and all 18 databases need over a thousand. In the next part, we write a Python script that generates them all, with every link and index, and validates the result.

The code for this series is in the okf-sql-knowledge repository.

Sources

Open Knowledge Format (OKF) specification, v0.2 — Google Cloud. Format rules used throughout: frontmatter and type (§4), links (§6), index files (§8), log files (§9), conformance (§11).
Your Agents' Knowledge Needs a File Format. Meet OKF — the introduction to OKF and its frontmatter fields.
LiveSQLBench — benchmark website and leaderboard (48% best success rate, checked October 2026).
LiveSQLBench on GitHub — BIRD team at HKU and Google Cloud. Code, evaluation and database setup.
LiveSQLBench-Base-Lite dataset — the 18 databases and 270 questions used in this post, and the note on withheld answers.
okf-skills — Marco Boffo, MIT license. The OKF validator (okf_validate.py, commit 8e31878).
okf-sql-knowledge — the code and guides for this series.

DEV Community: Lakhan Malviya

Turn Database Docs into Agent-Ready Knowledge (OKF Series, Part 3)

Set up the project and the database

The project layout

The database

Read the raw files into Python

Check what was read

Generate the table concepts

A helper for every concept file

The table module

The command

Generate the rule concepts and their links

Update the command

Generate the index files and validate

The final command

Validate

All 18 databases

What we built, and what we learned

What's next

Sources

Build a Knowledge Layer for SQL Agents with OKF (Part 2)

Is this a real problem?

What we'll build

Set up the project

Get the data

Read the three raw files

The schema: disaster_schema.txt

The column descriptions: disaster_column_meaning_base.json

The business rules: disaster_kb.jsonl

What we learned

Write your first concept files by hand

The table concept

The rule concepts

Add index files and a log

The root index

One index per folder

What the indexes make possible

The log

Validate the bundle, then break it on purpose

Break 1: remove the type

Break 2: add frontmatter to a folder index

What's next

Sources

The schema: `disaster_schema.txt`

The column descriptions: `disaster_column_meaning_base.json`

The business rules: `disaster_kb.jsonl`

Break 1: remove the `type`