Building TALON: Sub-Second Non-Generative Knowledge Graph Ingestion on a GTX 1070
Knowledge Graphs are rapidly becoming the preferred alternative to raw Vector Databases for Retrieval-Augmented Generation (RAG). By storing structured [Subject, Predicate, Object] triples, Knowledge Graphs eliminate the semantic ambiguity that causes traditional vector search to hallucinate.
However, Knowledge Graphs have suffered from a massive performance bottleneck: Ingestion Speed.
Until now, extracting structured facts from text required passing document blocks into generative LLMs (like Llama 3 or Qwen) and waiting for them to type out JSON token-by-token. This process takes 10 to 15 minutes per document, consumes heavy GPU VRAM, and frequently breaks due to JSON parsing errors.
To solve this bottleneck, I built TALON (Tensor-Accelerated Local Ontology Network) inside Hillock v0.2.2—a non-generative, open-source neuro-symbolic memory engine built in Python.
🧠 What is Hillock?
Hillock is a privacy-first local memory engine (AGPL-3.0) that replaces Vector Databases with three decoupled components:
-
SQLite Knowledge Graph: Stores deterministic
[Subject, Predicate, Object]triples. - Hebbian Plasticity Engine: Tracks gradient-free co-activation associations between entities over time.
- Hyperdimensional Computing (HDC / VSA): Uses CPU-bound vector symbolic algebra for hallucination defense and gating.
Repository: github.com/roandejager/Hillock
🦅 The TALON Architecture
Instead of relying on token generation, TALON uses pure tensor math and bi-encoder routing to achieve sub-second document relation extraction on consumer hardware (NVIDIA GTX 1070 / < 1GB VRAM).
TALON operates in three isolated CUDA stages:
Stage 1: Anaphora Resolution (Fastcoref)
Paragraphs are full of pronouns ("She", "He", "Their"). If you chunk text before resolving pronouns, downstream models lose the canonical head entity. TALON uses Fastcoref to replace all pronouns across full documents in ~300ms before text chunking.
Stage 2: Dynamic Predicate Routing (MiniLM-L6-v2)
Zero-shot relation classifiers get overloaded if you pass 100+ candidate labels at once. Stage 2 uses an 80MB MiniLM bi-encoder to scan input sentences and retrieve the Top-10 most relevant Wikidata predicates in < 2ms.
Stage 3: Zero-Shot Matrix Classification (GLiREL)
Stage 3 passes the sentence, spaCy entity spans, and Top-10 dynamic predicates into GLiREL (DeBERTa-v3). GLiREL performs single-pass latent span classification over entity pairs, extracting structured triples without generating a single text token.
📊 Benchmark Results (v0.2.2)
On our automated scientific evaluation harness (evaluate_hillock_PROTO_ish.py), upgrading to the TALON Engine produced massive accuracy jumps while running in sub-seconds:
| Metric | Legacy LLM Baseline | TALON Engine (v0.2.2) |
|---|---|---|
| Retrieval Accuracy | 10.0% | 50.0% |
| Extraction Recall | 13.6% | 50.0% |
| Gate Accuracy | 16.7% | 50.0% |
| Processing Speed | ~15 minutes | Sub-second Tensor Math |
🛠️ Try It Locally
Hillock is 100% open-source under AGPL-3.0. You can run it locally with PyTorch CUDA:
git clone https://github.com/roandejager/Hillock.git
cd Hillock
pip install -r requirements.txt
python main.py****
Check out the code and roadmap on GitHub: github.com/roandejager/Hillock
Top comments (0)