DEV Community: Roan de Jager

Building TALON: Sub-Second Non-Generative Knowledge Graph Ingestion on a GTX 1070

Roan de Jager — Tue, 11 Aug 2026 14:52:13 +0000

Building TALON: Sub-Second Non-Generative Knowledge Graph Ingestion on a GTX 1070

Knowledge Graphs are rapidly becoming the preferred alternative to raw Vector Databases for Retrieval-Augmented Generation (RAG). By storing structured [Subject, Predicate, Object] triples, Knowledge Graphs eliminate the semantic ambiguity that causes traditional vector search to hallucinate.

However, Knowledge Graphs have suffered from a massive performance bottleneck: Ingestion Speed.

Until now, extracting structured facts from text required passing document blocks into generative LLMs (like Llama 3 or Qwen) and waiting for them to type out JSON token-by-token. This process takes 10 to 15 minutes per document, consumes heavy GPU VRAM, and frequently breaks due to JSON parsing errors.

To solve this bottleneck, I built TALON (Tensor-Accelerated Local Ontology Network) inside Hillock v0.2.2—a non-generative, open-source neuro-symbolic memory engine built in Python.

🧠 What is Hillock?

Hillock is a privacy-first local memory engine (AGPL-3.0) that replaces Vector Databases with three decoupled components:

SQLite Knowledge Graph: Stores deterministic [Subject, Predicate, Object] triples.
Hebbian Plasticity Engine: Tracks gradient-free co-activation associations between entities over time.
Hyperdimensional Computing (HDC / VSA): Uses CPU-bound vector symbolic algebra for hallucination defense and gating.

Repository: github.com/roandejager/Hillock

🦅 The TALON Architecture

Instead of relying on token generation, TALON uses pure tensor math and bi-encoder routing to achieve sub-second document relation extraction on consumer hardware (NVIDIA GTX 1070 / < 1GB VRAM).

TALON operates in three isolated CUDA stages:

Stage 1: Anaphora Resolution (`Fastcoref`)

Paragraphs are full of pronouns ("She", "He", "Their"). If you chunk text before resolving pronouns, downstream models lose the canonical head entity. TALON uses Fastcoref to replace all pronouns across full documents in ~300ms before text chunking.

Stage 2: Dynamic Predicate Routing (`MiniLM-L6-v2`)

Zero-shot relation classifiers get overloaded if you pass 100+ candidate labels at once. Stage 2 uses an 80MB MiniLM bi-encoder to scan input sentences and retrieve the Top-10 most relevant Wikidata predicates in < 2ms.

Stage 3: Zero-Shot Matrix Classification (`GLiREL`)

Stage 3 passes the sentence, spaCy entity spans, and Top-10 dynamic predicates into GLiREL (DeBERTa-v3). GLiREL performs single-pass latent span classification over entity pairs, extracting structured triples without generating a single text token.

📊 Benchmark Results (v0.2.2)

On our automated scientific evaluation harness (evaluate_hillock_PROTO_ish.py), upgrading to the TALON Engine produced massive accuracy jumps while running in sub-seconds:

Metric	Legacy LLM Baseline	TALON Engine (v0.2.2)
Retrieval Accuracy	10.0%	50.0%
Extraction Recall	13.6%	50.0%
Gate Accuracy	16.7%	50.0%
Processing Speed	~15 minutes	Sub-second Tensor Math

🛠️ Try It Locally

Hillock is 100% open-source under AGPL-3.0. You can run it locally with PyTorch CUDA:

git clone https://github.com/roandejager/Hillock.git
cd Hillock
pip install -r requirements.txt
python main.py****

Check out the code and roadmap on GitHub: github.com/roandejager/Hillock

I built a local AI memory gate on a CPU, and my 7B model scored worse than my 1.5B model because it was too smart

Roan de Jager — Mon, 22 Jun 2026 18:24:41 +0000

Hi everyone,

I've been hacking on a personal, local project called Hillock. Honestly, it's very much a work in progress and it isn't some flawless breakthrough, but I wanted to see if we could build a lightweight, completely offline memory layer for local LLMs without the overhead of running a heavy neural vector database or wasting precious VRAM.

It is named after the biological Axon Hillock—the region of a human neuron that sums up incoming electrical charges and decides whether to fire (open the gate) or remain silent (block).

How the architecture works:

The Ground Truth (SQLite): Stores hard facts as simple database triples (Subject-Predicate-Object) so the system has a solid symbolic foundation.
The Synapses (Hebbian Plasticity): Tracks which concepts co-occur during a conversation to dynamically build gradient-free associative weights.
The Context (Hyperdimensional Computing): Maintains a 10,000-dimensional leaky context vector that rolls, binds, and accumulates history. This helps the system resolve pronouns (like "he/she") and decide when to block a query to prevent hallucinations.

The "Smarter Model, Lower Score" Paradox

I wrote a tough, 32-sentence scientific benchmark with complex sentence structures and hard negatives (like asking what Einstein discovered when the text only mentions Curie discovering radioactivity and Einstein working with her).

When I ran Qwen 2 (1.5B), it got around 50.0% Retrieval Accuracy. But when I upgraded to the much smarter Qwen 3 (5.2GB), the ingestion time jumped to 20 minutes on my local machine, and its score actually dropped to 25.0%!

Why? Because Qwen 3 is too expressive for my rigid evaluation script:

The test expected Marie_Curie born_in Poland. Qwen 3 extracted [Marie_Curie] -[spent_childhood_in]-> [Poland].
The test expected Albert_Einstein. Qwen 3 extracted [albert_einstein] (lowercase), which broke the exact-string checks.
The test expected compiler. Qwen 3 extracted [first_compiler].

So, while Qwen 3 populated the database with beautiful, highly accurate, and conversational triples (extracting up to 6 clean relations per block in a single pass), it got penalized by the rigid evaluation harness.

The codebase is written in pure Python, is fully open-source (under the AGPL-3.0 copyleft license), and is designed to run entirely offline on consumer hardware.

If anyone is interested in VSAs, alternative cognitive architectures, or has feedback on the HDC context-binding math, I'd love for you to check it out!

GitHub Repository: https://github.com/roandejager/Hillock

Hillock: A brain-inspired, CPU-bound memory gate for local LLMs

Roan de Jager — Tue, 16 Jun 2026 19:18:50 +0000

Hi everyone,

I've been hacking on a personal, local project called Hillock. Honestly, it's very much a work in progress and it isn't all that, but I wanted to see if we could build a lightweight, offline memory layer for local LLMs without the overhead of running a heavy neural vector database or wasting precious VRAM.

It is named after the biological Axon Hillock—the region of a human neuron that sums up incoming electrical charges and decides whether to fire (open the gate) or remain silent (block).

How the stack works:

Hard Facts (SQLite): Stores raw facts as simple database triples (Subject-Predicate-Object) so the system has a solid symbolic foundation.
Synapses (Hebbian Plasticity): Tracks which concepts co-occur during a conversation to dynamically build gradient-free associative weights.
Context (Hyperdimensional Computing): Maintains a 10,000-dimensional leaky context vector that rolls, binds, and accumulates history. This helps the system resolve pronouns (like "he/she") and decide when to block a query to prevent hallucinations.

The "Smarter Model, Lower Score" Paradox

I wrote a tough, 30-sentence scientific benchmark with complex sentence structures and hard negatives to see where this breaks on local hardware.

When I ran Qwen 2 (1.5B), it got around 50.0% Retrieval Accuracy. But when I upgraded to the much smarter Qwen 3 (5.2GB), its score actually dropped to 15.0%!

Why? Because Qwen 3 is too expressive for my rigid evaluation script:

The test expected Marie_Curie born_in Poland. Qwen 3 extracted [Marie_Curie] -[spent_childhood_in]-> [Poland].
The test expected Albert_Einstein. Qwen 3 extracted [albert_einstein] (lowercase), which broke the exact-string checks.
The test expected compiler. Qwen 3 extracted [first_compiler].

So, while Qwen 3 populated the database with beautiful, highly accurate, and conversational triples, it got penalized by the rigid evaluation harness.

The codebase is written in pure Python, is fully open-source (under the AGPL-3.0 copyleft license), and is designed to run entirely offline on consumer hardware.

If anyone is interested in VSAs, alternative cognitive architectures, or has feedback on the HDC context-binding math, I'd love for you to check it out!

GitHub Repository: https://github.com/roandejager/Hillock

Hillock: A brain-inspired, CPU-bound memory gate for local LLMs

Roan de Jager — Sun, 14 Jun 2026 21:04:21 +0000

Hi everyone,

I've been hacking on a local personal memory system called Hillock. Honestly, it's very much a work in progress and it isn't some flawless breakthrough, but I wanted to see if we could build a lightweight, completely offline memory layer for local LLMs without the overhead of running a heavy neural vector database or wasting precious VRAM.

The project is named after the biological Axon Hillock—the exact gatekeeper region of a human neuron that sums up incoming electrical charges and decides whether to fire (open the gate) or remain silent (block).

How the architecture works:

The Ground Truth (SQLite): Stores hard facts as simple database triples (Subject-Predicate-Object) so the system has a solid symbolic foundation.
The Synapses (Hebbian Plasticity): Tracks which concepts co-occur during a conversation to dynamically build gradient-free associative weights.
The Context (Hyperdimensional Computing): Maintains a 10,000-dimensional leaky context vector that rolls, binds, and accumulates history. This helps the system resolve pronouns (like "he/she") and decide when to block a query to prevent hallucinations.

The Honest Benchmarks (Yes, it breaks!)

I wrote a tough, 30-sentence scientific benchmark with complex sentence structures and hard negatives (like asking what Einstein discovered when the text only mentions Curie discovering radioactivity and Einstein working with her).

Running Qwen 1.5B locally on my computer, here is how it actually did:

Extraction Precision: 10.6%
Extraction Recall: 22.7%
Retrieval Accuracy: 30.0%
Gate Accuracy: 30.0%

Why are these scores low? Because a tiny 1.5B model completely trips over complex English grammar during ingestion (it gets confused and creates weird predicates). However, the actual HDC vector-matching itself is incredibly stable. I enforce a Constant-Component-Count of exactly 3 components per fact, which balances the vector norms and keeps retrieval highly reliable once the facts are actually in the database.

It is fully open-source (under the AGPL-3.0 copyleft license) and designed to run entirely offline on consumer hardware.

If anyone is interested in VSAs, neuro-symbolic AI, or has feedback on the HDC context-binding math, I'd love for you to check it out!

GitHub Repository: https://github.com/roandejager/Hillock

DEV Community: Roan de Jager

Building TALON: Sub-Second Non-Generative Knowledge Graph Ingestion on a GTX 1070

Building TALON: Sub-Second Non-Generative Knowledge Graph Ingestion on a GTX 1070

🧠 What is Hillock?

🦅 The TALON Architecture

Stage 1: Anaphora Resolution (Fastcoref)

Stage 2: Dynamic Predicate Routing (MiniLM-L6-v2)

Stage 3: Zero-Shot Matrix Classification (GLiREL)

📊 Benchmark Results (v0.2.2)

🛠️ Try It Locally

I built a local AI memory gate on a CPU, and my 7B model scored worse than my 1.5B model because it was too smart

How the architecture works:

The "Smarter Model, Lower Score" Paradox

Hillock: A brain-inspired, CPU-bound memory gate for local LLMs

How the stack works:

The "Smarter Model, Lower Score" Paradox

Hillock: A brain-inspired, CPU-bound memory gate for local LLMs

How the architecture works:

The Honest Benchmarks (Yes, it breaks!)

Stage 1: Anaphora Resolution (`Fastcoref`)

Stage 2: Dynamic Predicate Routing (`MiniLM-L6-v2`)

Stage 3: Zero-Shot Matrix Classification (`GLiREL`)