ChronoWeave: The Documentary

From Framework User to Framework Understander — A 4-Month Journey Through Gradient Descent, Transformers, and the Physics of History

---

"Every timeline you've ever seen is a lie of omission. It shows you WHEN things happened. It never shows you WHY. ChronoWeave is our attempt to fix that — and along the way, you're going to learn what actually happens inside a neural network when you call .backward()."

---

How to Use This Guide

This is not a tutorial you skim. It's a companion for a 3–5 month build. Read one section, do the exercise, hit the wall, climb over it, then come back. Every chapter follows the same rhythm:

• The Concept — the idea, explained with an analogy before a single line of code
• The Code Challenge — a skeleton you fill in, never a finished solution
• The Aha Moment — what you should see when it clicks
• The Extension — how this piece bolts onto the larger machine
• Resources — exact docs, not "just Google it"

And threaded throughout:

• 🧭 The Mentor Says — the advice I wish someone had given me
• 🕳️ The Rabbit Hole — optional, for when curiosity gets the better of you
• 😤 The Struggle — the wall you will hit, named in advance
• ✅ Checkpoint — stop and verify before you go further
• 💡 The Innovation — why this piece of the project is worth talking about in an interview

Let's begin.

---

CHAPTER 0: The Grand Vision — Seeing the Complete Picture

The Problem: Why Historical Timelines Are Broken

Open any history textbook's timeline. You'll see a horizontal line, some dots, some dates. 1914. 1917. 1929. 1939. It tells you when. It is silent on why.

The assassination of Archduke Franz Ferdinand didn't cause World War I in a vacuum — it was the spark that landed on a decade of alliance treaties, arms races, and colonial tension that had already turned the continent into dry kindling. A flat timeline shows you the spark. It hides the kindling.

What historians actually think in is a causal graph: event A enabled event B, which, combined with condition C, triggered event D. That graph is tangled, non-linear, and multi-dimensional — which is exactly why nobody draws it by hand. It's too much cognitive load.

ChronoWeave's bet: if a machine can extract the causal graph from raw text, and then lay it out so that causally-connected events cluster together and pull each other into readable arrangement, you get something a static timeline can never give you — a map you can explore, drag, and reorganize, where the layout itself is meaningful. Events that influence each other stay near each other. That's not a UI nicety. That's the entire value proposition.

The Solution: ChronoWeave's Architecture

Here's the complete system, before we write a single line of code:

┌─────────────────────────────────────────────────────────────────────┐
│                         THE USER'S BROWSER                           │
│  ┌──────────────┐        ┌────────────────────────────────────┐    │
│  │  Text Input   │──────▶│   React + D3.js Canvas               │    │
│  │  "Paste       │       │   (renders nodes = events,           │    │
│  │   history      │       │    edges = causal links)             │    │
│  │   here"        │       │                                       │    │
│  └──────────────┘        │   User drags a node ───────┐         │    │
│                            └─────────────────────────────┼─────────┘    │
└──────────────────────────────────────────────────────────┼──────────────┘
                    │ POST /extract                         │ WS: node_moved
                    ▼                                        ▼
┌─────────────────────────────────────────────────────────────────────┐
│                         FASTAPI BACKEND                              │
│                                                                        │
│  ┌────────────────────┐     ┌───────────────────────────────────┐  │
│  │  EXTRACTION ENGINE   │     │   SPATIAL INTELLIGENCE ENGINE       │  │
│  │  (Chapter 2)          │     │   (Chapter 3 — the heart of it)     │  │
│  │                        │     │                                      │  │
│  │  1. Split into         │     │  1. Build a graph: nodes=events,    │  │
│  │     sentences          │     │     edges=causal relations          │  │
│  │  2. Run NER            │────▶│  2. Init x,y for every node as a    │  │
│  │     (BERT) to find     │     │     TRAINABLE TENSOR                │  │
│  │     events, dates,     │     │     (requires_grad=True)            │  │
│  │     people, places     │     │  3. Define "Map Clarity Loss":      │  │
│  │  3. Run Relation       │     │     - causally linked nodes should  │  │
│  │     Extraction to      │     │       be CLOSE                     │  │
│  │     find "A causes B"  │     │     - unrelated nodes should be     │  │
│  │  4. Output: list of    │     │       FAR (repulsion)               │  │
│  │     (event, date,      │     │     - nothing should overlap        │  │
│  │      relation, event)  │     │  4. Run backprop on the             │  │
│  │     triples            │     │     COORDINATES (not the model's    │  │
│  │                        │     │     weights!) using a hand-written  │  │
│  │                        │     │     Adam optimizer                  │  │
│  │                        │     │  5. When user drags a node, PIN     │  │
│  │                        │     │     that node's coords, re-run      │  │
│  │                        │     │     optimization on everyone else   │  │
│  └────────────────────────┘     └───────────────────────────────────┘  │
│              │                                    │                      │
│              ▼                                    ▼                      │
│  ┌────────────────────────────────────────────────────────────────┐   │
│  │           PostgreSQL + pgvector (stores triples + embeddings)    │   │
│  └────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘

Data Flow: What Happens When a User Clicks "Generate"

Walk through it end to end, because this sequence is the skeleton every later chapter hangs off of:

1. User pastes text (say, three paragraphs on the causes of WWI) and clicks Generate.
2. Frontend POSTs the raw text to /api/extract.
3. Backend NER model (a fine-tuned BERT) tags every token: is it part of an EVENT, a DATE, a PERSON, a PLACE? Output: a list of entity spans.
4. Backend Relation Extraction model looks at pairs of entities in the same or nearby sentences and predicts a relation label: CAUSES, ENABLED_BY, PRECEDES, PART_OF, or NONE.
5. The result is a set of triples: (Assassination of Franz Ferdinand, CAUSES, Austria-Hungary ultimatum to Serbia). These get written to Postgres, and each event also gets a text embedding stored via pgvector (this lets us later cluster thematically similar events even without an explicit extracted relation).
6. The triples define a graph: events are nodes, relations are edges.
7. The Spatial Intelligence Engine initializes every node at a random (x, y) position — as a PyTorch tensor with requires_grad=True. This is the part that makes ChronoWeave unusual: the coordinates themselves are the parameters being learned, not neural network weights.
8. We define a custom loss function — the Map Clarity Loss — that penalizes bad layouts: causally-linked nodes that are far apart, unrelated nodes that overlap, edges that cross each other unnecessarily.
9. We run gradient descent (a hand-written Adam optimizer) for a few hundred steps. Each step: compute the loss, call .backward(), get gradients with respect to the x,y coordinates, nudge the coordinates a little in the direction that reduces loss. Repeat.
10. After convergence, the backend streams the final (node_id, x, y) positions back to the frontend over the initial REST response (or WebSocket, for the live re-layout case).
11. React + D3.js renders nodes at those coordinates, with edges drawn between causally-linked events, styled by relation type.
12. The magic moment: the user drags a node to a new spot because they disagree with the layout, or just want to explore. That node's position gets pinned (requires_grad=False for it, fixed at the drag target). The backend re-runs the optimizer on every other node, live, streaming intermediate frames over a WebSocket, so the user watches the rest of the graph flow and resettle around their edit — like iron filings reorganizing around a magnet you just moved.

That last step is the entire reason this project exists. A physics engine (force-directed graph layout, e.g. D3's built-in forceSimulation) can also do node repulsion and edge attraction — but it can't easily express arbitrary, learned, semantically-weighted objectives (e.g., "cluster by decade AND by causal chain AND penalize edge crossings, with weights that adapt based on graph density"). Gradient descent on a custom loss can. That's the pitch. We'll come back to defending it rigorously in Chapter 3.

Technology Stack: Why Each Piece Was Chosen

Layer	Choice	Why
ML core	PyTorch (from raw tensors, then nn.Module later)	You need autograd exposed at the tensor level to do coordinate optimization; PyTorch's dynamic graph makes this natural
NER / Relation Extraction	Fine-tuned BERT-base, later maybe REBEL for joint extraction	BERT is small enough to fine-tune on a single GPU and well documented; REBEL is a strong upgrade path for joint entity+relation extraction
Backend	FastAPI + WebSockets	Async-native, typed, and WebSocket support is first-class for the live re-layout streaming
Database	PostgreSQL + pgvector	Relational structure for triples, vector column for semantic similarity search on event embeddings
Frontend	React + D3.js	D3 gives you full control over force/position rendering; React manages component state and drag events
Deployment	Docker Compose → cloud (Render/Fly.io/AWS)	Reproducibility first, then scale

The Journey Ahead

Four phases, each roughly a month, building strictly bottom-up:

• Phase 1 (Weeks 1–4): You build the math — autograd, manual backprop, hand-rolled optimizers, a toy Transformer. No frameworks doing the work for you yet.
• Phase 2 (Weeks 5–8): You build the reader — BERT-based NER and relation extraction, a real data pipeline, a real database.
• Phase 3 (Weeks 9–12): You build the cartographer — the spatial intelligence engine. This is the conceptual core of the whole project and gets the most depth in this guide.
• Phase 4 (Weeks 13–16): You build the body — the full-stack app, the live drag-and-reoptimize loop, and you ship it.

🧭 The Mentor Says: Don't let the ambition of the final product distract you from the ugliness of the early steps. In week 2 you'll be manually computing gradients for a two-layer network with a pencil-and-paper feel to it, and it will seem impossibly far from "beautiful interactive graph app." That gap is supposed to be there. Every person who deeply understands PyTorch went through exactly this tunnel. You don't skip it by using nn.Module early — you skip understanding by doing that.

---

CHAPTER 1: The Foundation — Building Your Neural Engine

(Weeks 1–4 · Prerequisite: comfort with Python, basic linear algebra — matrix multiply, dot products — and derivatives from a first calculus course)

Week 1: Environment and the Shape of a Tensor

The Concept

A tensor is just a multi-dimensional array with two superpowers bolted on: it knows how to run on a GPU, and it can remember the sequence of operations that produced it, so it can later compute how a tiny nudge to its inputs would change its output. That second superpower is autograd, and it's the single idea Phase 1 exists to demystify.

Think of a tensor with requires_grad=True as a spreadsheet cell that doesn't just hold a number — it holds a number and a formula referencing other cells. Change an input cell, and every downstream cell knows exactly how much it would change, without you re-deriving the formula by hand. That "how much would it change" is the gradient.

The Code Challenge

Set up your environment first:

poetry init
poetry add torch numpy matplotlib jupyter
poetry shell

Then in a notebook, don't build anything yet — just observe autograd:

import torch

# TODO: create a tensor x = 3.0 with requires_grad=True
x = ...

# TODO: create y = x**2 + 2*x + 1
y = ...

# TODO: call y.backward() and print x.grad
# Predict on paper what dy/dx should be at x=3 BEFORE you run this
...

The Aha Moment

x.grad should equal 2*x + 2 = 8.0 — exactly the calculus-class derivative, computed automatically. The click isn't "wow it did calculus," it's realizing PyTorch didn't look up a symbolic formula for x**2 + 2*x + 1 — it recorded the sequence of primitive operations (pow, mul, add) as you executed them, and walked that sequence backward, applying the chain rule at each step. This recorded sequence is called the computation graph, and it's rebuilt fresh every time you run forward — which is why PyTorch is called "define-by-run."

The Extension

Every gradient ChronoWeave ever computes — whether it's a loss with respect to a BERT weight, or a loss with respect to a node's (x, y) position on the canvas — goes through this exact mechanism. There is no different code path for "coordinates" versus "weights." This is the whole trick behind Phase 3, twelve weeks from now.

Resources

• PyTorch autograd tutorial: https://docs.pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html

🕳️ The Rabbit Hole: Read about torch.autograd.grad() vs .backward() — the former lets you compute gradients without accumulating them into .grad, important later when you want gradients with respect to coordinates specifically, while leaving model weights untouched.

✅ Checkpoint 1.1: Explain, without looking anything up, why calling .backward() twice on the same graph (without retain_graph=True) throws an error. (Answer: the graph is freed after the first backward pass to save memory.)

---

Week 2: Manual Forward and Backward — No nn.Module

The Concept

nn.Module is a convenience wrapper. Underneath, a "layer" is nothing more than: take an input, multiply by a weight matrix, add a bias, pass through a nonlinearity. A "network" is several of those chained. "Training" is: compute a loss, get its gradient with respect to every weight, nudge each weight opposite to its gradient. Building this by hand once is the single highest-leverage exercise in this project.

The Code Challenge

Fit y = sin(x) on x in [-π, π] with manual parameters:

import torch
torch.manual_seed(0)

# Architecture: 1 -> 32 -> 1, tanh activation
W1 = ...  # TODO: (1,32), requires_grad=True, small random init
b1 = ...  # TODO: (32,)
W2 = ...  # TODO: (32,1)
b2 = ...  # TODO: (1,)

def forward(x):
    # TODO: z1 = x @ W1 + b1 ; a1 = tanh(z1) ; z2 = a1 @ W2 + b2
    ...
    return z2

def mse_loss(pred, target):
    # TODO: mean squared error, raw tensor ops
    ...

lr = 0.05
for step in range(2000):
    x = torch.linspace(-3.14, 3.14, 200).unsqueeze(1)
    y_true = torch.sin(x)
    pred = forward(x)
    loss = mse_loss(pred, y_true)
    # TODO: zero old grads, loss.backward(), manually update all 4 params
    #       (wrap update in `with torch.no_grad():`)
    ...
    if step % 200 == 0:
        print(step, loss.item())

The Aha Moment

Loss will not decrease if you forget to zero gradients — PyTorch accumulates them into .grad by default. Once burned by this, you'll never forget zero_grad() again — you'll understand why, not just cargo-cult it. Then plot forward(x) against sin(x): a sine wave emerges from 4 matrices you initialized as noise.

The Extension

This loop — forward, loss, zero grad, backward, manual update — is structurally identical to the loop you write in Phase 3 for optimizing node coordinates. Only what the "parameters" represent changes.

😤 The Struggle: You'll likely hit RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn, usually from overwriting a leaf tensor without torch.no_grad(). Print .requires_grad / .is_leaf on every parameter right before the failing line.

Resources

• PyTorch neural network basics: https://docs.pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html (concepts only this week)
• 3Blue1Brown, "Backpropagation calculus"

---

Week 3: Hand-Rolled SGD, Momentum, RMSprop, and Adam

The Concept

An optimizer answers: given a gradient, how exactly should I change the parameter? Plain SGD (subtract lr*grad) works but is slow and oscillates on ravine-shaped losses.

• Momentum — running average of past gradients, damping oscillation.
• RMSprop — running average of squared gradients per parameter; divides the step by its square root, shrinking effective learning rate for consistently-large-gradient parameters.
• Adam — momentum + RMSprop, plus bias-correction for early steps.

The Code Challenge

class ManualSGD:
    def __init__(self, params, lr=0.01, momentum=0.0):
        self.params = list(params)
        self.lr = lr
        self.momentum = momentum
        self.velocities = ...  # TODO: zeros_like buffer per param

    def step(self):
        with torch.no_grad():
            for p, v in zip(self.params, self.velocities):
                if p.grad is None: continue
                # TODO: v = momentum*v - lr*p.grad ; p += v
                ...

    def zero_grad(self):
        for p in self.params:
            if p.grad is not None: p.grad.zero_()


class ManualAdam:
    def __init__(self, params, lr=0.001, betas=(0.9, 0.999), eps=1e-8):
        self.params = list(params)
        self.lr, self.b1, self.b2, self.eps, self.t = lr, *betas, eps, 0
        self.m = ...  # TODO: zeros_like per param (first moment)
        self.v = ...  # TODO: zeros_like per param (second moment)

    def step(self):
        self.t += 1
        with torch.no_grad():
            for i, p in enumerate(self.params):
                if p.grad is None: continue
                g = p.grad
                # TODO: m[i] = b1*m[i] + (1-b1)*g
                # TODO: v[i] = b2*v[i] + (1-b2)*g**2
                # TODO: m_hat = m[i]/(1-b1**t) ; v_hat = v[i]/(1-b2**t)
                # TODO: p -= lr * m_hat / (sqrt(v_hat) + eps)
                ...

    def zero_grad(self):
        for p in self.params:
            if p.grad is not None: p.grad.zero_()

Re-run the Week 2 sine-fit with ManualAdam, compare convergence speed to vanilla SGD.

The Aha Moment

Adam converges in hundreds of steps where SGD needed thousands, and is far less sensitive to learning rate choice. This robustness is why you'll reach for it again in Phase 3, where the loss surface (Map Clarity Loss over 2D positions) is even messier.

The Extension

Your Phase 3 CoordinateAdam class will be a near copy of this one — same math, different thing being optimized.

💡 The Innovation (early preview): "I wrote my own Adam optimizer for coordinate updates because I needed to freeze individual coordinates mid-optimization when a user grabs a node — the standard torch.optim API doesn't expose that cleanly." Keep this for Chapter 3.

Resources

• Adam paper (Kingma & Ba, 2014): https://arxiv.org/abs/1412.6980
• Sebastian Ruder, "An overview of gradient descent optimization algorithms": https://www.ruder.io/optimizing-gradient-descent/

✅ Checkpoint 1.2: Plot loss curves for SGD, SGD+momentum, RMSprop, Adam — same problem, same init, same step count. If Adam isn't fastest, check bias-correction first.

---

Week 4: A Transformer From Scratch (Tiny, But Real)

The Concept

A Transformer's core operation: for every token, compute a weighted average of every other token's representation, where weights ("attention") are learned and reflect relevance. Everything else — multi-head, positional encoding, layer norm, feedforward — is scaffolding.

Analogy: a room full of people (tokens) forming an opinion on a topic — you weight each person's input by relevance. Query = what you're looking for; Key = what each token offers; Value = what it contributes if attended to.

The Code Challenge

import torch, math
import torch.nn.functional as F

def scaled_dot_product_attention(Q, K, V, mask=None):
    d_k = Q.shape[-1]
    scores = ...  # TODO: Q @ K.T / sqrt(d_k)
    if mask is not None:
        scores = scores.masked_fill(mask == 0, float('-inf'))
    weights = ...  # TODO: softmax(scores, dim=-1)
    output = ...   # TODO: weights @ V
    return output, weights
# Sanity check: verify weights.sum(dim=-1) is all 1.0

class TinyMultiHeadAttention(torch.nn.Module):
    def __init__(self, d_model, n_heads):
        super().__init__()
        assert d_model % n_heads == 0
        self.d_k, self.n_heads = d_model // n_heads, n_heads
        ...  # TODO: four Linear layers W_q, W_k, W_v, W_o

    def forward(self, x, mask=None):
        ...  # TODO: project, reshape to heads, attend per head, concat, W_o

class TinyEncoderLayer(torch.nn.Module):
    def __init__(self, d_model, n_heads, d_ff):
        super().__init__()
        self.attn = TinyMultiHeadAttention(d_model, n_heads)
        self.norm1 = torch.nn.LayerNorm(d_model)
        self.ff = torch.nn.Sequential(
            torch.nn.Linear(d_model, d_ff), torch.nn.ReLU(),
            torch.nn.Linear(d_ff, d_model))
        self.norm2 = torch.nn.LayerNorm(d_model)

    def forward(self, x, mask=None):
        ...  # TODO: x = norm1(x + attn(x)) ; x = norm2(x + ff(x))

Train on next-character prediction over a small corpus (a few historical-text paragraphs fits the theme).

The Aha Moment

Visualize attention weights for one sentence as a heatmap — rows should light up on semantically relevant tokens (e.g. a pronoun attending to the noun it refers to). Attention stops being a diagram and becomes something you watch your model do.

The Extension

In Chapter 2 you'll swap this toy encoder for a pretrained BERT, understanding exactly what happens inside every layer because you built the smallest version yourself.

Resources

• "The Annotated Transformer" (Harvard NLP): https://nlp.seas.harvard.edu/annotated-transformer/
• "Attention Is All You Need": https://arxiv.org/abs/1706.03762
• Jay Alammar, "The Illustrated Transformer": https://jalammar.github.io/illustrated-transformer/

🕳️ The Rabbit Hole: Attention has no inherent sense of token order. Read the sinusoidal positional encoding and ask why sine/cosine rather than a learned embedding.

✅ Checkpoint 1.3 — End of Phase 1: From memory, sketch a Transformer encoder layer's forward pass, and explain what breaks if you remove residual connections, layer norm, or the sqrt(d_k) scaling.

---

Phase 1 Milestone

By end of Week 4, from raw tensor operations, you have: (1) intuitive autograd understanding, (2) a hand-built 2-layer network with manual gradient updates, (3) three hand-built optimizers, (4) a working tiny Transformer with visualized attention. Your Week 3 optimizers become the Phase 3 coordinate optimizer almost verbatim; your Week 4 Transformer understanding is what lets you debug BERT rather than treat it as a black box.

---

CHAPTER 2: The Understanding — Teaching Your Engine to Read History

(Weeks 5–8 · Prerequisite: Phase 1 complete, comfort reading model architecture diagrams)

Week 5: Fine-Tuning BERT for Named Entity Recognition

The Concept

Your tiny Transformer from Week 4 learned character prediction from scratch on almost no data. BERT is the same architecture, scaled up (12 layers, 12 heads, 768-dim), pretrained on billions of words. Fine-tuning means: keep the pretrained weights as a strong starting point, attach a small task-specific head on top (here, a linear layer mapping each token's final hidden state to a label like B-EVENT, I-EVENT, B-DATE, O), and train the whole thing (or just the head) on your labeled data for a few epochs.

The reason this works with so little labeled data compared to training from scratch: BERT already learned general-purpose language structure from pretraining (grammar, word relationships, some world knowledge). Fine-tuning just teaches it to route that existing knowledge toward your specific labeling scheme.

The Code Challenge

from transformers import BertTokenizerFast, BertForTokenClassification
import torch

# TODO: load "bert-base-cased" tokenizer and BertForTokenClassification
#       with num_labels = len(your_label_list)
#       e.g. label_list = ["O","B-EVENT","I-EVENT","B-DATE","I-DATE","B-PERSON","I-PERSON"]
tokenizer = ...
model = ...

def tokenize_and_align_labels(text, word_labels):
    """
    text: list of words, e.g. ["The", "assassination", "of", "Franz", "Ferdinand"]
    word_labels: list of label ids, one per word
    Returns tokenized input + labels aligned to WordPiece subtokens
    (subtokens after the first get label -100 so they're ignored in the loss)
    """
    # TODO: tokenizer(text, is_split_into_words=True, ...) then use
    # .word_ids() to map each subtoken back to its source word
    ...

# TODO: set up a small labeled dataset (start with ~50-100 hand-labeled
# sentences from historical text — yes, you label it yourself first)
# TODO: standard fine-tuning loop: forward, CrossEntropyLoss (ignore_index=-100),
# backward, optimizer.step() -- use torch.optim.AdamW this time, not your manual one,
# since you've already proven you understand what it's doing

The Aha Moment

Run inference on a sentence the model never saw during fine-tuning and watch it correctly tag "the outbreak of war" as an EVENT span and "1914" as a DATE span, purely from ~100 examples. That's the pretraining transfer working — this would be essentially impossible to get right training from random initialization on 100 sentences.

The Extension

This NER output — spans tagged as EVENT, DATE, PERSON, PLACE — is exactly what Week 6's relation extraction will pair up into causal triples.

Resources

• Hugging Face token classification guide: https://huggingface.co/docs/transformers/tasks/token_classification
• BERT paper (Devlin et al., 2018): https://arxiv.org/abs/1810.04805

😤 The Struggle: Label alignment between words and WordPiece subtokens is the single most common source of silent bugs in NER fine-tuning — a misaligned label doesn't crash, it just quietly trains the model on garbage. Always spot-check tokenizer.convert_ids_to_tokens() against your aligned label array for a handful of examples before you trust any training run.

---

Week 6: Relation Extraction — Finding "A Causes B"

The Concept

NER tells you what the entities are. Relation extraction tells you how they relate. The simplest approach: for every pair of entities that co-occur in the same sentence (or a short window of sentences), feed their contextualized representations (plus the sentence itself) into a classifier that predicts one of a fixed set of relation labels: CAUSES, ENABLED_BY, PRECEDES, PART_OF, or NONE.

A more powerful approach — REBEL — does entity and relation extraction jointly, generating the full set of triples as a structured text sequence in one pass, rather than requiring you to first extract entities, then classify every pair. It's a strong upgrade path once your simpler pairwise classifier is working and you understand why the joint approach is more efficient (it doesn't blow up combinatorially with the number of entities in a sentence).

The Code Challenge

Start with the pairwise classifier — it's the better learning exercise, even though REBEL is more production-capable:

# For each entity pair (e1, e2) in the same sentence:
#   1. Mark their spans in the input with special tokens, e.g.
#      "[E1] The assassination [/E1] of Franz Ferdinand triggered
#       [E2] Austria-Hungary's ultimatum [/E2] to Serbia."
#   2. Run through BERT
#   3. TODO: take the hidden states at the [E1] and [E2] marker positions,
#      concatenate them, pass through a small classifier head
#      -> softmax over {CAUSES, ENABLED_BY, PRECEDES, PART_OF, NONE}

The Aha Moment

The output for that Franz Ferdinand sentence should be CAUSES with high confidence. Then feed it a sentence with two entities that are merely mentioned near each other with no causal link ("The war began in 1914. Assassinations were common in the region.") and watch it correctly predict NONE. That contrast — the model discriminating relevance, not just proximity — is the whole point of relation extraction over naive "entities near each other are related" heuristics.

The Extension

Every (entity_1, relation, entity_2) triple this produces becomes an edge in the causal graph that Phase 3's spatial engine will lay out.

Resources

• REBEL paper (Huguet Cabot & Navigli, 2021): https://aclanthology.org/2021.findings-emnlp.204/
• REBEL on Hugging Face: https://huggingface.co/Babelscape/rebel-large

🕳️ The Rabbit Hole: Read about "distant supervision" for relation extraction — a technique for auto-generating noisy training labels by aligning a knowledge base (like Wikidata) against text, instead of hand-labeling everything. Useful if your ~100 hand-labeled examples aren't enough once you scale up.

---

Week 7: The Data Pipeline — Text → Tokens → Entities → Relations → Storage

The Concept

A pipeline is only as trustworthy as its weakest stage boundary. This week isn't about new ML — it's about wiring Weeks 5 and 6 together into something that reliably takes raw pasted text and emits a clean, deduplicated, database-ready set of triples, with sane error handling for the inevitable garbage input (empty text, non-English text, text with no extractable events).

The Code Challenge

class ExtractionPipeline:
    def __init__(self, ner_model, relation_model, tokenizer):
        ...

    def extract(self, raw_text: str) -> list[dict]:
        """
        Returns: [{"subject": "...", "relation": "CAUSES", "object": "...",
                   "subject_date": "1914-06-28" or None, ...}, ...]
        """
        # TODO Step 1: sentence-split raw_text (spaCy or nltk sentence tokenizer)
        # TODO Step 2: run NER on each sentence -> entity spans
        # TODO Step 3: for entity pairs within a sentence AND across
        #      adjacent sentences (causal claims often span 2 sentences),
        #      run the relation classifier
        # TODO Step 4: filter relation predictions below a confidence
        #      threshold (start at 0.6, tune empirically)
        # TODO Step 5: deduplicate triples (same subject/object/relation
        #      appearing from overlapping sentence windows)
        # TODO Step 6: return structured triples
        ...

Set up PostgreSQL with pgvector:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE events (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    text TEXT NOT NULL,
    date_text TEXT,
    embedding VECTOR(768),  -- BERT's hidden size
    created_at TIMESTAMPTZ DEFAULT now()
);

CREATE TABLE relations (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    source_event_id UUID REFERENCES events(id),
    target_event_id UUID REFERENCES events(id),
    relation_type TEXT NOT NULL,
    confidence FLOAT
);

The Aha Moment

Paste in three unrelated paragraphs of history (say, one on WWI causes, one on the French Revolution, one on the fall of Rome) and watch the pipeline correctly keep the triples from each topic separate, with no spurious cross-topic relations — because relation extraction is confidence-thresholded and entity embeddings from unrelated topics don't get spuriously matched.

The Extension

This pipeline is the entire backend of the /api/extract endpoint from Chapter 0's data flow diagram. Everything downstream (Phase 3, Phase 4) consumes its output.

Resources

• pgvector docs: https://github.com/pgvector/pgvector
• SQLAlchemy + pgvector integration: https://github.com/pgvector/pgvector-python

✅ Checkpoint 2.1: Run your pipeline on a genuinely messy input (a Wikipedia paragraph with footnote markers, inconsistent date formats, nested clauses) and verify it degrades gracefully — no crash, just fewer/lower-confidence triples — rather than throwing an unhandled exception.

---

Week 8: Evaluation and Hardening

The Concept

An extraction pipeline that "looks right" on your three favorite test paragraphs is not the same as one that's reliable. This week is about building a small held-out evaluation set (hand-labeled, separate from training data) and computing real metrics: precision, recall, and F1 for both entity spans and relation classification.

The Code Challenge

def evaluate_ner(model, eval_sentences, eval_labels):
    # TODO: run inference, compute span-level (not token-level!) precision/recall/F1
    # Span-level means a predicted "B-EVENT I-EVENT" span only counts as
    # correct if it matches the gold span's exact boundaries, not just
    # individual token labels — this is a stricter and more meaningful metric.
    ...

def evaluate_relations(model, eval_pairs, eval_labels):
    # TODO: per-class precision/recall/F1, plus a confusion matrix
    # (CAUSES vs ENABLED_BY vs PRECEDES are semantically close and
    # commonly confused — expect this)
    ...

The Aha Moment

Your confusion matrix will likely show CAUSES and ENABLED_BY bleeding into each other — this is expected and informative, not a bug. It tells you those two categories are genuinely hard to distinguish from surface text alone (a human annotator would disagree with themselves on some of these too), which should inform how confidently the frontend displays that distinction later (e.g., maybe you merge them into one edge style with a subtler visual difference rather than two starkly different arrow types).

The Extension

Phase 2 milestone: a real, measured, imperfect-but-quantified extraction system, feeding real triples into a real database.

Resources

• "Named Entity Recognition" evaluation conventions (seqeval library): https://github.com/chakki-works/seqeval

Phase 2 Milestone

You now have a fine-tuned BERT NER model, a relation extraction classifier, a hardened pipeline connecting them, a Postgres+pgvector store, and honest evaluation metrics on held-out data. This is a legitimate, demoable NLP system on its own — worth pausing to appreciate before diving into Phase 3, which is where the project becomes genuinely novel.

---

CHAPTER 3: The Intelligence — Making Your Engine Think Spatially

(Weeks 9–12 · Prerequisite: Phase 1 and 2 complete. This is the conceptual core of ChronoWeave — take this chapter slowly.)

Why This Chapter Gets the Most Depth

Everything before this point — the Transformer, the NER model, the relation extractor — is, architecturally, "standard" deep learning applied to a specific domain. Impressive to build from scratch, but conceptually well-trodden. This chapter is where ChronoWeave does something genuinely uncommon: using gradient descent as a general-purpose layout algorithm, with coordinates as first-class trainable parameters. Take this slowly. It's the part of the project you'll actually be excited to explain in an interview.

Week 9: Coordinates as Parameters

The Concept

Every optimization problem in ML has the same shape: parameters, a loss function measuring how bad the current parameters are, and an optimizer that nudges parameters to reduce that loss. So far, "parameters" has meant neural network weights. Nothing in that shape requires the parameters to be weights. A parameter is just a tensor with requires_grad=True that appears somewhere in a differentiable computation whose output is your loss.

So: what if the parameters are the (x, y) positions of graph nodes on a canvas, and the loss is a hand-designed function that scores how "good" a layout is?

This reframes graph layout — traditionally solved with physics simulations (force-directed layout, spring-embedder algorithms) — as an optimization problem solvable with the exact same machinery you built in Phase 1.

The Code Challenge

Set up the skeleton, no loss function yet — just prove coordinates can be optimized at all:

import torch

num_nodes = 10
# TODO: initialize positions as a (num_nodes, 2) tensor, random in
# some reasonable range (e.g. -5 to 5), requires_grad=True
positions = ...

# Toy goal: pull every node toward the origin (0,0) — trivial loss,
# just to prove the plumbing works before you write anything smarter
def toy_loss(positions):
    # TODO: sum of squared distances from origin
    return (positions ** 2).sum()

optimizer = torch.optim.Adam([positions], lr=0.1)
for step in range(100):
    optimizer.zero_grad()
    loss = toy_loss(positions)
    loss.backward()
    optimizer.step()
    if step % 20 == 0:
        print(step, loss.item())

# TODO: plot positions before and after — every node should have
# collapsed toward (0,0)

The Aha Moment

Watching ten random points converge toward the origin using torch.optim.Adam — the exact same optimizer class you'd use to train a neural network — is the moment this project's central idea stops being abstract. You just used backpropagation to solve a geometry problem. No physics, no forces, no simulation of springs — just calculus.

The Extension

toy_loss is a stand-in. Next week you replace it with the real Map Clarity Loss — but the training loop around it doesn't change at all.

🧭 The Mentor Says: Resist the urge to jump straight to the full Map Clarity Loss this week. Build up loss terms one at a time, verify each one does what you expect in isolation (attraction only, then repulsion only, then combined), and only then combine everything. Debugging a five-term loss function that's never converged correctly at any point is miserable. Debugging a five-term loss function where you've verified each term separately is straightforward.

---

Week 10: The Map Clarity Loss

The Concept

A good layout satisfies several competing objectives simultaneously:

1. Attraction: causally-linked events should be close together (short edges are more readable)
2. Repulsion: all events should push apart from each other generally, so the graph doesn't collapse into a single point (this is what "attraction" alone would do — see Week 9's toy example)
3. Non-overlap: no two event boxes should visually overlap
4. Temporal ordering (optional but powerful): since these are historical events, you may want an additional soft constraint that events with earlier dates trend toward one side of the canvas — turning the layout into something between a pure causal graph and a timeline

Each of these becomes a differentiable term, and the total loss is a weighted sum:

L_total = w1 * L_attraction + w2 * L_repulsion + w3 * L_overlap + w4 * L_temporal

The weights (w1..w4) are hyperparameters you'll tune empirically — this is exactly analogous to loss weighting in multi-task neural network training, another well-known hard problem, so don't be surprised if getting a visually pleasing balance takes real iteration.

The Code Challenge

def attraction_loss(positions, edges):
    """
    edges: list of (i, j) node index pairs that are causally linked
    Pulls linked nodes together — squared distance, like a spring
    """
    i_idx = torch.tensor([e[0] for e in edges])
    j_idx = torch.tensor([e[1] for e in edges])
    # TODO: diff = positions[i_idx] - positions[j_idx]
    # TODO: return (diff ** 2).sum(dim=1).mean()
    ...

def repulsion_loss(positions, min_distance=2.0):
    """
    Pushes ALL pairs of nodes apart if they're closer than min_distance.
    This is O(n^2) -- fine for the node counts ChronoWeave targets
    (tens to low hundreds of events per graph), but know that this term
    is the one that would need approximating (e.g. Barnes-Hut) at scale.
    """
    n = positions.shape[0]
    # TODO: compute pairwise distance matrix
    # (hint: torch.cdist(positions, positions) does this in one call)
    dist_matrix = ...
    # TODO: for pairs closer than min_distance, penalize (min_distance - dist)^2
    # TODO: mask out the diagonal (distance of a node to itself = 0, don't penalize that)
    ...

def overlap_loss(positions, box_size=1.0):
    # TODO: similar to repulsion but specifically penalizes overlap of
    # fixed-size boxes -- can start as a simplified version of repulsion_loss
    # with min_distance = box_size, and refine later once you see real overlaps
    ...

def temporal_loss(positions, dates_normalized):
    """
    dates_normalized: tensor of shape (n,), each event's date mapped to
    [0, 1] (earliest event=0, latest=1)
    Soft-encourages x-coordinate to correlate with date
    """
    # TODO: target_x = dates_normalized * canvas_width
    # TODO: return ((positions[:, 0] - target_x) ** 2).mean()
    ...

def map_clarity_loss(positions, edges, dates_normalized, weights):
    w1, w2, w3, w4 = weights
    return (w1 * attraction_loss(positions, edges)
            + w2 * repulsion_loss(positions)
            + w3 * overlap_loss(positions)
            + w4 * temporal_loss(positions, dates_normalized))

The Aha Moment

Run the full optimization on a real extracted graph (from your Phase 2 pipeline!) with all four terms weighted, and watch causally-connected clusters visually separate from unrelated clusters, while individual nodes within a cluster stay legibly spaced apart. This is the actual "map" in ChronoWeave — and unlike a force-directed layout from a library, you can explain precisely why every node ended up where it did, because you wrote every term of the objective yourself.

The Extension

This is the function whose gradient — with respect to positions, not any neural network weight — drives literally everything the user sees on the canvas, and everything that happens when they drag a node in Chapter 4.

😤 The Struggle: The most common failure mode here is loss term imbalance — e.g. repulsion dominating so hard that attraction can't pull anything together, producing a uniform grid-like scatter with no visible clustering. When this happens, don't guess-and-check weights blindly. Log each individual loss term's value (not just the weighted sum) every N steps, and look at their relative magnitudes. If repulsion_loss is naturally 50x the scale of attraction_loss just from how you defined it (e.g. squared distances over many more pairs), your weights need to correct for that scale difference before they can express your actual priorities between the terms.

Resources

• Force-directed graph drawing (for conceptual contrast — read this to understand what you're deliberately doing differently): Fruchterman & Reingold, "Graph Drawing by Force-Directed Placement" (1991) — search for the PDF, it's a classic short paper
• torch.cdist docs: https://docs.pytorch.org/docs/stable/generated/torch.cdist.html

✅ Checkpoint 3.1: Take a graph with two clearly separate causal clusters (e.g. WWI causes and, unrelated, causes of the 2008 financial crisis, extracted from two different pasted texts) and verify optimization produces two visually distinct, non-overlapping clusters — not because you told it to, but because attraction pulls each cluster's internal nodes together while repulsion pushes the two clusters apart as aggregate masses.

---

Week 11: Custom Gradient Hooks and the Coordinate Optimizer

The Concept

Two refinements this week. First: gradient hooks. PyTorch lets you register a function that runs every time a gradient is computed for a specific tensor, which is how you'll implement per-node learning rate decay, gradient clipping for numerically unstable nodes, or debugging instrumentation (logging exactly which nodes have the largest gradients at each step — useful for diagnosing which part of the layout is "fighting" hardest).

Second: formalize your Phase 1 ManualAdam into a purpose-built CoordinateAdam that supports freezing individual coordinates — critical for Chapter 4's drag-and-reoptimize feature, where the dragged node must stay exactly where the user put it while every other node reoptimizes around it.

The Code Challenge

# Gradient hook example
def make_logging_hook(node_names):
    def hook(grad):
        # grad shape: (num_nodes, 2)
        # TODO: find and print the node with the largest gradient norm
        # this step -- useful for seeing which node is "hardest to place"
        ...
        return grad  # must return grad unchanged (or modified) -- don't return None
    return hook

positions.register_hook(make_logging_hook(node_names))

class CoordinateAdam:
    """
    Like your Phase 1 ManualAdam, but supports freezing a subset of
    coordinates (e.g. a node the user just dragged).
    """
    def __init__(self, positions: torch.Tensor, lr=0.05, betas=(0.9, 0.999), eps=1e-8):
        self.positions = positions
        self.lr, self.b1, self.b2, self.eps, self.t = lr, *betas, eps, 0
        self.m = torch.zeros_like(positions)
        self.v = torch.zeros_like(positions)
        # TODO: a boolean mask (num_nodes,) -- True = frozen, don't update
        self.frozen_mask = torch.zeros(positions.shape[0], dtype=torch.bool)

    def freeze(self, node_idx):
        self.frozen_mask[node_idx] = True

    def unfreeze(self, node_idx):
        self.frozen_mask[node_idx] = False

    def step(self):
        self.t += 1
        with torch.no_grad():
            g = self.positions.grad
            # TODO: zero out gradient rows for frozen nodes BEFORE the
            # Adam update, so their moment buffers don't drift either
            g = g.clone()
            g[self.frozen_mask] = 0.0

            self.m = self.b1 * self.m + (1 - self.b1) * g
            self.v = self.b2 * self.v + (1 - self.b2) * g ** 2
            m_hat = self.m / (1 - self.b1 ** self.t)
            v_hat = self.v / (1 - self.b2 ** self.t)
            update = self.lr * m_hat / (v_hat.sqrt() + self.eps)
            # TODO: apply update only to non-frozen rows
            update[self.frozen_mask] = 0.0
            self.positions -= update

    def zero_grad(self):
        if self.positions.grad is not None:
            self.positions.grad.zero_()

The Aha Moment

Freeze one node mid-optimization (simulate a user drag by just calling .freeze(node_idx) and manually setting that row of positions to a fixed target), re-run .step() for another 100 iterations, and watch every other node smoothly re-flow into a new stable configuration around the pinned node — while the pinned node itself doesn't move a single pixel, even though it still has a nonzero gradient every step.

The Extension

This freeze/unfreeze mechanism is exactly the backend logic Chapter 4's WebSocket drag handler calls into. You are not building new logic in Phase 4 — you're wiring this class up to a live event stream.

💡 The Innovation: This is the strongest, most concrete "why is this novel" answer for the whole project: "Standard force-directed layout libraries treat a dragged node as an external constraint bolted onto a physics simulation — usually by literally fixing its position and letting the simulation tick forward. My approach treats the drag as freezing one parameter's gradient within the exact same optimization loop used to generate the original layout, which means the re-layout objective is provably the same objective function, just re-solved under an added constraint — not a different, ad hoc mechanism." That's a real distinction, not a marketing line, and you can defend it because you wrote both halves.

Resources

• PyTorch hooks documentation: https://docs.pytorch.org/docs/stable/notes/autograd.html#hooks-for-saved-tensors
• register_hook API: https://docs.pytorch.org/docs/stable/generated/torch.Tensor.register_hook.html

---

Week 12: Debugging Gradient-Based Layout

The Concept

Gradient-based layout fails in ways that are different from — and often more confusing than — a standard training loop failing, because there's no "accuracy" metric to sanity check against, only a loss value and a visual result. This week is a deliberate practice week: you'll intentionally break your Week 10–11 system in several ways and learn to recognize the symptoms.

The Code Challenge — Diagnose Each of These Failure Modes

1. Everything collapses to one point. (Hint: check the sign and relative magnitude of your attraction vs. repulsion terms — is repulsion actually being computed, or is a masking bug zeroing it out?)
2. Layout oscillates forever, never converging. (Hint: learning rate too high for this loss surface, OR two loss terms directly fighting with equal and opposite gradients at every step — try lowering lr by 10x before assuming it's a logic bug)
3. NaN positions after N steps. (Hint: torch.cdist produces a zero distance for a node compared to itself; if your repulsion loss divides by distance anywhere instead of just using squared distance, you'll get a divide-by-zero on the diagonal. Always double-check you've masked the diagonal.)
4. One node flies off to extreme coordinates while everything else looks fine. (Hint: register a gradient hook — from Week 11 — on positions and print per-node gradient norms; the runaway node almost always has an anomalously large gradient from a bug specific to its edges, e.g. it participates in zero attraction edges so repulsion has nothing to balance against)

The Aha Moment

After deliberately inducing and then fixing all four failure modes above, run your full pipeline end-to-end — real text in, real triples extracted, real layout optimized — and watch it just... work, cleanly, without you needing to touch a single hyperparameter mid-run. That reliability is earned specifically by having debugged each failure mode once on purpose, rather than encountering them for the first time under deadline pressure in Phase 4.

The Extension

Phase 3 milestone: a working, debugged spatial intelligence engine that takes a causal graph and produces a legible 2D layout via gradient descent, with the ability to freeze nodes and re-optimize live.

🧭 The Mentor Says: Keep a running "failure log" as a markdown file next to your code: every bug you hit, what the symptom looked like, and what actually fixed it. Six months from now when this happens again in a different project, that log is worth more than any Stack Overflow search, because it's your debugging vocabulary for this specific class of problem (gradient-based systems with non-standard parameters), which almost nobody else has written down anywhere.

✅ Checkpoint 3.2 — End of Phase 3: Explain to a rubber duck (or a friend, or a mirror) why gradient descent on coordinates is a valid approach to graph layout at all — i.e., why the loss function's gradient with respect to position tells you a meaningful direction to move a node, given that "position" isn't something with an obvious ground truth the way a classification label is. (The core answer: there's no ground truth position, but there's a well-defined relative preference — closer is better for linked nodes, farther is better for unlinked ones — and gradient descent is a general algorithm for finding local optima of any differentiable preference function, not just supervised losses with ground truth targets.)

---

CHAPTER 4: The Body — Giving Your Engine a Web Presence

(Weeks 13–16 · Prerequisite: Phase 3 complete)

Week 13: FastAPI Backend and WebSockets

The Concept

Two distinct interaction patterns need two different transport mechanisms. The initial "paste text, click Generate" flow is a classic request/response — REST fits fine. But the drag-and-reoptimize flow needs the server to push a stream of intermediate frames to the client as optimization runs (so the user sees nodes flow smoothly into place rather than jumping instantly), which REST can't do — that's what WebSockets are for.

The Code Challenge

from fastapi import FastAPI, WebSocket
from fastapi.responses import JSONResponse

app = FastAPI()

@app.post("/api/extract")
async def extract(payload: dict):
    text = payload["text"]
    # TODO: run ExtractionPipeline (Chapter 2) -> triples
    # TODO: run spatial optimization (Chapter 3) to convergence
    # TODO: return {"nodes": [...], "edges": [...]}
    ...

@app.websocket("/ws/relayout/{graph_id}")
async def relayout(websocket: WebSocket, graph_id: str):
    await websocket.accept()
    # TODO: receive {"dragged_node_id": ..., "new_x": ..., "new_y": ...}
    drag_event = await websocket.receive_json()

    # TODO: load current positions for graph_id, freeze the dragged node
    #       at (new_x, new_y) using CoordinateAdam.freeze() from Chapter 3
    # TODO: run optimization step-by-step (not to convergence in one go!)
    #       and after every K steps, send the current positions:
    for step in range(200):
        # optimizer.step()...
        if step % 5 == 0:
            # TODO: await websocket.send_json({"positions": [...]})
            ...
    await websocket.close()

The Aha Moment

Open your browser's network tab, trigger a drag, and literally watch a stream of JSON frames arrive over the WebSocket connection every few milliseconds — the exact same optimization loop from Chapter 3, just now visible as it happens, one HTTP-adjacent message at a time.

The Extension

This is the wiring that turns Chapter 3's offline optimization script into a live, interactive feature.

Resources

• FastAPI WebSockets guide: https://fastapi.tiangolo.com/advanced/websockets/

---

Week 14: React + D3.js Rendering

The Concept

D3 excels at binding data to SVG elements and handling enter/update/exit transitions smoothly — exactly what you need when node positions update every few WebSocket frames. React manages component state and lifecycle; D3 manages the actual SVG manipulation and smooth transitions between positions. The common pattern: let React own the DOM structure (which nodes/edges exist), let D3 own the transitions (how a node's x/y animates from old position to new).

The Code Challenge

// TODO: a GraphCanvas component that:
// 1. Receives `nodes` and `edges` as props (from the /api/extract response)
// 2. Renders SVG circles for nodes, lines for edges, using D3 scales
//    to map data coordinates to screen coordinates
// 3. Uses d3.transition() to animate position changes smoothly whenever
//    the `nodes` prop updates (i.e., whenever a new WebSocket frame arrives)
// 4. Attaches d3.drag() behavior to each node, which on 'end' fires a
//    callback (e.g. onNodeDragged(nodeId, newX, newY)) that the parent
//    component uses to open the /ws/relayout WebSocket and stream in
//    the live re-layout frames from Week 13

function GraphCanvas({ nodes, edges, onNodeDragged }) {
  // TODO: useRef for the svg element, useEffect that runs D3 rendering
  // logic whenever `nodes` or `edges` change
}

The Aha Moment

Drag a node and watch not just that node move, but every connected node smoothly animate to its new position over the following second or two — not teleporting, not choppy, an actual fluid re-settling. This is the "magic moment" the entire project overview promised in Chapter 0, now real in a browser.

The Extension

Nothing further — this is the product. Everything from Chapter 1 onward has been building toward this specific ten seconds of interaction.

Resources

• "Learn D3: Joining Data" (Observable): https://observablehq.com/@d3/learn-d3-joining-data
• Amelia Wattenberger, "React + D3" (a well-regarded pattern guide for combining the two): https://wattenberger.com/blog/react-and-d3

😤 The Struggle: React re-rendering and D3 both wanting to own the DOM is a classic source of fights — D3 mutates elements directly, React expects to control them via its virtual DOM diff. The cleanest fix most people converge on: let React render the SVG container and static structure, but hand D3 a ref to manipulate node/edge elements directly inside a useEffect, and never let React's render also try to set those same attributes. Pick one owner per DOM property.

---

Week 15: Deployment

The Concept

Docker Compose ties your FastAPI backend, Postgres+pgvector database, and (optionally) a separate model-serving container into one reproducible unit you can run locally exactly as it'll run in the cloud — closing the "works on my machine" gap before it costs you a demo day.

The Code Challenge

# docker-compose.yml — TODO fill in the blanks
services:
  db:
    image: pgvector/pgvector:pg16
    environment:
      POSTGRES_PASSWORD: ...
    volumes:
      - pgdata:/var/lib/postgresql/data

  backend:
    build: ./backend
    depends_on:
      - db
    environment:
      DATABASE_URL: postgresql://...@db:5432/...
    ports:
      - "8000:8000"
    # TODO: mount your fine-tuned model weights as a volume rather than
    # baking them into the image -- keeps image builds fast during dev

  frontend:
    build: ./frontend
    ports:
      - "3000:3000"

volumes:
  pgdata:

The Aha Moment

docker-compose up on a completely fresh machine (or a cloud VM) and the whole system — model inference, database, backend, frontend — comes up correctly with zero manual steps beyond that one command. That's the difference between "a project on my laptop" and "a project."

The Extension

Deploy to Render, Fly.io, or a similar platform for a public demo URL — essential for the hackathon-ready, portfolio-ready version of this project.

Resources

• Docker Compose docs: https://docs.docker.com/compose/
• Render's Docker deployment guide: https://render.com/docs/docker

---

Week 16: Polish and the Full Demo

The Concept

The last week is deliberately not about new features. It's about the unglamorous 20% that makes a demo feel finished: loading states while extraction/optimization runs, graceful handling of bad input, a couple of pre-baked example texts a judge or interviewer can one-click try instead of needing to paste their own, and a README that explains the project in 30 seconds.

The Code Challenge

• Add a loading skeleton/spinner state to the frontend while /api/extract is in flight (this can take several seconds — a live optimization loop is not instant, and users need to know it's working, not frozen)
• Add 2–3 pre-loaded example texts (a good default: causes of WWI, causes of the French Revolution, causes of the 2008 financial crisis)
• Write a project README with: what it does, a GIF of the drag-and-reoptimize interaction, architecture diagram, how to run it locally

The Aha Moment

Show the finished product to someone who has never seen it, say nothing, and watch them paste in their own text and drag a node without prompting. If they intuitively understand what happened without you explaining it, the product design succeeded.

Phase 4 Milestone — Project Complete

A live, deployed, full-stack application: paste historical text, get an extracted causal graph laid out via gradient descent, drag any node and watch the rest of the graph re-optimize in real time — all running on infrastructure you understand end to end, built from raw tensor operations up.

---

EPILOGUE: Beyond ChronoWeave — What You Can Build Next

Portfolio Presentation

Lead with the live demo, not the tech stack list. The first ten seconds should be someone pasting text and dragging a node — let the magic moment sell itself before you explain any internals.

Research Paper Possibilities

The "gradient descent as general-purpose graph layout" idea, generalized beyond history specifically, touches on active research areas: differentiable graph drawing, learned layout objectives, and combining symbolic graph constraints with continuous optimization. A write-up comparing your Map Clarity Loss approach against classical force-directed layout on layout-quality metrics (edge crossing count, node overlap, cluster separation) would be a legitimate small research contribution, or at minimum a strong technical blog post.

Scaling the System

Current design targets tens to low-hundreds of nodes per graph (the O(n²) repulsion term is the binding constraint). Scaling further means either approximating repulsion (Barnes-Hut / quadtree methods, the same trick classical force-directed layout libraries use at scale) or moving to a hierarchical layout where clusters are optimized independently and then composed.

---

APPENDIX A: First Week Setup Guide (Day 1)

# 1. Install Python 3.11+, then:
curl -sSL https://install.python-poetry.org | python3 -
poetry --version   # verify install

# 2. Project scaffold
mkdir chronoweave && cd chronoweave
poetry init --name chronoweave-ml -n
poetry add torch numpy matplotlib jupyter transformers

# 3. Verify PyTorch sees your GPU (if you have one -- CPU is fine for Phase 1)
poetry run python -c "import torch; print(torch.cuda.is_available())"

# 4. Launch a notebook and run the Week 1 autograd exercise before doing
#    anything else -- confirm your environment works before building on it
poetry run jupyter notebook

If you don't have a GPU: everything in Phase 1 and most of Phase 3 runs fine on CPU (small networks, small graphs). Phase 2's BERT fine-tuning is the one place a GPU meaningfully helps — Google Colab's free tier is sufficient for the scale of fine-tuning this project needs.

---

APPENDIX B: Troubleshooting

Symptom	Likely Cause	Fix
Loss doesn't decrease at all	Forgot zero_grad() before .backward()	Gradients are accumulating across steps — add zero_grad() at the top of the loop
RuntimeError: ...does not require grad and does not have a grad_fn	A leaf tensor got overwritten outside torch.no_grad()	Check every in-place parameter update is inside with torch.no_grad():
Loss becomes NaN after N steps	Divide-by-zero, usually in a pairwise-distance loss term where the diagonal (self-distance = 0) isn't masked	Mask the diagonal before dividing by any distance term
Everything collapses to a point during layout optimization	Repulsion term isn't actually contributing (masking bug or weight of 0)	Log each loss term separately, verify repulsion is nonzero and comparable in scale to attraction
NER model predicts O for everything	Label misalignment between words and WordPiece subtokens	Spot-check tokenizer.convert_ids_to_tokens() against your label array on several examples
WebSocket disconnects mid-reoptimization	Sending too many frames too fast, or an unhandled exception mid-loop killing the coroutine	Throttle to sending every 5th step, wrap the optimization loop in try/except with a graceful websocket.close() on error
D3 nodes flicker or don't animate smoothly	React re-render fighting with D3's direct DOM manipulation	Ensure only one of React/D3 sets a given DOM attribute; do D3 updates inside useEffect, not render
Docker Compose backend can't reach the database	Using localhost instead of the service name in DATABASE_URL	Compose networking resolves service names (db), not localhost, between containers

---

APPENDIX C: Project Timeline (Text Gantt Chart)

Week:        1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16
Phase 1      ██ ██ ██ ██
  Autograd    ██
  Manual NN      ██
  Optimizers        ██
  Tiny Xformer         ██
Phase 2                  ██ ██ ██ ██
  NER fine-tune            ██
  Relation extract            ██
  Data pipeline                   ██
  Evaluation                         ██
Phase 3                              ██ ██ ██ ██
  Coords as params                     ██
  Map Clarity Loss                        ██
  Grad hooks + optimizer                     ██
  Debugging practice                            ██
Phase 4                                          ██ ██ ██ ██
  FastAPI + WS                                     ██
  React + D3                                          ██
  Deployment                                             ██
  Polish + demo                                             ██

---

APPENDIX D: Success Metrics — How Do I Know I Built It Correctly?

Phase 1 (correctness of understanding, not the app):

• You can derive, on paper, the gradient of MSE loss with respect to a 2-layer network's weights, without a reference
• Your manual Adam implementation converges at a comparable rate to torch.optim.Adam on the same toy problem
• You can explain attention's Q/K/V roles without analogy, in precise terms

Phase 2 (quantitative, on your held-out eval set):

• NER span-level F1 ≥ 0.75 on your hand-labeled eval set (BERT-base fine-tuned on ~100+ examples should clear this comfortably)
• Relation classifier per-class F1 ≥ 0.6 on CAUSES specifically (the class you care most about); lower is acceptable for rarer/harder classes like ENABLED_BY
• Pipeline handles at least 3 different genuinely messy real-world text samples without crashing

Phase 3 (behavioral, verified visually + numerically):

• Map Clarity Loss converges to a stable value (not oscillating) within 500 optimization steps on a graph of ~20 nodes
• Two independently-clustered causal groups in the same graph visually separate without being told to
• Freezing a node mid-optimization and resuming produces a stable re-layout where the frozen node's position never changes by more than floating-point rounding error

Phase 4 (product-level):

• End-to-end latency from clicking "Generate" to seeing a laid-out graph is under ~10 seconds for a few paragraphs of input
• Drag-and-reoptimize visually completes (settles, stops moving) within ~2 seconds
• A first-time user, with zero explanation, successfully pastes text and drags a node within 60 seconds of opening the app

---

APPENDIX E: Interview Prep — Explaining ChronoWeave to a Hiring Manager

The 30-second version:
"I built a system that reads historical text, extracts cause-and-effect relationships between events using a fine-tuned Transformer, and lays those events out on a 2D map using gradient descent instead of a physics engine — meaning the layout is generated by backpropagating a custom loss function with respect to node coordinates, the same way you'd train a neural network, except the 'parameters' are positions on a canvas instead of weights."

Anticipated follow-up questions and how to answer them:

• "Why not just use a force-directed graph library?" — Because I needed the layout objective to express more than physical forces can naturally capture: e.g., a soft temporal-ordering constraint alongside causal clustering, with tunable relative weights between competing goals, and the ability to freeze arbitrary subsets of nodes mid-optimization while re-solving the same objective for everyone else. Gradient descent on a custom differentiable loss generalizes to all of that; a spring simulation would need a bespoke mechanism bolted on for each one.

• "How did you handle relation extraction being noisy?" — I measured it rather than assumed it: held-out precision/recall/F1 per relation class, a confidence threshold before a triple gets displayed, and I designed the frontend to reflect uncertainty where the model itself is uncertain (e.g., visually softer distinction between relation types the model confuses most, per the confusion matrix).

• "What was the hardest bug?" — Talk about one specific failure mode from Chapter 3 Week 12 in detail (e.g., the NaN-from-unmasked-diagonal bug) — specificity here is what separates "I used ML" from "I understand ML."

• "Why build the optimizer from scratch instead of using torch.optim.Adam directly?" — Because I needed per-node freezing during live re-layout, which meant controlling exactly how the moment buffers and update step interacted with a frozen mask — not something the standard API exposes, and writing it myself meant I fully understood what I was modifying.

• "What would you do differently at scale?" — The repulsion loss term is O(n²); past a few hundred nodes I'd move to an approximate method (Barnes-Hut quadtree, the same technique classical force-directed layouts use), or a hierarchical approach that optimizes clusters independently before composing them.

---

End of guide. This document covers the full architecture, all four learning phases, and the operational appendices for building ChronoWeave. Treat each chapter's code skeletons as a starting point to fill in and argue with — the goal was never for you to run code someone else wrote, but to be the one who could have written PyTorch's autograd yourself, at least once, in miniature.