DEV Community: nightmoose

Ghostlog: a local séance for dead data pipelines

nightmoose — Sat, 31 Oct 2026 17:18:05 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Every data team has a graveyard. It holds the nightly job nobody remembers writing, the .sql file in legacy/, and the cron log that just stops one night in March. Someone always asks what that job did, and the person who knew left two reorgs ago.

Ghostlog holds a séance for that job. You point it at the remains, either an old cron or Airflow log or a deprecated SQL file, and it:

reads the remains: the job's name, how many nights it ran, its usual start time, every SQL statement it executed, and the error that ended it;
rebuilds the job's table lineage with sqlglot (raw.orders -> stg.orders -> mart.daily_revenue);
asks Gemma, running locally through Ollama, to let the job speak in the first person;
carves an ofrenda-style tombstone page with the job's years, an epitaph, its testimony, and its lineage.

It fits Día de los Muertos: the job gets remembered properly, and the next person who inherits the warehouse gets the context it took with it.

Demo

Summoning a nightly ETL job that died on 2025-03-14 when an upstream column disappeared. This is real Gemma output (gemma3:4b on a CPU-only Linux machine, about 8 seconds):

The tombstone it wrote:

A 60-second narrated walkthrough of a real run, summoning the job with Gemma and then opening its tombstone:

Live tombstone: nightmoose.github.io/ghostlog/sample/tombstone.html. All the samples are at nightmoose.github.io/ghostlog.

The raw transcripts and generated pages are committed in docs/sample/, including one run against a deprecated SQL file with a CTE and one run with --offline.

Code

nightmoose / ghostlog

A local séance for dead data pipelines: Gemma via Ollama + sqlglot lineage, fully offline.

Ghostlog

A local séance for dead data pipelines.

Point Ghostlog at the remains of a retired job (an old cron or Airflow log, or a deprecated .sql file) and it summons the job's ghost. Ghostlog reads the remains, rebuilds the job's table lineage with sqlglot and asks Gemma, running locally through Ollama, to let the job speak for itself in the first person Then it carves an ofrenda-style tombstone page you can open in any browser.

Everything runs on your own machine. Your logs and SQL stay on your laptop, and it keeps working with the Wi-Fi off.

Built for the DEV Hacktoberfest Open-Source AI Challenge, Week 4: Ghost in the Machine.

Quick start

git clone https://github.com/nightmoose/ghostlog
cd ghostlog
python -m venv .venv && . .venv/bin/activate
pip install -e .
# optional, for the ghost's own voice:
ollama pull gemma3:4b

ghostlog summon examples/legacy/nightly_orders_etl.log
# then open

…

View on GitHub

MIT licensed. It has one runtime dependency (sqlglot, MIT), 36 offline tests, and CI on Python 3.10, 3.12, and 3.13.

pip install -e .
ollama pull gemma3:4b
ghostlog summon examples/legacy/nightly_orders_etl.log

How I Built It

Ghostlog is Python, split into four small modules.

logparse reads the log line by line. It counts distinct run nights, finds the most common start time, deduplicates SQL statements, and turns the last error into a plain cause of death. For example, column "cust_id" does not exist becomes "upstream column 'cust_id' vanished."
lineage gives each INSERT, CREATE TABLE AS, MERGE, or UPDATE to sqlglot and records a source-to-target edge for each table it reads. CTE names are skipped, so WITH active AS (...) doesn't show up as a phantom table. SQL that fails to parse is reported rather than crashing the séance. The tombstone shows the longest source-to-sink chain.
medium sends the extracted facts to Gemma as JSON over Ollama's local HTTP API, using the standard library only. Ollama's JSON-schema format forces a reply with exactly three lines of testimony and three lines of epitaph.
tombstone writes one self-contained HTML file with inline CSS, no scripts, no external requests, and every value HTML-escaped.

What I learned from the model. My first runs used gemma3:1b. It had plenty of atmosphere but a loose grip on facts: it once said the job ran "for fourteen days each night." So I split the work. Code computes the facts, and the model only supplies the voice. On top of that, a grounding check rejects any reply where the ghost doesn't name its own job or says a number that isn't in the facts. When a reply fails the check, Ghostlog says why and falls back to a deterministic template. gemma3:4b passed the check on the sample log, so it became the default. The fallback also covers the case where Ollama isn't installed at all, so the séance always finishes.

Tests. The suite runs fully offline. A fixture fails any test that tries a real network call, and the model is exercised through fakes for valid, malformed, ungrounded, and unreachable replies. CI also smoke-tests the CLI on a runner with no model installed.

Why Does Open Innovation Matter?

Pipeline logs and SQL are some of the most sensitive text a company has. They hold table names, customer columns, and the shape of the business. With an open-weight model running locally, nothing leaves the laptop: no API key, no upload, no usage bill. It works on a plane with the Wi-Fi off.

Open weights also let me swap models with one flag (--model gemma3:1b, --model gemma3:4b, or anything else Ollama serves). That's how I found the right size for the job: I tried both locally and kept the one that stayed grounded. sqlglot, also open source, does the precise work of reading SQL across dialects, so the model never has to guess at lineage.

And because the whole stack is open, a tool for remembering abandoned software can't itself be abandoned by a vendor. If Ghostlog dies one day, anyone can dig it up and run it again.

The Spirit of It

The first time the 4b model answered "My existence ceased on 2025-03-14 due to the loss of cust_id," the job I'd made up felt like a real colleague. Every warehouse has a few jobs like that, and they deserve a marigold.

Prize Categories

Best Use of Gemma: Gemma (gemma3:4b, with gemma3:1b compared) runs locally through Ollama and gives each dead job its voice, constrained by a JSON schema and a grounding check.

MorningDrop: a local Gemma agent that checks last night's data drops, and gets checked itself

nightmoose — Thu, 22 Oct 2026 00:26:29 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

I'm a data engineer. Most of my mornings used to start the same way: open the folder where the nightly exports landed, open each CSV, scroll, squint, and try to work out whether anything was wrong before someone downstream noticed. Usually nothing was. That's what makes it so tiring: you do the same check every day so you can catch the one day something breaks.

MorningDrop does that check overnight so I don't have to. It's a small local agent you run from cron. When it wakes up it:

finds the files in a folder that are new since last night,
validates every row against a data contract with DataContractGate, offline,
hands a compact summary of the failures to Gemma running locally in Ollama, which writes a two-line morning summary and suggests fixes,
checks Gemma's answer with AISchemaGate before I ever see it, and
writes a Markdown report plus a two-line latest.txt I can read before coffee.

On a quiet night where everything passed, it doesn't even wake the model. Sloths conserve energy. 🦥

It's for anyone who gets data they don't control: vendor CSVs, storefront exports, CRM dumps, the "can you just drop it in the shared folder" files.

Demo

Recorded on a CPU-only box. The ~15–30 s Gemma wait is trimmed.

The sample drops in the repo have the kind of problems real exports have: a supplier that started sending usd instead of USD partway through the file, refunds arriving as quantity = -1, a customer with "usa" as a country code, and a brand-new shipments file nobody wrote a contract for. This is a real run with gemma3:4b on a CPU-only box:

$ morningdrop run
The orders_2026-10-24.csv file failed for 63 rows due to currency and quantity issues.
Prioritize fixing the currency and quantity validation rules in the orders contract.
report: reports/2026-10-25.md  (21.7s, summary by model)

The full report adds suggested fixes and the evidence behind them:

## Suggested fixes
- `orders_2026-10-24.csv` · `currency` · fix_data: Correct currency values to USD, EUR, or GBP.
- `orders_2026-10-24.csv` · `quantity` · fix_data: Ensure quantity values are greater than or equal to 1.
- `orders_2026-10-24.csv` · `customer_id` · fix_data: Validate customer_id against pattern: cus_[0-9]{5}

| File                       | Contract  | Rows | Passed | Failed | Status |
|----------------------------|-----------|-----:|-------:|-------:|--------|
| orders_2026-10-24.csv      | orders    |  240 |    177 |     63 | 🔴     |
| customers_2026-10-24.jsonl | customers |  120 |    116 |      4 | 🟡     |
| shipments_2026-10-24.csv   | —         |   80 |      0 |      0 | 🟡     |

### shipments_2026-10-24.csv
- No contract yet. Drafted one at contracts/_proposed/shipments.yaml; review it and move it into contracts/ to start enforcing.

The next night, after the exporter is fixed, the whole report is two lines and Gemma never gets called:

1 drop(s) checked overnight; every row passed.
Everything passed. Nothing to do.

Code

nightmoose / morningdrop

A local overnight agent: DataContractGate checks last night's data drops, local Gemma writes a two-line morning summary, AISchemaGate checks Gemma.

MorningDrop 🦥

Your data drops get checked overnight. You get two lines in the morning.

MorningDrop is a small local agent for anyone who wakes up to a folder of nightly exports: vendor CSVs, storefront dumps, CRM extracts. While you sleep it

finds the files that are new since last night,
checks every row against a DataContractGate contract, fully offline,
asks a local Gemma model (via Ollama) for a two-line summary and suggested fixes,
checks Gemma's answer with AISchemaGate before it reaches you, and
writes reports/YYYY-MM-DD.md, .json and a two-line latest.txt.

No rows leave your machine. No API keys. No per-call bill.

Recorded on a CPU-only box with asciinema + agg; the ~15–30 s Gemma wait is trimmed. Raw cast: docs/demo.cast.

$ morningdrop run
The orders_2026-10-24.csv file failed for 63 rows due to currency and quantity issues
Prioritize fixing the currency and quantity validation rules in

…

View on GitHub

MIT licensed, with tests (offline, using scripted fake models), CI on Python 3.9 and 3.12, and a CI job that installs Ollama and runs real Gemma end to end on a plain CPU runner.

How I Built It

The pipeline is boring on purpose. Everything that has a correct answer is done in plain code. The model only does the one thing it's good at: turning 60 × enum_violation on currency into a sentence a tired human will act on.

Validation: DataContractGate. Each dataset has a YAML contract (types, required fields, regex patterns, enums, ranges). The code imports the PyPI package named contractgate, which ships DataContractGate's validator; I use it offline, so no server and no network. CSV cells are coerced to the contract's types first. A value that doesn't parse stays a string, so the validator reports an honest type_mismatch instead of my code hiding it.
New datasets: draft a contract. When a file shows up with no contract, MorningDrop infers one (types, ranges, small enums, date patterns), compiles it with DataContractGate to prove it's valid, and saves it to contracts/_proposed/. It never enforces a contract you haven't reviewed.
The model: Gemma via Ollama. MorningDrop sends Gemma counts, field names, violation messages and the contract rule each one broke. It never sends whole rows, and passing rows never go to the model at all. The request uses Ollama's JSON-schema format so the model is steered toward the right shape from the start. Any OpenAI-compatible local server (llama.cpp, vLLM, LM Studio) works too.
Checking the model: AISchemaGate. This is the part I care about most. More on it below.
Scheduling: morningdrop run from cron, launchd or a systemd timer (examples in the repo), or morningdrop watch to poll.

The model's output gets a contract too

Small local models are great until they aren't. While building this I watched them:

gemma3:270m repeat my instructions back as its "summary" ("one sentence, what happened overnight (10-160 chars)"),
gemma3:1b return perfectly valid JSON that announced "shipments_2026-10-24.csv: 12 of 80 rows failed", for a file with no contract and no failures,
gemma3:4b suggest a quantity range of "1–1000" when the contract says 1–500. It had never been shown the contract, so I started including the rule each violation broke.

Shape-checking alone wouldn't have caught most of that. So the model's answer goes through two layers:

AISchemaGate validates the JSON against summary_contract.yaml: status must be green, yellow or red; each line must be 10–160 characters; every fix needs file, field, kind and suggestion. It's the same engine and the same contract format as the data side, pointed at LLM output.
Grounding checks tie the answer to the facts. The status must equal the worst file's status. The two lines must name the worst file. Every number in the text must already appear in the facts, which turned out to be the single most useful check. And every fix must point at a real file and a real failing field.

If either layer fails, the violations go back to Gemma as a repair prompt, with the temperature nudged up so a stuck model can get unstuck. If only some of the fixes are wrong, those are quarantined and the rest are kept, the same way DataContractGate quarantines bad rows. If the model still can't get it right, or Ollama isn't even running, a plain template writes the report from the same facts.

The result: every morning report is correct, and the model makes it nicer to read. Data contracts at the edges of the pipeline, and an output contract at the edge of the model.

With the final prompt, gemma3:4b passes on the first try in about 20–35 seconds on a CPU-only box. The 1B and 270M models still get caught and fall back to the template, which is exactly what should happen.

Why Does Open Innovation Matter?

For this tool, open weights aren't a nice-to-have. They're the reason it can exist.

The data never leaves the machine. These are customer exports: emails, order values, sometimes things covered by a DPA. "Send last night's customer file to a hosted API" is a conversation with legal. "Run Gemma on the box that already has the file" isn't.
It works at 5:30 AM with no internet and no keys. Nightly jobs fail in boring ways: an expired token, a rate limit, an outage on the provider's side. A local model on a cron job has none of those failure modes.
Set it up once, never pay per call. A job that runs every night forever should cost nothing to run every night forever.
I can pick the model size for the hardware, from a 270M model on a Raspberry Pi to 4B on a laptop, and swap Ollama for llama.cpp or vLLM by changing one line of config.
The checks are open too. DataContractGate and AISchemaGate are MIT-licensed, so the thing deciding whether the model's answer is trustworthy is code you can read, not another black box.

A closed API would probably write slightly nicer sentences. But it would make me ship my data somewhere else every night, and that's what I was trying to stop worrying about.

The Lazy Payoff

MorningDrop saves me 15–30 minutes every morning. It also checks more than I do by hand in that time: every row of every file against its contract, instead of the scroll-and-squint spot check I actually had time for, which is how 60 lowercase currency codes at the bottom of a 240-row file slip through.

On a clean night, the two lines in latest.txt are the whole job. On a bad night, the report names the file, the field, the row count and a first fix before I've opened anything. When a new dataset shows up, its first contract is already drafted for me to review.

Prize Categories

Best Use of Gemma: Gemma (via Ollama) writes the summary and the fix suggestions, and the whole design is built around making a small open-weight model safe to trust unattended.
Best Use of a .tech Domain: the project page lives at morningdrop.clankerspanker.tech.

"What breaks if this column changes?" Building LineageGate on FalkorDB

nightmoose — Sun, 18 Oct 2026 01:03:26 +0000

By Alex Suarez · written for the Graph Hacks: Context for AI Agents hackathon (WeMakeDevs × FalkorDB)

Live demo: https://lineagegate.fly.dev · Code: https://github.com/nightmoose/lineagegate

I've been a data engineer since 1997. The tools have changed many times over: cron and shell scripts, then SSIS, then ETL suites, then Hadoop, then the cloud warehouse with dbt on top. One conversation hasn't changed in all that time:

"Hey, we're changing amount to cents in the orders feed. That's fine, right?"

Nobody can answer that with confidence. The column feeds a staging model, which feeds three intermediate models, which feed a fact table, a reconciliation mart, a customer lifetime value model and five dashboards. Two of those dashboards belong to people who've never heard of the orders feed. Somewhere in there is a data contract that says "USD dollars, not cents", written after the last time this exact thing happened.

The second conversation is the 7 a.m. one: "The customer mart job failed. Why, and who do I page?" The error message names a failing test. The cause sits four hops upstream, in a CRM backfill. We've seen it before, but the write-up is in someone's old ticket and nobody remembers it.

Both questions are about relationships, not rows. That's why I built LineageGate on a graph.

Why a graph, specifically

You can answer "what's downstream of this column" with recursive SQL over an edge table. I've done it. It's painful to write, worse to maintain, and the answer comes back as a flat list of names. The flat list is the real problem: when I tell a finance lead their dashboard will break, the first thing they ask is "how?". I need the chain:

stg_orders.amount → int_order_revenue.gross_revenue → fct_orders.order_amount → fct_revenue_daily.revenue → mart_finance_revenue.revenue_usd → finance.revenue_daily

That chain is a path, and a graph database returns paths natively. Impact analysis is a variable-length traversal. Ownership is a multi-hop walk. "Have we seen this before?" is graph distance between today's failure and old incidents. Vector similarity can't give you a path, and a path is what earns trust.

The data model

LineageGate runs on a fully synthetic "Acme Analytics" warehouse: 111 nodes, 248 edges, 11 labels and 12 relationship types. It's small enough to read and big enough to be realistic. All lineage edges point upstream ("depends on"), so impact analysis means walking incoming edges.

flowchart RL
  Job>Job] -->|READS_FROM| Source([Source])
  Job -->|WRITES_TO| Table[Table]
  Job -->|REFRESHES| Model[Model]
  Table -->|HAS_COLUMN| Column[Column]
  Model -->|HAS_COLUMN| Column
  Model -->|READS_FROM| Table
  Model -->|DEPENDS_ON| Model
  Column -->|DEPENDS_ON| Column
  Dashboard[[Dashboard]] -->|DISPLAYS| Model
  Column -->|GOVERNED_BY| Contract{{Contract}}
  Test[/Test/] -->|GOVERNED_BY| Contract
  Test -->|READS_FROM| Column
  Incident((Incident)) -->|TRIGGERED_BY| Job
  Incident -->|VIOLATED| Contract
  Incident -->|RELATED_TO| Incident
  Model -->|OWNED_BY| Team[Team]
  Person[Person] -->|MEMBER_OF| Team

Two design choices mattered most:

Column-level lineage is first-class. Column -[:DEPENDS_ON]-> Column edges let impact be precise. Changing stg_orders.amount doesn't flag the support dashboard just because it shares a model with stg_orders.
Incidents are nodes, not documents. An incident links to the job that surfaced it, the contract it violated, and related incidents. Memory is part of the graph, so it can be queried like the graph.

Three queries doing the real work

Every tool the agent can call is a parameterised Cypher template. The model never writes Cypher itself. Here is the core of column impact, the path from a column to every dashboard it reaches:

MATCH (c:Column {fqn: $fqn})
MATCH p = (c)<-[:DEPENDS_ON*0..8]-(:Column)<-[:HAS_COLUMN]-(:Model)<-[:DISPLAYS]-(d:Dashboard)
RETURN d.id, d.name, length(p) AS hops,
       [n IN nodes(p) | {id: n.id, label: labels(n)[0], name: coalesce(n.fqn, n.name)}] AS pnodes,
       [r IN relationships(p) | {type: type(r), src: startNode(r).id, dst: endNode(r).id}] AS prels
ORDER BY hops

The two list comprehensions at the end matter more than they look. They project every path into plain node and edge maps on the server, so both the LLM and the UI receive the evidence in a form they can quote or draw.

Failed-job triage is one pattern from the failed job up to every ingestion job that feeds it:

MATCH p = (:Job {id: $id})-[:REFRESHES]->(:Model)-[:DEPENDS_ON*0..8]->(:Model)
          -[:READS_FROM]->(:Table)<-[:WRITES_TO]-(uj:Job)
RETURN uj.id, uj.status, pnodes, prels   // same path projection as above

For "which model would hurt most if it broke", I didn't write any graph math. I used FalkorDB's built-in algorithms:

CALL algo.pageRank('Model', 'DEPENDS_ON') YIELD node, score
RETURN node.id, score ORDER BY score DESC LIMIT 5

PageRank and algo.betweenness both put dim_customer at the top. That's the hub I planted in the seed data, and it's the model that would hurt most in real life. LineageGate then attaches the dashboard paths that make it critical, so even an algorithm result comes with evidence.

How Grok picks the graph tools

The agent loop is deliberately small: one Python module, no agent framework. It talks to xAI's OpenAI-compatible API and hands Grok seven tools:

impact_from_column
owners_for_node
paths_explain (k shortest paths via algo.SPpaths)
triage_failed_job
similar_incidents
record_incident
critical_assets

Each tool has a one-paragraph description. The system prompt sets a few hard rules:

Always call a tool before answering.
Quote at least one returned path word for word.
If a tool finds nothing, say the graph has no evidence and stop.

I never route by keyword. Ask "a customer job failed last night, have we seen it before, who should I page?" and Grok calls triage_failed_job with no arguments. That tool defaults to the most recent failure and already includes similar incidents. Ask about cents and dollars and it calls impact_from_column(table="stg_orders", column="amount").

Speed matters on camera. grok-4.3 gave great answers but took 5–26 seconds end to end. Switching the default to grok-4.20-0309-non-reasoning cut that to roughly 3.6–6.4 seconds per question in my dry run against the live site. With no API key, a deterministic offline router picks the tool from the question instead, so the demo never depends on the network.

Memory that survives the session

Hackathon Track 02 is about agent memory, and I wanted memory to be more than a chat transcript. record_incident MERGEs a new Incident node and links it into the lineage:

TRIGGERED_BY to the job
VIOLATED to each broken contract
RELATED_TO to its two most similar past incidents

similar_incidents then ranks incidents by graph distance from whatever you're asking about, combined with keyword overlap. Ask about stg_orders.amount in a brand-new session and the agent finds INC-0802 (the August cents-versus-dollars bug) through this path:

(Column stg_orders.amount) -[:GOVERNED_BY]-> (Contract orders_amount) <-[:VIOLATED]- (Incident INC-0802)

Nobody tagged that incident with the column name. The graph connects them through the contract. That's the kind of recall I've wanted in every incident review I've ever sat through.

Explainable by default, and honest when it can't answer

The UI is one HTML file served by FastAPI. Chat is on the left. The evidence panel on the right lists every tool the agent called and every path in the result, drawn left-to-right with vis-network. Click a path and it lights up. A second tab renders the live schema straight from FalkorDB (db.labels(), db.relationshipTypes() and a count of each label-type-label triple).

I spent as much time on the "no" cases as on the "yes" cases:

Misspelled column: the agent suggests close matches taken from the graph itself.
Ambiguous name: it lists the candidates instead of picking one.
Empty graph: it tells you to seed it.
Unknown column: ask about stg_orders.amount_eur and it says "the graph has no evidence" and offers stg_orders.amount. It doesn't invent a plausible-sounding blast radius.

An opt-in test sends an amount_eur question to the live model and asserts it abstains and prints no fabricated path.

Running it

It's one Fly.io machine running FalkorDB and the app together. On boot, it seeds the graph if it's empty, so a restart heals itself. Locally:

docker compose up -d
pip install -e '.[llm,api,dev]'
lineagegate seed && lineagegate serve

What I'd do next

Real ingestion: load lineage from dbt manifest.json and warehouse query logs instead of synthetic CSVs. The graph model wouldn't change.
Pre-merge checks: run impact_from_column against a pull request's changed columns and post the paths as a review comment before the change ships.
Persistent memory: add a volume and let the triage flow propose record_incident when an on-call engineer confirms a root cause.
Weighted paths: score edges by query volume, so "critical" reflects actual usage, not just structure.

Thirty years in, my data quality questions are still the same ones. What's new is that an agent can answer them, as long as the answer is a path through a graph it can show you.

Built in four days with FalkorDB, Python, FastAPI and xAI Grok. AI coding assistants helped write the code; see DISCLOSURE.md in the repo.

Demo video: https://youtu.be/ARw7zSb5Jdo