DEV Community: jc pong

OnCall Brain: Finding the Right Person, the Current Rule, and the Evidence

jc pong — Sat, 17 Oct 2026 07:58:54 +0000

An on-call knowledge graph that keeps document curation, technical responsibility, and policy history explicit.

A new on-call engineer can retrieve the right document and still make the wrong decision. The name beside a runbook might belong to its editor. A timeout setting might have been replaced. A confident chat message might contradict the operating procedure.

Those are the problems behind OnCall Brain, a project for FalkorDB Graph Hacks. It joins operations documents, organisation records, chats, and support tickets in a knowledge graph, then answers questions with source-line evidence.

All examples below come from the project's synthetic corpus for the fictional company AetherPulse. They are reproducible test cases, not reports from a production deployment.

The engineering question is specific. Can we make the relationships needed for an operational decision explicit enough that an answer can be inspected?

Where the data comes from

Picture a platform operations team. Users report problems through a ticketing system, here called Sentinel Desk: each ticket records the module, the first-line engineer it was assigned to, and how it was resolved. The real work happens elsewhere, in a dedicated channel on the company's internal instant-messaging tool, where first-line and Tier 2 engineers ask each other questions, claim tickets, hand them over, and announce rule changes. Around both sit formal documents (SOPs, a module registry, an escalation policy) and an organisation roster.

The synthetic corpus mirrors that setup: 10 formal documents, 18 days of channel logs, 95 tickets, and the org records.

Chat and tickets meet on the ticket ID. The ticket system says TCK-1014 was assigned to Alan Cox. The channel log adds two lines: Dominic Shaw writes, "I am transferring TCK-1014 to Colin on Sentinel Desk right now", and later Colin Marsh writes, "Verified TCK-1014 closure". Extraction turns those lines into HANDLED_BY edges on the same Ticket node, each citing its chat line and marked observation, next to the ticket system's own record. Across the corpus, the ticket system contributes 95 handler edges and chat adds 26 more; 21 tickets end up with handlers from both sources. That is what lets the graph answer "who has actually fixed this?" with people the ticket system alone would miss.

This join depends on someone typing the ticket ID. In practice, people often discuss a problem without naming the ticket, describing a symptom that resembles one ("the certificate handshake is timing out again"). OnCall Brain does not link those today: entities align only on identical IDs, so a resemblance never becomes a graph fact. At query time, keyword search over all evidence lines, ticket records included, can still surface the similar ticket and the chat thread side by side, but only as retrieved context, not as an edge, and not in the "who has fixed it" count. The natural extension is a similarity pass that proposes candidate SIMILAR_TO edges between chat threads and tickets, kept out of answers until a person approves them.

Chat also carries rules. The change of the default step timeout from 120 to 60 minutes, used below, was announced in the channel, not in a document, so it enters the graph as a SUPERSEDES edge with chat evidence.

The runbook editor is the wrong escalation target

One test asks whether Wesley Tate should log into a backend server and terminate a zombie worker when a Pipeline runner deadlocks.

Wesley is a plausible search result. He reviews Knowledge Hub articles and coordinates submissions. His name belongs near Pipeline documentation. But the module's technical lead is Colin Marsh.

The distinction is encoded directly in the graph. A Resource points to its curator through MAINTAINED_BY. A Person points to a Module through TECH_LEAD_OF. The ontology explicitly forbids deriving technical responsibility from document maintenance. Unregistered predicates are rejected, and each assertion's domain and range are validated.

The answer should correct the question's premise, direct the ticket to the Pipeline queue, and identify Colin. The walkthrough demonstrates that correction; the frozen evaluation also records this trap as a pass.

A text or vector search can retrieve the supporting passages. Similarity alone does not establish what Wesley's relationship to Pipeline permits him to do. A model could reason correctly from those passages, but it would have to reconstruct the responsibility distinction during generation. Here, that distinction is part of the retrieval contract.

Ticket history adds a second view. Someone who handled a module's tickets may offer practical help without being its official owner. OnCall Brain joins tickets to modules and handlers, then presents that history alongside ownership. It does not treat past handling as proof of current availability or authority.

The graph behind those distinctions

The relationships are not invented on top of the data; they are read off the structure the sources already have. The module registry is a table with a header note, "Review Cadence: Weekly review by Wesley Tate", and rows such as:

Module ID	Module Name	Tier 2 Lead	Content Platform
MOD-01	Pipeline Orchestration Engine	Colin Marsh	Knowledge Hub

A deterministic parser, with no LLM involved, maps one column to one relationship. The Tier 2 Lead column, defined in the same file as the technical owner accountable for runbook accuracy, becomes Colin Marsh -TECH_LEAD_OF-> MOD-01. The platform column becomes MOD-01 -HAS_RESOURCE-> Knowledge Hub. The review note becomes Knowledge Hub -MAINTAINED_BY-> Wesley Tate. Rosters and team tables are handled the same way: a Team Lead column becomes a HOLDS_ROLE edge to a role node.

What about a title the schema does not know, such as "Support Lead"? Nothing is inferred from it. A new column in a structured source is ignored until someone decides which relationship it means. In free text, the model must pick from the nineteen registered relationships, so "Ava is the support lead for Pipeline" can at most become a role assignment, and a role never implies TECH_LEAD_OF. Adding a relationship type is a schema decision with an explicit domain, range, and meaning. That is deliberate: "support lead" means escalation owner in one company and shift coordinator in another.

The free-text path has a limit worth stating. Quote validation proves that the cited line exists and says what the assertion quotes; it does not prove the model chose the right relationship type. That choice is guided by the predicate definitions in the extraction prompt, not checked by code.

This abbreviated model uses concrete source-line locators on its domain edges. registry means 08-module-knowledge-base-registry.md; chat and ticket locators name their files directly.

Colin Marsh (Person) --TECH_LEAD_OF [registry L17]--> MOD-01 (Module)
MOD-01 (Module) --HAS_RESOURCE [registry L17]--> Knowledge Hub (Resource)
Knowledge Hub (Resource) --MAINTAINED_BY [registry L9]--> Wesley Tate (Person)

TCK-1017 (Ticket) --ABOUT [tickets.jsonl L17]--> MOD-01 (Module)
TCK-1017 (Ticket) --HANDLED_BY [tickets.jsonl L17]--> Robert Coleman (Person)

Notice: 60-minute timeout --SUPERSEDES [chat_2026-08-20.md L28]--> Rule: 120-minute timeout

Evidence is not drawn above because it is not wired into the business graph. Each Evidence node stores a source file, line, and quoted text; each relationship stores the IDs of its Evidence nodes in an evidence property, and citations are resolved through those IDs. Business nodes carry Entity; Evidence nodes do not, keeping evidence records out of business traversals.

Every assertion carries an ID, authority, validity dates, state, and evidence references. Answer citations pair assertion IDs with file:line. That makes the unit of inspection a supported relationship, rather than an unexplained connection between names.

Yesterday's correct timeout can be today's wrong answer

The original corpus contains a change in the default Pipeline step timeout from 120 minutes to 60 minutes, effective 2026-08-20. Both values remain useful evidence. Only one belongs in the current default answer.

SUPERSEDES preserves that history. A visible replacement excludes its target from current answers while retaining the old rule, replacement, and effective date for inspection.

The freshness demo extends the example in an isolated graph. It copies the loaded graph, injects a notice replacing the 60-minute timeout with 30 minutes, and repeats the same read. The old card becomes superseded; the new notice and evidence appear. The resulting chain is 120 → 60 → 30.

The 30-minute rule is an injected demonstration, not an original AetherPulse policy. The documented verification covers graph reads and reset; it does not establish that generated answers or browser layout were tested. The demo page reads the graph without an LLM.

Retrieval by similarity can return every version of a timeout policy. Ranking the newest passage can help, but it still leaves questions about replacement scope and effective dates. A replacement edge records which rule changed. Future notices remain invisible until their effective date.

Visibility is enforced server-side at every traversal hop. A relationship must be approved, its start date must have arrived, and its end date must be absent or still ahead of the query time. A matching sentence cannot bypass those conditions.

A chat message confuses claiming with diagnosis

Another test starts with Luke Fischer's belief that a P3 ticket can remain unclaimed for four hours. The formal workflow training guide penalizes tickets left unassigned beyond 15 minutes. Sophia Evans's chat correction separates the deadlines explicitly. Claim within 15 minutes, provide an initial technical response within 30 minutes, and understand four hours as the initial diagnostic window.

These passages share the same vocabulary while describing different stages. A retrieved “P3, four hours” snippet can therefore reinforce the wrong premise even when it is topically relevant.

The graph's assertions retain authority and semantic slots. Its authority order is formal_rule, then practice, then observation. Chat defaults to observation; an announced policy can be practice, but chat is never automatically promoted to a formal rule.

Conflict checks require claims about the same subject and slot, mutually exclusive values, compatible conditions, and overlapping validity intervals. This matters because a claim deadline and a diagnostic deadline can coexist. The mistaken claim-window belief needs correction; the diagnostic window does not need deletion.

The answer can lay out the disagreement and cite both the mistaken chat statement and the governing evidence. The frozen Q13 result records that behavior as a pass. This is a useful retrieval advantage, not proof that vector search could never answer the question.

The model selects tools; the server owns Cypher

The model never writes executable Cypher. It selects tool arguments, registered predicates, direction, and hop count. Server-owned templates perform the queries.

For example, the following is copied verbatim from the README's query for oncall_brain/tools.py:module_expertise.

MATCH (k:Entity)-[a:ABOUT]->(m:Entity {id: $id}), (k)-[h:HANDLED_BY]->(p:Entity) WHERE a.state = 'approved' AND a.valid_from <= $now AND (a.valid_to IS NULL OR a.valid_to > $now) AND h.state = 'approved' AND h.valid_from <= $now AND (h.valid_to IS NULL OR h.valid_to > $now) RETURN k.id, k.kind, k.name, m.kind, m.name, p.id, p.kind, p.name, a.aid, a.authority, a.evidence, h.aid, h.authority, h.evidence

Both ticket relationships receive visibility checks, and both return evidence. My reason for preferring this boundary is reviewability. Predicate semantics and filtering can be checked in a fixed query rather than trusted to each generated query. The evaluation reports also record executed template names and tool arguments.

Ingest has a similar boundary. Structured organisation and ticket records are parsed without an LLM. Free text goes through model proposal, schema and evidence validation, entity alignment, and semantic annotation. Accepted assertions begin as candidates. A quoted evidence line that does not match its source causes rejection.

What the measurements actually support

The frozen deep-mode evaluation passed 15 of 17 gold questions. The walkthrough reports 14 of 17 for fast mode; the frozen report is specifically a deep-agent run. These are project test results on synthetic data, not a production accuracy claim or a measured victory over vector retrieval.

The failures are informative. Q1 lost required-point coverage. Q7 included all required points but returned insufficient_context where the card expected applicable. Retrieving facts and presenting the correct answer state remain separate problems.

A cold-cache full ingest took about 2 hours 10 minutes in the documented clean-checkout test, including provider timeouts. Cached completed slices help resume extraction, but they do not remove cold-start cost.

Conflict validation also has a limit. The Web check accepts a graph conflict record or at least two distinct source-line references in the Conflicts section. Counting locators does not independently prove that the passages contradict. Evidence makes an answer inspectable; it still needs interpretation.

Try the ownership trap, inspect its citations, then follow a replaced rule through its history. Those checks show what this graph contributes to an on-call decision.

Repository: OnCall Brain

Demo video: https://youtu.be/gifjF3u3s0c

AI development assistance included Claude Code for implementation, review, and fixes; Codex for features and documentation; Antigravity CLI for frontend pages; and Hunyuan hy4-preview, driven through Claude Code, for backend KB health and a conflict fix.
1

Update2Report: turning messy Friday updates into a client-ready weekly report with an open-weight model

jc pong — Fri, 16 Oct 2026 07:08:32 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 2 (Side Hustle).

What I Built

Update2Report takes the messy updates a small team posts in chat during the week, plus an optional task CSV, and turns them into a weekly report you can send to a client: a short summary, a printable HTML page, and an Excel workbook.

The person I built it for is the lead of a 5 to 20 person outsourcing or ops team. Every Friday they scroll back through stand-up messages, copy lines into a document, count what got done, and tidy the wording. That is about two hours of work that nobody enjoys and nobody bills for. I think that is a task someone would pay to stop doing.

The thing I cared about most was trust. A report that sends a client "5 docs written" when the team wrote 4 is worse than no report. So the model does the language work, and plain Python does the counting and checking.

Demo

Live demo: http://8.138.157.149

Paste updates like these (all fictional):

Alice: finished login page redesign, 2 bugs fixed, blocked on API spec.
Bob: migrated billing cron to new scheduler; load test pending.
Chen: wrote 5 onboarding docs; started FAQ, 40% done.

The generated summary kept the 2, the 5 and the 40%, and did not add any people, projects or numbers that were not in the input. You also get a statistics table with Project, Owner, Completed, In progress and Blockers.

A few things about the demo itself:

There are buttons for cached sample inputs. They cost nothing and always work.
Live generation has a per-IP hourly cap and a daily cap, and I can switch it off with one setting. When it is off, or when the model endpoint is unreachable, the page says cached samples still work instead of showing a stack trace.
Please do not paste anything sensitive into a public demo.

Code

jsonpang / update2report

title	Update2Report
emoji	📋
colorFrom	blue
colorTo	indigo
sdk	gradio
sdk_version	5.50.0
python_version	3.11
app_file	app.py
pinned	false
license	mit

Update2Report

Turn messy team notes into a printable weekly HTML report and an Excel workbook. Open-weight models are the core of Update2Report—they extract structured tasks from unstructured updates and draft natural summaries, while deterministic Python code computes trusted statistics and cross-checks every number in the summary against ground truth. The public demo runs DeepSeek's open-weight DeepSeek-V4.1-Flash (MIT) through an OpenAI-compatible gateway, with OpenAI's open-weight gpt-oss-120b (Apache 2.0) as the fallback model; point LLM_BASE_URL at another provider or a local Ollama instance to use a different backend. All bundled Northwind Studio data is fictional.

Run locally

git clone https://github.com/jsonpang/update2report.git
cd update2report
python -m pip install -r requirements.txt
Copy-Item .env.example .env
# Edit .env with your own endpoint, key and model, then start:
python

…

View on GitHub

MIT licensed. The Python package is in update2report/, the Gradio app is app.py, and there are 52 tests.

How I Built It

Stack. Python, Gradio for the UI, and an OpenAI-compatible client for the model. The model is DeepSeek's open-weight DeepSeek-V4.1-Flash, licensed under MIT, with OpenAI's open-weight gpt-oss-120b (Apache 2.0) as the fallback. The model connection is three environment variables, LLM_BASE_URL, LLM_API_KEY and LLM_MODEL, so swapping the model or the host means changing those and no code. The public demo has a few more settings of its own (rate limits and a switch to pause live generation). An optional LLM_FALLBACK_MODEL retries once on a second model if the first is rate-limited or down.

Split of work. The model does two jobs: pull structured tasks (project, owner, status, blockers) out of free text, and write the summary in either an internal or a client-facing tone. Everything that can be a fact is computed in code.

Checks around the model.

Extraction validation. The extracted JSON is validated deterministically. If it is malformed, the model gets one correction attempt.
Number preservation. Every number and percentage in the source must appear in the summary. If one is missing, the summary is regenerated once. If it is still missing, the report appends a "Numbers from source:" list, so a figure is never silently dropped.
No invention. Any number in the summary, digits or words like "three", must exist in the source, and capitalized names in the summary (people, projects) must appear in the source too. Otherwise the draft is rejected and rewritten once with the reason attached. The name check is a simple word match, so it catches made-up names, not every wrong attribution.
Statistics from code. The Project / Owner / Completed / In progress / Blockers table is computed in Python, not written by the model. If a CSV is supplied, chat items and CSV tasks are deduplicated by task name within each project, so the same task is not counted twice.

These checks do not prove that extraction is semantically perfect. They make sure the numbers a client sees match what the team wrote, and when something is off, the failure is visible.

AI assistance. I used AI coding assistants (Codex and Claude) while writing the code and tests. I reviewed and ran the results myself, and the design above is mine.

Why Does Open Innovation Matter?

Privacy is the main reason, and cost is second.

A weekly report contains client names, delivery status and sometimes complaints. A team lead may not be allowed to paste that into a hosted chatbot, and a client may not want them to. Because the model is open-weight and the app speaks to any OpenAI-compatible endpoint, you can point LLM_BASE_URL at a local Ollama server and the notes never leave your machine. With a closed model that option does not exist.

Cost is the other half. The task is small: a few thousand characters in, a few hundred out, once a week. A small open model on hardware you already own is enough, and there is no per-seat or per-token bill to explain to a client. I also did not have to build around one vendor's API, which matters for a tool I may want to sell or hand over.

Where it did not do better: a larger hosted model writes smoother prose. That is why the checks exist. I would rather have a slightly plainer summary with correct numbers.

My First User

I do not have a paying user yet. Before the deadline I plan to ask a few team leads to run it on a real week of their own notes, and I will update this post with what they say, including whether anyone would pay.

What's Next

Test with local Ollama models of different sizes and record how often the number check has to fall back to the "Numbers from source" list.
Support more input formats, such as exported chat logs.
Offer a hosted version for people who do not want to run it themselves, and a self-hosted setup for people who care most about privacy.

Prize Categories

None that I am aware of fit, so I am not listing any.