DEV Community: Shikhar Verma

Precedent: an agent that knows which Solidity security advice has been overruled

Shikhar Verma — Mon, 05 Oct 2026 05:34:05 +0000

This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content

What I Built

Security advice for Solidity goes stale, and stale advice in this field is expensive. Docs, EIPs, audit reports and old blog posts contradict each other, and much of that disagreement is just time passing: the compiler and the EVM changed underneath the advice. A keyword search or a plain LLM returns the outdated answer with full confidence, because both only see text, not the fact that one piece of text replaced another for a given version.

Precedent treats security guidance like case law. Every claim carries a date and a compiler-version scope, and newer rulings overrule older ones. Ask "is transfer() still safe on 0.8.28?" and you get:

a ruling for your exact version,
a docket: what the older advice said (struck through) and what overruled it, each linked to its original source,
a contested status when sources genuinely disagree and nothing controls, instead of a guess.

It covers [[5]] patterns in depth: [[LIST, e.g. ETH sends with transfer/send, selfdestruct, SafeMath, ERC-4626 inflation attacks, and tx.origin as a no-change control]]. The control case matters: it checks that the agent doesn't invent drift where none exists.

Precedent summarizes published sources. It is not an audit, and every verdict says so.

Who it's for: Solidity developers who need a citable answer for their compiler version, reviewers doing quick triage, and hackathon builders who don't want to ship a pattern that was deprecated last year.

Demo

Live: [[DEPLOYED URL]]

[[VIDEO EMBED OR GIF: a verdict resolving, with overruled claims struck through and the controlling claim highlighted]]

Try these questions [[replace with the questions that work well in your build]]:

[[Question 1: a stale-trap pattern with a version]]
[[Question 2: a genuinely contested pattern]]
[[Question 3: the stable control, which should show no drift]]

Code

[[GITHUB REPO URL]]

How It Works

Question + compiler version
        │
        ▼
  Precedent agent (Vercel AI SDK, multi-step tool loop)
        │
        ├──► Context MCP endpoint A (Knowledge Base)
        │       what do the sources say? where do they conflict?
        │
        ├──► Context MCP endpoint B (dataset, GROQ)
        │       verified claims in scope for this version,
        │       with supersession and sources
        ▼
  Verdict: status, stance, claim IDs only
        │
        ▼
  UI fetches claim text, dates, URLs from Sanity by ID
        │
        ▼
  Ruling card + docket timeline

Identify the pattern and version from the question and the version selector. If no version is given, the agent assumes the latest in the dataset and says so.
Read the sources through the Knowledge Base endpoint to see what they say and where they conflict.
Query the claims through the GROQ endpoint: the verified claims whose version range covers the version, plus which claims supersede which.
Resolve. Controlling claims are in-scope claims that no other in-scope claim supersedes. If their stances agree, the status is settled. If they differ, it's contested and no winner is picked. If none exist, it's out of scope.
Return IDs, not prose. The agent emits a structured verdict containing claim IDs. The UI then loads the text, dates and URLs from Sanity.

Step 5 is deliberate. The model never writes a source, a URL or a date, so it can't fabricate a citation. The API route also re-checks that every ID exists and recomputes the status itself; on a mismatch the user sees an error, not a wrong answer.

How I Used Sanity

The agent reads Sanity through two Context MCP endpoints, because they do different jobs. An endpoint backed by a Knowledge Base serves Knowledge Base mode, and an endpoint backed by a dataset serves GROQ mode, so I use one of each.

Endpoint A: Knowledge Base

I pointed Sanity Context at [[N]] source documents: [[e.g. Solidity docs and changelogs, EIPs, OpenZeppelin docs and release notes, N public audit reports]]. Sanity distills them into a navigable Knowledge Base where each entry stays linked to the source it came from, and where conflicting claims surface side by side with their sources. [[Add what you saw: how many entries it produced, an example of a contradiction it surfaced on its own, and any decision you made about a conflict.]]

I stayed well within the beta's ~150-document budget on purpose: a small, curated set of primary sources beats a large noisy one.

Endpoint B: the dataset, in GROQ mode

Knowledge Base mode can't express "this claim applies only to 0.8.x and was replaced by that one", so I modeled that as structured content:

Type	What it holds
`pattern`	A named pattern, a short summary, and aliases
`source`	Title, URL, publisher, kind (docs, EIP, audit, release notes, blog), publish date, license
`claim`	A paraphrased statement (280 characters max), a stance (safe, unsafe, deprecated, mixed), `fromVersion` / `toVersion`, `supersedes[]`, `contradicts[]`, and a confidence flag

Compiler versions are stored as numbers (for example 0.8.28 becomes 8028) so GROQ can compare them. Only claims I had checked by hand against the primary source are marked verified, and only those are used. [[N]] claims across [[5]] patterns.

The ruling is computed, not stored. This query returns every in-scope claim and which other claims supersede it:

*[_type == "claim" && confidence == "verified" && pattern._ref == $patternId
  && (!defined(fromVersion) || fromVersion <= $v)
  && (!defined(toVersion)   || toVersion   >= $v)]{
    _id, statement, stance,
    "source": source->{title, url, publishedAt},
    "supersededBy": *[_type == "claim" && ^._id in supersedes[]._ref]._id
  }

The controlling claims are the ones with no in-scope claim in supersededBy. Because the ruling is derived, fixing one claim or adding a newer one changes every affected answer immediately, with no re-curation.

Why structure matters here

Keyword search finds passages that mention a pattern. It has no way to know that one passage replaced another for a specific compiler version. The supersedes chain and the version scope are the content model, and the agent's correctness depends on them. The Knowledge Base shows what the sources say; the dataset says which statement controls.

Tools the agent used

[[List the actual tool names each endpoint exposed, discovered at runtime, and what the agent used each one for. Example format: tool_name (endpoint A): used to ... ]]

The Context endpoints are read-only, so the agent can't change the content. Claims are seeded by a script and edited in Studio.

Does It Actually Beat the Alternatives?

I wrote [[N]] "Stale-Trap" questions where the obvious answer is outdated: [[N]] stale traps, [[N]] genuinely contested cases, [[N]] stable controls, and [[N]] out-of-scope. I labeled the ground truth by hand from the verified claims, then ran three systems with the same model, same output schema, and [[3]] runs per question:

A: keyword search over the raw sources (top passages given to the model, no tools)
B: Knowledge Base only (Endpoint A without the claims layer)
C: Precedent (both endpoints)

Because every system returns the same structured verdict, grading is deterministic rather than a model judging a model: an answer is stale if it matches a superseded claim, and a citation is correct if it includes a controlling claim.

System	Stale-answer rate (lower is better)	Cites the controlling source	Says "contested" when it should
A: keyword search	[[X%]]	[[X%]]	[[X%]]
B: Knowledge Base only	[[X%]]	[[X%]]	[[X%]]
C: Precedent	[[X%]]	[[X%]]	[[X%]]

[[Two or three sentences in your own words: which system did best, where B failed and why, and one specific question that shows the difference.]]

Design Decisions

IDs only from the model. Prevents fabricated citations.
Contested is a first-class outcome. A security tool that picks a winner between genuinely conflicting sources is worse than one that says so.
Verified claims only. Unreviewed claims never reach the ruling path.
No fallback to model memory. If the endpoints fail, the user gets an error, not a guess from the model's training data.
Small and deep. [[5]] patterns done properly beat 50 done loosely.
A control case. tx.origin has no drift, so a correct agent should report a stable answer.

Guardrails

Precedent never writes exploit code and never audits user code. Every verdict shows "Summarizes published sources. Not an audit." Claims are my paraphrases with links to the originals, and I used permissively licensed sources where possible.

Challenges and What I Learned

[[Write 3 to 4 honest items from your actual build. Prompts to answer: What broke first? What did the Knowledge Base do better or worse than expected? Where did the model misread supersession, and how did you fix it? How long did verifying claims by hand take, and what did you find?]]

Limits, Honestly

[[N]] questions is a small sample, so treat the percentages as indicative, not conclusive.
Only [[5]] patterns are covered.
Claims are paraphrased and hand-verified, so errors are possible. Check the linked sources before relying on a ruling.
[[Knowledge Bases are in beta; mention anything that didn't work as expected.]]

What's Next

More patterns and more eras of the EVM
Paste a code snippet and detect which patterns it touches
A compare mode: the same pattern across two compiler versions
[[Anything else you genuinely plan]]

Built with: Next.js, TypeScript, Vercel AI SDK, Sanity Studio and Sanity Context, [[model name and provider]], deployed on [[Vercel]], with Google Antigravity as the coding environment. (keep only if true)

Sanity Project Details

Project ID: [[YOUR SANITY PROJECT ID]] (dataset: [[DATASET NAME]])
[[OR: public dataset URL, and confirm the dataset is public]]

First Light: The Privacy-First Morning Brief & Radar for Hackathon Hunters

Shikhar Verma — Mon, 05 Oct 2026 05:18:31 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

My friend Anant is a machine learning engineer, backend enthusiast, and avid hackathon hunter. Between tracking global tech affairs, web3 releases, ML benchmarks, hackathon registration windows, and daily personal tasks, his mornings were overwhelmed by a storm of notification badges, spam newsletters, and calendar alerts. He needed a way to wake up, glance at his phone, and see only the top 5 things that actually matter to him today—without shipping his entire personal calendar, private to-do list, and unreleased project ideas to a third-party cloud LLM.
To solve this, I built First Light: a privacy-first, 100% locally-run morning intelligence brief and hackathon radar. Every morning at 6:30 AM, First Light:

Gathers events from local ICS calendars, markdown to-do files, RSS news feeds, and a curated hackathon radar (DEV, Devpost, MLH, Kaggle, and Web3/ETHGlobal).
Mathematically calculates urgency windows and deduplicates cross-posted hackathons across platforms.
Uses a local open-weight model (Google Gemma 2B) to synthesize the noise into a concise, prioritized 5-bullet morning brief tailored to Anant's profile.
Delivers a native, lock-screen Web Push notification to his phone via a self-hosted PWA, complete with an interactive dashboard and an ElevenLabs audio briefing. > "Waking up to just 5 bullet points that actually matter to me—instead of 40 noisy emails and calendar popups—has completely changed my mornings. And knowing it's all running locally on my own machine makes me trust it enough to feed it my real daily to-do list." — Anant

Demo

Video Walkthrough: Watch the Full Demo on Google Drive
Live Interface & Features:
- 🔔 Native PWA Web Push: Lock-screen notification delivered directly to mobile without third-party messaging apps.
- 🌅 Dawn-Themed Dashboard: Frosted glass UI displaying the AI briefing, deadline countdowns, and urgency meters.
- 🎧 First Light Audio: On-demand text-to-speech briefing player powered by ElevenLabs.
- 🎯 Hackathon Radar: Filterable list of competitions with direct registration links and AI "Should I enter?" verdicts.
- ✅ Interactive To-Dos: Real-time task completion synced directly to the local SQLite database.

Code

Shikhyy / firstlight

First Light 🌅

A privacy-first, locally-run morning brief and hackathon radar. First Light cuts through the noise by summarizing your calendar, markdown to-do lists, RSS feeds, and upcoming hackathons into a single, personalized push notification every morning at 6:30 AM.

All data fetching, deduplication, and AI summarization (using gemma:2b via Ollama) runs 100% locally on your machine. Your schedule never leaves your device.

Features

Hackathon Radar: Aggregates and deduplicates hackathons from DEV, Devpost, MLH, Kaggle, and Web3 sources.
Local AI Summarization: Uses Ollama to read your top events and tell you why they matter based on your configured interests.
Privacy-First: No cloud APIs for your personal data.
PWA Web Push: Sends a native lock-screen notification to your phone without relying on third-party apps.
Deterministic Math: LLMs are bad at math, so all deadline urgency and countdowns ("closes in 13h") are calculated entirely in Python before…

View on GitHub

Repository URL: https://github.com/Shikhyy/firstlight

How I Built It

First Light is engineered in Python with an installable CLI (pip install -e .), backed by SQLite and a lightweight Flask PWA with a custom Service Worker.

1. Open-Source AI Architecture

Local-First Core: We run Google's Gemma 2B (gemma:2b) locally via Ollama. It runs on consumer hardware (such as an Apple Silicon Mac) with near-zero latency and zero operational cost.
Pydantic Structured Outputs: The pipeline enforces structured JSON schema validation (BriefSummary) directly on the model's output to guarantee schema compliance.
Graceful Cloud Fallbacks: For cloud deployments where local Ollama is inaccessible, First Light seamlessly falls back to Hugging Face Serverless Inference (google/gemma-2-2b-it) or Groq, keeping the architecture strictly on open Gemma models.

2. The "Code Does Dates, AI Judges" Architecture

LLMs frequently hallucinate calendar math and relative dates. First Light enforces a strict architectural separation:

Deterministic Engine (Python): Date parsing, deadline countdowns (e.g., "closes in 13h"), urgency scores ([0.0, 1.0]), and deduplication (canonical URL normalization + RapidFuzz token matching) are calculated purely in Python.
Judgment Engine (Gemma): The LLM receives pre-calculated facts and strictly evaluates relevance: "Why does this matter to Shikhar based on his interests in ML, Web3, and Backend?"

3. Resilient Pipeline & Fail-Safe Delivery

Each data connector (DEV, Devpost, MLH, Kaggle, RSS, Calendar) is isolated in a try/except sandbox; if one scraper fails, the pipeline logs the error in a source_runs health table and continues uninterrupted.
If Ollama is offline or times out, the pipeline triggers a deterministic fallback generator, ensuring a morning brief is always delivered on time.

Why Does Open Innovation Matter?

Open innovation and open-weights models are essential for personal software like First Light:

Absolute Data Privacy: Your personal calendar, daily tasks, health appointments, and unreleased project ideas are deeply intimate data. Routing this stream through proprietary closed APIs means sharing your life with centralized corporate servers. With Gemma running locally, your personal data literally never leaves your device.
Zero Marginal Cost & No Rate Limits: Closed APIs charge per token, rate-limit automated jobs, and risk breaking changes. Open models allow automated morning pipelines and background cron jobs to run indefinitely for free.
Resilience & Vendor Independence: Open innovation ensures First Light will function offline, on an airplane, or years in the future without risk of API deprecation, subscription price hikes, or account suspensions.

My Agent Session

First Light was designed, implemented, and tested through an intensive pair-programming session using Google Antigravity:

Orchestrated the full project lifecycle: data connectors, SQLite schema design, fuzzy deduplication algorithms, Pydantic schema validation, and PWA Web Push implementation with VAPID crypto keys.
Iterated rapidly on prompt engineering and deterministic fallback logic to guarantee sub-minute pipeline execution on lightweight hardware.

Prize Categories

We are entering the following partner categories:

Gemma: Core intelligence is built entirely around Google's gemma:2b via Ollama for fast, local, privacy-first summarization, with Hugging Face Serverless integration for google/gemma-2-2b-it.
ElevenLabs: Integrated ElevenLabs Text-to-Speech into firstlight/delivery/audio.py to synthesize a natural, spoken morning briefing playable directly from the PWA dashboard.
Render: Provided a turnkey render.yaml Infrastructure-as-Code Blueprint with persistent disk mounts to deploy First Light as a cloud web service and automated cron worker with a single click.
Sentry Agent Tracing: Integrated sentry-sdk to trace pipeline transactions, monitor scraper reliability, and observe local LLM inference performance.