DEV Community

Romil Patel
Romil Patel

Posted on

Does This CVE Affect Me? I Built an AI Agent That Shows Its Evidence

Sanity Challenge Path One Submission

This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content.

What I Built

I built AI Security Advisory Assistant, an evidence-first tool for researching known vulnerabilities in npm packages.

A developer enters a package, an optional installed version, and a security question. The app returns documented affected ranges, source-listed fixes, deployment conditions to verify, and links to original advisories. Its Coverage and Activity views show which sources and operations actually completed.

The app also accepts a package-lock.json and checks the exact installed versions of direct and transitive dependencies against published advisories. This is an advisory lookup, not a scan of application code or a guarantee that a deployment is safe.

Demo

The app currently runs locally and does not yet have a public deployment.

Step-by-step screenshots

These screenshots show a local research session and a sample dependency-file check.

1. Start a security research question

Open the Research page. Enter the software name and, if known, its installed version.

Empty security research form

2. Ask a specific, versioned question

For this example, enter Next.js, version 14.2.24, and ask whether CVE-2025-29927 applies, what fixes are listed, and which deployment conditions need review. Select Run research.

Research form filled with the Next.js version and CVE question

3. Read the research overview

The overview shows advisory findings, affected version ranges, documented fixes, and conditions to verify. Open the original advisory before acting on a result.

Research overview with advisory findings and fixes

4. Check coverage before trusting the answer

The Coverage tab records the resolved npm package, how many live advisories were retrieved and matched, whether the Knowledge Base check completed, and the timing of each source. A failed source means incomplete coverage.

Coverage tab showing completed OSV and Knowledge Base checks

5. Inspect the evidence

The Sources tab shows the retrieved Knowledge Base entry and links to original advisory records. I use those links to verify claims and deployment conditions against their sources.

Sources tab with Knowledge Base and original advisory links

6. Review the activity trail

The Activity tab shows the operations performed for this request, including package resolution, live lookup, Knowledge Base reads, and record verification.

Activity tab showing the operations performed

7. Pick a sample lockfile

The repository includes a demo package-lock.json. I use it to demonstrate the Dependencies feature.

Sample package-lock.json in File Explorer

8. Upload the lockfile

Open Dependencies and choose the lockfile, or drag and drop it onto the upload area. The app parses the JSON as data; it does not install packages or run scripts.

Dependencies upload area for package-lock.json

9. Review exact dependency matches

The result groups matching source records by installed package and version. Each advisory has a link for reviewing its listed fix and conditions.

Dependency results grouped by package and installed version

How It Works

Architecture

flowchart TB
  User["Your question"] --> App["Next.js app"]
  App --> Live["OSV.dev"]
  App --> Curated["Sanity Knowledge Base"]
  App --> Model["DeepSeek V4.1 Flash"]
  Live --> Check["Verify evidence"]
  Curated --> Check
  Model --> Check
  Check --> Answer["Cited answer"]

The model helps choose relevant Knowledge Base paths and passages. Application code checks the retrieved evidence and version conclusions before presenting the result.

Code

AI Security Advisory Assistant on GitHub

The repository includes the application, Sanity schema, advisory importer, sample lockfile, setup guide, screenshots, and verification record.

How I Used Sanity

These screenshots show my Sanity setup. Account details, identifiers, and trial information are obscured in the screenshots where visible. No token is shown.

1. Keep the existing project and dataset

I configured the app and local Sanity Studio for my existing AI Security Advisory Assistant project and its production dataset. I did not create a second project or dataset. The project settings screen confirms which project I used.

Redacted Sanity project settings for AI Security Advisory Assistant

2. Import selected source advisories

data/advisory-sources.json lists original GitHub Security Advisories. npm run ingest fetches and validates them as a dry run; npm run ingest -- --apply writes reviewed records into production using an Editor token kept in my local environment.

At the time of capture, the curated collection contained 122 published advisory documents. The organization activity view records that I attached the dataset to the Security Advisories Knowledge Base and created the Security Advisor Context MCP endpoint.

Redacted Sanity activity showing dataset attachment and Context MCP endpoint creation

3. Build a navigable Knowledge Base

In Sanity Context, I selected the production dataset as the source and built its entries. At the time of capture, the screen showed 122 source documents, a Ready source, and 28 generated entries.

Documents are the stored advisories. Entries organize related information into shorter, cited paths that the agent can navigate. Their counts differ because an entry can draw on multiple documents, and the counts can change after a rebuild.

Sanity Context source showing 122 documents and 28 ready Knowledge Base entries

4. Retrieve and verify at request time

The Next.js server connects to the Security Advisor MCP endpoint and calls initial_context and knowledge_base_read. It uses a separate project Viewer token to read the published advisory records behind cited entries.

The model helps select relevant paths and passages. Application code checks that paths, quotes, source links, and version conclusions agree with the records. The server also queries OSV.dev for live npm advisories, so results are not limited to the curated documents.

5. Refresh after imports

When source records change, I rerun ingestion, rebuild the existing Knowledge Base, wait for Entries up to date, and run a live test and browser query. The Context endpoint and Viewer tokens stay server-side. The Editor token is only for importing, never for the public app.

6. Inspect the generated evidence

The Entries view groups advisories by package and topic. This Next.js entry brings related cache-poisoning and XSS records together with affected ranges, fixes, conditions, and numbered source citations. The agent can navigate to a relevant entry through MCP instead of reading every source document.

Sanity Knowledge Base entry grouping Next.js advisories with version details and citations

Sanity Project Details

  • Project ID: o7qa6o3y
  • Dataset: production
  • Knowledge Base: Security Advisories
  • Context MCP endpoint: Security Advisor

Built With

Next.js, React, TypeScript, Node.js, Sanity Content Lake, Sanity Context MCP, OSV.dev, GitHub Security Advisories, Vercel AI SDK, DeepSeek V4.1 Flash through Token Harbor, Vitest, and Playwright.

Final Thoughts

For setup, start with Setup. For the exact tests performed, see VERIFICATION.md.

Thanks for reading.

Top comments (0)