DEV Community: vinmay

ClearToList: Is that thrift find recalled? A photo check against CPSC recalls with Gemma 4

vinmay — Sat, 17 Oct 2026 16:35:22 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

A $5 bassinet at a thrift store can be a recalled product that's illegal to resell, and nothing on it tells you.

That's the problem I am trying to solve. Selling a recalled product is illegal in the U.S. even secondhand and even if you're one person flipping stuff on Facebook Marketplace on weekends. A lot of what gets recalled is exactly the stuff that piles up at thrift stores like cribs, bassinets, sleepers, high chairs, strollers, toys.

I pulled every CPSC recall since 2015 to see how bad it is. About 1 in 5 is a kids’ product, and more than half of those (55%) don’t list a model number or UPC at all in their product data. The only way to know is what the thing looks like.

Every recall tool I found wants text. A brand, a model number, a SKU from your store catalog. A reseller's haul has none of that. No box, no SKU, half the time you don't even know the brand.

So I built ClearToList. You take a photo of the item, it finds the closest recalls. Then you take a photo of the label and it settles it by model number or UPC. You get one of three answers, shown as a price-dot sticker on your own photo:

🟥 Recalled: a model number or UPC on the item matches a recall. Don't list it.
🟨 Check the label: it looks like a recalled product. That alone means nothing, so it asks for a label photo.
⬜ No match found: nothing matched. This never means "safe". There's no green sticker on purpose.

Only Recalled is an alarm. My first version also alerted when something just looked like a recalled product. Turns out a lot of normal white cribs look like recalled white cribs and that evaluation talked me out of it.

Buyers get it too. Looking at a Marketplace listing? Screenshot it and check before you pay. If it looks like a recalled product, the page gives you a message to send the seller asking for a label photo. Resellers can check a whole haul at once (up to 10 photos) and get one report as CSV or PDF.

Who pays for this

Nobody yet, and nothing is charged today. Here's what I'd test first:

Free: buyers and casual checks, a few photo checks a day. Parents shouldn't pay to find out if a crib is recalled.
Reseller, $9/month: unlimited photo checks, bigger hauls, alerts when something you already checked gets recalled later, and a dated "checked" record for every item you list.
Store, $29/month per location: bulk intake scanning for consignment and kids' resale shops, several staff logins and a daily pull list. They take in used baby gear from strangers every day and carry the most risk.
Marketplace licensing: the long shot. A check built into a resale app's listing flow.

A scan on open models costs a fraction of a cent, so even the $9 tier works with very few customers. These prices are my guess, not tested.

Demo

Real results from the live site, I just cut the waiting. Demo photos: Wikimedia Commons (ajay_suresh, Sally V, Hardyplants, קרלוס הגדול), CC BY 2.0 / CC BY-SA / CC0.

Try it yourself: cleartolist.onrender.com

No sign-up. Three sample photos run real scans, and everyone gets free photo checks every day. Want to use it regularly? Drop your email for an invite code and I'll send one within a day. Model number, UPC and brand lookup is always open.

Take a photo, pick one from your library, or try a sample. The label is step 2.

The label's model number matches a recall. Red sticker, what to do whether you're selling or buying, and the recall itself.

Looking similar is never an alarm. Yellow sticker, the closest recalls to compare, and a button to add the label. (Play yard photo: Famartin, CC BY-SA 4.0, via Wikimedia Commons.)

Code

vinmay / cleartolist

ClearToList

Check secondhand products against U.S. product recalls before you list them, children's products first.

Resellers buy unboxed, unbranded secondhand items with no SKU, so catalog-based recall tools can't help them. And selling a recalled consumer product is illegal no matter how small the seller. ClearToList works in two photos: a photo of the item finds the closest CPSC recalls, and a photo of its label settles it by model number or UPC. You get one of three verdicts:

Verdict	Meaning
🟥 CONFIRMED	A model number or UPC read off the item matches a recalled product. Don't list it.
🟧 POSSIBLE MATCH	The item looks similar to the recalls shown side by side with your photo. That alone doesn't mean it's recalled: photograph the label and scan it.
⬜ NO MATCH FOUND	No match in CPSC recalls. That does not mean the item is safe.

Only CONFIRMED is an alarm…

View on GitHub

How I Built It

Data

All CPSC recalls since 2015 come from the CPSC Recalls API: about 3,400 in scope and around 12.7k product photos. The ingest only pulls what changed, backs off politely and falls back to a cache if the API is down. A GitHub Actions job refreshes it every night and redeploys the site's data. Car seats are out because NHTSA handles those, not CPSC.

The scan

flowchart LR
    P[Item photo] --> A[SigLIP 2<br/>image search over<br/>~12.7k recall photos]
    P --> G[Gemma 4 reads the photo<br/>JSON: item type, brand,<br/>text, model no., UPC]
    G --> E[Exact match<br/>model no. / UPC]
    G --> T[EmbeddingGemma<br/>text search]
    A --> F{Weighted rank fusion}
    T --> F
    E -->|boost to top| F
    F --> S[Top 3 shortlist]
    S --> V{Rule}
    V -->|code matches| C[Recalled]
    V -->|closest photo very similar| M[Check the label<br/>show shortlist,<br/>ask for label photo]
    V -->|otherwise| N[No match found]
    L[Label photo] --> G2[Gemma 4 reads<br/>model no. / UPC] --> E2[Exact match] --> C

Gemma 4 E4B reads the photo and returns a JSON: item type, brand, any visible text, model number, UPC.
Search. SigLIP 2 searches the recall photos, model numbers and UPCs are matched exactly, brands are matched, and EmbeddingGemma searches the recall text. Weighted reciprocal rank fusion combines it all. For a normal product photo the image leads. Text only leads when the photo is basically a label and Gemma actually read words off it.
A dumb rule decides, not a model. Code match means Recalled, unless the brand on the item contradicts the recall. Very similar closest photo means Check the label. Otherwise, No match found. I wanted the answer to come from something I can read in ten lines and not from a model non-deterministically.

Picking the reader

I tried Gemma 4 E2B and E4B on 30 held-out photos. First prompt was bad because Gemma kept saying "baby product" instead of "bassinet". Asking for a specific item type got both to 15/30, and E4B was faster (3.6 s vs 6.2 s median), so E4B it is. A full scan is about 7s.

One gotcha is SigLIP on the GPU left too little room for Gemma and EmbeddingGemma to stay loaded, so Ollama swapped them on every single scan and added 25s. So, I had to move SigLIP to the CPU and that solved the problem and it's still well under a second there.

The site and not getting wrecked by a public upload form

The site runs on Render's free tier and runs no models itself. Photo checks go into a small in-memory queue, and a separate GPU worker running the open models connects out to the site over signed HTTPS to pick up jobs. It accepts no incoming connections. It sends back only recall ids and matched codes, and Render re-checks any code against its own copy of the data before it ever shows Recalled.

Security

Uploads are checked by their actual bytes, capped at 8 MB and ~30 megapixels, and re-encoded, which strips GPS and anything hidden in the file.
Photos live in memory only and get deleted right after the check.
Per-person rate limits, a daily cap on free checks, a small queue that says no when it's full, and a kill switch for uploads.
Worker requests are HMAC-signed with a timestamp and nonce.
Anything Gemma reads off a label is treated as untrusted. It can't set the verdict, and the page sits behind a strict Content Security Policy.
No cookies, no IPs, no tracking. Just anonymous daily counts and a "Was this helpful?" button.

Evaluation

Every test query is a held-out recall photo roughed up to look like a phone shot (crop, rotation, dim light, blur), and that photo plus every near-duplicate of it is hidden from the index. The only way to score is to find other photos of the same recall. For negatives I used licensed Wikimedia photos of normal, non-recalled products in the same categories. Rules were picked on a dev split and the test ran once.

There are three different numbers here and they're easy to mix up, so here's what each one means for someone holding the item:

From one item photo	Kids' products	Other categories	Label close-ups
Flagged "Check the label" with the right recall	34%	37%	15%
Right recall shown at all (incl. grey "closest recalls to check")	63%	38%	57%
Right recall somewhere in the internal top 3 (not shown to you)	67%	64%	60%
Recalled verdicts (right / total)	0 / 0	2 / 3	2 / 2
False Recalled alarms on normal lookalikes	0%	0%	n/a

The top row is the one that matters: from a photo alone, ClearToList points you at the right recall about a third of the time. The 67% in the third row only says the search is good enough to build on. I hide most of that shortlist on purpose, because showing it every time flagged almost every normal product.

The grey suggestions in row two raise it to 63% for kids' products, but 98% of normal lookalikes get suggestions too. That's why they're grey and never called a match.

The label photo is what makes it reliable. Recalled needs a model number or UPC off the item. Of the five Recalled verdicts in the test, four were correct. None of the 124 tested lookalikes triggered a Recalled verdict. These are encouraging early results, but the sample is too small to establish real-world reliability. Cribs and sleepers only get 45% top 3 from the image, because one white crib looks a lot like another. On label close-ups, Gemma reading the label finds the recall where image search can't (60% vs 18%).

I also set a kill criterion before building the app: about 60% internal top-3 on kids' products, or I drop the idea. The spike hit 71% and the test landed at 67-69%. That number answers "is the search good enough to build on", not "what does a user see". The user number is the 34%.

It took three runs to get these numbers. Run 1 had a Gemma "verifier" that was right about half the time and raised alarms on 31% of normal kids' items, so I cut it. The full run history is in the Full evaluation.

Why Does Open Innovation Matter?

The math only works if a scan is basically free. A reseller might check hundreds of items a week. On open models a scan costs a fraction of one API call, and there's no per-call bill to scale with usage. That's the difference between "free for parents" being a real promise or a trial.
Photos of people’s stuff stay private. People photograph things in their homes, sometimes with their kids' stuff around. Those photos go to my worker, get checked and get deleted. They never get sent to a third-party model API.
I could actually tune it. Forcing strict JSON out of Gemma, swapping E2B for E4B, moving SigLIP to the CPU, picking thresholds on a dev split: all of that is versioned in the repo, so nothing shifts under the evaluation because a vendor updated a model. And the whole pipeline can be fine-tuned on recall photos later.

My First User

It went live this weekend, so I don't have a real user story yet. I'll update this section with actual numbers and whatever people tell me.

If you flip on eBay, Poshmark or Marketplace or you're buying used baby gear, run one item through it and tell me in the comments what it got right or wrong.

Prize Categories

Gemma: Gemma 4 reads the item and its label, which is the core of the whole thing. EmbeddingGemma powers the text search.
Render: the hosted site and job queue, deployed from render.yaml.
GitHub Copilot (GitHub Actions): Actions is what keeps ClearToList current without me touching it. A scheduled workflow runs every night at 09:17 UTC: it pulls only the recalls CPSC added or changed since the last run, rebuilds the site’s SQLite database (full-text search plus exact model-number and UPC tables), publishes it as a data-latest release asset and calls Render’s deploy hook, so the live site picks up a new recall the next morning. A second workflow runs ruff, the full pytest suite and a gitleaks scan over the whole git history on every push. During launch weekend a third one pings the site every 10 minutes so the free instance doesn’t fall asleep on a visitor.

What's next

Per-category thresholds to cut down noise outside kids' products (sports gear and kitchen stuff are the worst)
More recall sources: EU Safety Gate (it has photos) and Health Canada
Real thrift store photos in the eval
More scanning capacity by running the same worker on rented GPUs
Alerts that re-check your past items when a new recall drops

This is a screening aid only, not legal advice. Recall data is from the U.S. Consumer Product Safety Commission. Not affiliated with or endorsed by CPSC.

Doppel: find two different things that look similar, outside

vinmay — Mon, 12 Oct 2026 04:07:28 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Doppel is a walking game about lookalikes. You take a photo of anything outside then walk about 50 meters, then photograph another one which could be its twin: a different kind of thing that looks the same (a pinecone and an artichoke, or moss and broccoli). EmbeddingGemma 2 runs on your phone which judges how alike the two photos look and shows its reasons.

It's for anyone on a walk who wants a reason to look at things more closely around them. You can do three pairs a day for 10 to 15 minutes then it tells you to stop. You can't lose and you have fun looking around during your walk and your photos never leave the phone.

Demo

Live: https://doppel-ws59.onrender.com (Chrome on Android or desktop; needs WebGPU; downloads a 388 MB model once, then works offline)
No time to go outside? Try a demo round: one full round with bundled CC0 photos and the real model, no camera needed, nothing saved. On a desktop it's the first button you see.
Video:

Code

vinmay / doppel

Doppel started as a fork of Sense Hunt and Warmr (same author, same challenge window).

Doppel

Find two different things that look the same. Outside.

Photograph anything outdoors. Walk about 50 metres. Then find its twin: a different kind of thing that looks the same, like a pinecone and an artichoke, moss and broccoli, or bark and elephant skin. EmbeddingGemma 2 runs on your phone and judges how alike the two photos look, and it shows its reasons:

Doppel sees: what it thinks each photo is ("pinecone ↔ artichoke"), so you know it understood you.
A twin score from 0 to 100, in four bands: Uncanny, Cousins, Distant, Strangers.
What they share: "Both: ridged · layered · wrinkled", the visual qualities both photos have.

Three pairs a day, about 10 to 15 minutes, then "Done for today". You can't lose. Photos never leave the phone, and after the first load…

View on GitHub

How I Built It

How I got here

Doppel was my third try at this challenge.

Sense Hunt. A daily list of prompts ("a tree with peeling bark"); you photograph one and the model checks it. It worked but it was getting too hard as the prompt generator needed to have a controlled list of words and something achievable for the player.
Warmr. Hot/cold for the real world: find a hidden daily secret using photos. It depended too much on where you lived. If your street didn’t have the secret, you couldn’t win.
Doppel. I was then looking for a judgement that could be done similarly by a player as well as the model. We are very good at "these two look alike" games and so is image embedding. Doppel is that judgment.

How it works

EmbeddingGemma 2 came out the day after the challenge opened. It can embed both photos and text into the same space and it's small enough to run in the browser on WebGPU and Doppel leans on it for everything.

The score is just how close your two photos are scaled to 0-100 which is calibrated on 23 of my own outdoor photos.

The rest comes from comparing photos to text. At build time I embedded a list of about 200 everyday things ("a tire", "moss") with a few things you shouldn't photograph (screens, rooms, people), and 45 visual words like "round" or "ridged". The closest thing is what "Doppel sees". A visual word shows up under "What they share" when it fits both photos noticeably better than usual. The same lists suggest what to look for on your walk ("Try: seed pod · corn on the cob · seashell") and catch you photographing the same kind of thing twice.

A pair of very different things that still look alike gets a small bonus, so a pinecone and an artichoke beat a cloud and fog. The "Need an idea?" hints were written by Gemma 4 E4B, running locally through Ollama.

Nothing leaves the device. Your photos and scores live in IndexedDB, and a service worker keeps the app and the model available offline. The colors come from Sanzo Wada's A Dictionary of Color Combinations (1933).

On my walk

I played a full day in my neighborhood with three pairs and these are the results Doppel showed me -

Tire ↔ manhole cover: 49, Distant. Both round, ridged, gray. Fair.
Power lines ↔ street lamp: 37, Distant. Doppel's verdict: "If you squint. Hard."
Manhole cover ↔ sidewalk: 83, Uncanny. The one that surprised me: up close, the cover and the speckled concrete really do read as the same texture.

One observation I had was on my Pixel 8 Pro each photo takes 15-20 seconds to read which is much slower than the 1.7 s on a laptop. It turned out fine because that's time spent looking around instead of at the screen.

Does it work?

These numbers come from the real model running in a browser.

I tested on 44 CC0 photos with the real model in a headless browser on a laptop iGPU. (docs/eval.md):

	Result
Twins vs strangers, mean score	58.6 vs 19.9
Twins scoring Cousins or better	6 of 8
Strangers scoring Distant or worse	8 of 8
Same-kind pairs refused	3 of 4
Screens, rooms, documents, blurry photos refused	4 of 4
Photo embed time (laptop iGPU)	median 1.7 s

The first run passed five of the six original criteria and only 5 of 8 twins reached Cousins (moss and broccoli sat at exactly 50).

Why Does Open Innovation Matter?

It works on a walk with no signal. The model is open-weight and small enough (388 MB) to run in the browser on WebGPU, so after one download Doppel works in airplane mode, in a park, on a trail.
Your photos never leave your phone. A game that asks you to photograph your street all day shouldn't upload your street. Here there's nothing to upload to: no server, no account.
It costs nothing to run. No API bill per photo means no ads, no subscription, no limits.
The model is swappable. The game only needs "embed a photo" and "embed a text". The calibration script re-derives every number from photos, so a better open model is a re-run, not a rewrite.

Prize Categories

Best Use of Gemma: EmbeddingGemma 2 (released Oct 6) is the core of the game: every photo is embedded on-device in the browser (LiteRT-LM, WebGPU), and the score, the labels, "What they share" and the twin suggestions all come from it. Gemma 4 E4B, run locally through Ollama, wrote the idea hints.
Best Use of Render: Doppel is deployed on Render as a static site (render.yaml in the repo). Render serves the app and the LiteRT-LM WebGPU runtime that each phone downloads to run EmbeddingGemma 2 locally.

Your `pip install` Just Stole Your SSH Keys: The LiteLLM Supply Chain Attack Explained

vinmay — Tue, 24 Mar 2026 20:19:13 +0000

A single pip install litellm==1.82.8 was enough to drain everything off your machine. No suspicious imports. No weird prompts. Just a package install, and your AWS credentials, SSH keys, and API keys were already heading to an attacker's server.

Here's what happened, why it's scary, and what you can actually do about it.

What Happened

On March 24, 2026, LiteLLM version 1.82.8 landed on PyPI with a malicious file bundled inside: litellm_init.pth.

That .pth extension is why this attack is so nasty.

Python automatically runs .pth files in your site-packages directory every time the Python interpreter starts. No import needed, no user interaction. The attacker hid a double base64-encoded payload inside this file. The moment Python ran, the payload ran too.

What did it grab? Pretty much everything:

All your environment variables (OPENAI_API_KEY, AWS_SECRET_ACCESS_KEY, all of it)
SSH private keys (~/.ssh/id_rsa, id_ed25519, and more)
AWS, GCP, Azure, and Kubernetes credentials
Git credentials and .gitconfig
Shell history (~/.bash_history, ~/.zsh_history)
Docker configs, npm tokens, database passwords
Crypto wallet files

Then it encrypted everything with AES-256, wrapped the key with a hardcoded RSA-4096 public key, and shipped it all off to https://models.litellm.cloud/. Note that domain: litellm.cloud, not the real litellm.ai. Classic.

Why the Scale Is Scary

LiteLLM gets 97 million downloads per month. That alone is a huge problem.

But supply chain attacks don't stop at the direct install. They travel through dependency trees. If you installed dspy, langchain, or any of the other popular AI packages that depend on litellm>=1.64.0, you were also exposed without ever typing pip install litellm yourself.

The attack was only live for about an hour. It got discovered almost by accident: a developer's machine ran out of RAM and crashed because the payload was executing inside Cursor through an MCP plugin that pulled in litellm as a transitive dependency. A bug in the attacker's own code gave them away.

If that bug wasn't there, this could have run quietly for days or weeks across thousands of CI/CD pipelines, dev machines, and prod servers. Nobody would have noticed.

The Real Problem: You Can't See What's Inside Your Dependencies

When you run pip install something, you're not just installing one thing. You're pulling in a whole tree of packages, and any one of them could be compromised.

This isn't new, but it's getting worse as the AI package ecosystem keeps exploding. New packages, new versions, new dependencies dropping every single day. The attack surface is growing way faster than anyone can audit it.

We're taught to think of dependencies as a good thing. Reusable building blocks, standing on the shoulders of giants, all that. The LiteLLM incident is a reminder that every dependency is also a trust decision, and most of us are making those decisions without really thinking about it.

What You Should Actually Do

If you installed litellm 1.82.8:

Check for litellm_init.pth in your site-packages/ directory
Rotate everything: every API key, SSH key, and cloud credential that was on that machine
Check any CI/CD environment where litellm might have been installed too

Going forward:

Pin your dependencies. Exact version locks (==) in production instead of >=. It won't stop a poisoned release from getting in if you're on that version, but it stops silent upgrades pulling in something bad later.
Use lockfiles. pip-compile, poetry.lock, uv.lock, whatever fits your setup. Know exactly what you're running.
Audit transitive dependencies. pip-audit and safety scan your full dependency tree for known issues. Worth running in CI.
Don't pip install as root. Limits how much damage a compromise can actually do.
Keep an eye out for .pth files. They're a legit Python feature, but they're also a perfect delivery mechanism for malware. If you see one in site-packages from a package you don't recognise, that's worth investigating.

Can We Do Better Than Grep?

Most of the advice above is reactive. It helps you recover or reduce damage after something gets in. What's actually hard is knowing before you run anything that a package can even reach your credentials.

This is the gap I've been trying to close with something I'm building called ReachScan. The idea is pretty simple: instead of just matching against a list of known bad packages, it maps what a codebase or its dependencies can actually reach: filesystem paths, environment variables, system resources. If a package has no business touching ~/.ssh/, you should know that before it runs, not after.

It won't catch everything. But knowing the capability surface of what you're about to install is a lot better than just hoping nothing in the tree is malicious.

The Uncomfortable Truth

Karpathy put it well after this incident: the way we think about dependencies needs to change. The whole "building pyramids from bricks" model assumes the bricks are trustworthy. In 2026, that's a harder assumption to stand behind.

That doesn't mean stop using dependencies. That's not realistic. It just means:

Be deliberate about what you pull in
Actually understand what each dependency can do on your machine
Have a rotation plan for credentials that treats compromise as a when, not an if

The LiteLLM attack got caught by luck. The next one might not be.

I built "npm audit" for AI agents

vinmay — Sat, 21 Mar 2026 02:23:50 +0000

I was adding MCP tools to a project when I realized something uncomfortable: I had no idea what the code I was installing could actually do.

The README said "connects Claude to Blender." What it didn't say was that one of the registered tools passes a raw string parameter to Python's exec() with no builtin restriction. The LLM doesn't get "Blender API access." It gets full Python execution on the host machine.

I wanted a way to know this before running the code. So I built one.

What reachscan does

reachscan is a static analysis CLI for Python and TypeScript/JavaScript AI agent codebases. Point it at a repo, a PyPI package, or an MCP endpoint, and it tells you:

What the code can do (shell exec, file access, network calls, credential access, dynamic code execution)
Which of those capabilities the LLM can actually trigger (reachability analysis)
The exact call path from the LLM entry point to the dangerous code

pip install reachscan

# Scan a GitHub repo
reachscan https://github.com/user/repo

# Scan a PyPI package before installing
reachscan pypi:some-agent-package

# Scan local code
reachscan ./my-agent

That's it. No config, no API keys, no cloud service. It runs offline and produces a report in about 2 seconds.

The problem

When you give an LLM tools, you're granting it real-world capabilities like file access, shell commands, network calls, credential reads. Most frameworks make it easy to add tools and hard to audit what you've exposed.

Here's real code from a popular MCP server:

@mcp.tool()
def execute_blender_code(ctx: Context, code: str) -> str:
    """Execute arbitrary Python code in Blender."""
    blender = get_blender_connection()
    result = blender.send_command("execute_code", {"code": code})

That code: str parameter? It ends up here:

exec(code, {"bpy": bpy})  # No __builtins__ restriction

namespace = {"bpy": bpy} looks like a sandbox. It isn't. Without explicitly setting __builtins__, Python injects the full builtins module. The LLM can import os, run subprocess, read your files — anything.

Here's what reachscan shows for this server:

  DYNAMIC  exec()                  server.py:431  reachable
           path: execute_blender_code → send_command → execute_code

  EXECUTE  subprocess.run()        addon.py:89    reachable

  SEND     requests.post()         server.py:198  reachable
           path: generate_3d_model → _call_api

  SECRETS  os.environ[...]         server.py:12   module_level

The reachable tag is the key part. It means the LLM can trigger this code through a registered tool and not just that the code exists somewhere in the repo. module_level means it runs on import. unreachable means the code exists but no LLM call path leads to it.

How it works (briefly)

Detectors scan the AST for 7 capability categories: EXECUTE, READ, WRITE, SEND, SECRETS, DYNAMIC, AUTONOMY
Entry point detection finds the functions exposed to the LLM — @tool, @mcp.tool(), @function_tool, BaseTool subclasses, etc. across LangChain, OpenAI Agents SDK, MCP, Pydantic AI, CrewAI, Semantic Kernel, and AutoGen
Call graph + BFS traces up to 8 hops from each entry point to determine which capabilities are actually reachable
Every finding gets one of 5 states: reachable, unreachable, module_level, unknown, no_entry_points

The false positive rate is 0.47% across 1,912 labeled findings on 10 real-world repos. I care about this number a lot because a noisy scanner is a useless scanner.

Why I built it

The short version: I was evaluating third-party MCP servers and realized there was no npm audit equivalent for AI agent code. I could run pip audit to check for known vulnerabilities in dependencies, but nothing told me "this package gives the LLM shell access on your machine."

The existing tools I found either:

Require API calls per scan (expensive, not offline)
Produce flat capability lists without reachability context (noisy)
Don't handle the MCP/agent-specific entry point patterns

So I built the tool I wanted.

What it found across 50 real MCP servers

I ran reachscan against 50 of the most popular MCP server repos:

1 in 3 has shell execution capability
1 in 3 has outbound network I/O
1 in 4 accesses credentials from environment variables
10 of 50 had 4+ capabilities active simultaneously

The highest-risk combination: credential access + network egress. That appeared in 8 of 50 repos. If the LLM can read your AWS keys AND make HTTP calls, that's an exfiltration path.

Not all of these are bugs. An AWS MCP server should talk to AWS. The question is whether the LLM can misuse those capabilities — and whether you know about them before you deploy.

Try it

pip install reachscan

# Scan any GitHub repo
reachscan https://github.com/ahujasid/blender-mcp

# Scan a PyPI package before installing
reachscan pypi:openai-agents

# JSON output for CI
reachscan . --json --severity high

Apache 2.0, pure Python, runs offline. No API keys, no cloud service.

If something looks wrong — false positive, missed pattern, bad output — open an issue.

GitHub: vinmay/reachscan
PyPI: reachscan
Full scan results (50 repos): Medium writeup

I scanned 50 MCP servers to see what they can actually do — here's what I found

vinmay — Thu, 12 Mar 2026 03:35:44 +0000

One of the 50 MCP servers I scanned gives the LLM a full Python shell
on your machine. The tool is called execute_blender_code. The exec()
call has no builtin restriction. I verified it — imports, file reads,
subprocess execution all work.

That's what I built reachscan to find.

The problem

MCP servers aren't plugins in a sandboxed browser extension model. They
run as normal OS processes with your user permissions. If a server calls
subprocess.run(), the LLM can trigger shell commands. If it calls
exec() without restricting builtins, the LLM gets Python execution on
your machine.

Most people don't know which of the servers they're running fall into
which category.

What reachscan does

It's a static analysis CLI for Python and TypeScript/JavaScript agent
codebases. It maps seven capability categories:

Capability	What it means
`EXECUTE`	Shell execution via subprocess, os.system
`READ/WRITE`	Local filesystem access
`SEND`	Outbound HTTP, sockets
`SECRETS`	Env var credential access
`DYNAMIC`	exec(), eval(), importlib
`AUTONOMY`	Background threads, autonomous loops
Entry points	LLM-callable tool registrations

The key distinction from a linter: reachscan tracks reachability.
A subprocess.run() buried in dead code is a different risk than one
called directly from a registered MCP tool.

Install it:

pipx install reachscan
reachscan https://github.com/any-org/any-mcp-server

False positive rate: 0.47% across a labeled corpus of ~3,900 findings.

The headline finding: blender-mcp

blender-mcp has a registered
MCP tool called execute_blender_code:

@mcp.tool()
def execute_blender_code(ctx: Context, code: str) -> str:
    """Execute arbitrary Python code in Blender."""
    blender = get_blender_connection()
    result = blender.send_command("execute_code", {"code": code})

That string travels over a local TCP socket to the Blender addon:

def execute_code(self, code):
    namespace = {"bpy": bpy}
    exec(code, namespace)  # ← line 431

Why `namespace = {"bpy": bpy}` doesn't protect you

This looks like a sandbox. It isn't.

When you call exec(code, namespace) without setting
namespace["__builtins__"], Python automatically injects the full
builtins module. I verified this:

namespace = {"bpy": object()}

exec('import os; print(os.getcwd())', namespace)
# → /home/user/...  ✓

exec('print(open("/etc/hostname").read())', namespace)
# → my-machine  ✓

exec('import subprocess; r = subprocess.run(["id"], \
capture_output=True, text=True); print(r.stdout)', namespace)
# → uid=1000(user) gid=1000(user)  ✓

The LLM doesn't just get Blender API access. It gets full Python
execution on the host.

The same primitive, done right

Finding exec() in a scan result doesn't automatically mean critical.
Two projects in this dataset handle it correctly:

awslabs/mcp — strips builtins before exec():

namespace['__builtins__'] = _SAFE_BUILTINS
# Removes: __import__, exec, eval, compile, open, getattr...

StarRocks MCP — validates the AST before eval():

validate_plotly_expr(plotly_expr)  # must be exactly px.<method>(...)
fig = eval(plotly_expr, {"px": px}, local_vars)

Same pattern. Very different trust boundary.

Results across 50 repos

1,139 findings across 18 repos with non-zero activity
1 in 3 repos has shell execution capability
1 in 4 accesses credentials from environment variables
10 of 50 had 4+ capabilities active simultaneously

The TypeScript gap

22 repos showed 0 capability findings — many are TypeScript-only.
reachscan detects TypeScript entry points (452 found) but doesn't yet
analyze TypeScript function bodies. "Clean" for a TS-heavy repo means
"no Python capability findings" — not verified safe.

TypeScript capability analysis is next.

Main takeaway

Adoption is moving faster than visibility. Treat MCP servers like
privileged code, not plugins. Audit tool boundaries before you deploy.

Try it on any MCP server or agent repo:

pipx install reachscan
reachscan https://github.com/any-org/any-mcp-server

GitHub: github.com/vinmay/reachscan

Full writeup with all 50 results and methodology:
Medium article

DEV Community: vinmay

ClearToList: Is that thrift find recalled? A photo check against CPSC recalls with Gemma 4

What I Built

Who pays for this

Demo

Code

vinmay / cleartolist

ClearToList

How I Built It

Data

The scan

Picking the reader

The site and not getting wrecked by a public upload form

Security

Evaluation

Why Does Open Innovation Matter?

My First User

Prize Categories

What's next

Doppel: find two different things that look similar, outside

What I Built

Demo

Code

vinmay / doppel

Doppel

How I Built It

How I got here

How it works

On my walk

Does it work?

Why Does Open Innovation Matter?

Prize Categories

Your `pip install` Just Stole Your SSH Keys: The LiteLLM Supply Chain Attack Explained

What Happened

Why the Scale Is Scary

The Real Problem: You Can't See What's Inside Your Dependencies

What You Should Actually Do

Can We Do Better Than Grep?

The Uncomfortable Truth

I built "npm audit" for AI agents

What reachscan does

The problem

How it works (briefly)

Why I built it

What it found across 50 real MCP servers

Try it

I scanned 50 MCP servers to see what they can actually do — here's what I found

The problem

What reachscan does

The headline finding: blender-mcp

Why namespace = {"bpy": bpy} doesn't protect you

The same primitive, done right

Results across 50 repos

The TypeScript gap

Main takeaway

Why `namespace = {"bpy": bpy}` doesn't protect you