DEV Community

Cover image for NVIDIA SkillSpector: Should You Scan Your AI Agent Skills Before Installing Them?
ArshTechPro
ArshTechPro

Posted on

NVIDIA SkillSpector: Should You Scan Your AI Agent Skills Before Installing Them?

#ai #agents #agentskills #programming

If you have been using Claude Code, Codex CLI, Gemini CLI, or any agent framework that supports "skills," you have probably installed a few skills from a marketplace or a random GitHub repo without reading every line of code inside them. Most people do. The skill promises to help with PDF generation, data analysis, or some other task, you drop it into your project, and you move on.

NVIDIA's new open source tool, SkillSpector, exists because that habit is riskier than it looks. This article walks through what SkillSpector does, how to set it up, and whether it is worth adding to your workflow.

What SkillSpector actually does

SkillSpector is a security scanner purpose-built for AI agent skills rather than general source code. It runs a two-stage pipeline:

  1. Static analysis — fast, regex and AST-based pattern matching that looks for dangerous code patterns (exec, eval, subprocess, obfuscated payloads), taint flows from sensitive sources to network or execution sinks, YARA signature matches for known malware/webshell/cryptominer patterns, and dependency checks against the OSV.dev vulnerability database.
  2. Optional LLM semantic analysis — a second pass where a language model evaluates context and intent, filters out false positives from the static stage, and explains findings in plain language. The prompt used for this step includes anti-jailbreak protections so a malicious skill cannot talk its way out of being flagged.

The static stage alone covers a wide net (prompt injection, data exfiltration, privilege escalation, supply chain issues, excessive agency, output handling, system prompt leakage, memory poisoning, tool misuse, rogue-agent behavior, trigger abuse, dangerous code execution, taint tracking, and MCP-specific issues like tool poisoning and least-privilege violations). Adding the LLM pass is what pushes precision up meaningfully, since static pattern matching alone tends to over-flag.

Every scan ends with a risk score from 0–100 and a severity label, so instead of reading a wall of findings you get a clear signal: safe, use caution, or do not install.

Setting it up

You do not need an NVIDIA account or any paid API to get value out of this. Here is the fastest path.

1. Clone and install 

git clone https://github.com/NVIDIA/skillspector.git
cd skillspector

# Create and activate a virtual environment
uv venv .venv && source .venv/bin/activate
# or, without uv:
# python3 -m venv .venv && source .venv/bin/activate

# Install
make install
# or, if you want to contribute / run tests:
make install-dev
Enter fullscreen mode Exit fullscreen mode

The project targets Python 3.12+ and is Apache 2.0 licensed, so there is no licensing friction for commercial use.

2. Run your first scan 

# A local skill folder
skillspector scan ./my-skill/

# A single SKILL.md file
skillspector scan ./SKILL.md

# A skill hosted on GitHub
skillspector scan https://github.com/some-user/some-skill

# A zipped skill package
skillspector scan ./my-skill.zip
Enter fullscreen mode Exit fullscreen mode

That is the whole entry point. No config file is required for a basic static scan.

3. Prefer Docker? Skip Python entirely

If you would rather not set up a Python environment, the repo ships a Dockerfile based on the official python:3.12-slim-bookworm image:

docker run --rm -v "$PWD:/scan" skillspector scan ./my-skill/ --no-llm
Enter fullscreen mode Exit fullscreen mode

4. Turn on LLM-powered analysis (optional but recommended)

Static analysis alone is fast but can be noisy. Adding an LLM pass improves accuracy and gives you readable explanations for each finding. SkillSpector supports three providers out of the box, plus anything OpenAI-compatible (including local models via Ollama or vLLM):

Provider Env var for the key Where it runs
OpenAI OPENAI_API_KEY api.openai.com or any compatible endpoint
Anthropic ANTHROPIC_API_KEY api.anthropic.com
NVIDIA build.nvidia.com NVIDIA_INFERENCE_KEY build.nvidia.com

Example with Anthropic:

export SKILLSPECTOR_PROVIDER=anthropic
export ANTHROPIC_API_KEY=sk-ant-...
skillspector scan ./my-skill/
Enter fullscreen mode Exit fullscreen mode

Or point it at a local model with no API key at all:

export SKILLSPECTOR_PROVIDER=openai
export OPENAI_API_KEY=ollama
export OPENAI_BASE_URL=http://localhost:11434/v1
export SKILLSPECTOR_MODEL=llama3.1:8b
skillspector scan ./my-skill/
Enter fullscreen mode Exit fullscreen mode

If you just want the fast static pass without any model calls, add --no-llm to any command.

5. Pick an output format that fits your workflow 

skillspector scan ./my-skill/ --format json --output report.json       # automation
skillspector scan ./my-skill/ --format markdown --output report.md     # review docs
skillspector scan ./my-skill/ --format sarif --output report.sarif     # CI/CD and IDE tooling
Enter fullscreen mode Exit fullscreen mode

The SARIF output is worth calling out specifically: it plugs straight into GitHub code scanning, VS Code, and most CI pipelines that already understand SARIF from other security tools, which makes it realistic to wire this into a pull request check rather than running it manually every time.

6. Use it from Python directly

If you want to embed scanning inside your own tooling rather than shelling out to the CLI, the workflow is exposed as a LangGraph graph:

from skillspector import graph

result = graph.invoke({
    "input_path": "/path/to/skill",
    "output_format": "json",
    "use_llm": True,
})

print(f"Risk Score: {result['risk_score']}/100")
print(f"Severity: {result['risk_severity']}")
Enter fullscreen mode Exit fullscreen mode

Reading the results

Scores map to four bands:

  • 0–20 (LOW) — Safe
  • 21–50 (MEDIUM) — Caution
  • 51–80 (HIGH) — Do not install
  • 81–100 (CRITICAL) — Do not install

Each finding in the report points to the exact file and line, names the pattern that triggered it, and (when LLM analysis is enabled) explains why it matters in a sentence or two. That last part is what makes the tool usable by people who are not security specialists — you do not need to know what a taint-flow chain is to understand "this code reads your environment variables and sends them to an external server."

Where it falls short

The project is upfront about its limitations, and they are worth knowing before you rely on it as your only line of defense:

  • It is static analysis at its core, so it does not execute the skill to observe runtime behavior.
  • Non-English content can slip past pattern matching.
  • Anything hidden inside an image cannot be inspected.
  • Encrypted or compiled code is opaque to the scanner.
  • The live CVE lookup (via OSV.dev) needs outbound network access; offline environments fall back to a much smaller built-in list.

None of this is unusual for a static scanner, but it means SkillSpector is a strong filter, not a guarantee.

What others are saying

This tool has been getting attention quickly since release. Developer Jacob Bennett, writing on his blog, described the gap NVIDIA addressed as a significant security blind spot for agent skills, and suggested the scanner is a good candidate to wire into CI for organizations that share skills internally. That lines up with how the tool is actually designed to be used: not as a one-time check, but as a recurring gate before a skill gets trusted.

Is it worth a try?

For a few specific situations, yes, clearly:

  • You install skills from public marketplaces or random repos and have never audited what is inside them.
  • Your team shares internal skills and you want a lightweight gate before something gets merged or distributed.
  • You already use SARIF-based scanning in CI and want this to slot in alongside your existing security tooling.
  • You want a quick second opinion before running a skill that asks for broad tool access or touches credentials.

The setup cost is low. A static scan needs nothing beyond a Python virtual environment, runs in seconds, and requires no API keys. Adding the LLM pass takes one extra environment variable and a key for whichever provider you already use, including a fully local option through Ollama if you would rather not send any code to an external API. The license is permissive, the CLI is simple enough to run once and forget, and the output formats mean it fits into an existing pipeline instead of becoming a new manual chore.

The honest caveat is that this is a young project (the GitHub repository is only a few weeks old at the time of writing), so expect the pattern set and accuracy to keep evolving. It is also not a replacement for actually reading a skill's code if it is going to run with elevated privileges. But as a first-pass filter that takes a few minutes to set up and catches a meaningful share of real issues, it is a reasonable addition to any workflow where you are installing code you did not write and trusting it with system access.

Quick reference 

git clone https://github.com/NVIDIA/skillspector.git
cd skillspector
uv venv .venv && source .venv/bin/activate
make install

skillspector scan ./my-skill/ --no-llm        # fast static check
skillspector patterns                          # list all 64 detection patterns
Enter fullscreen mode Exit fullscreen mode

If you try it on a skill you already have installed, it might be worth checking what comes back before you run that skill again.

Top comments (0)