That amazing skill you shared only runs on your machine. I built a linter for it — then found I was 85% wrong

I mass-produced skills by saying "turn this into a skill." Then I shared them

With Claude Code or Codex, any procedure that worked can become a reusable SKILL.md just by asking "turn what we just did into a skill." I did this for image generation, deploy steps, everything. It was great.

Where it fell apart was handing those skills to teammates. They ran on my machine and silently died on theirs. Every time I dug in, it was the same shape:

• Output path hardcoded to C:\Users\atlan\Downloads\out.png (my home directory)
• The body calls codex exec ... with no instruction anywhere for installing that CLI
• Assumes OPENAI_API_KEY is already set, with no guidance when it isn't
• A model id like gpt-image-2 written inline

All of them are the author's environment baked in. None of them raise an error — they just fail quietly in the next person's hands, which is the worst kind.

So I built carrylint, a linter that fails CI on exactly this.

The standard made the format portable. Whether the contents run is a separate question

In December 2025, Anthropic made Agent Skills an open standard. One SKILL.md now runs across Claude Code, Codex, Gemini CLI, Cursor, Copilot, and 20-odd others. Format portability is solved.

But a standard only guarantees the shape of the container. If the inside holds an absolute path or an undeclared CLI, the container is valid and the contents still don't run for anyone else. I couldn't find a tool looking at that.

• reflint (also mine) asks: do the references exist?
• skills-lint (also mine) asks: do skills collide, is the frontmatter valid?
• carrylint asks: do the references resolve in a different environment, under a different model?

The failure class is inverted. The others check "is this correct as specified." carrylint checks "will the next person install this and have it actually run."

What it catches

Run it against a deliberately non-portable sample:

✗ examples/bad/leaky-image-gen/SKILL.md — 3 errors / 3 warnings
  ✗ :16  [abs-path] machine-specific absolute path `C:\Users\alice\Downloads\out.png`
         — will not resolve on anyone else's machine (use a relative path or {baseDir})
  • :16  [undeclared-cli] calls `codex` but never declares or installs it
  ✗ :22  [abs-path] machine-specific absolute path `C:\Users\alice\Downloads\out.png`
  • :25  [provider-env] assumes `OPENAI_API_KEY` is set
  ✗ :27  [placeholder] unresolved placeholder `<FILL_ME>` left in a shipped file
  • :29  [todo] TODO/FIXME marker left in a shipped file

carrylint: 3 errors / 3 warnings
exit code: 1

Rules are split by severity. False positives are what get a linter uninstalled, so only things the next person will hit, with no room for interpretation, are error (which fails the PR). I moved that line a lot after shipping — see below.

Severity	Rules
error	Author-environment absolute paths (C:\…, /Users/<realname>/); unfinished markers (<FILL_ME>, REPLACE_ME). $HOME, ~, YOUR_API_KEY, /path/to/ are excluded — they're portable or a documentation convention
warn	Undeclared external CLIs (host commands like claude mcp add excluded); raw provider-specific env references; leftover TODO:
opt-in	Hardcoded model ids (claude-*, gpt-*) — deliberate pinning is legitimate, so off by default

No LLM and no API key at runtime — pure static analysis. Since the criterion is "are you locked into one environment or model", it works identically whether the skill was written by Claude or Codex. The tool embodies the thing it checks for.

Then I ran it against 230 real skills and learned I was 85% wrong

This is the part I most want to be honest about. I had shipped it, but something nagged before I promoted it: my own examples and my own tests passing proves nothing. So I collected 230 real SKILL.md / AGENTS.md files from public GitHub repos and ran carrylint against them untouched.

The good half. Skills that only run for their author are genuinely out there. One PPT-generation skill hardcoded /Users/guohao/Documents/... — the author's Mac path — into the body. Another used C:/Users/vudrk/Desktop/AI Projects/ as the base for every script. None of these error out; they die quietly for the next person. carrylint's reason to exist was sitting in the real data.

The bad half. About 85% of my errors were false positives. I was flagging Bearer YOUR_API_KEY (the standard API-docs convention for "put your key here") as unfinished. I was flagging claude mcp add — the host itself — as an undeclared CLI. I was flagging $HOME/..., which resolves per user and is the portable way to write it, as a machine-specific absolute path. I'd written that false positives are a linter's only cause of death, and then nearly shipped promotion on top of an 85% false-positive rate.

Luckily the real data named exactly what to fix. v0.1.1 corrected four things: $HOME/~/generic names are portable; placeholders narrowed to genuine fill-me markers; host CLI setup commands (claude mcp add etc.) excluded; the home-relative path rule dropped entirely. Re-run against the same 230: error false positives went from ~85% to near zero, and every true positive survived. I then ran it against 70 repos it had never seen to check I hadn't overfit (3% fired, all genuine). The real examples I found are bundled in the repo as regression tests.

One lesson: your own tests passing is not evidence of correctness. You find out by running against real data — and by doing it before you promote, not after.

Honestly: this niche is already crowded

One more admission. Partway through designing this I went looking, and there were already 7+ linters for SKILL.md as of 2026. My own skills-lint is one of them. This was not an empty lot.

But I read the ones I could find, and none of them looked at whether the contents actually run somewhere else. They stop at spec compliance and frontmatter. So carrylint does only that. Teams that mix Claude and Codex and distribute skills to each other are honestly still rare — the demand may be slightly ahead of its time. Still, if "I shared it and it only ran for me" has ever stung you, the low-noise error rules (sharpened by that audit) should earn their place.

Summary

The standard made SKILL.md portable as a format. Whether it runs is a different question. carrylint fails CI on that difference.

- uses: hyuga611/carrylint@v0     # in CI

npx @hyuga/carrylint              # right now, locally

Repo: https://github.com/hyuga611/carrylint

Comments

[Reid Marlow]

This is exactly the failure mode that scares me with shared agent skills. The format standard gets people to the starting line, but the portable part is really the dependency contract. Absolute paths, hidden CLIs, and magic env vars are the stuff that makes a reusable skill become a local shell script with nicer packaging.

[hyuga]

"Dependency contract" is a better name for it than anything I came up with — stealing that.

The 230-repo audit gave me a rough ranking of which parts of that contract actually break. Absolute paths were the one unambiguous killer, and they really are out there: /Users/guohao/Documents/... and C:/Users/vudrk/Desktop/AI Projects/ sitting in the body of skills people shared. Nothing errors — it just quietly does nothing for the next person.

Hidden CLIs and magic env vars turned out to be harder, which is why they're warnings and not errors. Statically, codex exec (a real undeclared dependency) and claude mcp add (the host you're already running in) look identical, and Bearer YOUR_API_KEY looks exactly like an unfinished placeholder. I was failing CI on all three until the audit talked me out of it.

The part that can be enforced is the declaration itself: name the CLI in frontmatter (requires:) or install it in the body and the linter goes quiet, leave it implicit and you get told. Your comment made me go re-read my own implementation of that, and it only recognised the single-line form — declare two dependencies as a YAML list, the way anyone would actually write it, and it still warned you. So the people writing the contract properly were the ones getting flagged for it. Fixed on my end, going out in the next patch. Thanks for the nudge.

[Yunetzi]

Love the candor. The real win is a tiny linter that flags non-deterministic steps and missing tests before shipping. Ship small, test hard, learn fast—humility pays.

[hyuga]

Thanks — glad the candid part landed.

Non-determinism is a rule I keep sketching and haven't shipped: I can't find a signal that doesn't fire on every model call, and the audit left me cautious about noisy rules. The testing half I've had more luck with — for a skill it usually turns into a read-back: re-fetch the external state after a write rather than trusting that the step reported success. carrylint only covers the portability side today, so both of those are still open ground.