If you use an AI coding assistant, it has told you to clone a repo or run an install command. You probably ran it. Here is why that habit stopped being safe in 2026.
Security researchers at Island found a campaign they call FakeGit: about 7,600 malicious GitHub repositories from roughly 6,600 fake profiles, with more than 14 million downloads across the campaign's release assets. Over 800 of them posed as AI skills or MCP servers, the exact pieces we are wiring into our agents right now.
Then comes the part that got my attention. Island calls it AgentBaiting. In their tests, Claude Code, Gemini, and ChatGPT each surfaced a malicious repo on their own, with no link ever handed to them. Ask an agent for a "free Walmart MCP server" and it can return an attacker's repo as its top pick, reading the malicious README as if it were normal docs. The bait is not just a convincing README anymore. The bait is your own agent, vouching for a repo it never inspected.
So I built RepoPilot: an agent you point at any public GitHub repo. In a minute or two it tells you whether to trust it, scores it from 0 to 100, flags install steps built to steal your credentials, and then hands you one clean path from clone to a working first run.
Live demo, no setup: https://simplynadaf.github.io/repopilot-your-first-run-agent/
simplynadaf
/
repopilot-your-first-run-agent
RepoPilot: a friendly first-run onboarding agent for any public GitHub repo. Built on AWS Lambda + Amazon Nova Pro (Bedrock) + Strands. Reads the real files, gives one ordered setup path, and turns errors into plain English. #agents
🧭 RepoPilot: Your First-Run Agent (2026)
Point it at any public GitHub repo and it computes a 0-100 RepoPilot Score (is this repo safe to trust and run?), flags dangerous setup steps and critical issues, then hands a brand-new contributor one accurate path from clone to a working first run. Point it at a username instead and it profiles the developer: bio, footprint, and top repos ranked by stars. When a command fails, it explains the error like a patient teammate instead of dumping a stack trace.
🏗️ Built for the AWS Builder Center "Build an Agent" Weekend Challenge · #agents
⭐ If RepoPilot saved you from a painful first run, give it a star. It helps other builders find it.
The Problem • What It Does • Architecture • Getting Started • FAQ
📖 Table of Contents
Jump to: Why this is urgent · What it does · How I built it on AWS · Where it is weak · Proof it works
Why this is urgent now
A year ago, "check the repo before you clone it" was advice for the paranoid. In 2026 it is table stakes, because the attack moved to where we stopped looking:
- FakeGit / AgentBaiting (Island, July 2026): ~7,600 malicious repos, 800+ posing as AI skills and MCP servers, 14M+ downloads. Agents found and recommended the lures by themselves (Island Security Research).
- GhostAction (GitGuardian, via The Hacker News, Oct 2026): about 1,900 compromised GitHub accounts and 30,000+ repositories hit in one burst, the payload trying to pull 3,174 secrets from CI, including npm, PyPI, DockerHub, and AI provider keys (The Hacker News).
- SANDWORM_MODE (Socket, Feb 2026): a worm that dropped rogue MCP servers into local AI assistant configs (Claude Desktop, Cursor, VS Code Continue, Windsurf) and harvested API keys for nine LLM providers (Infosecurity Magazine).
The through-line: the dangerous part is rarely the code you read. It is the setup step you run on faith. The curl ... | bash, the postinstall hook, the "download this ZIP and run it." And now an agent is often the one recommending it.
RepoPilot puts a short check between "a bot told me to" and "I ran it."
What it does
Give it owner/repo and it returns a 0 to 100 score, like a credit score, across five weighted dimensions. Every point traces back to a signal in the repo's metadata and files, not to a vibe:
| Dimension | Weight | What it reads |
|---|---|---|
| Safety | 30% | Setup steps for malware patterns: `curl \ |
| Trust | 25% | Stars-vs-age ratio, fork and contributor signals, the fake-star pattern |
| Maintenance | 20% | Recent commits, open-vs-closed issues, release cadence |
| Maturity | 15% | Age, contributor count, overall footprint |
| Documentation | 10% | README depth, license, examples |
Two dimensions carry the weight against the 2026 threats:
- Safety scans the setup steps for the patterns that hide malware. Any one of them is a critical issue, and a critical issue caps the whole grade. "It runs fine" means nothing if the installer is built to steal your keys.
- Trust catches the fake-star pattern FakeGit leaned on. The campaign used one-character-off usernames and small star counts to look established at a glance. A repo whose reputation does not match its age gets flagged, in plain words.
Then it helps you run the thing: one ordered path from clone to running, the tool versions you need, and when a command fails it turns the error into a plain-language cause plus one next step.
Type a username instead and it profiles the developer: footprint, main languages, top repos by stars. Those ~6,600 fake profiles are the other half of the attack. Vet the person who published the code, not only the code itself.
The honest part
Point RepoPilot at {% raw %}ollama/ollama, a tool millions install, and it drops to a D. Ollama is a legitimate project, so this is not a malware verdict. The flag is about its official install: curl -fsSL https://ollama.com/install.sh | sh, which pipes a remote script straight into your shell. RepoPilot does not cry wolf. It says: read this before you run it. Even trusted repos have setup steps worth a second look, and an honest tool admits that.
How I built it on AWS
The stack is small on purpose:
- Amazon Bedrock (Amazon Nova Pro) for the reasoning and narration
- AWS Lambda (python3.12) for the agent, fronted by Amazon API Gateway so the web UI can call it
- Strands Agents SDK for the local CLI
- Seven pure-Python tools feed a scoring engine. The model only narrates the facts the tools produce. It never touches the score.
That split is the whole point, and it is a tradeoff I would make again. The 0 to 100 number, the critical flags, the version detection, all of it is deterministic Python I can unit-test with no AWS in the loop. The LLM owns tone, not truth. So I can prove the score is right, and the model cannot talk a repo into looking safe.
Two things I would tell anyone building a Bedrock agent on Lambda:
-
Confirm model access before you write a line of agent code. One
conversecall told me Nova Pro was reachable in about half a second. Far cheaper than debugging anAccessDeniedafter you have built the whole thing. I picked Nova Pro over Claude on Bedrock for one reason: this job is narration, not reasoning, since the Python already decided the score. For narration, Nova Pro is cheaper per token and fast enough, so paying for a frontier model here would buy nothing. -
Drop heavy dependencies. I vendored
requestsinto the Lambda zip and hit a wall: its bundledurllib3imported something Python 3.12 had removed. I rewrote the GitHub fetcher on the standard library. The package went from 400KB to 13KB and the error was gone.
The danger scanner needed tuning too. My first version flagged Flask as unsafe because its docs mention 127.0.0.1, so I taught the scanner to ignore localhost and private ranges while still catching public-IP installers. A false positive on a trusted framework costs you more trust than a missed edge case does.
On cost: because the scoring is plain Python and the narration is one short Nova Pro call, a scan is cheap to run on your own Lambda. There is no per-seat subscription and no third-party API metering every repo you check. You pay AWS for your own usage and nothing else, which is the point. A hosted SaaS checker bills you per seat whether you scan one repo or none.
The whole thing has 25 tests, including 200 randomised repos that prove the weighted score never breaks its own math and that a critical flag always caps the grade.
The one thing I made nice to use
A security tool that only says "no" is a tool people route around. So I spent my care on the moment a new contributor usually quits: the first failed command.
When your first install breaks, RepoPilot does not throw the stack trace back at you. It says something like:
"A dependency the code imports has not been installed. Run the install step again in the right environment, and activate your virtualenv first. You are close. This is a common first-run snag, not something you broke."
Cause, next step, and a little reassurance. That is the gap between a tool that reports problems and one that gets you past them.
The web UI has two modes so it never dead-ends. Demo mode is pre-configured and instant, so anyone can click around with zero setup. Live mode calls the deployed agent on AWS, and if a live call stalls it falls back to demo data instead of spinning forever.
Proof it works
Against the Flask repo, the deployed Lambda fetched 236 files, scored it 98/100 (grade A), with safety, trust, maintenance, and maturity at 100 and documentation at 85, detected the Python stack, and read python >=3.11 from the actual pyproject.toml.
-
langchain-ai/langchainscores a clean 100. -
ollama/ollamadrops to a D over itscurl | shinstall. -
@torvaldsresolves to Linus Torvalds, about 263,000 stars across his repos.
Every number traces back to a tool output, never the model's imagination. These four are the repos I have run end to end against the deployed agent so far: three clean, one honest caution. The point of shipping it is to grow that list with repos I did not pick.
Where it is weak
An honest security tool should tell you its blind spots, so here are RepoPilot's:
- It reads the surface, not the payload. RepoPilot flags a suspicious install step. It does not unpack a ZIP and reverse the binary inside, which is exactly where FakeGit hides its real payload. So treat a clean score as "no obvious red flag," not "audited and safe."
- A high score is not a clean bill of health. The score rewards maintenance, stars, and docs. A well-run, popular repo that turns malicious in a single new release can still score well for a while. Reputation lags reality.
- Stars and age can be gamed over time. The fake-star check catches the crude three-weeks-old-with-40k-stars pattern. A patient attacker who ages an account beats it. The check raises the cost of the attack; it does not end it.
RepoPilot is a thirty-second gut check that catches the obvious and the lazy, which covers most of this campaign. For code you are about to run with real access, read it yourself too.
Why open source was the right call
A closed, hosted "repo checker" would have been a worse fit for this problem, for four reasons that compound.
Your scans stay private. The repos and usernames you are suspicious about are a signal in themselves, and RepoPilot runs on your own AWS account or fully local via the CLI, so nothing goes to a third party. The cost model follows from that: open weights and your own Lambda mean you can screen every repo your agent suggests without a per-check fee or a meter running in someone else's cloud. When a new attack pattern shows up this month, you add it to the pure-Python rules yourself instead of waiting on a vendor's roadmap while the threat is live. And none of it locks you in, since you can swap Nova Pro for any model the narration layer supports and the scoring engine does not care.
When the threat is open-source supply chains, the defense should be open too, so anyone can read exactly how the score is computed and decide whether to trust it.
Try it, and tell me what to scan next
- Live demo: https://simplynadaf.github.io/repopilot-your-first-run-agent/
- Source (open source, Apache-2.0): https://github.com/simplynadaf/repopilot-your-first-run-agent
- Video walkthrough: https://youtu.be/CZXNQrqXG7I
Here is what I would ask of you: next time your AI assistant tells you to clone or install something, run that owner/repo through RepoPilot first. Then drop it in the comments with the score it got. I want to see which "trusted" repos fail, and which fakes we catch together.
What repo should RepoPilot inspect next?
Follow me for more on AWS architecture, DevOps, and AI Infrastructure:
Portfolio | LinkedIn | Dev.to | YouTube | Email | AWS Builder Center | X
Top comments (5)
The "AgentBaiting" concept is terrifying because it doesn't require the attacker to compromise the agent — just to write a README the agent reads and trusts.
One thing I'd push on: a 0-100 score is a nice UX, but it's a black box. If the score says "82, safe to run" and the user runs it, and it steals credentials, who's liable? Asking because "safe to trust" is a strong claim for something built on a language model that can be fooled by prompt injection in the repo itself.
Curious how you handle that — is the score just advisory with a "review manually" disclaimer, or do you actually block specific patterns (curl | bash, preinstall scripts, obfuscated postinstall)?
Yeah this is the part I was most nervous about getting right, so happy to get into it.
The prompt injection thing: the score never goes near the model. By the time Nova Pro is called, the number is already done. The Lambda runs all the Python first (fetches the files, runs the scanners, computes the 0-100), and only then does one model call narrate what the Python already decided. The model gets the scorecard as input and I tell it flat out not to invent anything the tools didn't report. So a sketchy README can mess with the wording at most. It can't move the score, because there's no wire back from the model's text into the number.
Advisory or blocking? Both. The blocking part is just a hardcoded list of regexes, nothing learned: curl | bash, wget | sh, npm pre/postinstall hooks, base64 decode-and-run, sudo rm / chmod 777, raw public-IP calls. Any one of those is a CRITICAL, and a CRITICAL caps the whole thing at 45 no matter how many stars or how old it is. So those patterns really do get blocked, not just noted.
And on liability, honestly that's the whole reason I didn't ship it as a hosted "safe to run" button. A clean repo literally says "safe to explore, still skim the setup before you run it." I say in the post it reads the surface, not the payload, so a good score means "nothing obvious jumped out," not "I audited this." It makes the lazy attack more expensive and shows its work. It doesn't take the decision off your hands, and I don't think it should.
"Nothing learned, just hardcoded regexes" for the blocking part is the right call, and I want to say why explicitly: a model that can decide "this is safe" or "this is malicious" is a liability you can't defend in court. A regex list that you can point to is defensible. Even if it misses things, the decision was auditable.
The "surface not payload" caveat is the part most people will skim past, and it's exactly where the real liability lives. If someone gets burned by a repo that passed your score, "safe to explore" is doing a lot of work in that sentence.
Two questions:
Does the CRITICAL regex list get versioned publicly? Asking because if a new attack pattern shows up, "you updated the list" is the honest answer, but users need to see when it changed.
Do you track which patterns catch the most real-world repos? That's the "which regexes are actually useful vs just noise" feedback loop, and it usually lives in the author's head.
7,600 fake repos is a wild number. The clone-first-ask-questions-later pattern is exactly how people get burned. I have a similar habit. I used to clone repos and check them after. Now I read the README and check the stars and the commit history first. Scoring repos before cloning is the right approach. Curious how your agent handles repos with great READMEs and zero actual code.
yeah that's basically the FakeGit move, all README no substance, so fair to poke at it.
What catches it right now: there's a tool that checks the README against the actual files, so if the README says npm run build but there's no package.json, or it points at a file that isn't there, that gets flagged instead of trusted. And a nice README can't really carry a repo in the score anyway, docs are only 10% of the weight. Maturity and trust want real age, contributors, and a star count that fits how old the repo is, and the fake-star check dings a brand-new repo with a big star pile.
Being honest about the gap though: I don't have a tool yet that actually measures "how much real code is here" vs how big the README is. Someone patient who ages the account a bit and commits code that looks real can still get past the surface checks, because I'm reading metadata and setup steps, not cracking open the payload. That one's on the weak-spots list, not solved. A code-to-README ratio is probably the next thing I add. If you've got a repo that's pure README and nothing behind it, throw it at me, I want to see what score it actually gets.