Sarvar Nadaf

for AWS Community Builders

Posted on Oct 13 Originally published at builder.aws.com AI-assisted

I built an AI agent that scores any GitHub repo 0-100 before you clone it

#aws #ai #agents #discuss

You see a repo in a reel. Someone says "just use this one for agents." You clone the URL without thinking. In 2026 that reflex got dangerous: attackers planted more than 17,000 fake GitHub repos posing as AI agent and MCP projects, with install steps that steal your credentials. The README is the bait.

So for the AWS Builder Center "Build an Agent" weekend challenge, I built RepoPilot: a working agent you point at any public GitHub repo that tells you, in seconds, whether you should trust it, then helps you run it.

Originally published on AWS Builder Center as my entry for the "Build an Agent" weekend challenge. Resharing it here for the Dev.to community.

Live demo: https://simplynadaf.github.io/repopilot-your-first-run-agent/

simplynadaf / repopilot-your-first-run-agent

RepoPilot: a friendly first-run onboarding agent for any public GitHub repo. Built on AWS Lambda + Amazon Nova Pro (Bedrock) + Strands. Reads the real files, gives one ordered setup path, and turns errors into plain English. #agents

🧭 RepoPilot: Your First-Run Agent (2026)

Point it at any public GitHub repo and it computes a 0-100 RepoPilot Score (is this repo safe to trust and run?), flags dangerous setup steps and critical issues, then hands a brand-new contributor one accurate path from clone to a working first run. Point it at a username instead and it profiles the developer: bio, footprint, and top repos ranked by stars. When a command fails, it explains the error like a patient teammate instead of dumping a stack trace.

🏗️ Built for the AWS Builder Center "Build an Agent" Weekend Challenge · #agents

⭐ If RepoPilot saved you from a painful first run, give it a star. It helps other builders find it.

The Problem • What It Does • Architecture • Getting Started • FAQ

📖 Table of Contents

View on GitHub

What it does

Give it owner/repo and it returns a 0-100 score, computed like a credit score across five weighted dimensions. Each point traces to a real signal in the repo's metadata and files:

Dimension	Weight	What it reads
Safety	30%	Setup steps for malware patterns: `curl \
Trust	25%	Stars-vs-age ratio, fork/contributor signals, the fake-star pattern
Maintenance	20%	Recent commits, open-vs-closed issues, release cadence
Maturity	15%	Age, contributor count, overall footprint
Documentation	10%	README depth, license, examples

Safety scans the actual setup steps for the patterns that hide malware: {% raw %}curl ... | bash, npm postinstall hooks, base64 blobs, raw-IP calls. Any one is a critical issue, and a critical issue caps the whole grade because "it runs fine" means nothing if the installer is designed to steal your keys.
Trust catches the fake-star pattern: a three-week-old repo with 40,000 stars is a red flag, and it says why.

Then it helps you run the thing: one ordered path from clone to running, the exact tool versions you need, and when a command fails it turns the error into a plain-language cause plus one next step.

Type a username instead and it profiles the developer: footprint, main languages, top repos by stars. Vet the person, not just the code.

The honest part: point it at ollama/ollama, a tool millions install, and it drops to a D, because the official install really is curl -fsSL https://ollama.com/install.sh | sh. It does not call ollama malware. It says "this pipes a remote script into your shell, read it first." Even trusted repos have setup steps worth a second look.

How I built it

The stack is deliberately small:

Amazon Bedrock (Amazon Nova Pro) for the reasoning / narration
AWS Lambda (python3.12) for the agent, fronted by Amazon API Gateway so the web UI can call it
Strands Agents SDK for the local CLI
Seven pure-Python tools feed a scoring engine; the model only narrates the facts, it never touches correctness

Two things I would tell anyone building a Bedrock agent on Lambda:

Confirm model access before you write code. One converse call told me Nova Pro was reachable in half a second. That is cheaper than debugging an AccessDenied after you have built everything.
Drop heavy deps. I vendored requests into the Lambda zip and hit a wall: the bundled urllib3 imported something Python 3.12 removed. I rewrote the GitHub fetcher on the standard library. The package went from 400KB to 13KB and the problem vanished.

The danger scanner also needed tuning. My first version flagged Flask as unsafe because its docs mention 127.0.0.1, so I taught the scanner to ignore localhost and private IP ranges while still catching real public-IP installers. Fewer false alarms, more trust a false positive on a trusted framework erodes confidence faster than a missed edge case.

The whole thing is unit-tested without AWS: 25 tests, including 200 randomised repos to prove the weighted score never breaks its own math and a critical flag always caps the grade. Keeping the tools pure and deterministic means the LLM is only responsible for tone, not truth.

The one thing I made delightful

The challenge asked for an agent people actually enjoy using, and offered "a helpful error message" as an example. So I spent my care on the moment a new contributor usually gives up: the first failed command.

When your first install explodes, RepoPilot does not echo the stack trace back at you. It says something like:

"A dependency the code imports has not been installed. Run the install step again in the right environment, and activate your venv first. You are close this is a common first-run snag, not something you broke."

Cause, next step, and a bit of reassurance. That is the difference between a tool that reports problems and one that helps.

I also gave the web UI two modes so the delight survives contact with reality. Demo mode is pre-configured and instant anyone can click around with zero setup and nothing ever hangs. Live mode calls the real deployed agent on AWS, and if a live call ever stalls it falls back to demo data rather than spinning forever. The agent is never allowed to dead-end on you.

Proof it works

Against the real Flask repo, the deployed Lambda fetched 236 files, scored it 98/100 (grade A) with safety, trust, maintenance, and maturity at 100 and documentation at 85, detected the Python stack, and read python >=3.11 straight from the real pyproject.toml. Point it at langchain-ai/langchain and it scores a clean 100. Point it at ollama/ollama and it correctly drops to a D over the real curl | sh install. Type @torvalds and it profiles Linus Torvalds 263,000 stars across his repos. Every number traces back to a tool output, not the model's imagination.

How it will help you

Next time a post tells you to clone something, run it through RepoPilot first. You get a trust read before you type git clone, a plain-language heads-up on any risky install step, and a clean first-run path if the repo checks out. For maintainers, the score is a mirror: it shows exactly what hurts your repo's trust (no license, stale, thin docs) in points you can fix.

It runs on AWS Free Tier, and the code is open source (Apache-2.0).