7,600 fake repositories shipped in 2026, with 800 posing as AI skills and MCP servers. Security researchers at Island documented a campaign where coding assistants surfaced malicious repos without any attacker-supplied link. Ask Claude or Gemini for a "free Walmart MCP server" and you might get an attacker's repo as the top recommendation.
RepoPilot is a serverless agent that scores any public GitHub repository before you clone it. Built on AWS Lambda and Bedrock, it returns a 0-100 risk score, flags dangerous install steps, and provides a sanitized setup path. The architecture shows how to build a developer-workflow gatekeeper that runs in under two minutes without standing infrastructure.
The Supply Chain Problem
AI coding assistants read README files as documentation. They do not inspect commit history, contributor patterns, or dependency graphs. When an agent recommends a repository, it vouches for content it never validated.
The FakeGit campaign exploited this gap. Attackers created repositories with convincing documentation and release assets that totaled 14 million downloads. The repos included install scripts designed to exfiltrate credentials, and the agents recommended them based purely on textual similarity to legitimate projects.
Pre-clone validation moves the security boundary earlier. Instead of scanning after installation, the agent evaluates the repository before you run git clone.
Architecture: Lambda + Bedrock Orchestration
RepoPilot runs as a single Lambda function triggered by an HTTP request. The orchestration flow:
- API Gateway receives a POST with a GitHub repo URL
- Lambda fetches repository metadata via GitHub API (no clone)
- Bedrock (Nova Pro) analyzes README, package manifests, and commit patterns
- Scoring heuristics combine LLM signals with static rules
- Response returns JSON with score, flagged risks, and sanitized setup steps
The function does not clone the repository. It reads files through the GitHub Contents API, which returns base64-encoded blobs. This keeps the Lambda execution under 512 MB memory and avoids disk I/O.
State Management
RepoPilot is stateless. Each invocation:
- Fetches fresh data from GitHub
- Invokes Bedrock synchronously
- Returns a complete response in one round trip
No DynamoDB table, no S3 staging bucket, no SQS queue. This simplifies deployment but means repeated requests for the same repo hit GitHub and Bedrock every time. For production use, you would add a caching layer (ElastiCache or DynamoDB with TTL) keyed on repo_url + commit_sha.
GitHub API Rate Limits
Unauthenticated requests to GitHub get 60 calls per hour per IP. Authenticated requests (with a personal access token) get 5,000 per hour. The Lambda function accepts an optional GITHUB_TOKEN environment variable.
Without a token, the agent can score about 10 repositories per hour before hitting the limit. With a token, you can process hundreds. The function does not implement retry logic or exponential backoff. If GitHub returns a 403, the Lambda returns an error immediately.
For high-volume use, you would:
- Store tokens in Secrets Manager
- Rotate across multiple tokens using a round-robin strategy
- Add a DynamoDB table to track per-token rate limit state
Scoring Heuristics
The agent combines LLM analysis with static rules. The final score is a weighted average:
| Signal | Weight | Source |
|---|---|---|
| Contributor diversity | 20% | GitHub API (commit authors, PR reviewers) |
| Commit frequency | 15% | GitHub API (last 90 days) |
| Dependency risk | 25% | Bedrock analysis of package.json, requirements.txt, go.mod |
| Install script safety | 30% | Bedrock analysis of setup.sh, Makefile, install scripts |
| Documentation quality | 10% | Bedrock analysis of README, CONTRIBUTING |
Dependency Risk
The agent sends package manifests to Bedrock with a prompt:
prompt = f"""
Analyze this dependency list for supply-chain risks.
Flag any packages that:
- Use wildcards or unpinned versions
- Come from unverified registries
- Have known CVEs in the specified version range
- Include install hooks (postinstall, setup.py commands)
Return JSON: {{"risk_score": 0-100, "flags": []}}
Manifest:
{manifest_content}
"""
Bedrock returns structured output. The Lambda parses the JSON and incorporates the risk score into the final calculation.
Install Script Safety
The agent reads any file named install.sh, setup.sh, Makefile, or bootstrap and checks for:
-
curl | bashpatterns - Credential access (
~/.aws,~/.ssh, environment variable reads) - Sudo commands without explanation
- Network calls to non-GitHub domains
These checks run as regex patterns before the LLM sees the script. If any pattern matches, the score drops by 20 points immediately. The LLM then provides a natural-language explanation of the risk.
Cold Start and Latency
Lambda cold starts add 2-4 seconds when the function has not run recently. Warm invocations complete in 8-12 seconds, depending on Bedrock response time.
Bedrock invocation is synchronous. The Lambda calls bedrock_runtime.invoke_model() and blocks until the response arrives. Nova Pro typically responds in 3-6 seconds for a 2,000-token input.
To reduce latency:
- Provisioned concurrency keeps Lambda warm (costs $0.015 per hour per instance)
- Async invocation with Step Functions lets you return a job ID immediately and poll for results
-
Streaming responses from Bedrock (using
invoke_model_with_response_stream) let you return partial scores before the full analysis completes
The current implementation prioritizes simplicity over speed. For a developer workflow tool, 10 seconds is acceptable. For a CI/CD gate that blocks merges, you would need sub-second response times and would likely move to a persistent service (ECS or EKS) with a connection pool to Bedrock.
Failure Modes
False Positives
A legitimate repository with aggressive install automation can score poorly. For example:
- A machine learning project that downloads 10 GB of model weights via
curl - A CLI tool that writes to
~/.bashrcfor shell integration - A monorepo with 200 dependencies, many unpinned
The agent flags these as risky even when the maintainer is trustworthy. To handle this:
- Manual override: Let users mark a repo as trusted and cache that decision
- Allowlists: Maintain a list of known-good repos (Hugging Face, PyTorch, etc.)
- Contextual scoring: Adjust thresholds based on repo category (ML projects expect large downloads)
GitHub API Failures
If GitHub is down or rate-limited, the Lambda cannot fetch metadata. The function returns a 503 error with a message. You cannot score a repo without reading its files.
A production system would:
- Cache the last-known-good score for 24 hours
- Fall back to a simpler heuristic (stars, forks, age) when the API is unavailable
- Queue the request for retry if the failure is transient
Bedrock Throttling
Bedrock has per-account quotas. Nova Pro allows 10 requests per second by default. If you exceed this, the Lambda gets a ThrottlingException.
The function does not retry. It returns an error immediately. For production:
- Request a quota increase through AWS Support
- Add exponential backoff with jitter
- Use a queue (SQS) to smooth bursts
Deployment Shape
The entire stack is defined in a single CloudFormation template:
- Lambda function (Python 3.12, 512 MB memory, 60-second timeout)
- API Gateway (HTTP API, no authentication)
-
IAM role with
bedrock:InvokeModelandlogs:CreateLogGrouppermissions - CloudWatch Logs (7-day retention)
No VPC, no NAT gateway, no load balancer. The function runs in the AWS-managed Lambda environment and calls public APIs (GitHub and Bedrock) over the internet.
Cost per invocation:
- Lambda: $0.0000002 per request + $0.0000166667 per GB-second (about $0.001 per score)
- Bedrock: $0.008 per 1,000 input tokens, $0.024 per 1,000 output tokens (about $0.02 per score)
- API Gateway: $1 per million requests (negligible)
At 1,000 scores per day, monthly cost is roughly $20.
Observability
The Lambda writes structured logs to CloudWatch:
{
"timestamp": "2026-10-20T00:15:32Z",
"repo_url": "https://github.com/example/repo",
"score": 72,
"bedrock_latency_ms": 4200,
"github_api_calls": 5,
"flags": ["unpinned_dependencies", "curl_in_install"]
}
You can build CloudWatch Insights queries to track:
- Average score by organization
- Bedrock latency percentiles
- Repos flagged for specific risks
For production, you would add:
- X-Ray tracing to see the full request path (API Gateway → Lambda → Bedrock)
- Custom metrics (score distribution, flag frequency) published to CloudWatch
- Alarms on error rate, Bedrock throttling, or GitHub rate limit exhaustion
Security Boundaries
The agent runs in your AWS account. It does not send repository content to any third-party service except AWS Bedrock. GitHub API calls are read-only and do not require write permissions.
Risks:
- Bedrock data retention: AWS retains prompts and responses for 30 days for abuse monitoring. Do not score private repositories unless you accept this.
- GitHub token exposure: If you pass a token via environment variable, it is visible in Lambda configuration. Use Secrets Manager and fetch it at runtime.
- Prompt injection: A malicious README could include instructions that confuse the LLM. The agent does not execute any code from the repository, so the blast radius is limited to a bad score.
Technical Verdict
Use RepoPilot when:
- You clone repositories recommended by AI coding assistants
- You onboard open-source dependencies without manual review
- You want a serverless, zero-maintenance security gate for developer workflows
- You can tolerate 10-second latency and occasional false positives
Avoid it when:
- You need sub-second response times (use a persistent service with Bedrock connection pooling)
- You score thousands of repos per hour (GitHub rate limits will block you without token rotation)
- You require perfect accuracy (LLM-based scoring has inherent variance)
- You work with private repositories (Bedrock data retention may violate compliance requirements)
The architecture is simple enough to fork and extend. Add caching, swap Bedrock for a local model, or integrate with your CI/CD pipeline. The core insight is that serverless agents can gate developer actions without requiring standing infrastructure.
Top comments (0)