DEV Community: Joe Slade

Giving AI Agents a Verdict on Repo Health—Actor #4 in My Apify Portfolio

Joe Slade — Fri, 12 Jun 2026 00:38:09 +0000

Your AI agent will recommend a library that hasn't shipped a commit in over a year—and never flinch. It can't tell a thriving project from a dying one, so it treats a vibrant repo and an abandoned one as equally safe to build on. That's how stale dependencies sneak into production: not through carelessness, but because nothing in the loop ever asked is this thing still maintained?

Developers ask it constantly—they just do it by hand, badly. One pattern I kept seeing while researching this: people lean on a single signal and know it's a bad one. As one developer put it on r/selfhosted: "if a github repo hasn't been updated in the past 2 years I tend to consider it abandoned… but there are some projects older than this that I'd still happily use." Last-commit date is a blunt instrument. So are stars. So is open-issue count—"maybe the maintainers don't think they're as important; the issue tracker will often be swamped." Everyone has a heuristic; nobody has a verdict.

So I built GitHub Repo Intelligence MCP—the fourth actor in my Apify portfolio. Point it at any owner/name or GitHub URL and it returns one of four verdicts — Actively maintained, Slowing, At-risk, or Likely abandoned—backed by the metrics and the pinned thresholds that produced it. It runs as an MCP server, so an AI agent can call it mid-task instead of guessing.

Here's how it's built—and the design decision the whole thing rests on.

The Core Design Decision: The Verdict Is the Product

Most GitHub tooling for agents hands the model a pile of API data and leaves the conclusion to guesswork. The official GitHub MCP server will happily fetch you commits, issues, and PRs—but "here are 300 issues" is not an answer to "should I depend on this?" The model still has to invent a judgment, with no consistent rubric, every single time.

I inverted that. The actor's whole job is to do the synthesis and return the conclusion, with the evidence attached:

get_repo_health          — headline verdict + per-dimension breakdown + rationale
get_activity_metrics     — commit cadence, releases, momentum trend
get_issue_health         — maintainer response time + stale-backlog ratio
get_pr_health            — merge rate + oldest open PR
get_contributor_insights — contributor count + bus-factor flag

Four dimensions—activity, issues, PRs, contributors — collapse into one verdict. Crucially, the verdict isn't a black-box score. Every response ships the metrics and the pinned thresholds behind it, so a human (or an agent) can audit why, not just read a number. Judgment, not raw access.

And it's deterministic: the same repo state and the same config always produce the same verdict. No per-call heuristic drift, no "the model felt differently this time." That property matters more for an agent tool than it sounds—it means the verdict is reproducible, testable, and safe to build a workflow on.

The Architecture: One Fetch, Five Tools

The naive way to build five tools is five sets of API calls. That's slow, and against GitHub's rate limits it's actively hostile.

Instead, a single GitHub GraphQL query pulls everything the five tools need in one round trip. The result is cached and shared, so drilling from the headline verdict (get_repo_health) down into a dimension (get_issue_health) costs zero extra upstream calls—it's reading from the same cached fetch. GraphQL is what makes that possible: one precisely-shaped request instead of a dozen REST round-trips for commits, issues, PRs, releases, and contributors.

The verdict logic itself is pure: metrics in, thresholds applied, verdict out. That makes it trivially testable without touching the network—the same discipline I've carried through every actor in this portfolio.

The Gotcha: Bring a GitHub Token (Here's Why)

The failure most people will hit first isn't in my code—it's the thing that makes the whole tool fall over if you skip it: GraphQL rejects unauthenticated requests outright. GitHub's REST API gives anonymous callers a tiny quota; the GraphQL API gives them nothing. No token, no data.

It gets worse on Apify specifically. Actors run from shared cloud egress, so if anonymous requests were allowed, every Apify user would be drawing from the same pooled anonymous IP quota—which burns to zero almost instantly. The symptom an unprepared user hits is a wave of rate-limit errors that seem to come out of nowhere and have nothing to do with their own usage.

So the input schema makes a GitHub token a first-class, walked-through step rather than an afterthought buried in a README. A fine-grained token with public-repo read access is enough—sixty seconds in GitHub settings—and it lifts you to 5,000 requests/hour that are yours, not shared. The lesson I keep relearning across this portfolio: the scariest setup failures are the ones that look like the tool is broken when really it's the environment you didn't prime.

A Real Run

I pointed it at two repos that make the contrast obvious—one thriving, one famously frozen—against the live endpoint.

vercel/next.js:

{
  "repo": "vercel/next.js",
  "verdict": "Actively maintained",
  "activity": "Healthy",
  "issues": "Healthy",
  "pullRequests": "Moderate",
  "contributors": "Healthy",
  "rationale": "Overall: activity is Healthy; weakest dimension is pull requests (Moderate)."
}

moment/moment:

{
  "repo": "moment/moment",
  "verdict": "Likely abandoned",
  "activity": "Likely abandoned",
  "issues": "At-Risk",
  "pullRequests": "At-Risk",
  "contributors": "Healthy",
  "rationale": "Overall: activity is Likely abandoned; weakest dimension is issues (At-Risk)."
}

Two repos, two unambiguous verdicts—and the rationale tells you which dimension drove each one. Then drill into the activity behind moment's verdict, no extra fetch:

{
  "repo": "moment/moment",
  "commits90d": 0,
  "commits365d": 0,
  "releaseCount365d": 0,
  "momentumTrend": "steady"
}

Zero commits in a year. Zero releases. Steadily zero.

Now—moment is the honest, interesting case, and it's worth not glossing over. Moment isn't broken. Its maintainers deliberately declared it legacy years ago and pointed people to lighter alternatives. It's still downloaded millions of times a week, which is exactly why contributors reads Healthy—a deep historical base. A single-signal tool that only checked "is it popular?" would wave it through. A single-signal tool that only checked "recent commits?" would call it dead without nuance.

This is the whole point of a transparent, multi-dimensional verdict: it flags moment as Likely abandoned on the signals that matter for new adoption (no active development), while the per-dimension breakdown and rationale let you—or your agent—apply judgment. The verdict says "don't start something new on this." That's the right call. And you can see exactly why it said it.

It Runs as a Standby MCP Server

The actor is a **standby actor—a persistent, always-warm HTTP server exposing an MCP endpoint over Streamable HTTP. Practically, that means it drops straight into the MCP configurator flow: add it to your agent, and get_repo_health is a tool the model can reach for the moment it's about to recommend a dependency. No cold-start per query.

Because it's always warm, you can also schedule it—re-check the repos your project depends on and get flagged when one of them starts slowing, before "slowing" becomes "abandoned."

What's Next

The verdict logic today is threshold-based: it reads quantifiable signals—cadence, response times, merge rates, contributor depth—and maps them to a verdict with pinned, transparent cutoffs. That's fast, deterministic, and free to run, and it's the right default.

The honest frontier is the gap between signals and context. A repo can be quiet because it's dying—or because it's genuinely finished, like moment. A maintainer can be slow because they've checked out, or because they're mid-rewrite on a branch. Threshold logic gets you a defensible verdict; a contextual layer (an optional semantic pass over release notes, pinned issues, and maintainer statements) is where the next iteration earns its cost.

This actor also pairs with the rest of the portfolio—same philosophy every time: focused, testable, composable. Small actors you wire into a pipeline, each doing one thing with a clear opinion, instead of one monolith that does everything adequately and decides nothing.

Try It

Live on the Apify Store:
👉 apify.com/joeslade/github-repo-intelligence-mcp

The input schema walks you through the GitHub token and the five tools. If you wire it into an agent workflow—or you've got a repo whose verdict surprises you—drop a comment. I'd like to hear which dimension called it.

How I Built a SERP Topic Gap Analyzer on Apify (And What the Prefill Bug Taught Me)

Joe Slade — Wed, 06 May 2026 02:18:11 +0000

Most content gap tools work backwards. They start with keyword data and try to reverse-engineer what your competitors rank for. I wanted to start from the other direction: given a set of real SERP results, which topics are your competitors covering that you're not?

That question led to the SERP Topic Gap Monitor — a deterministic, composable Apify actor that takes pre-fetched SERP data as input, runs a topic extraction and gap-scoring pipeline, and returns a ranked list of coverage gaps your site is missing.

Here's how it's built and what I learned along the way.

The Core Design Decision: Accept Data, Don't Fetch It

The first architectural choice was also the most important: the actor doesn't scrape Google. It accepts pre-fetched SERP results as structured input.

This sounds like a limitation. It's not.

Fetching Google reliably is a whole problem on its own — proxies, rate limits, bot detection, constantly shifting HTML structure. Bundling that complexity into a content analysis tool creates a fragile system where the analysis breaks every time Google changes its layout.

By accepting SERP data as input, the actor is decoupled from any specific data source. You can feed it results from Google Search API, SerpAPI, Apify's own SERP scrapers, or a fixture file you built by hand. The analysis stays stable regardless of where the data comes from.

The input schema is straightforward:

{
  "targetDomain": "yourblog.com",
  "serpResults": [
    {
      "keyword": "best nootropics",
      "results": [
        {
          "url": "https://healthline.com/...",
          "title": "...",
          "snippet": "...",
          "pageContent": "..."
        }
      ]
    }
  ]
}

pageContent is optional — when it's present, the topic extraction is richer. When it's not, the actor falls back to title and snippet.

The Pipeline: Four Pure Functions

The analysis runs as a linear pipeline of pure functions:

tokenize → topics → gaps → score

tokenize — takes raw text (title, snippet, page content), strips HTML, lowercases, removes punctuation, filters a bundled English stopword list (~170 words, NLTK/Snowball-based), and returns a token array.

topics — aggregates tokens across a result set into a frequency map. Higher frequency = more likely to be a genuine topic signal vs. noise.

gaps — compares your domain's topic set against the competitor topic set. Any topic present in competitor results but absent from your pages is a gap.

score — assigns a gapScore to each gap:

gapScore = uniqueCompetitorPages / totalUniqueCompetitorPages

A score of 1.0 means every competitor page in the result set covers this topic. Your site covers none of them. That's where you start writing.

The formula is simple on purpose. No black box, no magic weighting. If you understand the ratio, you understand the output.

Test-First, All the Way Down

Because all the logic is pure functions, the entire pipeline is fully testable without network access. I wrote 54 Vitest specs across 5 files before writing a line of implementation.

The test suite covers:

Tokenization edge cases (empty strings, HTML entities, mixed case, punctuation boundaries)
Stopword filtering (ensuring common words don't inflate topic scores)
Gap detection (correct identification of covered vs. uncovered topics)
Scoring math (boundary conditions at 0 and 1.0)
End-to-end pipeline integration with fixture SERP data

Running npm test gives you full confidence the analysis is correct before you push anything to the platform. This matters more than you'd think — debugging a scoring error in a live Apify run is significantly slower than catching it in a local test.

The Bug That Took Me Three QA Failures to Find

After publishing, I got a notice from Apify: the actor had been flagged "Under Maintenance" after three consecutive automated QA failures.

The error wasn't in the analysis code. It was in input_schema.json.

Apify's automated QA system runs each actor using the prefill values from the input schema as test input. If required fields don't have prefill values, the QA run receives undefined for those fields — which causes the actor to fail immediately.

My targetDomain and serpResults fields had no prefill. So every QA run hit this:

Actor.fail("targetDomain is required")

The fix took five minutes. Adding a prefill to both fields:

"targetDomain": {
  "type": "string",
  "prefill": "myblog.com"
},
"serpResults": {
  "type": "array",
  "prefill": [ /* 2-keyword fixture array */ ]
}

Build 0.1.4 deployed, QA passed, maintenance flag removed.

The lesson: Always set prefill for every required field in your Apify input_schema.json. The QA system can't test what it can't input. I've carried this forward to every actor since.

A Real Run

I ran it against one of my own sites — peakhealthprovisions.com, a wellness/nootropics site — using two keywords and five competitors (Healthline, Examine, WebMD, MedicalNewsToday, NootropicsExpert).

Results: 20 gaps, 0 topics covered.

nootropic   → gapScore: 1.0  (5/5 competitors)
nootropics  → gapScore: 1.0  (5/5 competitors)
cognitive   → gapScore: 0.8  (8 unique pages)
memory      → gapScore: 0.7  (7 unique pages)
focus       → gapScore: 0.6  (6 unique pages)

Every competitor is covering these topics. The site isn't covering any of them. That's not a content calendar suggestion — that's the content calendar.

What's Next

The actor's design creates an explicit upstream dependency: you need SERP data to feed it. One logical next step is a dedicated SERP actor with a clean JSON output schema designed to pipe directly into the Gap Monitor — one actor fetches, one actor analyzes, composable by design.

There's also a "Works well with" section in the README that cross-references the Changelog Triage Agent — another actor in the portfolio for teams who want to monitor API changelogs for breaking changes. Different use case, same philosophy: focused, testable, composable.

Try It

The actor is free to use on the Apify Store:
👉 apify.com/joeslade/serp-topic-gap-monitor

The README has a full walkthrough of the input format, output schema, and how to source SERP data to feed it. If you're building it into a content pipeline or have questions about the architecture, drop a comment — happy to dig into it.

Tags: apify seo typescript webdev devtools

What's Next for PolicyPilotPro?

Joe Slade — Sat, 05 Jul 2025 23:01:07 +0000

This is a submission for the World's Largest Hackathon Writing Challenge: After the Hack.

PolicyPilotPro addresses a critical need in the startup ecosystem by making compliance accessible, automated, and aligned with the fast-paced nature of technology companies. But this hackathon project is just the beginning.

Immediate Roadmap:

Market Validation: Target 10-20 high-urgency customers actively preparing for SOC 2 certification or Series A funding
Feature Enhancement: Expand compliance framework database and implement advanced analytics
Partnership Development: Build relationships with compliance consultants and startup accelerators

Long-Term Vision:

With a clear development roadmap and pricing strategy aligned with customer value ($99-$499/month based on company size), PolicyPilotPro is positioned to capture significant market share in the growing GRC and legal tech markets. Our target: $100K+ MRR within 18 months, with the potential to scale into a $10M ARR business.

The Bigger Picture:

This isn't just about building another SaaS product. It's about democratizing compliance for the next generation of innovative companies, ensuring that regulatory requirements become a competitive advantage rather than a growth barrier.

PolicyPilotPro represents the future of startup operations: AI-powered, proactive, and perfectly integrated into the workflows that drive innovation forward.

Built with Bolt.new - Powered by AI - Designed for the Future

PolicyPilotPro was built entirely using Bolt.new in less than 48 hours

Joe Slade — Sat, 05 Jul 2025 22:48:03 +0000

This is a submission for the World's Largest Hackathon Writing Challenge: Building with Bolt.

What PolicyPilotPro Does

PolicyPilotPro is an AI-powered compliance automation platform that transforms regulatory burden into strategic advantage for startups and growing companies. Think of it as having a compliance co-pilot that never sleeps, never misses a regulatory update, and never lets you fly blind into legal turbulence.

Core Capabilities:

Intelligent Document Scanning: Upload your existing policies, tech stack documentation, and workflows—our AI instantly identifies compliance gaps across multiple frameworks
Real-Time Policy Generation: Need a GDPR privacy policy? SOC 2 documentation? Our AI generates customized, legally-informed policies in minutes, not months
Regulatory Change Monitoring: Stay ahead of evolving regulations with proactive alerts that assess impact on your specific business context
Workflow Integration: Seamlessly connects with Notion, Linear, and Google Drive to make compliance part of your existing processes

The Problem We Solve:

In a market projected to grow by $44.22 billion from 2025-2029, startups are caught in a compliance paradox. They need robust regulatory frameworks to secure funding, enterprise deals, and SOC 2 certification, but they lack the resources for dedicated legal teams. PolicyPilotPro bridges this gap, serving as a "compliance insurance policy" for fast-moving teams who can't afford to slow down but also can't risk regulatory missteps.

How We Built It

The Tech Stack Symphony:

Bolt.new as our primary development platform, enabling rapid full-stack application creation
Supabase for scalable database architecture and real-time functionality
AI Integration leveraging GPT-4o for document analysis and policy generation
Modern UI/UX with responsive design optimized for startup workflows

The beauty of this approach? What traditionally would require months of development, multiple team members, and significant infrastructure investment was accomplished in a single weekend by leveraging the democratized power of AI-assisted development.

Challenges We Ran Into

The Full-Stack Reality Check: While I have a solid background in software development, my expertise has primarily been in front-end engineering. It's been years since I've navigated the complexities of full-stack architecture, database design, and server-side logic.

But here's where the paradigm shift becomes tangible. Bolt.new and Supabase didn't just make these challenges manageable—they made them disappear. Creating database schemas, implementing third-party integrations, and building server-side edge functions became as intuitive as describing what I wanted to accomplish.

The Compliance Complexity: Building a platform that handles multiple regulatory frameworks (SOC 2, GDPR, CPRA, AI-specific regulations) while maintaining accuracy and legal validity required a careful balance between automation and human oversight.

Accomplishments We're Proud Of

Speed of Innovation: Developing a complex, fully functional MVP application over a single weekend isn't just impressive—it's a testament to how AI-powered development tools are redefining what's possible for individual creators and small teams.

Market Validation: Our solution addresses a real pain point in the $46.76 billion legal technology market, specifically targeting the underserved startup segment.

Technical Achievement: Successfully integrating AI-powered document analysis, policy generation, and regulatory monitoring into a cohesive platform that actually works—not just a demo, but a production-ready solution.

User Experience: Creating an intuitive interface that makes complex compliance concepts accessible to non-legal professionals, complete with dashboard analytics, compliance scoring, and actionable recommendations.

What We Learned

This hackathon became a masterclass in modern development capabilities. I discovered that Bolt.new, combined with providers like Supabase, ElevenLabs, and other GTM tech solutions, can deliver real-world results with dramatically less time and fewer resources than traditional development approaches.

Key Insights:

AI-assisted development isn't just faster—it's fundamentally different, enabling individual creators to build enterprise-grade solutions
The barrier between idea and implementation has essentially vanished for those willing to embrace these new tools
Complex business logic, database relationships, and user interfaces can be created through conversational development