DEV Community

Aman
Aman

Posted on

I Built an Autonomous GitHub Trending Tracker with Star Velocity & MCP Topic Radars 🚀

#ai #python #github #opensource

Hey DEV Community! 👋

Today I'm excited to open-source GitTrends Intelligence — an autonomous engine and interactive web dashboard designed to track breakout GitHub projects, viral star velocities, and cutting-edge open-source tools.

🧐 Why I Built This

GitHub's native trending page is great, but it has significant limitations for developers trying to catch breakout tech early:

  1. No Star Velocity: You can't see the exact velocity (+stars gained in the last 24h) directly on an interactive interface.
  2. Missing AI & MCP Taxonomies: There are no dedicated topic filters for the fastest-growing ecosystems right now — like Model Context Protocol (#MCP), Coding Agents, Local LLMs, and AI Video Generation.

I wanted a zero-maintenance, automated system that gives developers real-time intelligence on what is truly going viral across GitHub.

🛠️ Tech Stack & How It Works

I built the entire project with zero external server costs using a fully automated architecture:

1. Python Intelligence Engine (engine.py)

  • Scrapes trending repos across 6 major ecosystems (Overall, Python, JavaScript, TypeScript, Go, Rust).
  • Uses connection pooling, rotating headers, and fallback heuristics to extract repo metrics safely.
  • Classifies projects using description semantic scanning for tags like #MCP, #Coding-Agents, #AI-Video, and #Security.

2. Zero-Maintenance Cloud Automation

  • Scheduled GitHub Actions runs automatically every 12 hours (at 00:00 UTC and 12:00 UTC).
  • Automatically executes the engine in GitHub's cloud, generates daily historical archives (YYYY-MM/), generates an RSS 2.0 feed (feed.xml), and commits updates autonomously.

3. Lightweight Client-Side Web Dashboard

  • Pure Vanilla HTML, CSS & JavaScript — no heavy frameworks or complicated build steps.
  • Features real-time search, category tabs, star velocity sorting, table vs card view modes, and 1-click clone command copy.
  • Deployed directly on GitHub Pages with automatic cache-busting.

🌐 Live Demo & Source Code

💬 Looking for Your Feedback!

Since this is an active open-source project, I'd love to hear from the community:

  • What additional language categories or data metrics would you like to see tracked?
  • If you find this helpful for discovering new open-source projects, consider dropping a ⭐ on GitHub!

Happy building! 💻

Top comments (2)

Collapse
 
raknaos profile image
Baptiste Le Bouquin •

The star-velocity angle is the right one — GitHub's trending page conflates absolute stars with momentum, and the two diverge fast. A 40k-star repo gaining 50 a day is history; a 200-star repo gaining 80 a day is where the ecosystem is actually moving.

How are you computing the 24h velocity? The search API doesn't sort by star delta, so I'm guessing you take periodic snapshots and diff them — if so, does the snapshot cadence live in GitHub Actions or an external cron? I ask because I've run daily trending scrapers before and the flaky part was never the parsing, it was the silently rate-limited 403s at 3am that left holes in the delta series.

One thing worth watching as the radar matures: separating promo spikes (a front-page HN thread, a newsletter mention) from sustained adoption. Velocity catches both, but they mean very different things if you're hunting for tools to actually adopt — plenty of repos trend for 48 hours on launch buzz and go quiet a month later. A cheap decay check (is it still gaining 7 days after the spike?) filters most of that.

The MCP topic filter is a smart cut, by the way — that ecosystem moves faster than any tag-based feed can track.

Collapse
 
aman_social profile image
Aman •

Thanks for the thoughtful feedback! You hit the nail on the head regarding momentum vs. absolute star count.

To answer your question: the scheduler runs entirely via GitHub Actions on a 12-hour cron (0 0,12 * * *), with manual workflow_dispatch enabled for ad-hoc runs.

For the 24h delta, GitHub renders a "stars today" metric directly on the trending cards (article.Box-row), which the engine parses and normalizes alongside daily JSON snapshots archived in the repo.

To avoid the dreaded 3am 403/429 silent failures you mentioned, engine.py uses:

  1. A resilient requests session mounted with exponential backoff retries (HTTPAdapter handling 429/5xx).
  2. Rotating browser User-Agents with randomized jitter delays between requests.
  3. Automated commit-and-push steps inside Actions to record each run into data/daily/ without relying on an external DB.

And that 7-day decay check suggestion is brilliant. Separating launch-day spikes (HN/Product Hunt buzz) from sustained developer adoption would add huge signal. Definitely adding that to the roadmap!