Every morning I had the same routine: open Hacker News, GitHub Trending, ArXiv, TechCrunch, five AI lab blogs, and three newsletters. Thirty minutes later, I had read the same Claude announcement eight times across different sources and still missed the one ArXiv paper that actually mattered to my stack.
A conversation with a friend at Meta confirmed it: even people inside the industry cannot keep up. The firehose is useless. We need filtered, scored, summarized intelligence.
So I built it.
What ScanBrief Does
[ScanBrief](https://scanbrief.dev) is an automated intelligence pipeline that runs every morning at 6:30 AM. It:
- **Ingests** from 20+ high-signal sources — Hacker News (API), GitHub Trending (scrape), ArXiv, OpenAI blog, Google AI blog, TechCrunch, Ars Technica, The Verge, and more coming weekly.
- **Deduplicates** — URL hash matching plus fuzzy title similarity. The same story appearing on HN, Reddit, Twitter, and three newsletters gets collapsed into one item.
- **Scores** — Each item gets a composite score (0-100) based on source authority, novelty, velocity (how fast it is spreading), and relevance to your tech stack.
- **Summarizes** — Claude AI generates 2-sentence summaries for the top 15 items. Enough context to decide if you need the full article.
- **Delivers** — Clean markdown brief to Discord, email, Slack, or via API. Read it in 5 minutes over coffee.
Why Not Just Use a Newsletter?
Newsletters are humans picking links on their schedule. That is fine for casual reading. But for builders who need to stay current on signals that affect their actual codebase:
- **Newsletters miss things** — human bandwidth is finite. ScanBrief scans 92+ items per run.
- **No deduplication** — TLDR, The Rundown, and Morning Brew all cover the same story separately. You read it three times.
- **No personalization** — they cannot score by what affects YOUR projects and YOUR stack.
- **Fixed schedule** — ScanBrief delivers when you need it, how you need it.
The Tech Stack
Deliberately simple:
- **Python** — ingesters, dedup engine, scorer, brief formatter
- **Claude AI (Haiku)** — summarization via CLI
- **SQLite** — item storage, dedup history, brief archive
- **Discord webhooks** — delivery (email and API coming in Sprint 3)
- **Cron** — 6:30 AM daily, runs in ~2 minutes end-to-end
Total infrastructure cost: roughly $2/month. All source APIs are free. AI summarization runs on Claude CLI. Hosting is a single Linux server.
What Makes Dedup Hard (and Valuable)
Deduplication is the killer feature and the hardest engineering problem. Stage 1 catches exact URL matches across sources. Stage 2 uses fuzzy title matching (85%+ similarity threshold) to merge "Claude 5 Released" with "Anthropic Launches Claude 5." Future sprints add semantic embeddings for complete rephrases and cross-language duplicates.
Getting this right is the difference between a useful brief and another noise source. We are conservative on thresholds — better to show a near-duplicate than to accidentally merge two different stories.
Coming Next: Project Relevance
The feature I am most excited about is project relevance mapping. ScanBrief will scan your actual codebase and tech stack, then flag when a trending signal affects YOUR projects. "New Claude version released" automatically flags every project that uses the Claude API. "GitHub ships AI code review" flags your CI/CD pipeline.
No newsletter can do this. It requires knowing what you are building.
Pricing
TierPriceWhat You Get
Free$0Daily email brief, top 5 items, 3 tracks
Pro$9/moFull brief, all tracks, emerging signals, custom sources, API access
Team$29/moShared dashboard, team tracks, Slack/Discord webhooks, priority support
**Early subscribers get Pro features free for 3 months.**
Try It
Head to [scanbrief.dev](https://scanbrief.dev) and sign up for the free tier. You will get your first brief tomorrow morning.
Signal compounds. Noise compounds too. Choose which one you are stacking.
**Built by [Prime Automation Solutions](https://primeautomationsolutions.com)** — We build autonomous systems that run while you sleep. From network automation to AI pipelines, we help businesses eliminate manual work.
Originally published at https://primeautomationsolutions.com
Top comments (0)