Designing a YouTube Digest for Signal Over Noise

#dataengineering #automation #appliedai #python

Why I Built This

I follow a lot of YouTube channels around data, analytics, and AI. Over time, it became overwhelming. New videos every day, long talks, podcasts, webinars. Though all were useful, watching them still demanded time and focus.

I tried the usual things, and none of it really solved the problem:

saving videos for later
creating playlists
watching at higher speed (watching at 1.5x or 2x)
asking an LLM to “summarize this video” one by one

At some point, I realized the issue wasn’t YouTube. It's how I was consuming it. I didn’t need to watch everything. I needed a better way to decide what actually deserves my attention.

What I Wanted Instead

Instead of watching more videos or reading longer summaries, I just wanted to stay informed, quickly scan and move on.

The goal was simple: Cut the noise & focus on the signal !

Tell me what’s new
Highlight ideas, tools, or patterns
Help me decide watch or skip

If a video introduces something genuinely interesting, I want to know. If it’s repetitive or low signal, I’m fine missing it. Automation should handle the volume, and I should only step in only when needed.

The Core Idea

The idea was simple: let automation handle the heavy lifting.

Instead of manually opening videos;

The system picks up newly uploaded videos from channels I follow
Passes the video URL directly to an LLM (Gemini)
The model processes the video and returns a short, structured summary in a format I control.

The focus isn’t on full summaries, but on highlights. What’s new, what stands out, and what’s worth attention.

The result is a compact digest I can scan in minutes and decide whether a video deserves deeper time.

How the System Works (At a High Level)

The system runs as a small pipeline made up of independent steps.
Each step is designed to work on its own, which makes the system easy to extend, replace, or run independently.

It starts by identifying newly uploaded videos from the YouTube channels I’m interested in.
Each video is checked and classified (shorts/long) so the system knows how to handle it.
For longer videos, the video URL is passed to an LLM, which returns a concise summary based on a predefined structure.
For Short videos (less than 3 minutes), The title and the link is captured.
Once processing is complete, everything is grouped by channel and compiled into a single digest.

The final output is sent as an email, but it can also be stored for later use or extended further as the project evolves.

Key Design Decisions

I wanted this system to stay simple and efficient. The goal wasn’t to build a complex platform, but something reliable that could run regularly without much attention.

1. Conscious API usage:
Since YouTube APIs are quota-based, the system is designed to fetch only what’s necessary and avoid redundant calls wherever possible.

2. Keep the processing steps loosely coupled:
Each part of the pipeline can evolve on its own. Whether that’s how videos are fetched, how summaries are generated, or how the output is delivered, etc.

3. Signals over Noise:
The summaries are intentionally short. The goal is not to capture everything, but to surface what’s new or meaningful. If something doesn’t add signal, it doesn’t earn space in the digest.

What I Intentionally Didn’t Build

The intent of this system is to solve my problem first. Also, validate whether this idea works. If it works, I wanted to make the foundations built right for scale.

So, I didn't;

Build a UI or dashboard: The output is meant to be consumed quickly, and an email works well for that. Adding a frontend would have increased complexity without improving the core value.
Add a database to park the data
Make this a real-time system: A daily or periodic digest is enough.

These decisions helped keep the system small, focused, and easy to evolve.

What I Learned Along the Way

Channel Subscriptions: I realized that I'm subscribed to 74 channels. Seeing that number in the API response was honestly surprising and immediately explained why things felt overwhelming.
Gemini Cost (vertex AI): Be really cautious of passing very long videos. I once did a 2 hour 40 minute video and it was $1.
Youtube Data API Quota : It's kind of free with very generous free limit of 10K units a day (each API hit is 1-100 units), which works well for this use case. It’s a soft limit, though you can request more, but it requires approval.
Noise in the data: Initially the summaries were very long & not useful. I iterated on the prompt and output structure to get to the signal.

What's Next

I’d like to explore the following by running this system for next few weeks;

Refine how different types of content are handled. Not every video needs the same level of detail, and there’s room to tune this further.
Track Ai cost
Look for optimizations

Final Thoughts

This started as an idea to see if automation could help, As the project evolved it became a pet project, I gained more confidence in the approach and eventually turned into an useful system that I actually rely on.

I don’t watch fewer videos because of this. I just watch the right ones & it saves a lot of time and mental energy.

If you’re in a similar situation with YouTube or long-form content in general, this approach might be worth trying.

If you’ve tried something similar or have ideas to build on this, I’d love to collaborate.