Building an Edge AI Pipeline to Automate Podcast Notes

When I first launched my podcast, I was entirely focused on the recording. I thought the real work was prepping questions, hitting record, and editing out the "ums" and "ahs." I was wrong. The most exhausting part of the process happened long after the microphone was turned off.

It was the administrative graveyard: writing show notes, pulling quotes, guessing timestamps, and staring at a blank screen trying to write a title that didn't sound incredibly boring. I was spending two hours on post-production paperwork for every one hour of recording. It was draining my creative energy and making me actively dread publishing the next episode.

So I decided to build something.

The Hidden Cost of Podcast Post-Production

The problem with podcast post-production isn't just that it takes time; it's the severe context switching. As a host, you're in a creative, conversational headspace. Forcing your brain to immediately pivot into the role of a meticulous, SEO-conscious editor is jarring.

When you ignore this friction, the costs compound. First, you risk creator burnout. Second, your publishing schedule becomes inconsistent because the "boring stuff" keeps getting pushed to the bottom of the to-do list.

But perhaps the biggest hidden cost is discoverability. Podcast platforms are essentially search engines. Your title and description are the primary vectors for SEO. If you slap together a weak title and a one-sentence description because you're tired, your episode is practically invisible. The audio might be pristine, but if the metadata is an afterthought, no one will ever click play.

Moving the Heavy Lifting to the Edge

I knew AI could handle the heavy lifting of summarization and formatting, but I needed to think carefully about the architecture. Initially, I spun up a standard serverless function to process transcripts. It worked, but the cold starts were noticeable, and passing large text payloads back and forth to a central server added latency. I wanted the tool to feel instantaneous.

This led me to edge computing. I built PodCrisp on Cloudflare Workers, leveraging their AI inference capabilities to run models right at the edge, geographically close to the user.

The workflow for generating show notes is straightforward but relies heavily on precise prompt engineering. Instead of just asking the AI to "summarize this," I force it into a specific structural output. Here is a simplified look at the Cloudflare Worker handling the transcript-to-notes pipeline:

// Show notes generation pipeline (Cloudflare Workers + AI)
export default {
  async fetch(request, env) {
    const { transcript } = await request.json();

    // Call AI model for structured show notes
    const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        {
          role: 'system',
          content: `Convert podcast transcripts to show notes:
## Episode Summary (2-3 sentences)
## Key Takeaways (5-7 bullet points)
## Timestamps (estimate from content flow)
## Resources Mentioned`
        },
        { role: 'user', content: transcript }
      ],
      max_tokens: 800,
    });

    return new Response(JSON.stringify(response), {
      headers: { 'Content-Type': 'application/json' },
    });
  },
};

In this setup, the worker receives the raw transcript via a JSON payload. We pass it to the @cf/meta/llama-3-8b-instruct model. The magic is in the system prompt. By explicitly defining the markdown structure—Summary, Key Takeaways, Timestamps, and Resources—we prevent the model from just outputting a wall of text. We also cap the max_tokens to 800 to keep the response snappy and focused.

The trade-off with edge AI is that you're often working with highly optimized, smaller models compared to the massive parameter models you might run on a dedicated GPU server. This means you can't just rely on the model's sheer size to understand complex instructions. You have to compensate with rigorous prompt engineering. The system prompt in the code above isn't just a suggestion; it's a strict set of constraints that forces the model to behave predictably.

Running this inference at the edge means we avoid the round-trip latency to a centralized GPU cluster, making the user experience significantly smoother.

Practical Results and Workflow Expansion

Moving to the edge was a significant improvement for the perceived performance of the tool. What worked surprisingly well was the AI's ability to estimate timestamps. Even without explicit audio markers in the text, the model can infer where topics shift based on conversational flow and generate a rough timestamp list. It's not perfect, but it gives editors a substantial head start.

Beyond just show notes, I realized the entire pre- and post-production workflow needed help. That's why I expanded the toolset to include an episode script outline generator for the planning phase, and an episode description writer. I also added SEO keyword suggestions specifically tailored for podcast episode pages, because standard web SEO doesn't always translate perfectly to audio discovery.

If I were to change one thing right now, it would be adding niche-specific prompt templates. A true crime podcast needs a very different title and description tone than a B2B SaaS interview. I'm currently experimenting with passing a "niche" variable into the system prompt to adjust the vocabulary and hook style dynamically.

Crafting the Perfect Title

While show notes were my initial focus, crafting the actual episode title is another massive bottleneck. A good title needs a curiosity gap, a clear benefit, and the right tone—all without crossing into clickbait territory.

To help with that, I extracted the title generation logic into a standalone, free tool. You can try the PodCrisp Podcast Title Generator to get multiple style options for your next episode. It's completely free, requires no login, and runs right in your browser.

---

Full disclosure: I'm the developer of PodCrisp. I built this because I needed it myself.