How I Built an Edge AI Pipeline for Podcast Show Notes

When I first launched my tech podcast, I thought the hard part was getting guests and recording the audio. I was wrong. The real bottleneck hit me three episodes in: the post-production metadata. I was spending nearly two hours after every recording session just trying to write a compelling title, summarize the episode, and format the show notes. I was burning out before the episode even went live. So I decided to build something.

Let's talk about why this matters. In podcasting, discoverability is everything. If your title is generic, it gets lost in the feed. If your show notes are just a raw transcript dump, listeners won't engage. But as developers, we usually start podcasts because we love the technical deep dives, not because we love writing SEO-optimized marketing copy. The cost of ignoring metadata is low listener retention, but doing it manually drains the creative energy you need for the actual episodes. I needed a way to turn raw transcripts into structured, engaging content without staring at a blank text editor for an hour.

I knew I needed LLMs for this, but I wanted to avoid the latency and egress costs of traditional cloud API routing. I decided to build the pipeline on Cloudflare Workers, utilizing their edge AI inference. This keeps the processing close to the user and simplifies the architecture.

To set this up, you need to enable the AI binding in your wrangler.toml file. This allows your worker to call models like Llama 3 directly without managing external API keys in your code.

The first piece of the puzzle was the title generator. Prompt engineering is all about constraints. If you just ask an LLM for a title, it gives you something bland. You have to enforce rules. Here is the prompt structure I use to generate multiple style options:

// Prompt engineering for podcast title generation
function buildTitlePrompt(topic, tone, count) {
  return {
    system: `You are an expert podcast producer. Generate compelling titles.
Rules:
- 5-10 words each
- Use curiosity gaps, numbers, or strong hooks
- Match the tone: ${tone}
- No clickbait that overpromises
- Format as numbered list`,
    user: `Episode topic: ${topic}

Generate ${count} title options that make someone want to click play.`
  };
}

// Example: generate 5 titles for a bootstrapping episode
const prompt = buildTitlePrompt(
  "How to start a podcast with zero budget",
  "inspirational",
  5
);

Notice how the system prompt explicitly forbids clickbait and enforces a word count. This prevents the model from generating overly long, academic titles that perform poorly in podcast directories.

The second piece is the show notes pipeline. This is where the heavy lifting happens. We take the raw transcript and pass it to a model running on Cloudflare Workers. The trick here is forcing a strict markdown structure so the output is immediately usable for a CMS or website.

// Show notes generation pipeline (Cloudflare Workers + AI)
export default {
  async fetch(request, env) {
    const { transcript } = await request.json();

    // Call AI model for structured show notes
    const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        {
          role: 'system',
          content: `Convert podcast transcripts to show notes:
## Episode Summary (2-3 sentences)
## Key Takeaways (5-7 bullet points)
## Timestamps (estimate from content flow)
## Resources Mentioned`
        },
        { role: 'user', content: transcript }
      ],
      max_tokens: 800,
    });

    return new Response(JSON.stringify(response), {
      headers: { 'Content-Type': 'application/json' },
    });
  },
};

By defining the exact markdown headings in the system prompt, the model knows exactly how to structure the output. The max_tokens limit ensures we don't get runaway generation if the transcript is particularly chatty.

Moving to edge AI changed the game for latency. The show notes generation feels nearly instantaneous compared to my previous centralized API setup. What surprised me most was how well the model handled timestamp estimation. By prompting it to look for conversational shifts and topic changes, it actually generates reasonably accurate timestamps without needing a separate audio alignment tool.

If I were to change one thing, it would be breaking the show notes into smaller chunks for very long episodes. Pushing a two-hour transcript through an 800-token output limit means it sometimes glosses over the middle section. Chunking the transcript by topic first, or increasing the token limit for larger models, would solve this.

I ended up packaging these exact edge AI workflows into a platform called PodCrisp. It handles title generation, show notes, script outlines, and SEO keywords, all running on Cloudflare Workers. If you're dealing with the same post-production bottleneck, I've made the Podcast Title Generator completely free to use. You can try it out and see how the prompt engineering works in practice at https://podcrisp.com/free-tools/title-generator.

---

Full disclosure: I'm the developer of PodCrisp. I built this because I needed it myself.