How I Built an Edge AI Pipeline for Podcast Show Notes

I used to spend hours after recording a podcast episode just staring at a blank screen, trying to write show notes and come up with a catchy title. I would record a great conversation, export the audio, and then the real grind would begin. Transcribing, summarizing, formatting, and brainstorming titles took twice as long as the actual recording. I realized I was spending eighty percent of my time on admin work and only twenty percent on creating content. So I decided to build something.

The friction of post-production is the silent killer of indie podcasts. When you are bootstrapping a show, every hour counts. If the administrative overhead of turning a raw recording into a publishable episode is too high, you will eventually burn out. The cost of ignoring this is not just lost time; it is lost consistency. Podcasters need to publish regularly to grow, but the manual labor of writing descriptions, extracting timestamps, and generating SEO-friendly titles creates a massive bottleneck. We needed a way to automate the boring parts without sacrificing the authentic voice of the show.

I wanted to build a tool that could handle this pipeline efficiently, without the massive latency or costs of traditional cloud AI setups. This led me to Cloudflare Workers and their AI inference capabilities. Running AI at the edge means the processing happens close to the user, drastically reducing latency for text generation tasks. It also keeps compute costs predictable, which is crucial when you are building a tool for other indie creators.

Let us look at how I structured the show notes generation pipeline. The goal was to take a raw transcript and force the LLM to output a highly structured Markdown format. Prompt engineering is crucial here. If you just ask for show notes, you get a messy wall of text. You need strict system instructions to enforce the schema.

// Show notes generation pipeline (Cloudflare Workers + AI)
export default {
  async fetch(request, env) {
    const { transcript } = await request.json();

    // Call AI model for structured show notes
    const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        {
          role: 'system',
          content: `Convert podcast transcripts to show notes:
## Episode Summary (2-3 sentences)
## Key Takeaways (5-7 bullet points)
## Timestamps (estimate from content flow)
## Resources Mentioned`
        },
        { role: 'user', content: transcript }
      ],
      max_tokens: 800,
    });

    return new Response(JSON.stringify(response), {
      headers: { 'Content-Type': 'application/json' },
    });
  },
};

In this Cloudflare Worker, the fetch handler receives the raw transcript via a POST request. We then pass it to the Llama 3 8B Instruct model. Notice the system prompt: it explicitly defines the exact Markdown headers we want. By constraining the output format in the system message, the model knows exactly how to structure the response. We cap max_tokens at 800 to keep the response concise and prevent the model from rambling, which also helps control inference costs.

For the title generator, the approach is slightly different. We are not just summarizing; we are trying to generate hooks. I built a specific prompt engineering function to handle different tones and enforce rules that prevent generic clickbait.

// Prompt engineering for podcast title generation
function buildTitlePrompt(topic, tone, count) {
  return {
    system: `You are an expert podcast producer. Generate compelling titles.
Rules:
- 5-10 words each
- Use curiosity gaps, numbers, or strong hooks
- Match the tone: ${tone}
- No clickbait that overpromises
- Format as numbered list`,
    user: `Episode topic: ${topic}

Generate ${count} title options that make someone want to click play.`
  };
}

// Example: generate 5 titles for a bootstrapping episode
const prompt = buildTitlePrompt(
  "How to start a podcast with zero budget",
  "inspirational",
  5
);

This function dynamically builds the prompt based on the user's input. The key here is the negative constraint: "No clickbait that overpromises." LLMs naturally lean towards hyperbole, so explicitly forbidding it in the system prompt forces the model to rely on curiosity gaps and strong hooks instead. Beyond just titles and show notes, I also integrated episode script outline generators to help plan the conversation before hitting record, and SEO keyword suggestions to help the episode pages rank better in search engines. All of these features share the same underlying edge AI architecture.

Moving the inference to the edge was a huge win for the user experience. The time-to-first-token dropped significantly compared to my initial tests on standard serverless functions. What surprised me most was how well the 8B model handled the structured show notes when given a rigid system prompt. It rarely hallucinated timestamps or broke the Markdown formatting.

If I were to change one thing, I would probably implement a streaming response for the show notes generator. Right now, it waits for the full generation to complete before returning the JSON, which can feel like a slight delay for very long transcripts. Adding streaming would make the UI feel much more responsive while the user waits.

I have packaged the title generation logic into a free tool so other podcasters can test it out without needing to set up their own infrastructure. You can try the Podcast Title Generator at https://podcrisp.com/free-tools/title-generator to see how different tones affect the output for your next episode.

---

Full disclosure: I'm the developer of PodCrisp. I built this because I needed it myself.