Building Edge AI Inference to Automate Podcast Show Notes

When I first launched my tech podcast, I thought the hard part was getting guests and recording the audio. I was wrong. The real bottleneck hit me the moment I hit "stop recording." I suddenly had a 45-minute audio file and zero energy to write the show notes, extract timestamps, or brainstorm a compelling title. I was spending three hours per episode just on administrative text. I tried plugging my transcripts into generic AI chatbots, but they lacked the specific context needed for podcast formatting. They would hallucinate timestamps or write titles that sounded like clickbait. So I decided to build something.

The Problem

The post-production admin for podcasts is a massive cognitive drain. When you finish a deep, technical conversation, your brain is fried. Asking it to immediately pivot into analytical summarization and SEO optimization is a recipe for burnout.

If you ignore this step, your podcast suffers. Poor titles mean lower click-through rates in directories like Apple Podcasts and Spotify. Missing or messy show notes mean your episode is invisible to search engines. But if you spend too much time on it, you resent the process and eventually quit. The cost of ignoring it is poor discoverability; the cost of over-investing is creator fatigue. We needed a way to bridge the gap between raw audio and polished metadata without losing the human element.

Technical Approach

I started building PodCrisp to solve this. My main technical goal was to keep the AI inference fast and reliable. I chose to build the backend on Cloudflare Workers, utilizing their edge AI capabilities. Running inference at the edge means the AI models are hosted close to the user, drastically reducing latency and eliminating the cold-start issues common in traditional serverless functions.

To set this up, you need to enable the AI binding in your wrangler.toml file:

[ai]
binding = "AI"

This binds the AI namespace to your worker, allowing you to call the models directly without managing external API keys in your environment variables.

Let's look at how I approached the prompt engineering for the title generator. The key is constraining the AI so it doesn't just generate generic fluff.

// Prompt engineering for podcast title generation
function buildTitlePrompt(topic, tone, count) {
  return {
    system: `You are an expert podcast producer. Generate compelling titles.
Rules:
- 5-10 words each
- Use curiosity gaps, numbers, or strong hooks
- Match the tone: ${tone}
- No clickbait that overpromises
- Format as numbered list`,
    user: `Episode topic: ${topic}

Generate ${count} title options that make someone want to click play.`
  };
}

// Example: generate 5 titles for a bootstrapping episode
const prompt = buildTitlePrompt(
  "How to start a podcast with zero budget",
  "inspirational",
  5
);

By explicitly defining the rules in the system prompt—like word count limits and banning overpromising clickbait—the output becomes much more usable.

For the heavier lifting, like converting a full transcript into structured show notes, I built a specific pipeline using Cloudflare Workers AI. I opted for the @cf/meta/llama-3-8b-instruct model because it handles structured formatting and instruction-following exceptionally well.

Here is the core of the show notes generation worker:

// Show notes generation pipeline (Cloudflare Workers + AI)
export default {
  async fetch(request, env) {
    const { transcript } = await request.json();

    // Call AI model for structured show notes
    const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        {
          role: 'system',
          content: `Convert podcast transcripts to show notes:
## Episode Summary (2-3 sentences)
## Key Takeaways (5-7 bullet points)
## Timestamps (estimate from content flow)
## Resources Mentioned`
        },
        { role: 'user', content: transcript }
      ],
      max_tokens: 800,
    });

    return new Response(JSON.stringify(response), {
      headers: { 'Content-Type': 'application/json' },
    });
  },
};

There are a few architectural trade-offs here. By capping max_tokens at 800, I ensure the response remains concise and prevents the model from rambling, which is crucial for show notes. However, this means if a transcript is incredibly long, the model might summarize the end too briefly. To mitigate this, I chunk longer transcripts before passing them to the prompt.

Practical Results

Moving the AI inference to the edge was the most impactful decision I made. The show notes generation feels nearly instant, which keeps the creator in a state of flow rather than waiting on a loading spinner.

What surprised me most was how much the structured system prompts improved the timestamp generation. Initially, the AI would just make up random times. By forcing it to "estimate from content flow" and pairing it with a strict markdown format in the system prompt, the hallucinations dropped significantly. It doesn't replace manual verification, but it gives creators a highly accurate starting point.

If I were to change one thing, I would add more granular control over the summary depth. Some technical episodes need a longer summary, while a casual interview might only need two sentences. Adding a "detail level" parameter to the prompt builder is on my roadmap.

Wrapping Up

If you are currently drowning in post-production admin, I'd love for you to try the tools I've been building. You can start by using the free Podcast Title Generator to see how constrained prompt engineering can yield better, more natural-sounding titles for your next episode.

You can try it here: https://podcrisp.com/free-tools/title-generator

---

Full disclosure: I'm the developer of PodCrisp. I built this because I needed it myself.