How I Built an Edge AI Pipeline to Rescue My Podcast Workflow

When I first launched my indie development podcast, I was completely in love with the recording process. Sitting down with a microphone, diving deep into architecture decisions, and chatting with fellow devs was a blast. But the moment the recording stopped, the dread set in.

I was struggling with the post-production admin. Writing a catchy title, listening back to a 45-minute audio file to pull out timestamps, and formatting show notes took me nearly two hours per episode. It was draining my energy and making me want to quit before I even hit publish.

So I decided to build something.

The Cost of Ignoring the Admin Work

It is easy to dismiss show notes and titles as minor details, but they are actually the engine of your podcast's discoverability. A poorly named episode gets ignored in RSS feeds. A lack of structured show notes means you miss out on SEO traffic from people searching for the specific tools or concepts you discussed.

The real cost of ignoring this workflow, however, is burnout. When the friction between hitting "stop" on your recorder and publishing the episode is too high, you start delaying releases. Inconsistent publishing schedules kill audience trust. I realized that if I wanted to keep podcasting, I had to automate the tedious parts without sacrificing the quality of the output.

Building an Edge AI Pipeline

My goal was to create a pipeline that could take a raw transcript and instantly output a polished title, a structured summary, and timestamped show notes. I chose to build this on Cloudflare Workers, utilizing their AI inference capabilities at the edge.

Why the edge? Traditional AI APIs often route requests through centralized data centers, which can introduce latency. By running inference at the edge, the processing happens closer to the user, resulting in a much snappier experience. Plus, Cloudflare's Workers AI platform makes it incredibly straightforward to integrate open-source models without managing GPU infrastructure.

Prompt Engineering for Titles

The first hurdle was the "blank page" syndrome for episode titles. I needed a way to generate titles that were catchy but not overly clickbaity. The secret here is strict system prompting.

Here is the function I use to build the prompt for the title generator:

// Prompt engineering for podcast title generation
function buildTitlePrompt(topic, tone, count) {
  return {
    system: `You are an expert podcast producer. Generate compelling titles.
Rules:
- 5-10 words each
- Use curiosity gaps, numbers, or strong hooks
- Match the tone: ${tone}
- No clickbait that overpromises
- Format as numbered list`,
    user: `Episode topic: ${topic}

Generate ${count} title options that make someone want to click play.`
  };
}

// Example: generate 5 titles for a bootstrapping episode
const prompt = buildTitlePrompt(
  "How to start a podcast with zero budget",
  "inspirational",
  5
);

By explicitly defining the rules—like word count limits and forbidding overpromising clickbait—the AI outputs much more usable options. The tone variable allows me to shift the style depending on the episode, whether it is a serious technical deep dive or a casual chat.

Structuring Show Notes at the Edge

The heavier lift was the show notes generator. A raw transcript is just a wall of text. To make it useful, the AI needs to parse the conversational flow and extract specific metadata.

Here is the core of the Cloudflare Worker that handles this pipeline:

// Show notes generation pipeline (Cloudflare Workers + AI)
export default {
  async fetch(request, env) {
    const { transcript } = await request.json();

    // Call AI model for structured show notes
    const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        {
          role: 'system',
          content: `Convert podcast transcripts to show notes:
## Episode Summary (2-3 sentences)
## Key Takeaways (5-7 bullet points)
## Timestamps (estimate from content flow)
## Resources Mentioned`
        },
        { role: 'user', content: transcript }
      ],
      max_tokens: 800,
    });

    return new Response(JSON.stringify(response), {
      headers: { 'Content-Type': 'application/json' },
    });
  },
};

The key takeaway from this code is the system prompt structure. Instead of just asking the model to "summarize this," I forced it to output specific Markdown headings. This prevents the model from just giving a generic paragraph and forces it to logically separate the summary, the key takeaways, and the resources. Setting max_tokens to 800 ensures the response doesn't get cut off while still providing enough depth for a standard episode.

When designing this pipeline, I had to weigh the trade-offs between using a massive proprietary model versus a smaller, edge-optimized model like Llama 3 8B. For podcast show notes, the 8B parameter model is more than capable of understanding context and extracting structured data. It keeps the inference fast and cost-effective. I keep the architecture modular so I can swap the model identifier in the env.AI.run call without rewriting the surrounding pipeline logic if I ever need heavier reasoning.

Practical Results and Iterations

Moving the inference to the edge worked better than I initially expected. The UI feels nearly instantaneous, which keeps the creative momentum going right after a recording session.

What surprised me the most was how much the structured system prompts reduced hallucinations. In my early tests, if I just asked for "show notes," the AI would sometimes invent resources that were never mentioned in the transcript. By explicitly asking for "Resources Mentioned" and grounding the prompt in the transcript text, the accuracy improved significantly.

If I were to change one thing about the current architecture, it would be adding more granular control over the summary length. Sometimes a 45-minute technical episode needs a slightly longer summary to capture the nuance, and I am currently tweaking the token limits and prompt instructions to handle variable episode lengths more gracefully.

Try the Free Title Generator

If you are a fellow podcaster struggling with the post-production admin, or just a developer curious about edge AI pipelines, I would love for you to try out the tools I have built.

You can start with the free Podcast Title Generator to see how the prompt engineering works in practice. It lets you input your topic and tone to generate multiple title options instantly.

Check it out here: https://podcrisp.com/free-tools/title-generator

---

Full disclosure: I'm the developer of PodCrisp. I built this because I needed it myself.