How I Built an Edge AI Prompt Engine for Podcast Creators

When I first started recording episodes for my indie dev podcast, I assumed the hardest part would be the technical setup—mic gain, audio routing, or editing out my umms. I was wrong. The real bottleneck turned out to be the post-production admin. Specifically, staring at a blank screen trying to write a compelling title, followed by the dread of writing detailed show notes.

I’d spend an hour just tweaking the episode title, trying to find the balance between catchy and accurate. Then I’d have to listen to the entire episode again just to pull out timestamps and key takeaways for the description. It was draining, and it was eating into the time I should have been spending on actual content.

So I decided to build something.

The Problem with Podcast Metadata

It’s easy to dismiss titles and show notes as an afterthought, but they are the primary discovery mechanism for your show. If your title is bland, nobody clicks. If your show notes lack structure, search engines can't index your content effectively, and listeners can't find the specific segments they care about.

The cost of ignoring this is high: great episodes simply die in obscurity.

As a developer, my instinct was to automate this using Large Language Models. But simply dumping a massive raw transcript into a generic prompt rarely works. The output is usually verbose, hallucinates timestamps, or generates titles that sound like clickbait. The challenge wasn't just connecting to an AI model; it was engineering the prompts and the infrastructure to handle the context reliably and quickly.

A Technical Approach: Edge AI and Prompt Engineering

To solve this, I built PodCrisp, relying on Cloudflare Workers to run AI inference at the edge. Running inference at the edge means the AI processing happens in data centers physically close to the user, drastically reducing latency and avoiding the cold-start penalties of traditional serverless functions.

Let’s break down the two main technical components: title generation and show notes extraction.

1. Engineering the Title Prompt

Generating a good podcast title requires strict constraints. If you just ask an AI to write a title, it will give you a paragraph. You need to force it into a specific format and tone.

Here is the prompt engineering logic I use for the title generator:

// Prompt engineering for podcast title generation
function buildTitlePrompt(topic, tone, count) {
  return {
    system: `You are an expert podcast producer. Generate compelling titles.
Rules:
- 5-10 words each
- Use curiosity gaps, numbers, or strong hooks
- Match the tone: ${tone}
- No clickbait that overpromises
- Format as numbered list`,
    user: `Episode topic: ${topic}

Generate ${count} title options that make someone want to click play.`
  };
}

// Example: generate 5 titles for a bootstrapping episode
const prompt = buildTitlePrompt(
  'How to start a podcast with zero budget',
  'inspirational',
  5
);

Notice the explicit rules in the system prompt. By limiting the word count and explicitly forbidding overpromising clickbait, the model is forced to be punchy and honest. The tone variable allows us to shift the output from educational to conversational without rewriting the core logic.

2. The Show Notes Pipeline

For show notes, the challenge is structuring a massive wall of text into a readable format. We need a summary, key takeaways, estimated timestamps, and resources.

Here is how the Cloudflare Worker handles the transcript processing:

// Show notes generation pipeline (Cloudflare Workers + AI)
export default {
  async fetch(request, env) {
    const { transcript } = await request.json();

    // Call AI model for structured show notes
    const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        {
          role: 'system',
          content: `Convert podcast transcripts to show notes:
## Episode Summary (2-3 sentences)
## Key Takeaways (5-7 bullet points)
## Timestamps (estimate from content flow)
## Resources Mentioned`
        },
        { role: 'user', content: transcript }
      ],
      max_tokens: 800,
    });

    return new Response(JSON.stringify(response), {
      headers: { 'Content-Type': 'application/json' },
    });
  },
};

There are a few key decisions here. First, I'm using Llama 3 8B via Cloudflare's AI bindings. It’s a highly capable model for structured extraction. Second, notice the system prompt explicitly asks the model to estimate from content flow for timestamps. Since text transcripts don't contain actual audio timing data, asking the AI for exact seconds will result in hallucinations. Telling it to estimate based on the conversational flow yields much more reliable markers for the listener. Finally, capping max_tokens at 800 prevents the model from rambling and keeps the output concise.

Practical Results and Lessons Learned

Moving the inference to the edge was a game-changer for the user experience. For example, generating show notes from a long transcript might take 10-15 seconds on standard cloud providers. With Cloudflare Workers, it can consistently finish in under 3 seconds.

What surprised me most was how much the model respects the negative constraints in the prompt. Initially, I struggled with the AI generating titles that were too sensational. Adding the explicit rule against clickbait immediately cleaned up the output.

If I were to rebuild this today, I would likely implement a secondary validation step. Sometimes the AI misses a resource mentioned in passing if it wasn't explicitly named. Adding a lightweight secondary keyword extraction pass could catch those edge cases, though it would add a bit of latency.

Putting It Into Practice

Building the backend was only half the battle; the frontend needed to be just as intuitive for creators who aren't necessarily developers. I've packaged the title generation logic into a free, easy-to-use interface so you can test the prompt engineering yourself without writing any code.

You can try the Podcast Title Generator right here: PodCrisp Free Title Generator. It lets you input your topic, select a tone, and see how the structured prompts perform in real-time.

Podcasting is hard enough without letting the metadata slow you down. Hopefully, seeing the architecture and prompt structures behind these tools gives you some ideas for your own projects, or at least saves you some time on your next episode.

---

Full disclosure: I'm the developer of PodCrisp. I built this because I needed it myself.