When I first started podcasting, I thought the hard part was hitting "record." I quickly learned that the actual bottleneck isn't the recording; it's the aftermath. After a long session, the last thing I want to do is stare at a blank screen trying to write a catchy title, format messy show notes, or draft an engaging episode description.
I was spending hours just trying to do the administrative work that makes a podcast discoverable. I tried using generic AI wrappers to help, but they were often slow, felt disconnected from the actual context of podcasting, and required a lot of manual copy-pasting to get the formatting right. I needed something tailored specifically to the audio-to-text workflow. So I decided to build something.
The Problem with Podcast Post-Production
The cost of ignoring post-production admin is higher than most indie podcasters realize. Listeners absolutely judge a book by its cover. If your episode title is bland, your click-through rate drops. If your show notes are a wall of unformatted text, your SEO suffers and listeners can't easily find the resources you mentioned.
From a technical perspective, solving this with AI introduces a new set of trade-offs. When you process a 5,000-word transcript to generate structured show notes, you are dealing with significant input and output token counts.
If you rely on traditional centralized cloud APIs, every request has to travel to a central data center, be processed by a massive model, and return. For a tool generating show notes, this latency adds up quickly, making the user experience feel sluggish. On the flip side, running local models is great for privacy but incredibly difficult to scale and often lacks the nuanced language understanding required for good copywriting.
There was a clear gap for a specialized, fast, and context-aware AI pipeline that could handle the heavy lifting of podcast formatting without making the user wait.
Evaluating the Technical Approach: Centralized vs. Edge
When building PodCrisp, I had to choose an architecture for the AI pipeline. The two main approaches I evaluated were centralized cloud APIs (like calling a massive proprietary model from a standard serverless function) versus edge inference (running optimized models directly on Cloudflare Workers).
Centralized APIs offer massive model sizes, which is great for complex reasoning. However, they introduce latency and egress costs. Every request travels across the backbone network.
Edge inference, on the other hand, runs the model on servers geographically closer to the user. The trade-off is that you are generally limited to smaller, highly optimized models (like Llama 3 8B). But for specific, structured tasks like formatting show notes or generating titles, a well-prompted 8B model often outperforms a poorly prompted 70B model, with a fraction of the latency.
I decided to build the pipeline on Cloudflare Workers using their AI inference capabilities. This allowed me to keep the compute close to the user while leveraging models fine-tuned for instruction following.
Here is a look at the core of the show notes generation pipeline:
// Show notes generation pipeline (Cloudflare Workers + AI)
export default {
async fetch(request, env) {
const { transcript } = await request.json();
// Call AI model for structured show notes
const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
messages: [
{
role: 'system',
content: `Convert podcast transcripts to show notes:
## Episode Summary (2-3 sentences)
## Key Takeaways (5-7 bullet points)
## Timestamps (estimate from content flow)
## Resources Mentioned`
},
{ role: 'user', content: transcript }
],
max_tokens: 800,
});
return new Response(JSON.stringify(response), {
headers: { 'Content-Type': 'application/json' },
});
},
};
The magic here isn't just in the infrastructure; it's in the prompt engineering. By forcing the system prompt to output specific Markdown headers, the model is constrained to a structure that is immediately useful for podcasters. I also set max_tokens to 800 to ensure the model doesn't ramble, keeping the output concise and focused.
Practical Results and Trade-offs
Moving to edge inference yielded some interesting results. What worked exceptionally well was the latency. Because the inference happens at the edge, the time-to-first-token is noticeably faster, making the UI feel much snappier when generating episode descriptions or script outlines.
What surprised me was the capability of the 8B model. I initially worried that a smaller model would hallucinate resources or fail to capture the nuance of the episode summary. However, by strictly defining the output structure in the system prompt, the model stayed highly focused. It reliably extracted key takeaways without inventing facts, provided the transcript was clear.
Of course, there are trade-offs. If I had to do it over, I would change how the model handles timestamps. Currently, the AI estimates timestamps based on the content flow and text length. While it's a helpful rough guide, it isn't a perfect substitute for actual audio-aligned timestamps. I'm currently exploring ways to feed chunk-level metadata into the prompt to improve timestamp accuracy without sacrificing the speed of the edge pipeline.
Try the Free Title Generator
If you are currently struggling with the blank page syndrome for your next episode, I put together a free tool to help you get unstuck. It uses a similar edge-based approach to generate multiple title options based on your topic and desired tone, giving you a solid starting point so you can get back to recording.
You can try the free Podcast Title Generator here: https://podcrisp.com/free-tools/title-generator
Full disclosure: I'm the developer of PodCrisp. I built this because I needed it myself.
Top comments (0)