DEV Community: elizabeththomas7

From Piper to Polly: How I Built a Production-Ready Text-to-Speech API (and Everything That Broke Along the Way)

elizabeththomas7 — Thu, 14 May 2026 04:30:13 +0000

A walkthrough of building a voice AI backend — through three TTS providers, a chunking problem, Redis caching, distributed locks, and a thundering herd.

The Idea

I wanted to read long articles without staring at a screen. The concept was simple: paste an article, get back an MP3. Building it turned out to be an education in the real-world constraints of TTS APIs — character limits, latency, cost, and what happens when 50 users click Play on the same article at the same moment.

Here's the full journey, told through the architecture decisions that actually mattered.

Iteration 1 — Piper TTS: Free, Local, and Immediately Limiting

The first version ran Piper — an open-source, offline neural TTS engine. You spin up a process, feed it text, get back a WAV file. No API keys, no cost, no network round-trips.

What worked: It ran entirely on my machine. Zero latency on credentials. Perfect for prototyping.

What broke: Piper is a local binary. It has no concept of concurrency — one synthesis job at a time. Voice quality, while decent, was noticeably robotic on longer prose. And crucially, the model files are large. Deploying this to a server meant bundling hundreds of megabytes of model weights and a native binary per target platform.

The real killer was the character limit. Piper (like all neural TTS systems) struggles with very long inputs. A full 2000-word article would either fail silently or produce garbled audio near the end. That problem — long text — became the thread I'd keep pulling on through every subsequent iteration.

The exit criterion: I needed a hosted API with a predictable quality ceiling and a clear path to production.

Iteration 2 — ElevenLabs: Great Voice, Brutal Cost at Scale

ElevenLabs produces genuinely impressive voice output. The SDK is well-designed:

from elevenlabs.client import ElevenLabs

client = ElevenLabs(api_key=api_key, timeout=timeout_sec)
audio_iter = client.text_to_speech.convert(
    voice_id,
    text=text,
    model_id=model_id,
    output_format=output_format,
)
data = b"".join(audio_iter)

You stream back an iterator of bytes and concatenate. Clean, fast, and the voice quality is excellent.

What worked: Plug-and-play integration. The voices sound human. Developer experience is top-tier.

What broke: The free tier evaporates fast. A single medium-length article at 1500 characters per minute of audio burns through credits quickly. If you're building something for real users — or even testing seriously — the cost curve is steep.

There was also the same character-limit problem: ElevenLabs has a per-request text limit. A long article needs to be split before you even call the API. I'd deferred solving this from the Piper days, but here it became unavoidable.

The exit criterion: I needed either cheaper synthesis or a way to make the expensive calls worth their cost. That meant solving chunking first, then caching second.

The Architecture Pivot: Chunking the Article

Before switching providers, I had to solve the fundamental problem: a full article can be 10,000+ characters, and every TTS provider has a per-request limit (Amazon Polly's is 3,000 characters for standard voices, for example).

The solution is a text chunker that splits on sentence boundaries — never in the middle of a sentence — and targets a configurable chunk size:

_SENTENCE_RE = re.compile(r"(?<=[.!?])\s+")

def chunk_text(text: str, target_chars: int = 2500, max_chars: int = 4000) -> list[str]:
    text = re.sub(r"\s+", " ", text).strip()
    sentences = _SENTENCE_RE.split(text)
    chunks: list[str] = []
    buf: list[str] = []
    size = 0

    for sentence in sentences:
        add = len(sentence) + (1 if buf else 0)
        if size + add > max_chars and buf:
            chunks.append(" ".join(buf))
            buf = [sentence]
            size = len(sentence)
        else:
            buf.append(sentence)
            size += add
        if size >= target_chars:
            chunks.append(" ".join(buf))
            buf = []
            size = 0

    if buf:
        chunks.append(" ".join(buf))
    return chunks

Two thresholds, not one:

target_chars (default 2500): the soft target. Once a chunk reaches this, close it and start a new one.
max_chars (default 4000): the hard ceiling. If the next sentence would push past this, flush first even if target_chars hasn't been reached.

This means every chunk is a coherent set of complete sentences, never a mid-sentence cut, and always within the provider's hard limit.

Once you have chunks, you synthesize each one independently and stitch the resulting MP3 files together with ffmpeg's concat demuxer.

Iteration 3 — Amazon Polly: The Right Economics

With chunking solved, I switched the synthesis backend to Amazon Polly. The economics are hard to beat:

Standard voices: 5 million characters/month, free, permanently.
Neural voices: 1 million characters/month free for the first 12 months, then pay-as-you-go.

For a personal reading assistant or a low-to-medium traffic app, the standard tier is effectively free forever.

The full request flow at this point:

POST /tts/from-text  { "text": "..." }
│
▼
chunk_text()  →  [chunk_0, chunk_1, chunk_2, ...]
│
▼  (for each chunk)
polly.synthesize_speech()  →  chunk_N.mp3
│
▼
ffmpeg concat  →  combined.mp3
│
▼
FileResponse  (streamed back to client, temp dir cleaned up in background)

This works. A 3000-word article (~18,000 characters) splits into roughly 7 chunks. Each Polly call takes 0.5–2 seconds. Total latency: 4–12 seconds depending on network and chunk count.

The problem with this: every request re-synthesizes everything from scratch. The same New York Times article, requested by 100 different users, triggers 700 Polly calls. That's wasteful, slow, and eventually expensive.

Iteration 4 — Redis Cache: Stop Paying for the Same Sentence Twice

The insight: text is deterministic. The same input sentence will always produce the same audio bytes from the same voice/engine/region combination. This is a perfect caching problem.

The cache key encodes everything that affects the output. Including voice ID, engine, and region in the key means that if you switch from Joanna/standard to Matthew/neural, you automatically get cache misses — you never accidentally serve audio from the wrong voice.

The loop with Redis, before locking:

for i, chunk in enumerate(chunks):
    h = hash_chunk(chunk)
    key = _redis_chunk_key(chunk_hash=h)
    cached = r.get(key)

    if cached:
        hits += 1
        mp3_path.write_bytes(cached)   # cache hit: instant
    else:
        misses += 1
        mp3_path = polly_synth_chunk_mp3(...)  # cache miss: call Polly
        r.set(key, mp3_path.read_bytes(), ex=cache_ttl_sec)

This is dramatically better than nothing. A second request for the same article is now pure cache reads — sub-100ms total instead of 4–12 seconds.

But there's a race condition hiding here.

Iteration 5 — The Thundering Herd Problem

Imagine a popular article gets published. Fifty users open it and click Play simultaneously. Here's what happens without locking:

All 50 requests call r.get(key) — all get None (cold cache).
All 50 requests call polly.synthesize_speech() for the exact same chunks.
All 50 requests write the same bytes to Redis.
You just made 350 redundant Polly calls (50 users × 7 chunks) and wasted 349/350ths of them.

It's expensive, it stresses the upstream API, and it can push you into Polly's throttling limits.

The fix is a distributed synthesis lock in Redis.

Iteration 6 — Redis Distributed Lock: One Synthesis Per Chunk

The pattern: before calling Polly, try to atomically acquire a lock on the synthesis of that chunk. Only one worker wins the lock. Everyone else waits for the winner to finish and populate the cache.

lock_key = f"{cache_key}:synth-lock"
lock_ttl = max(180, int(polly_timeout * 2) + 30)  # generous TTL
got_lock = r.set(lock_key, b"1", nx=True, ex=lock_ttl)

nx=True means "only set if not exists" — this is atomic in Redis. Exactly one caller gets True; all others get None.

The full per-chunk decision tree:

┌─────────────────────────────────────────────┐
│  r.get(cache_key)                           │
│  ├── HIT  → use cached bytes, continue      │
│  └── MISS → try to acquire synth-lock       │
│             ├── GOT LOCK                    │
│             │   → call Polly                │
│             │   → write result to cache     │
│             │   → release lock              │
│             └── LOCK HELD BY ANOTHER        │
│                 → wait with exponential     │
│                   backoff for cache to      │
│                   populate                  │
│                 ├── cache appeared → HIT    │
│                 └── timeout → try lock      │
│                     once more, or 503       │
└─────────────────────────────────────────────┘

The wait function uses exponential backoff with a cap:

def _redis_wait_for_chunk(r, value_key, *, deadline_monotonic):
    backoff = 0.05
    max_backoff = 0.5
    while time.monotonic() < deadline_monotonic:
        data = r.get(value_key)
        if data:
            return data
        time.sleep(backoff)
        backoff = min(max_backoff, backoff * 1.25)
    return None

Start polling at 50ms, grow by 25% each iteration, cap at 500ms. This keeps Redis query volume low while still responding promptly when the synthesis finishes.

The full route handler handles all three outcomes:

for i, chunk in enumerate(chunks):
    h = hash_chunk(chunk)
    key = _redis_chunk_key(chunk_hash=h)

    # 1. Cache hit - instant return
    cached = r.get(key)
    if cached:
        hits += 1
        mp3_path.write_bytes(cached)
        continue

    # 2. Try to become the synthesizer
    lock_key = _redis_synth_lock_key(chunk_cache_key=key)
    got_lock = r.set(lock_key, b"1", nx=True, ex=lock_ttl)
    if got_lock:
        misses += 1
        try:
            mp3_path = polly_synth_chunk_mp3(text=chunk, ...)
            b = mp3_path.read_bytes()
            if len(b) <= max_cached_bytes:
                r.set(key, b, ex=cache_ttl_sec)
        finally:
            r.delete(lock_key)   # always release, even on error
        continue

    # 3. Someone else holds the lock - wait for their result
    deadline = time.monotonic() + lock_ttl + wait_extra_sec
    waited = _redis_wait_for_chunk(r, key, deadline_monotonic=deadline)
    if waited:
        hits += 1
        mp3_path.write_bytes(waited)
        continue

    # 4. Wait timed out - try to acquire lock one more time
    if r.set(lock_key, b"1", nx=True, ex=lock_ttl):
        # ... synthesize and cache (same as case 2)
        continue

    # 5. Still locked after full wait - give up gracefully
    raise HTTPException(status_code=503, detail="TTS busy synthesizing this segment; retry shortly.")

The finally: r.delete(lock_key) is the most important line. Whether Polly succeeds, errors, times out, or raises an exception, the lock is released. Without this, a failed synthesis leaves the lock held until TTL expiry, blocking all subsequent requests for that chunk for potentially minutes.

Handling Scale: The Full Picture

With caching and locking in place, the behavior under load becomes predictable.

Warm cache (article seen before):
All chunks are in Redis. Every request is N × r.get() + ffmpeg concat + FileResponse. Latency drops to under 300ms for most articles. No Polly calls at all.

Cold cache, 50 simultaneous users (thundering herd):

1 request wins the lock per chunk → calls Polly, writes to cache, releases lock.
49 requests wait on _redis_wait_for_chunk → find cached bytes as soon as the winner finishes.
Total Polly calls: N chunks (7 for our example), not 50 × N = 350.
You can verify this in logs: chunk cache stats hits=49 misses=1 per chunk.

Memory guard:

if len(b) <= max_cached_bytes:
    r.set(key, b, ex=cache_ttl_sec)

Chunks larger than MAX_CACHED_CHUNK_BYTES (default 5MB) are synthesized but not cached. A pathologically long chunk from unusual input won't fill Redis memory.

The Final Architecture Diagram

Client
│  POST /tts/from-text  { text: "..." }
▼
FastAPI  (backend/main.py)
│
├── chunk_text()  →  [chunk_0 .. chunk_N]
│                    (sentence-boundary splitting)
│
└── for each chunk:
      │
      ├── SHA-256 hash  →  cache key
      │
      ├── Redis GET
      │   ├── HIT  →  write bytes to disk
      │   └── MISS
      │         ├── SET NX (acquire synth lock)
      │         │   ├── GOT LOCK
      │         │   │   → Amazon Polly synthesize_speech()
      │         │   │   → write MP3 bytes to disk
      │         │   │   → Redis SET (cache result, 30-day TTL)
      │         │   │   → Redis DEL (release lock)
      │         │   └── LOCK HELD
      │         │       → exponential backoff poll
      │         │       → cache appeared → write bytes to disk
      │         │       → timeout → retry lock → 503
      │
      └── ffmpeg concat  →  combined.mp3
            │
            └── FileResponse  (audio/mpeg)
                background: shutil.rmtree(tmp)

What I'd Do Differently

Async synthesis. The current implementation is synchronous — the HTTP request blocks until all Polly calls return and ffmpeg finishes. For a public API, I'd move to a job queue (Celery, ARQ, or even a simple Redis list): accept the article, return a job ID immediately, poll or subscribe for the result. This eliminates timeout risk on slow connections.

Streaming audio. Instead of waiting for all chunks before returning, you can stream chunk_0 to the client while chunk_1 is still synthesizing. This cuts perceived latency significantly for long articles.

Persistent cache storage. Redis in-memory is fast but expensive per GB at scale. For audio bytes that are valid for months, consider offloading cached chunks to S3 or R2 (using Redis only for the lock and a pointer/URL, not the raw bytes).

Key Takeaways

All TTS providers have character limits. Design your chunker before you pick a provider, not after.
Text synthesis is deterministic. The same text from the same voice always produces the same bytes. Cache aggressively.
Cache keys must include all synthesis parameters. Voice ID, engine, and region are part of the key — not just the text hash.
The thundering herd is real. Without a distributed lock, a cold-cache spike causes N × concurrent_users upstream calls. Redis SET NX is the right primitive for this.
Always release locks in finally blocks. A failed synthesis that doesn't release its lock blocks every subsequent request for that chunk until TTL expiry.

What Reddit Can Teach Us About Women’s Watch Preferences (Python + NLP Project)

elizabeththomas7 — Fri, 08 May 2026 07:05:09 +0000

Most “what watch should I buy?” discussions online skew heavily male. A friend wanted to launch a women’s watch, so I helped with a small data analysis.

In this post I’ll walk through a small but complete Python pipeline I built:

Scrape relevant posts and comments from Reddit with no API keys
Filter out irrelevant posts (e.g. men asking for themselves)
Run NLP analysis: sentiment, brands, features, prices, keywords, clustering, topic modeling
Generate visualizations and CSVs you can explore further

Everything here is powered by standard Python libraries: requests, pandas, nltk, scikit‑learn, and wordcloud.

1. Collecting Reddit data without API keys

We didn’t use the official Reddit API; instead we hit the public JSON endpoints directly using requests.

At the top of reddit_json_scraper.py we define search URLs across multiple subreddits:

Each URL returns a JSON blob; we wrap that in a helper:

Reddit’s listing JSON has a fairly nested structure, so we created extract_post_data to normalize it into a flat dictionary with the fields we actually care about (ID, subreddit, title, body, score, comment count, timestamps, etc.):

def extract_post_data(post_json):
    """
    Extract relevant information from a Reddit post JSON
    """
    try:
        data = post_json['data']

        return {
            'post_id': data.get('id', ''),
            'subreddit': data.get('subreddit', ''),
            'title': data.get('title', ''),
            'text': data.get('selftext', ''),
            'author': data.get('author', ''),
            'score': data.get('score', 0),
            'upvote_ratio': data.get('upvote_ratio', 0),
            'num_comments': data.get('num_comments', 0),
            'created_utc': data.get('created_utc', 0),
            'created_date': datetime.fromtimestamp(data.get('created_utc', 0)).strftime('%Y-%m-%d %H:%M:%S'),
            'url': f"https://reddit.com{data.get('permalink', '')}",
            'post_url': data.get('url', ''),
            'is_video': data.get('is_video', False),
            'over_18': data.get('over_18', False)
        }

The main collection loop simply iterates through all search URLs, fetches JSON, and appends normalized posts into a list:

We also fetch comments for the most “interesting” posts, sorted by engagement (score + num_comments), by hitting each post’s .json endpoint and walking the comment tree.

At the end of main() we save everything to CSV and run a quick text summary (brand and keyword counts, simple price stats).

2. Filtering: keeping posts that are really about women’s watches

Search results are noisy. Some posts mention “women” but are actually men asking for themselves.
filter_posts.py applies a simple but effective regex filter.We flag posts that contain phrases like “as a man” or “for men”:

…but we keep posts that clearly talk about buying for a woman, e.g. “gift for my wife”:

NON_FILTER_PATTERNS=r"(for|gift|buying|getting|choosing|help).*(mum|mom|mother|wife|girlfriend|partner|daughter|sister|woman|female|her|she)"

filter_check combines title and text, applies these patterns, and filtered_posts_csv writes a cleaned filtered_posts.csv. This becomes the starting point for our analysis.

3. Analyzing the conversations with WatchDataAnalyzer

The main analysis lives in watch_analyzer.py as a single class:

Load the filtered posts and comments
Combine titles, bodies, and comment text into all_text
Set up NLTK and VADER sentiment

3.1. Light text cleaning

We remove URLs and normalize whitespace, then build a combined_text column per post:

3.2. Sentiment on posts and comments

Using VADER, we compute a compound score and label each post/comment as positive, neutral, or negative:

self.posts_df['sentiment_scores']=self.posts_df['combined_text'].apply(lambda x: self.sia.polarity_scores(x))
self.posts_df['sentiment_compound']=self.posts_df['sentiment_scores'].apply(lambda x: x['compound'])
self.posts_df['sentiment_label']=self.posts_df['sentiment_compound'].apply(
    lambda x: 'positive' if x>0.05 else ('negative' if x<-0.05 else 'neutral')
)

We do the same for comments and then plot the distribution, saving sentiment_dist.png.

3.3. Brands, price ranges, and features

We look at three practical angles:

Brand mentions — a curated list from Titan and Seiko to Rolex and Omega, counted across all text.

def extract_brands(self):
    # Common watch brands
    brands = [
        'casio', 'seiko', 'citizen', 'timex', 'fossil', 'orient', 'tissot',
        'michael kors', 'daniel wellington', 'mvmt', 'skagen', 'swatch',
        'rolex', 'omega', 'cartier', 'tag heuer', 'breitling', 'patek philippe',
        'audemars piguet', 'vacheron constantin', 'baume mercier', 'longines',
        'hamilton', 'bulova', 'invicta', 'bering', 'titan', 'fastrack',
        'sonata', 'maxima', 'hmt', 'raymond weil', 'zenith', 'iwc'
    ]
    brand_mentions={}
    # ...

Price — regexes to capture Indian price patterns with ₹/rs/inr or “rupees”, then bucketed into budget/mid‑range/premium/luxury ranges.

def extract_prices(self):
    # Patterns for price extraction
    patterns = [
        r'(?:₹|rs\.?|inr)\s*(\d+(?:,\d{3})*(?:\.\d+)?)',
        r'(\d+(?:,\d{3})*(?:\.\d+)?)\s*(?:₹|rs\.?|inr)',
        r'(\d+(?:,\d{3})*(?:\.\d+)?)\s*(?:rupees|rupee)',
    ]

    all_prices=[]

    for text in self.all_text:
        for pattern in patterns:
            matches=re.findall(pattern,str(text),re.IGNORECASE)
            for match in matches:
                price_str=match.replace(',','').replace('.','')
                all_prices.append(int(price_str))

    ranges = {
            'Budget (<₹5,000)': sum(1 for p in all_prices if p < 5000),
            'Mid-range (₹5,000-₹20,000)': sum(1 for p in all_prices if 5000 <= p < 20000),
            'Premium (₹20,000-₹1,00,000)': sum(1 for p in all_prices if 20000 <= p < 100000),
            'Luxury (>₹1,00,000)': sum(1 for p in all_prices if p >= 100000)
        }

Features — categories like size, material, movement, style, strap, and “features” (water resistance, sapphire, chronograph, etc.), each with their own keyword list.

This gives a quick picture of which brands dominate, what price bands people discuss, and which attributes come up most.

3.4. Keywords, clusters, and topics

Using scikit‑learn:

TF‑IDF keywords — we build a TfidfVectorizer over combined_text
and save the top terms to keywords_tfidf.csv.

def extract_keywords(self):
    self.preprocess_all_text()

    vectorizer=TfidfVectorizer(
        max_features=80,
        stop_words='english',
        min_df=2
    )

    texts=self.posts_df['combined_text'].fillna('').tolist()
    X=vectorizer.fit_transform(texts)

    feature_names=vectorizer.get_feature_names_out()

    scores = X.mean(axis=0).A1

    # Create keyword dataframe
    keywords_df = pd.DataFrame({
        'keyword': feature_names,
        'tfidf_score': scores
    }).sort_values('tfidf_score', ascending=False)

Clustering — we cluster posts into 5 groups using K‑Means over TF‑IDF vectors, then inspect top words per cluster.

def cluster_posts(self, n_clusters=5):
    """
    Cluster posts based on text similarity
    """
    # ...
    vectorizer = TfidfVectorizer(
        max_features=50,
        stop_words='english',
        min_df=2
    )

    texts = self.posts_df['combined_text'].fillna('').tolist()
    X = vectorizer.fit_transform(texts)

    # K-means clustering
    kmeans = KMeans(n_clusters=n_clusters, random_state=42, n_init=10)
    clusters = kmeans.fit_predict(X)

    self.posts_df['cluster'] = clusters

Topic modeling — we run LDA/NMF over the same vectors to discover high‑level themes (“budget gifts”, “small wrists and office wear”, “sporty/outdoor”, etc.).

def topic_modeling(self, n_topics=5, method='lda'):
    """
    Perform topic modeling using LDA or NMF
    """
    # ...
    vectorizer = TfidfVectorizer(
        max_features=100,
        stop_words='english',
        ngram_range=(1, 2),
        min_df=2,
        max_df=0.95
    )
    # ...
    if method.lower() == 'lda':
        model = LatentDirichletAllocation(n_components=n_topics, random_state=42)
    else:  # NMF
        model = NMF(n_components=n_topics, random_state=42)
    # ...
    for idx, topic in enumerate(model.components_):
        top_words = [feature_names[i] for i in topic.argsort()[-10:][::-1]]
        print(f"\n   Topic {idx + 1}: {', '.join(top_words)}")

4. Putting it all together

The generate_report() method runs the full pipeline:

Preprocess text
Run sentiment, brand/feature/price extraction
Compute keywords, clusters, and topics
Generate a word cloud and sentiment plot
Save everything to CSVs you can open in Excel or a notebook

def generate_report(self):

    # 1. Pre- processing the text 
    self.preprocess_all_text()

    # 2. Analyze sentiment
    sentiment_df=self.analyze_sentiment()

    # 3. Brands mentions
    brands=self.extract_brands()

    # 4. Features
    features = self.extract_features()

    # 5. Prices
    prices = self.extract_prices()

    # 6. Keywords
    keywords_df = self.extract_keywords()

    # 7. Clustering
    clusters = self.cluster_posts(n_clusters=5)

    # 8. Topic Modeling
    print("\n🔍 Running topic modeling (this may take a moment)...")
    topic_model, vectorizer = self.topic_modeling(n_topics=5, method='lda')

    # 9. Visualizations
    print("\n🎨 Creating visualizations...")
    self.create_wordcloud('wordcloud.png')
    self.plot_sentiment_distribution('sentiment_dist.png')

    # Save results ...

It’s a compact example of how to go from raw Reddit JSON to structured insights about a very specific question: what are people really saying when they talk about women’s watches ?