DEV Community

Cover image for Building Saaram: How I Built a Low-Resource Telugu News Summarizer with FastAPI, mT5, and Adaptive AI Routing
Hariharan Narlakanti
Hariharan Narlakanti

Posted on

Building Saaram: How I Built a Low-Resource Telugu News Summarizer with FastAPI, mT5, and Adaptive AI Routing

#python #nlp #ai #webdev

Saaram (సారం) is a production-ready AI platform for Telugu news summarization and text-to-speech built using FastAPI, mT5 transformers, morphology-aware TF-IDF, adaptive inference routing, and neural speech synthesis.

Saaram delivers deep abstractive summarization while strictly respecting the memory constraints of free-tier cloud infrastructure. The research behind this project was submitted to CIS 2026 (Springer LNNS).

Introduction

Building real-world AI applications is rarely about simply grabbing the largest model on Hugging Face. It is an exercise in practical software engineering: balancing accuracy, latency, memory footprints, cold-start delays, and cloud hosting budgets.

When I set out to build Saaram (సారం)—which translates to "essence" in Telugu—my goal was to solve a specific engineering problem in Indic Natural Language Processing (NLP): How can we deliver high-quality Telugu text summarization while guaranteeing 100% reliability on low-resource, free-tier CPU cloud infrastructure?

Saaram Web Platform

In this case study, I'll share the architectural design, the morphology-aware tokenization strategy, the adaptive routing heuristics, the benchmark results, and the production trade-offs that made Saaram possible.

Key Features

  • 📰 Multi-Mode Summarization: Summarize raw Telugu text, BBC Telugu news URLs, or live RSS feeds.
  • 🤖 Hybrid NLP Engine: Seamlessly combines statistical character $n$-gram TF-IDF with neural abstractive mT5 models.
  • 🧠 Adaptive Runtime Router: Automatically switches execution paths based on real-time RAM availability and text score variance.
  • 🔊 Saaram Radio (TTS): Streams neural Telugu audio across Full Bulletin, Brief Bulletin, and Headlines modes.
  • ⚡ Asynchronous FastAPI Service: Dockerized backend deployed on Hugging Face Spaces with lazy model loading.
  • 🛡️ Fault-Tolerant Fallback: Automatically degrades to statistical summarization if system memory is constrained, preventing 500 server errors.

The Problem: Why Indic NLP Needs Custom Engineering

Large Language Models (LLMs) have transformed text processing for English. However, applying standard off-the-shelf NLP tools to low-resource Dravidian languages like Telugu introduces two major engineering bottlenecks:

1. Telugu is Highly Agglutinative

In agglutinative languages, root words change significantly through affixation to express grammatical cases, tenses, and locations.

Consider how the Telugu root word రాష్ట్రం (state) inflects across different contexts:

  • రాష్ట్రం (state)
  • రాష్ట్రంలో (in the state)
  • రాష్ట్రానికి (to the state)

Standard word-level tokenizers treat these three inflected surface forms as completely unrelated dictionary entries. This fragments vocabulary frequency counts, causing traditional statistical models (like standard TF-IDF) to underestimate the importance of recurring core concepts.

2. Free Cloud Infrastructure Has Strict Memory Limits

Neural transformer models like mT5 generate fluent abstractive summaries, but they require substantial computational resources. Running inference on mT5 Base or fine-tuned variants consumes 540–800 MB of peak RAM.

On free-tier cloud hosting (such as CPU Docker containers on Hugging Face Spaces), concurrent user requests can easily trigger Out-Of-Memory (OOM) process crashes.

This forced a fundamental engineering question:

Can we build a hybrid system that delivers neural abstractive quality when resources are available, but gracefully degrades to a fast statistical fallback when system memory is constrained?

System Architecture

Saaram follows a decoupled, three-layer production architecture:

  • Frontend: React, Vite, Tailwind CSS, and Framer Motion (Hosted on Vercel)
  • Backend Service: FastAPI, Uvicorn, and PyTorch (Containerized via Docker on Hugging Face Spaces CPU)
  • Speech Engine: Microsoft Edge Neural TTS (te-IN-ShrutiNeural)

System Architecture

The React frontend communicates asynchronously with FastAPI endpoints. The backend orchestrates article extraction, text normalization, dynamic model routing, summary generation, and audio synthesis before returning structured JSON payloads.

The Hybrid Summarization Pipeline

Instead of forcing users to guess which model to run, Saaram exposes four distinct execution modes:

⚡ FAST — Morphology-Aware TF-IDF

Uses character $n$-gram tokenization (slice range of 2 to 4 characters) combined with a position-aware sentence weighting strategy that gives higher importance to early lead sentences.

  • Best for: Short news snippets, instant responses, and low-memory environments.
  • Inference Latency: Sub-5 milliseconds (< 0.005 seconds).

⚖️ BALANCED — Pretrained mT5 Base

Uses csebuetnlp/mT5_multilingual_XLSum to generate abstractive summaries with strong contextual understanding (BERTScore: 0.7273).

🎯 HIGH QUALITY — Fine-Tuned mT5 (8-bit LoRA)

A Parameter-Efficient Fine-Tuned (PEFT) checkpoint trained on Kaggle Tesla T4 GPUs using 8-bit quantization (bitsandbytes).

  • Low-Rank Adaptation (LoRA) adapter layers configured with rank 16 and alpha 32 were injected into self-attention Query and Value projections.
  • Configured only 1.77 million trainable parameters (representing just 0.3029% of the total 584M model weights), keeping memory overhead minimal.

🧠 AUTO — Adaptive Dynamic Routing

The core innovation of the application is the Adaptive Router. Instead of asking users to manually toggle models, the router inspects every incoming request using three dynamic heuristics:

  1. Available System Memory: Queries system memory using psutil. If available RAM drops below 400 MB, the router forces execution to TF-IDF V2 to prevent an OOM server crash.
  2. Input Word Count: Texts under 75 words route to TF-IDF V2 (summarization is trivial); texts over 200 words route to mT5.
  3. TF-IDF Score Variance: Computes sentence TF-IDF score contrast. A high variance (≥ 0.04) indicates clear, distinct lead sentences, routing to TF-IDF V2. A low variance (< 0.008) indicates homogeneous text requiring neural abstractive re-generation.

Saaram Execution Modes UI

Solving Agglutination with Character N-Grams

Standard word-level TF-IDF fails on Telugu because of affixation. To solve this without adding the latency of a heavy morphological analyzer, I implemented character $n$-gram tokenization using scikit-learn's analyzer='char_wb' with a range of 2 to 4 characters.

Here is how character $n$-grams bridge agglutinative inflections:

Word-Level TF-IDF (V1):
  "రాష్ట్రంలో"    <--- ZERO MATCH --->    "రాష్ట్రానికి"
  (Fragmented into 2 unrelated vocabulary entries)

Character N-Gram [2,4] (V2):
  "రాష్ట్రంలో"   ===[ Shared Root Slices: "రాష్", "రాష్ట్", "రాష్ట్రం" ]===>   "రాష్ట్రానికి"
  (Connects both inflected surface forms back to the core root concept!)
Enter fullscreen mode Exit fullscreen mode

In our evaluation, switching from word-level V1 to character $n$-gram V2 produced a 26.6% relative improvement in ROUGE-2 scores, proving that statistical methods can remain highly effective when tailored to specific language structures.

🎙️ Saaram Radio: Neural Speech Generation

Reading summaries on mobile screens isn't always convenient. Saaram integrates a text-to-speech pipeline that converts generated summaries into natural Telugu audio using Microsoft's Edge Neural TTS (te-IN-ShrutiNeural).

Users can choose between three audio bulletin modes:

  • Full Bulletin: Reads the complete summary text.
  • Brief Bulletin: Reads the headline plus the top two key sentences.
  • Headlines Mode: Reads a quick title checklist.

The audio is synthesized on the backend and streamed directly as an MP3 file to the custom frontend audio player.

Saaram Radio UI

Research Evaluation & Benchmark Results

We evaluated all five system configurations on the complete 1,302-sample XL-Sum Telugu test split.

Model ROUGE-1 ROUGE-2 ROUGE-L BERTScore BLEU METEOR Peak RAM
TF-IDF V1 (Word-level) 0.0709 0.0128 0.0554 0.6626 0.0060 0.0744 165.8 MB
TF-IDF V2 (Char N-gram) 0.0811 0.0162 0.0628 0.6671 0.0070 0.0879 165.8 MB
mT5 Base 0.1610 0.0478 0.1420 0.7273 0.0330 0.1180 544.8 MB
mT5 Fine-tuned (LoRA) 0.1639 0.0496 0.1452 0.7272 0.0336 0.1210 799.6 MB
Auto Router 0.1562 0.0458 0.1372 0.7241 0.0331 0.1158 164.8 MB

Key Research Lessons:

  • Documenting Negative Results: Paired statistical hypothesis testing (Wilcoxon signed-rank test, p = 0.887) showed that 8-bit LoRA fine-tuning produced almost identical BERTScore values to mT5 Base. One plausible explanation for this parity is that the base multilingual checkpoint (csebuetnlp/mT5_multilingual_XLSum) was already pre-trained on XL-Sum splits including Telugu, meaning additional in-distribution LoRA adaptation primarily refines surface phrasing rather than shifting contextual semantic representations.
  • Quality Retention: The Auto Router retained 99.56% of mT5 Base's semantic quality (BERTScore: 0.7241 vs. 0.7273) while routing requests away from heavy neural execution under memory constraints.

Engineering Challenges & Fault Tolerance

Deploying transformer models on free CPU instances introduces unique infrastructure hurdles:

  • Free-Tier Memory Limits: Transformer inference consumes up to 800 MB RAM.
  • Cold-Start Latency: Lazy model loading on CPU instances takes 15–30s on cold boot.
  • Fault-Tolerant Degradation: The application automatically falls back to TF-IDF whenever transformer memory limits are exceeded or checkpoint loads fail. Instead of throwing unhandled 500 server errors, the system flags the JSON payload with "degraded_mode": true and continues serving users cleanly.

Tech Stack Overview

  • Frontend: React, Vite, Tailwind CSS, Framer Motion (Vercel)
  • Backend Orchestration: FastAPI, Pydantic, Uvicorn (Hugging Face Spaces Docker)
  • Machine Learning: PyTorch, Hugging Face transformers, peft (LoRA), bitsandbytes, scikit-learn
  • Speech Engine: Microsoft Edge Neural TTS (te-IN-ShrutiNeural)

What I Learned

This project taught me far more than NLP algorithms. I gained practical experience with:

  • Designing asynchronous FastAPI architectures
  • Managing Transformer memory constraints in production
  • Quantized LoRA fine-tuning and subword tokenization
  • Empirical benchmark methodology and statistical significance testing
  • Building fault-tolerant web applications for low-resource languages

Perhaps the biggest lesson was this:

Engineering isn't about using the largest model. It's about making the right trade-offs for the problem you're solving.

Resources & Live Links

Final Thoughts

Saaram started as an effort to address Telugu NLP gaps and evolved into a complete research and engineering effort combining NLP, backend development, performance profiling, and system design.

If you have suggestions, questions, or ideas for improvement, I'd love to hear them in the comments below! If you enjoyed reading about the project, consider starring the GitHub repository—it helps others discover the project and supports low-resource Indic NLP development.

Thank you for reading!

Top comments (0)