DEV Community: Ardhansu Das

Integrating the OpenAI API the Right Way — Streaming, Rate-Limiting, and Prompt

Ardhansu Das — Wed, 08 Jul 2026 06:35:15 +0000

Integrating the OpenAI API the Right Way
Calling openai.chat.completions.create() is the easy 10% of building an AI-powered product. The hard 90% is making it feel instant, not breaking under load, and getting consistent output from a model that's fundamentally non-deterministic. I ran into all three of these while building an AI tool that generates formal letters from user input using GPT.
Here's what actually mattered.

Streaming isn't optional for UX If you're generating anything longer than a sentence, a blocking request feels broken — users stare at a spinner for 5-10 seconds with no feedback. Streaming turns that dead time into something that reads like the model is "thinking out loud," which massively changes perceived performance even though total latency is the same. javascriptconst stream = await openai.chat.completions.create({ model: "gpt-3.5-turbo", messages: [{ role: "user", content: prompt }], stream: true, });

for await (const chunk of stream) {
const token = chunk.choices[0]?.delta?.content || "";
process.stdout.write(token); // or push to a WebSocket/SSE connection
}
On the frontend, this pairs naturally with Server-Sent Events or a WebSocket connection so tokens render as they arrive instead of waiting for the full response.

Rate-limiting protects you, not just OpenAI It's tempting to think rate-limiting is only about respecting OpenAI's API limits. In practice, it's just as much about protecting your infrastructure and your bill from a single user (or a bug in your own retry logic) hammering the endpoint. A simple token-bucket approach per user goes a long way: javascriptconst buckets = new Map();

function allowRequest(userId, limit = 5, windowMs = 60_000) {
const now = Date.now();
const bucket = buckets.get(userId) || { count: 0, resetAt: now + windowMs };

if (now > bucket.resetAt) {
bucket.count = 0;
bucket.resetAt = now + windowMs;
}

if (bucket.count >= limit) return false;

bucket.count += 1;
buckets.set(userId, bucket);
return true;
}
For production, swap the in-memory Map for Redis so limits hold up across multiple server instances — an in-memory bucket resets the moment you scale horizontally.

Prompt templates need version control, not vibes Once you have more than two or three prompts, "tweak it until it feels right" stops scaling. What worked well was treating prompt templates as versioned, parameterized objects rather than inline strings scattered across the codebase: javascriptconst templates = { formalLetter: { version: "v3", system: "You are a professional correspondence writer. Be concise, formal, and grammatically precise.", build: ({ recipient, purpose, tone }) => Write a formal letter to ${recipient} regarding ${purpose}. Tone: ${tone}., }, // ...9 more templates }; This made it possible to A/B test prompt changes, roll back a regression instantly, and keep a clean diff history of why a prompt changed — something you lose completely when prompts are just strings buried in route handlers.
Treat the model's output as untrusted input This one's easy to skip until it bites you. Model output can include unexpected formatting, occasional hallucinated details, or — if you're not careful with your system prompt — instructions the user tried to sneak into the input. Always:

Sanitize before rendering (especially if output ever touches HTML)
Validate structure if you expect JSON (models drift; don't trust it blindly)
Set a max_tokens ceiling so a runaway generation doesn't blow your latency or cost budget

javascripttry {
const response = await openai.chat.completions.create({
model: "gpt-3.5-turbo",
messages,
max_tokens: 800,
});
const content = response.choices[0].message.content;
// sanitize/validate before using content
} catch (err) {
// handle rate limit (429), timeout, and malformed-response cases separately
}
The takeaway
None of this is exotic — it's the same discipline you'd apply to any third-party API. But it's easy to skip when a demo works fine with one user and no load. The gap between "works in a demo" and "works in production" for LLM features is almost entirely streaming, rate-limiting, and prompt versioning — not the model call itself.

I write about backend systems, applied AI, and the engineering details that don't make it into the docs.

Building an NLP Pipeline That Actually Understands Offer Text (with spaCy)

Ardhansu Das — Wed, 08 Jul 2026 06:34:21 +0000

Building an NLP Pipeline That Actually Understands Offer Text
Most "extract structured data from unstructured text" tutorials stop at named entity recognition and call it a day. In practice, that's maybe 30% of the problem. The real work is building a pipeline that can reliably pull out conditional information — things like "eligible only for orders above ₹999" or "valid till stocks last" — from messy, inconsistent text scraped off the web.
Here's how I approached this while building an NLP pipeline to extract eligibility conditions and offer insights from scraped promotional text.
The problem with naive NER
Out-of-the-box spaCy models are great at tagging ORG, MONEY, DATE, and PERCENT entities. But offer text isn't clean prose — it's fragments, abbreviations, and domain-specific phrasing:
"Flat 20% off* | Min order ₹499 | T&C apply | Valid on select cards only"
A generic model will happily tag 20% and ₹499 as entities, but it has no idea that one is a discount and the other is a threshold condition, or that "select cards only" is a restriction that changes the entire meaning of the offer.
Combining rule-based parsing with entity extraction
The fix isn't to throw more data at a bigger model — it's to combine spaCy's statistical NER with rule-based matching for domain-specific patterns:
pythonimport spacy
from spacy.matcher import Matcher

nlp = spacy.load("en_core_web_sm")
matcher = Matcher(nlp.vocab)

Pattern: "min order" / "minimum order" followed by a currency amount

min_order_pattern = [
{"LOWER": {"IN": ["min", "minimum"]}},
{"LOWER": "order", "OP": "?"},
{"IS_CURRENCY": True, "OP": "?"},
{"LIKE_NUM": True}
]
matcher.add("MIN_ORDER", [min_order_pattern])

def extract_conditions(text):
doc = nlp(text)
matches = matcher(doc)
conditions = []
for match_id, start, end in matches:
span = doc[start:end]
conditions.append(span.text)
return conditions
This gets you a huge accuracy boost on structured sub-phrases without needing a labeled dataset or fine-tuning a transformer.
Where entity extraction still earns its keep
Rule-based matching handles known patterns well, but you still need statistical NER for the long tail — brand names, card issuer names, dates written in inconsistent formats. The trick is layering them:

Rule-based matcher — catches conditions, thresholds, restrictions (high precision, domain-specific)
Statistical NER — catches entities you can't enumerate in advance (brands, dates, amounts)
Post-processing merge — resolves overlaps and attaches conditions to the entities they modify

pythondef parse_offer(text):
doc = nlp(text)
entities = [(ent.text, ent.label_) for ent in doc.ents]
conditions = extract_conditions(text)
return {
"entities": entities,
"conditions": conditions,
"raw": text
}
Lessons that don't show up in the docs
A few things that saved a lot of debugging time:

Normalize before you tag. Scraped text has inconsistent casing, stray HTML entities, and unicode symbols like ₹ that trip up tokenizers if you don't clean them first.
Version your matcher patterns. As you scrape more sources, you'll keep discovering new phrasings. Treat your rule set like code — test it, don't just append to it.
Precision over recall for conditions. A false positive (extracting a wrong condition) is worse than a missed one, since downstream systems often act on these conditions automatically.

Why this matters beyond offers
The same layered approach — statistical NER for open-ended entities, rule-based matching for domain patterns, and a merge step to reconcile them — generalizes to almost any "extract structured facts from semi-structured text" problem: parsing job postings, extracting clauses from contracts, or pulling structured fields out of support tickets.
If you're building something similar, I'd genuinely recommend starting with the rule-based layer before reaching for a fine-tuned model. It's faster to iterate on, easier to debug, and often gets you 80% of the way there for a fraction of the engineering cost.

I write about backend systems, NLP, and applied ML. Follow along for more.

Why Data Structures Still Matter When You're Doing Machine Learning

Ardhansu Das — Wed, 08 Jul 2026 06:32:50 +0000

It's easy to assume DSA is "just for interviews" once you're writing model.fit() and letting a library handle the rest. But the moment your ML system has to run on real, large-scale, latency-sensitive data — not a clean Kaggle CSV — data structures and complexity analysis stop being theoretical and start being the difference between a system that scales and one that falls over.
A few places where this showed up directly in projects I've built.

Preprocessing at scale is a data structures problem Feature engineering on large datasets is full of classic DSA patterns hiding in plain sight:

Deduplication of scraped or user-submitted records → hash sets, O(n) instead of O(n²) pairwise comparison
Sliding-window features (rolling averages, time-based aggregations) → deques for O(1) amortized window updates instead of recomputing from scratch
Frequency-based features (word counts, category counts for encoding) → hash maps, but watch out for high-cardinality categorical features blowing up memory

pythonfrom collections import deque

def rolling_average(stream, window_size):
window = deque(maxlen=window_size)
for value in stream:
window.append(value)
yield sum(window) / len(window)
A naive rolling average recomputed from a slice on every step is O(n·k). The deque version is O(n) total. On a dataset with millions of rows, that's the difference between a preprocessing job that finishes in minutes versus one that doesn't finish at all.

Graphs show up more than you'd expect
Any time your data has relationships — user-item interactions, dependency chains, entity co-occurrence — you're working with a graph whether you call it that or not. Recommendation systems, fraud-detection pipelines, and even NLP dependency parsing all lean on graph traversal and shortest-path concepts:
pythondef bfs_connected_component(graph, start):
visited = {start}
queue = deque([start])
component = []

while queue:
node = queue.popleft()
component.append(node)
for neighbor in graph[node]:
if neighbor not in visited:
visited.add(neighbor)
queue.append(neighbor)

return component
Understanding why BFS is O(V + E) — not just being able to write it — matters when you're deciding whether a graph-based feature is even feasible to compute on a graph with millions of edges.
Complexity analysis decides your architecture, not just your code
This is the part that's easy to underrate. Knowing Big-O isn't just about passing a coding interview — it directly informs decisions like:

Whether to precompute and cache a similarity matrix (O(n²) space) or compute similarities on demand
Whether a k-nearest-neighbors lookup needs an approximate structure (KD-tree, ball tree, or an ANN library) once your dataset crosses a certain size, because brute-force O(n) search per query stops being viable
Whether a batch job's O(n log n) sort-and-merge step will actually finish inside your maintenance window as data grows

None of these are "algorithm interview" questions. They're architecture decisions that determine whether your ML pipeline runs in 10 minutes or 10 hours next quarter when the dataset doubles.

A concrete example: efficient top-k without sorting everything A common ML pipeline task — get the top-k highest-scoring items from a large candidate set — gets solved wrong surprisingly often: pythonimport heapq

def top_k(candidates, k):
# candidates: list of (score, item) tuples
return heapq.nlargest(k, candidates, key=lambda x: x[0])
Sorting the entire candidate list is O(n log n). A heap-based nlargest is O(n log k) — a meaningful difference when n is in the millions and k is small, which is exactly the shape of most recommendation and ranking problems.
The takeaway
Libraries like scikit-learn and TensorFlow abstract away a lot, but they don't abstract away the cost of how you get data into and out of them. The preprocessing, feature engineering, and serving layers around a model are still ordinary software engineering — and that's where DSA fundamentals quietly decide whether your ML system is production-ready or just a notebook that happens to work on a laptop.

I write about backend systems, applied ML, and the engineering fundamentals that hold it all together.

🧠 Deep Learning: Or How I Learned to Stop Worrying and Love the Matrix Multiplication

Ardhansu Das — Mon, 12 May 2025 06:52:08 +0000

Welcome to the magical, GPU-heated world of Deep Learning — where we teach machines to think, sort of, by throwing math, data, and a terrifying number of layers at them until they give us something cool like a cat detector or ChatGPT.

Let’s be honest: deep learning sounds like something you'd do during therapy. But no — it’s just machine learning’s more expensive, more dramatic sibling who needs a Tesla V100 GPU to feel alive.

🤖 What Is Deep Learning?
Deep Learning is like that student who didn’t pay attention all semester but still aces the final because they “intuitively figured it out.” Instead of writing rules, we throw data at neural networks and let them figure things out on their own. And somehow, they do. Sort of.

At the heart of it are neural networks, which are vaguely inspired by the human brain — if your brain only did matrix multiplication and silently judged your batch size choices.

*🧱 The Building Blocks of a Modern Deep Learning Breakdown:
*🔹 1. Neurons and Layers
A neuron in deep learning is a function that says: “Give me a number, and I’ll pass it through this mystical activation function until it looks fancy.”

Stack these neurons into layers. Stack those layers into models. Stack those models into an identity crisis when the loss doesn't converge.

🔹 2. Activation Functions
You’ve probably heard of ReLU — the one that just zeroes out negative numbers like a savage. There’s also sigmoid and tanh, which are great if you're nostalgic for the 90s and vanishing gradients.

🔹 3. Loss Function
This is literally the model’s “How wrong am I?” function. The goal is to minimize loss, but most of the time it just minimizes your will to debug.

🔹 4. Backpropagation
Imagine teaching a dog to sit by yelling “no” every time it gets it wrong, but instead of a dog it’s math, and instead of “no” it’s derivatives.

**🧪 So, What Have I Done With Deep Learning?
**Oh, you know — just the usual:

Built a face mask detection system using ResNet50, because if you’re not using a heavyweight model to check for tiny strips of fabric on faces, are you even doing deep learning?

Fiddled with OpenCV until my webcam gave me PTSD.

Watched training loss go down like my hopes and dreams… only to watch validation accuracy crash like my laptop running 50 epochs.

**🧩 Why Is Deep Learning So Hard Yet Addictive?
**Because there's always that 1% chance that after 7 hours of training and 8 Red Bulls, your model might actually work. And when it does? You feel like an AI god. Until it classifies a banana as a gun. Again.

**📈 Final Thoughts (Because This Blog Needs a Conclusion)
**Deep learning is a lot like dating: it requires patience, constant tweaking, and sometimes ends in heartbreak because “the weights didn’t align.” But if you keep at it, feed it enough data, and don’t mind being ghosted by your GPU, it can actually do some amazing things.

So, here’s to the next model. The next dataset. The next long night spent tuning hyperparameters only to realize… you forgot to normalize your inputs. Again.

Stay caffeinated, stay curious, and may your gradients never vanish.

AI Meets Safety: Real-Time Face Mask Detection with Python

Ardhansu Das — Tue, 06 May 2025 17:44:24 +0000

In the post-pandemic world, face masks have become essential. As a Full-Stack Developer exploring AI and Computer Vision, I decided to build a real-time face mask detection system using Python, OpenCV, and Deep Learning.

Here’s a breakdown of how I built it and what I learned.

🚀 Project Overview
The project captures live video from your webcam and uses a trained deep learning model (MobileNetV2) to determine if a person is wearing a mask or not. The model then overlays a label on the video feed in real-time.

🧠 Tools & Technologies Used
Python

OpenCV – for video capture and image processing

TensorFlow/Keras – for training the model

MobileNetV2 – a lightweight CNN model for fast inference

Matplotlib & NumPy – for data analysis and preprocessing

🛠️ How It Works
Data Collection
I used a dataset with two categories: with mask and without mask. You can find such datasets on Kaggle or build your own.

Model Training
I used MobileNetV2 as the base model and fine-tuned it with our dataset. After several epochs, the model reached over 95% accuracy.

Face Detection
Using OpenCV’s built-in face detector (cv2.CascadeClassifier), we detect faces in each video frame.

Prediction & Display
Each detected face is passed to the model. Based on the output, a green label (“Mask”) or red label (“No Mask”) is drawn on the frame.

📦 Key Learnings
Real-time applications require optimization — MobileNetV2 helped due to its lightweight architecture.

OpenCV is powerful for computer vision tasks, and integrating it with deep learning opens up many possibilities.

Proper dataset preparation (balanced, augmented, preprocessed) significantly boosts model performance.

📁 GitHub Repository
You can find the full code and instructions

ardhansu / Face-Mask-Detection

📌 Final Thoughts
This was my first step into combining computer vision and deep learning for something impactful. It’s exciting to see how AI can be used in practical, real-time applications like this.

I’d love to hear your feedback! Let me know if you build on this or have suggestions for improvements. 👇