DEV Community: Sidharth Devaraj

Don't Skip DSA: A Real-World Optimisation Win

Sidharth Devaraj — Fri, 29 Aug 2025 10:42:22 +0000

There's a common sentiment in our web and mobile development world:

"Do I really need Data Structures and Algorithms (DSA)? Isn't that just for MAANG interviews?"

My experience says yes.

I want to share a story from one of my projects where a fundamental understanding of DSA saved my day in production.

The Problem: Optimising Our Cash Pool

I was working on an application where we managed invoice payments. We had a specific, real-world optimisation problem:

"Given a fixed pool of cash, how do we pay as many outstanding invoices as possible to maximise the total amount paid, without exceeding the available cash?"

Sounds like a fairly standard business requirement, right?

At first glance, one might start thinking about complex loops, sorting strategies, or even greedy approaches. Many developers might jump straight into writing custom, bespoke logic.

The "Aha!" Moment: Pattern Recognition

But as I looked at the problem, something clicked. I realised this wasn't a unique, never-before-seen challenge.

This was a classic instance of the Knapsack Problem.

Let's break down why:

The "Knapsack": Our fixed cash pool
The "Items": Each individual invoice
The "Weight" / "Value": The amount of each invoice (since our goal was to maximise the total amount paid, the invoice amount served as both its "weight" in the budget and its "value" towards our goal)
The Constraint: Invoices had to be paid in full (0/1 choice – either an invoice is paid or it's not; no partial payments)

The Solution: Leveraging Proven Algorithms

Once I recognised this underlying pattern, the path forward became incredibly clear.

Instead of spending hours or days trying to invent a new, possibly inefficient algorithm, I knew I could leverage well-established solutions for the Subset Sum Problem.

This meant:

Speed of Development: I wasn't reinventing the wheel. I could recall or quickly look up the optimal approaches for this class of problem.
Efficiency and Reliability: I could implement an algorithm that was already proven for its performance characteristics, ensuring our cash allocation was truly optimised.
Maintainability: Future developers (or even myself later) would recognise the pattern and the algorithmic choice, making the code easier to understand and maintain.

This allowed me to ship a critical feature quickly and with confidence in its efficiency.

Why This Matters for Developers Like Us

This experience solidified a key principle for me: a strong foundation in core computer science principles is an incredible superpower in real-world development.

It's not about memorising every algorithm, but about:

Pattern Recognition: Learning to identify the fundamental structure of a problem, even when it's cloaked in business jargon.
Leveraging Existing Knowledge: Knowing that many problems have already been solved, and understanding which solutions apply.
Building Robust Solutions: Choosing algorithms with known performance characteristics rather than creating ad-hoc, potentially inefficient ones.

So, next time you're wondering if that DSA course or LeetCode challenge is really worth it, remember that it's training your brain to be a better, more efficient problem-solver in the long run.

What About You?

Have you had a similar "aha!" moment where a seemingly academic concept from DSA (or another CS theory) was the perfect solution to a real-world problem in your projects?

👉 Share your stories in the comments below!

Beyond CRUD: System Design Lessons From a Video Re-Streaming Control Platform

Sidharth Devaraj — Thu, 28 Aug 2025 08:46:29 +0000

Most web apps boil down to CRUD - Create, Read, Update, Delete.

But every now and then, a project comes along that breaks the monotony. Instead of just wiring up endpoints and database rows, you need to start thinking about systems — processes, configs, orchestration, resilience.

This post is a reflection on one such project I worked on. A client in the video re-streaming¹ space wanted a control dashboard for their customers — something simple on the surface:

"Let users toggle their streams on and off."

Sounds easy, right? Just a button, an API call, done.

Except… those toggles had to orchestrate system processes, job queues, config updates, and real-time feedback loops — all while hiding that complexity behind a clean web UI.

It turned out to be one of the most fascinating technical problems I’ve worked on. Let me walk you through the challenges we hit and the solutions we designed.

1. State Management: Database vs. System

Most web apps can treat the database as the ultimate source of truth. Not here.

The streaming process only respected its configuration file, which lived on disk. That file dictated whether a stream was active or not.

This raised a tricky problem:

If the database said "stream = on" but the file said otherwise, who do we believe?
How do we keep both in sync without introducing drift?

Approach

We treated the configuration file as the source of truth.
The API server became a translation layer: user actions → update config file → reflect state in DB.
On startup, the system reconciled the DB against the file to re-establish a consistent state.

This ensured that if the process restarted or a manual change occurred, we didn’t blindly trust stale DB values.

2. Multi-Step Orchestration

Toggling a stream wasn’t as simple as flipping a flag. The real sequence looked like this:

Update the configuration file.
Restart the streaming process.
Wait until the process stabilized and confirmed readiness.

Each step could fail. For example:

File write errors (permissions, disk issues).
Process restart failure.
Process enters a crash loop.

Approach

We built an asynchronous job queue with retries and compensation.
Each toggle request became a background job.
Jobs moved through states: PENDING → RUNNING → COMPLETED/FAILED.
If a step failed, the job queue retried or rolled back gracefully (e.g., restore old config).
The system ensured no two jobs interfered, keeping operations atomic from the user’s perspective.

This was crucial for stability—otherwise a single half-completed toggle could corrupt the live state.

3. Real-Time Feedback to Users

Users didn’t want to "fire and forget." They wanted to know if their toggle request succeeded.

A plain HTTP request wasn’t enough because the job was async. Returning "OK" immediately would lie — the process might still be restarting.

Approach

We built a persistent WebSocket channel between server and client.
When a toggle was requested, the client got back a jobId.
The UI subscribed over WebSocket to that job’s status.
As the background job progressed, the server pushed updates (in progress…, completed, or failed).
The UI displayed real-time status, so customers always knew the outcome.

This turned a frustrating black box into a transparent, transactional experience.

Sequence diagram of the workflow can be viewed here

4. Key Takeaways

This project was a reminder that:

Source of truth matters. Sometimes it’s not the database.
Asynchronous orchestration is unavoidable when dealing with external systems.
Real-time feedback builds trust. Users don’t just want features—they want visibility into what’s happening.
Architecture resilience > code cleverness. The hardest part wasn’t writing code, it was designing a system that wouldn’t collapse under edge cases.

Final Thoughts

What looked like a trivial feature—"toggle a stream"—actually turned into a system design problem, touching on state synchronisation, async orchestration, and real-time communication.

For me, this was one of the most rewarding technical challenges I’ve worked on. It taught me that the real engineering battles often happen at the intersection of web applications and the messy, stateful systems they control.

💬 I’d love to hear from others:

Have you ever faced a "simple" feature request that turned into a full-on system design challenge?

¹ Video re-streaming: The process of taking a live video feed from one source and re-broadcasting it to multiple platforms (e.g., streaming a live event to YouTube, Facebook, and Twitch simultaneously).

From Role-Based Chaos to Resource-Based Bliss: My REST API 'A-Ha!' Moment 💡

Sidharth Devaraj — Tue, 26 Aug 2025 12:02:36 +0000

Hey everyone,

I want to share a confession from years ago, back when I was just starting out with REST API design.

I thought I was being super clever and organised by designing my API endpoints around user roles. My brain went straight for /admin/... for all things admin and /customer/... for customer-related actions. Felt neat, right? 😅

Then came the inevitable moment when both admins and customers needed to interact with the same data (users, for example), and my perfectly segmented system crumbled fast. 🤦‍♂️

That's when the lightbulb finally went off: resource-based RESTful design isn't about who is accessing the data, it's fundamentally about the data itself. This simple but profound shift saved me from countless headaches and taught me a crucial lesson in API architecture.

Instead of having separate, duplicated logic, focusing on resources like /users, /products, and /orders, and then implementing proper authentication and authorisation to control access based on roles, made everything so much cleaner and more maintainable.

Anyone else have a similar early "a-ha!" moment in their dev journey? What's a mistake you made that ended up teaching you a valuable lesson? Share your stories below! 👇