DEV Community: Bhavya Thakkar

From Recursion to Backtracking

Bhavya Thakkar — Sun, 01 Mar 2026 11:23:11 +0000

You already know recursion. Here's the one idea that unlocks everything else.

1 — The Gap Nobody Talks About

You understand recursion. You've done tree traversals, written Fibonacci, maybe even memoized things. Recursive functions feel natural — break the problem down, solve the smaller version, build back up.

But then someone says "generate all permutations" or "find all valid partitions" and something shifts. You know the shape of the answer vaguely. You sit down to write it. And the code doesn't come.

It's not that you're missing a data structure or a formula. The gap is conceptual — and it's smaller than you think.

Recursion computes. Backtracking explores. Pure recursion commits to a path and returns a value. Backtracking explores a path, and if it doesn't work out, undoes the choice and tries another. That undo step is the entire difference. Once you internalize it, the code starts writing itself.

To make the idea clearer: imagine you're in a maze. The goal isn't to escape, it's to find a cat stranded somewhere inside. This subtle shift in goals probably made you think "cool, I'll just explore all the paths then." That's the crux of backtracking vs recursion. Now extend the example: what if you had to give me the exact sequence of turns for every possible path to the cat?

2 — How to Know You're Looking at a Backtracking Problem

Before writing a single line of code, you need to recognize what kind of problem you're facing.

Backtracking signals:

The problem asks for all solutions, not just one
You're building something incrementally and need to abandon partial builds
There's a constraint that can fail mid-way through construction
Keywords: generate all, find all permutations, all valid combinations, all ways to partition

Pure recursion signals:

You need a single answer
Clear top-down decomposition — the answer is defined in terms of the same function on a smaller input
Nothing is shared between branches, no exploration of alternatives

Key question to ask yourself: Am I computing a value — or am I exploring a space of possibilities? If you're exploring, you almost certainly need backtracking.

Remember the maze — if you just needed the cat's coordinates, you'd stop at the first path that works. But returning all possible paths means exploring every route — and when a path leads nowhere, you backtrack to the last crossroads and try another branch.

3 — The Anatomy of Backtracking

Every backtracking solution follows the same skeleton:

function backtrack(state, choices):
  // base case: we've built a complete valid state
  if state is complete:
    collect or return state

  for each choice in choices:
    make choice              // modify state
    backtrack(updated state, remaining choices)
    undo choice              // ← restore state — this is backtracking

Three steps repeat for every choice: make, recurse, undo. The undo step is what separates backtracking from plain recursion. Without it, you aren't exploring alternatives — you're contaminating future branches with the choices of past ones.

What "make" and "undo" look like concretely depends on the problem — adding and removing from a list, marking and unmarking a cell — but the pattern is always the same.

4 — Recursion vs Backtracking — Side by Side

The best way to understand backtracking isn't to study it in isolation — it's to watch pure recursion fail at a problem that needs it, then see exactly how backtracking fixes the breakage.

Example 1 — Pure recursion succeeds: Count all subsets

How many subsets does [1, 2, 3] have? For each element, we either include it or we don't:

function countSubsets(index):
  if index == length:
    return 1

  return countSubsets(index + 1)   // exclude
       + countSubsets(index + 1)   // include

Notice what we're not doing. We're not building anything. No shared state between calls. Each recursive call lives in its own world and returns a number upward. Both branches are completely independent — they never interfere with each other.

When recursion works cleanly, it's usually because each call is stateless — it receives inputs, does work, returns a value. Nothing is shared.

Example 2 — Pure recursion breaks: Generate all subsets

Now change one thing. Instead of counting, collect the actual subsets:

current = []
result  = []

function findSubsets(index):
  if index == length:
    result.add(current)       // feels right. it isn't.
    return

  current.add(nums[index])
  findSubsets(index + 1)      // "include" branch

  findSubsets(index + 1)      // "exclude" branch — but current still has nums[index]!

Let's trace through what actually happens for [1, 2]:

findSubsets(0)
  → add 1.  current = [1]
  → findSubsets(1)               // include branch
      → add 2.  current = [1, 2]
      → base case: result = [[1, 2]]          ✓
      → findSubsets(2)           // "exclude 2" branch
          → base case: current is STILL [1, 2]
            result = [[1, 2], [1, 2]]         ✗
  → findSubsets(1)               // "exclude 1" branch
      → current is STILL [1, 2] from before...

The "exclude" branch never actually excluded anything. It inherited the dirty state left behind by the "include" branch. Both branches share the same current, and there's no mechanism to clean it up between them.

The diagnosis: The recursion structure is correct. The problem is shared mutable state. Pure recursion has no mechanism to undo that.

Think of it this way — pure recursion is like each call working on its own notepad. The moment you introduce a shared structure like current, you've moved to a shared whiteboard. Every call reads and writes to the same surface. Without a rule that says "erase before the next person writes", it's chaos. Backtracking is that rule.

The fix — backtracking enters

function backtrack(index, current):
  if index == length:
    result.add(copy of current)   // snapshot, not reference
    return

  // include branch
  current.add(nums[index])
  backtrack(index + 1, current)
  current.remove(last)            // ← undo — clean the whiteboard

  // exclude branch — current is clean now
  backtrack(index + 1, current)

One line — current.remove(last) — is the entire difference. It erases the contamination before the exclude branch runs. Both branches now see a clean state.

Notice also that we collect a copy of current, not current itself. If you're wondering why — hold that thought. It's one of the most common bugs in the pitfalls section.

5 — The Two Flavors

Not all backtracking problems have the same shape. Once you've recognised that a problem needs backtracking, the next question is: what kind?

Picking from a pool (permutations / combinations) — you have a set of items and you're choosing from them. Tracking what's been used is the key mechanic — visited array or a start index that prevents reuse.

Splitting a structure (slicing / partitioning) — you have one string or array and you're cutting it at different points. A moving start index is the key mechanic — you never "undo" the pointer, you just try different cut points from the same position.

[Expand with side-by-side pseudocode skeletons showing the structural difference between the two]

6 — Common Mistakes and Pitfalls

These are the bugs that appear even when you understand backtracking conceptually. Each one is subtle enough to survive a quick read of the code.

1 — Not restoring state (the bleed bug)

You add to current inside the loop but forget to remove afterward. The state from one branch bleeds into the next. Every subsequent branch starts dirty.

The fix: every make step must have a matching undo step — always, unconditionally.

for each choice:
  current.add(choice)
  backtrack(...)
  current.remove(last)   // never skip this

2 — The copy trap

You collect current directly into result instead of a snapshot. Since current keeps mutating, every entry in result ends up pointing to the same final state — a list full of identical, wrong results.

result.add(current)           // all entries become identical
result.add(copy of current)   // correct — snapshot at this moment

3 — Confusing permutation structure with partition structure

Permutation problems track which items have been used (visited array or used set). Partition problems track where you are in the structure (a start index). Using the wrong mechanic gives you either wrong results or runaway recursion.

Ask yourself: am I picking from a pool, or am I cutting a single sequence?

4 — Forgetting the base case is a collection step

In problems that ask for all solutions, the base case isn't just a termination signal — it's where you capture the result. Returning early or returning a boolean and forgetting to collect means valid solutions silently disappear.

5 — Off-by-one in slicing

In partitioning problems, the start index or slice boundary is off by one. You either process an empty prefix (producing invalid empty partitions) or skip the last valid cut point. Trace through the first and last iterations by hand before trusting the loop bounds.

7 — A Mental Checklist

Does the problem ask for all solutions, not just one? → likely backtracking
Is there an undo step? → backtracking. No undo → pure recursion.
Am I picking from a pool or cutting a structure? → determines the flavor
Am I collecting all results or stopping at first? → determines base case behavior
Did I copy before collecting? → prevents the copy trap
Does every make step have a matching undo? → prevents the bleed bug

Kafka Self-Healing Cluster:

Bhavya Thakkar — Sun, 05 Oct 2025 08:57:18 +0000

Hey there, welcome back to the 2nd episode of the monthly misadventures of a regular dev. Thanks for sticking around, especially after how rough the last one was. Let’s get straight to the point, have you ever had your organization look at the Confluent or Kinesis MoM bills and think, “Why can’t we just self-host this again?” You tried explaining that scaling and meeting downtime SLAs would be tough, especially with a lean team, but those warnings fell on deaf ears. Now you’re facing the consequences, those infrequent outages with your self-hosted Kafka servers.Those unwelcomed weekend outages. Don’t worry, I’ve been there too. In this post, I’ll share how I tackled this challenge. The implementation details? They’re reserved for an upcoming blog.

What’s the issue:
Our self-hosted Kafka cluster has experienced intermittent downtimes caused by corrupted log files. Although these failures are infrequent, they require manual developer intervention to resolve—typically SSHing into the affected node, purging corrupted data, and restarting the broker.
Proposed Solution:
We plan to add a sidecar to each Kafka node that exposes an API interface on the node for health checks, purging, starting, stopping, and restarting brokers. This sidecar abstracts the complexity of the underlying infrastructure and Kafka internals, giving developers a simple way to interact with each node.
The second component is a centralized controller, hosted separately. It polls the health status of individual nodes, every 5 seconds. If a broker becomes unresponsive, the controller will trigger an automated purge and restart sequence, retrying up to three times. If the broker remains unavailable, a cluster-wide purge and restart will be attempted. Should that fail, an infrastructure operations alert will be sent, tagging the tech team to manually intervene.
Although rare—over 1.5 years of managing our Kafka server, purging and restarting has always resolved availability issues,we included these escalation steps to prepare for unexpected scenarios. Besides polling, the controller also offers an API to manage cluster-wide operations with the same commands available to the sidecars.
Limitations:
This solution effectively monitors and maintains broker health but currently lacks dynamic load management. We run a fixed number of brokers and controllers and do not support automatically adding or removing nodes based on load. However, in our production environment, we remain well within available compute capacity.
Another limitation is visibility. While the system operates reliably, it lacks monitoring tools such as connection failure metrics, utilization stats, or a dashboard—interactions are limited to the API interface, with no UI available.As they say if you cant validate or monitor it running smoothly it isn't running smoothly enough
Future Scope:
Future improvements will focus on adding a visibility dashboard with metrics, a user interface for cluster interaction, and support for dynamic cluster resizing via API and UI.

The Dashboard Dilemma

Bhavya Thakkar — Fri, 05 Sep 2025 13:00:21 +0000

Introduction
Sweaty palms, jittery hands, typing away at midnight to ship a crucial feature to improve team productivity, we've all been there. This could be avoided by building a reliable and easy to use dashboard that gives insights on KPIs, makes management smoother, makes any strategic pivots data-driven, and iterations faster. The problem lies in what we want to track, how frequently, how stringently, and how do we present it in a quick yet reliable fashion. I was in the same conundrum.
During August last year, after basic infrastructure and backend setup was done for our in-house ERP, I was tasked with any upcoming improvements and feature extensions.
Imagine this, you are in the spotlight of the financial year-end party all thanks to one feature that quadrupled the sales team's efficiency, I was stoked-that's an understatement; I wanted this every bit of it, even the possibility of failing miserably.

But little did I know one of those features was going to be an organization-level dashboard involving complex KPIs, like call efficiency, call times, and all of it preferably real-time. So this blog is going to be about how I went about it, what were the trade offs I chose, what I could have done better, what I actually got right, and things I didn't.

Freshness SLA & Its Impact
So what exactly is freshness SLA, and why does it matter so much? Let's take an example: if you were working at an HFT (high frequency trading) firm, if the prices that you execute your purchase/sell orders on are even a second old, it's going to cost the company probably millions of dollars, and on the stark end of the spectrum if you were working on a government project of verifying a user's document for vehicle registration you would be fine even if the system took, let's say an hour.
What changed in these 2 examples is the requirement of frequent data updates/syncs, that's freshness SLA, and freshness is simply how old or new the data that you possess is
The one thing about any business and any data-driven decision is, if you ask them what they want, they will most definitely answer with something along the lines of
"The most recent accurate data, as quick as possible"
The problem with these statements is I don't have anything to latch on to, nothing to map my user expectations, and nothing to derive my KPI. Luckily I was smart enough to ask the right question,

Is it okay if the data is 15 mins stale?

The answer to that question most definitely won't be a simple yes or no, and that's the beauty of it-it will give you insights on what can you get away with, it will be more like

15 mins is too old, we are willing to accept 5 mins stale data>

15 mins is fine but no more delays than that

or a
rigid real-time requirement, but then the expectation for performance would be lowered. I was working with a max staleness of 15 minutes and a sub-3 second response time, any slower and it would be a bad UX, and preferably sub-1 second responses.

figure 1

This gives me quite a bit of wiggle room, even though the data as shown in figure 1 is quite complex to begin with. So let's break it down: each user is assigned a designation, and a team which can change with time due to promotions, and internal movements. We need to extract information about KPI on customer state, derived from call_history, and whether the customer is closed or not, based on the invested flag in the customer table. All simple and straightforward so far, but we need to do it for each user in all the teams, over the time period specified, and aggregate under the correct hierarchy, this also includes contributions by past team members.
(user here refers to the stakeholders of our ERP system—members of the sales team)
Final data would look something like this.

{
  "key": 1,
  "child": [
    {
      "key": 2,
      "child": [
        {
          "key": 3,
          "child": [
            {
              "key": 4,
              "warm_calls": 2,
              "dialled_calls": 100,
              "call_efficiency": 5,
              "closures": 1,
              "sip_setup": 1,
              "designation": "RM"
            },
            {
              "key": 5,
              "warm_calls": 2,
              "dialled_calls": 100,
              "call_efficiency": 5,
              "closures": 1,
              "sip_setup": 1,
              "designation": "RM"
            }
          ],
          "warm_calls": 4,
          "dialled_calls": 200,
          "call_efficiency": 5,
          "closures": 2,
          "sip_setup": 2,
          "designation": "team-lead"
        }
      ],
      "warm_calls": 4,
      "dialled_calls": 200,
      "call_efficiency": 5,
      "closures": 2,
      "sip_setup": 2,
      "designation": "manager"
    },
  ],
  "warm_calls": 4,
  "dialled_calls": 200,
  "call_efficiency": 5,
  "closures": 2,
  "sip_setup": 2,
  "designation": "avp"
}

sample data for 1 team consisting of 2 sales reps

We were mainly considering 3 approaches, as any lean startup running on a time crunch, our first idea was caching, but the problem was caching the whole data (as shown in sample data) would be expensive, largely due to the amount of recursive hierarchy that we need to maintain for team-based KPIs. Not to mention we would need to invalidate cache, and if a cache miss occurs, the DB query is still inefficient. We have neither optimized the query, nor have we reduced the data that query processes, so no real improvement on the first hit. This didn't sit right with me and if I had done the easy thing, and kept the TTL as 15 minutes and called it a day, it would be a band-aid fix waiting to explode in my face. Caching as a solution would work for frequently accessed time frames like daily or weekly data, but the moment we switch to say more infrequently accessed data like yearly, quarterly, or half-yearly we run into the problem of frequent cache misses

The second approach was to use batch processing to precompute this data, although much better than the quasi improvement of caching, it had its own challenges setting up complex ETL pipelines, separate infrastructure setup, a lot of initial configuration, and still it isn't a perfect fit for the real-time requirement. Not to mention that if setup is anything short of perfect it would introduce data inconsistencies, and any minor schema change would need us to go through this whole process again, not ideal for development velocity

The third approach we had was to use streaming of events from our mobile ERP app to directly populate web dashboards, fast, quick, reliable, initial infra setup in terms of Apache Kafka is required. This can be circumvented by using cloud solutions like Confluent and Kinesis, these would be plug and play. We only had to worry about triggering events and consuming them, but the problem is the data we required was too complex, mandating complex aggregations and joins. While data points sent by mobile client could be used to calculate KPIs, processing it in event-driven architecture would mean computing it as we receive the event making refreshes much slower than the required 3 seconds
So what I ended up doing was to use a combination of these 3 to find the sweet spot between these approaches. I wanted a streamlined approach similar to Redis, with cost and accuracy of batch processing and ease of setup similar to streaming

The Incremental Refresh Approach
So I lied, I didn't use all three, I held back caching. I just couldn't come to terms with a pseudo real-time dashboard, I used streaming, specifically a queue, Kafka to be even more specific, to ingest real-time events into the DB. These events included metrics like call times, leads closed, leads transitioning to different states in the funnel, so my database has fresh data, what about the aggregation problem, you ask. That's where realizing that all the data required for a real-time organizational dashboard mostly resides in the backend server, I would say mostly because some of the transactional data would be async, including but not limited to payment_status, fulfillment_timestamp. This doesn't affect the problem statement much because in such dashboards you work with an unsaid assumption, whatever data is present within the system, you crunch the numbers and give it to business teams. This context setting was required to appreciate a rather simple solution, materialized views, all your aggregations could be pre-computed and stored as a materialized view.

The problem with materialized view is that it doesn't refresh with updates, an intuitive solution would be to use hooks and refresh the view. That's only true if you serve all your data through views to the dashboard, now let's think about this problem in a broader perspective. Does historical data change, the answer to that in my case was no. So can I save all the data until yesterday in a view and refresh it through a cron? Yes, and compute only today's slice irrespective of time frame. The aggregation considers significantly less data, and queries can be optimized to give sub-second results.
Now coming to another downfall of views, they lock the whole table while they are refreshed, not good. So we need to run it in a cron, during phases with lower loads, I hear you, didn't I frown upon additional infra, and that's where a nifty little extension on PostgreSQL comes to save the day, pg_cron to be specific. So how it works is you configure your usual cron but on your db in a different schema, you just provide the refresh view command as the command to be executed to the cron config and whether it should be active or not.

This is a clean solution, no additional infra overhead, no compromise on real-time SLA, minimal cost increase, if any due to views, and the best part it's elegant and easy to manage. The next section goes over how to setup pg_cron for your use case, I refrained from going on in-depth on how to make materialized views, this blog is already running too long and setting them up is really straightforward. While pg_cron does require some work if you don't self-host your servers and use a cloud provider, we use AWS RDS so all of this info will be in context to that but I will attach references on how to achieve the same on the other 2 major cloud providers, once pg_cron has been added as an extension in postgres, process after that is really straightforward.

PG Cron Setup RDS
pg_cron is supported on RDS for PostgreSQL versions 12.5 and above.

Modify Parameter Group
Go to the AWS RDS Console.
Navigate to Parameter Groups on the sidebar.
If you don’t have a custom parameter group, create one by copying the default group associated with your DB instance.
Edit the custom parameter group. Find the parameter shared_preload_libraries.
Add pg_cron to this parameter’s value (append it if other libraries exist).
Save the changes.
Apply the Parameter Group to Your DB Instance
Modify your RDS instance to use the new/modified parameter group.
A database restart is required for the changes to take effect. Restart your instance.
Create the pg_cron Extension in the Database

Connect to your PostgreSQL instance as a user with rds_superuser privileges (default user postgres usually has these).

CREATE EXTENSION pg_cron;

This will enable the pg_cron background worker and create the necessary objects, typically in the postgres database by default.
Schedule Jobs

SELECT cron.schedule('job_name', 'cron_schedule', 'command_to_run');

For example:

SELECT cron.schedule('refresh_materialized_view', '*/15 * * * *', 'REFRESH MATERIALIZED VIEW my_mat_view');

Managing Jobs
To unschedule jobs:

SELECT cron.unschedule(job_id);  
SELECT cron.unschedule('job_name');

Job details and histories can be viewed in the cron.job_run_details table in the database, like so

SELECT * FROM cron.job_run_details
WHERE jobid = your_job_id;