DEV Community: Siddharth Pandalai

The captain routes. The captain never rows.

Siddharth Pandalai — Mon, 05 Oct 2026 04:02:23 +0000

Two runs died on rate limits in the same week.

Not because the work was too big. Because the most expensive model in the fleet was doing the rowing.

The wrong mental model

Model tiers arrive looking like a quality ladder. A small one, a medium one, a large one. So the instinct is obvious: the work matters, use the best one you can afford.

That instinct is what empties a budget, because it treats capability as the only axis and ignores volume completely. Capability is what you need for one hard decision. Volume is what you need for four hundred easy ones. Those are different problems and they want different answers.

They are roles, not ranks.

Mechanical work stays a small-model job no matter how important the surrounding project is. A rename, a format, a lint sweep, an extraction. Importance does not transfer down into the difficulty of the individual step.
Bulk execution is a mid-tier job, and this is where almost all real work actually lives. Twenty files, one pattern.
Orchestration and judgement is where the largest model earns its price. Deciding what the work is. Splitting it. Ruling on the ambiguous case that the spec did not cover.

The failure has a name

An orchestrator that starts doing the work itself is a captain picking up an oar. The ship stops steering at precisely the moment the volume shows up, which is precisely when steering was the thing you were paying for.

I wired my own harness this way early on. Top tier as planner and executor, because the work mattered and I wanted the best output at every step. It read as thoroughness at the time.

Two waves died on rate limits inside one week. The work was not unusually large. Every one of a few hundred mechanical steps had gone through the most expensive path available, and the budget was gone long before the actual judgement calls arrived. The runs did not fail at the hard part. They never reached the hard part.

The fix was one sentence, and its grammar mattered

The fix was not a smaller model. It was making delegation mandatory wording in the orchestrator's prompt rather than advisory:

Delegate execution. Do not perform it.

Not "prefer to delegate". Not "you may delegate where appropriate". Those get read as permission, and permission loses to the more concrete instruction sitting next to it.

A model told to plan and execute will execute. Executing is specific and immediate; delegating is abstract and deferred. When two instructions compete, the concrete one wins, and "helpful" resolves toward doing the visible thing now.

This is the same failure as any under-specified interface. "You are the orchestrator" does not imply "and therefore you do not also do the work" to something built to be maximally useful. If a constraint matters, it has to be stated as a constraint, not implied by a role name.

The routing rule

The version I run now:

Pick the cheapest tier that covers the work. Not the best available. The cheapest sufficient one.
Escalate exactly one rung, and only on verified failure or genuine spec ambiguity.
Never escalate on volume. Volume is the signal to go down a tier and parallelise, not up. This is the one people get backwards, and it is the expensive one.
Keep the top tier's budget for deciding. Its value is entirely in where it points. Every token it spends on execution is a token not spent on the call only it can make.

This generalises past agents

The shape is not new. It is the same reasoning behind not running your analytics job on the primary database, not putting a senior engineer on a mechanical refactor, and not calling a paid geocoding API in a loop when a local lookup table covers ninety percent of cases.

The resource with the highest capability usually also has the lowest throughput and the highest cost per unit. Spending it on volume is how systems fall over, and it always feels like diligence while you are doing it.

The takeaway

Tiers are roles, not ranks. Pick the cheapest one that covers the work, escalate one rung at a time and only on verified failure, and never escalate because there is a lot of something.

Then write the constraint into the prompt as a constraint. "The captain never rows" has to be said out loud, because a captain built to be helpful will absolutely pick up an oar, and it will look like enthusiasm right up until the ship stops.

🌀 Iteration 17 of **The Loopdown. field notes from an engineer who writes.
Series: **Notes From The Loop* · Featuring: The Fleet*
← Previously in this series: Plausible is worse than wrong
📚 The full series: Notes From The Loop

Follow the loop → LinkedIn · dev.to · GitHub

I audited my own migrations. It was not fine.

Siddharth Pandalai — Sat, 03 Oct 2026 04:38:54 +0000

Forty-seven schema crossings in one app. Fifteen with a test behind them.

I would have told you, confidently, that it was fine. I have been quoting the migration count as a strength.

A migration is the only code that gets one attempt

Everything else you ship has a second chance. A bad screen gets a hotfix. A wrong API call gets retried. A crash gets a patch release and an apologetic changelog entry.

A migration runs once, on a device you cannot see, against data you cannot reproduce, at a moment you do not choose. If it is wrong, the user's data is not broken. It is gone. There is no second attempt because the schema has already moved, and the row you needed to read in order to recover is the row that did not survive.

And yet it is routinely the least tested code in the codebase, because it looks trivial. One ALTER TABLE. Ship it.

The audit

I ran a count across Mileway this week:

47 Migration objects
schema version 48
8 instrumentation test files, covering 15 version pairs

So roughly two thirds of the crossings have never once been executed against a real snapshot of the schema they claim to upgrade. They are trusted on read. Someone looked at the SQL, thought "yes, that looks right," and shipped it. Usually me.

The number itself did not bother me as much as what happened when I sorted it.

Coverage is the wrong instrument

Percentage tells you nothing here, because migrations are not uniformly dangerous. Adding a nullable column is close to free. Touching a constraint is not. So I sorted the untested ones by what they actually do:

Foreign key changes (5 to 6, 16 to 17). SQLite cannot alter a constraint in place. The only way through is to create a new table with the new definition, copy every row across, drop the old one, and rename. That copy is a full data movement wearing the costume of a schema tweak. If a column order is off or a type coerces, rows land wrong and nothing throws.

A data-moving migration (32 to 33). INSERT INTO ... SELECT. This one does not change shape, it transforms content. A mismatched column order here loses no rows at all, which is exactly why it is the worst case: you get a full table of confidently wrong values and no error anywhere.

A drop (47 to 48). Irreversible by construction. Nothing to say about this one except that it should have had a test first.

Four migrations. That is the actual risk surface, out of thirty-five untested ones. Test those and you have bought most of the safety available for a fraction of the work.

Two ways to write the test, and they are not equal

The standard path is MigrationTestHelper, which needs exported schemas:

ksp { arg("room.schemaLocation", "$projectDir/schemas") }

helper.createDatabase(TEST_DB, 16).apply {
    execSQL("INSERT INTO trips (id, distance_m) VALUES (1, 4200)")
    close()
}
val db = helper.runMigrationsAndValidate(TEST_DB, 17, true, MIGRATION_16_17)

The second path is the one I am actually on, and I only rediscovered it reading my own tests. This project runs exportSchema = false, which blocks MigrationTestHelper entirely. So the existing tests build the old schema by hand and drive it through a real SQLite connection:

val connection = BundledSQLiteDriver().open(path)
connection.execSQL("CREATE TABLE media_library (...)")   // the v39 shape, written out
connection.execSQL("INSERT INTO media_library VALUES (...)")
// then run MIGRATION_39_40 and assert

That works. It is a genuine test. But the two are not equivalent and the difference is worth being precise about, because it is the sort of thing you only notice when it bites.

MigrationTestHelper builds version 16 from the schema Room actually exported at version 16. The hand-written version builds it from your memory of version 16. If that memory is wrong, and after forty-odd crossings it eventually will be, the test constructs a table that never existed, migrates it successfully, and passes. Green the whole way, while production carries a shape your test has never seen.

So: hand-seeding is a valid answer to "exportSchema is off". It is not a free one. You are trading a verified starting point for a remembered one, and the interest on that trade compounds with every version.

If you can turn schema export on, turn it on. If you cannot, at least write the seed SQL by copying from the migration that created the table rather than from memory, and say so in a comment, which is what my better tests already do.

Assert the data, not the schema

Whichever path you take, the important line is not the migration call. It is the query afterwards.

db.query("SELECT distance_m FROM trips WHERE id = 1").use { c ->
    assertTrue("the row did not survive the crossing", c.moveToFirst())
    assertEquals(4200, c.getInt(0))
}

runMigrationsAndValidate proves the resulting schema matches what Room expects. It says nothing about whether your rows are still there or still correct. Only reading one back does that.

For the data-moving ones, assert the count as well. If 1,200 rows go in, 1,200 come out, and if they do not you want a red test rather than a support ticket in four months.

And never leave fallbackToDestructiveMigration() in a shipping build. It converts "my migration has a bug" into "the user's data is deleted, silently, and the app looks fine". A development convenience that reads, in production, as a data-loss feature.

The rules, short version

Export schemas if you can. MigrationTestHelper builds the old version from what Room actually exported. Hand-seeding builds it from what you remember, and after forty crossings those drift.
Sort by what it touches, not by count. Constraints, data movement, drops. Everything else is bookkeeping.
Assert data, not just schema. runMigrationsAndValidate checks shape. Read a row back to check truth.
Count rows through a data move. In equals out, or the test fails.
No destructive fallback in release. Ever.
Write the test before the migration ships, because after it ships you are no longer testing, you are doing forensics.

The takeaway

I did not find a bug this week. I found thirty-five crossings I had never rehearsed, and four of them touch the things that actually eat data.

The uncomfortable part is that the count was a number I had been proud of. Forty-seven migrations sounds like maturity. It is only maturity if you can say what happens when each one runs, and until this week I could not.

The boat only rows one way. Pay the toll before you get on it.

🌀 Iteration 16 of **The Loopdown. field notes from an engineer who writes.
Series: **Crossing The Schema* · Featuring: The Ferryman*
📚 The full series: Crossing The Schema

Follow the loop → LinkedIn · dev.to · GitHub

Plausible is worse than wrong

Siddharth Pandalai — Thu, 01 Oct 2026 06:57:24 +0000

The API returned 200. The article published. Every single tag was silently dropped, and nothing anywhere said so.

I found it two days later, by accident, looking at something else.

Wrong code gets caught. Plausible code ships.

A junior engineer fails in a way you can see. The code looks off, the naming is strange, something in the shape of it makes you slow down and read twice. That instinct is most of what code review actually is.

Borrowed work does not fail like that. Ask a model for something and it returns code that looks exactly right, because looking right is the thing it is best at. It compiles. It runs. It returns success. When your instructions run out, it does not stop and ask. It fills the gap with the most plausible thing, because being helpful is the only mode it has.

So the defect never lands in the syntax, which is where review is looking. It lands in the seam between what you asked for and what you meant, and it survives review precisely because it reads well.

Two from the same week, both in the tooling that publishes this series.

One: the line that returned 200 and did nothing

The publisher pushes each article to the dev.to API. The line read:

tags: tags.join(",")

Look at it. Tags are a list. The API wants tags. Join them. I would have approved this in a pull request without a second thought, and so would you.

That endpoint wants an array. Hand it a string and it does not reject the request, does not warn, does not fail. It returns 200 OK, publishes the article, and drops every tag on the floor. Four tags, every post, silently.

There was no error to read. The publish log said published, because it was. The only way to catch it was to fetch the article back and look at what was actually on it:

const landed = response.json?.tags || []
if (tags.length && !landed.length) {
    warn(`tags were rejected (sent ${tags.join(",")})`)
}

That check is three lines and I did not write it until after the damage, because the call had already told me it worked.

Two: the snippet that fit perfectly and taught the wrong thing

This one is worse, and it is the one I keep thinking about.

Part of the tooling picks a code snippet out of each article to render onto a slide. I specified it plainly: take the first code block that fits the slide.

It did exactly that. Perfectly. Every time.

On a post about the five second window you get to call startForeground(), it selected a WorkManager snippet. Real API, correct Kotlin, lifted from the right file, would compile. Also completely unrelated to the point the slide was making. That deck was one push away from teaching a few thousand people the wrong lesson inside a beautifully rendered panel.

Nobody lied to me. I asked for a block that fits. I meant a block that demonstrates the claim. The gap between those two sentences is the entire bug, and it is invisible in the output, because the output looks great.

Specify the predicate, not the shape

"A block that fits" is a shape. Shapes are easy to describe and easy to satisfy, which is exactly why they are dangerous: a shape can be satisfied perfectly by something useless.

"A block that contains the API this slide is about" is a predicate. It can be checked. It can fail. Something that fails loudly is worth more than something that succeeds vaguely.

The fix was one filter:

.filter((b) => matcher.test(b.join("\n")))

and a deliberate decision that when nothing matches, the slide falls back to plain text rather than showing code that argues for something else. Four posts now have no code slide. That is the correct outcome. Four honest gaps beat four confident wrong answers.

Verify the outcome, not the call

This generalises well past AI, and it is the part worth stealing.

A 200 means the server received your request. It does not mean the server did what you wanted. A successful INSERT means a row exists, not that it holds what you think. A migration that does not throw is not a migration that preserved your data.

// the call succeeded
val response = api.updateProfile(profile)
if (response.isSuccessful) return Result.success(Unit)

// what you actually wanted to know
val saved = api.getProfile()
check(saved.displayName == profile.displayName) {
    "server accepted the update and ignored displayName"
}

You will not do this everywhere, and you should not. Do it at the boundaries where a silent no-op is expensive: anything that publishes, anything that pays somebody, anything that migrates data, anything you cannot easily inspect afterwards.

Review the seams, not the syntax

The practical version, for when you are reviewing work you did not personally type:

Read the specification, not the diff. The diff is the answer. The bug is usually in the question. Ask what was actually requested, and whether the code answers that or something adjacent.
Find where the instructions ran out. Every generated block has a boundary past which nobody said anything specific. That is where the plausible filler lives. Go there first.
Distrust code that returns success. Especially against an API you do not control. Confirm the effect happened.
Make the check mechanical. An assertion, a linter rule, a test. Your attention is the one component in this system guaranteed to degrade, and it degrades fastest exactly when the output looks good.

The takeaway

Borrowed work does not fail like bad work. It fails like confident work that answers a question slightly next to the one you asked.

Specify the predicate, not the shape. Verify the outcome, not the call. Write the check down as something that runs, because the day you stop reading carefully is not announced in advance.

It will do anything you can describe. The trouble is what happens where you stopped describing.

🌀 Iteration 15 of **The Loopdown. field notes from an engineer who writes.
Series: **Notes From The Loop* · Featuring: The Borrowed Hand*
📚 The full series: Notes From The Loop

Follow the loop → LinkedIn · dev.to · GitHub

Your thresholds do not belong in constants

Siddharth Pandalai — Mon, 28 Sep 2026 07:21:01 +0000

Every tuning change needed a code review, a release, and a week of store rollout. So we stopped tuning, which is the worst possible outcome.

Eighteen guesses in a const val

Our location pipeline accumulated roughly eighteen numbers that shaped its behaviour:

speed band boundaries (walking, cycling, driving)
a minimum displacement gate for each band
a stationary speed and jitter threshold
a rolling history window size and its movement threshold
a hard teleport gate
four time-gap tiers, each with a relaxed speed cap
a maximum plausible gap distance

All const val, scattered through the processor. Every one of them was a guess. Informed, tested, reasonable, and still a guess about how phones behave in the hands of thousands of drivers.

The kind of thing you obviously want to revisit once real data arrives.

The cost of revisiting

Except revisiting meant: edit code, open a PR, get review, cut a build, push to the store, wait for rollout. A week at best to move a number by 0.5.

The predictable result is that nobody moved it. Threshold discussions became opinion versus opinion, because gathering evidence and shipping a correction cost more than the disagreement was worth. The numbers ossified and then got defended.

Frozen numbers are not stable numbers. They are unexamined ones.

The unglamorous refactor

@Serializable
data class AbnormalDetectionConfig(
    // speed-band jitter gates
    val walkingMaxMps: Double = 2.5,
    val cyclingMaxMps: Double = 7.0,
    val walkingJitterM: Double = 2.0,
    val cyclingJitterM: Double = 3.0,
    val drivingJitterM: Double = 5.0,
    val stationarySpeedMps: Double = 1.2,
    val stationaryJitterM: Double = 1.2,
    // movement-history window
    val speedHistorySize: Int = 5,
    val movementHistoryMps: Double = 1.5,
    // instant-teleport hard gate
    val spikeHardGateM: Double = 5_000.0,
    // gap-recovery tiers
    val gapMinSec: Long = 30L,
    val gap5mSec: Long = 300L,
    val gap1hSec: Long = 3_600L,
    val gap6hSec: Long = 21_600L,
    val gapTier5mMps: Double = 150.0,
    val gapTier1hMps: Double = 100.0,
    val gapTier6hMps: Double = 60.0,
    val gapMaxDistanceM: Double = 10_000.0,
) {
    companion object {
        /** Exactly today's constants: the no-override baseline. */
        val DEFAULT = AbnormalDetectionConfig()
    }
}

Injected into the processor:

class LocationProcessor(
    private val abnormalConfig: AbnormalDetectionConfig = AbnormalDetectionConfig.DEFAULT,
    // ...
)

That is the entire change. No new abstraction, no strategy pattern, no plugin system. One data class and a constructor parameter.

Two details that made it land

The defaults are byte-identical to the old constants. This makes the refactor provably behaviour neutral: every existing test passes untouched, and the diff is a pure move. That is the only kind of large refactor that gets approved without argument, and it is worth some care to achieve.

It stages. Debug settings could override the config the same week. Later, server-driven config became a change to where the object is constructed, not to the algorithm. The processor never learns where its numbers came from, which is exactly the property you want.

Worth being clear about what this is not. It is not a feature-flag system, a rules engine, or remote code execution. It is a struct of numbers with sane defaults. The moment tuning parameters become a DSL, you have built a second product nobody asked for.

The line I would draw

A const val is for something that genuinely cannot change: metres in a kilometre, the number of milliseconds in a second, a protocol constant.

A threshold in a heuristic is a hypothesis about the world. Hypotheses must be cheap to revise, or they quietly turn into dogma defended by whoever picked them.

If changing a number in your system requires a release, you will guess instead of measure. Fix the cost of changing it, and measuring becomes the easier path.

🌀 Iteration 14 of **The Loopdown. field notes from an engineer who writes.
Series: **Chain Of Custody* · Featuring: The Archivist*
← Previously in this series: Invariants are cheap. Silent corruption is not.
📚 The full series: Chain Of Custody

Follow the loop → LinkedIn · dev.to · GitHub

Invariants are cheap. Silent corruption is not.

Siddharth Pandalai — Sat, 26 Sep 2026 04:01:31 +0000

We had five numbers that were supposed to add up. Nothing in the codebase checked that they did.

The relationship nobody wrote down

A tracked journey produces five distance figures: original, cleaned, mock, abnormal and spike. Anyone on the team could tell you how they relate:

cleaned = original - (mock + abnormal)

With one wrinkle. spike is not in that equation, because spike distance is never folded into original in the first place, so subtracting it again would double count. It is tracked separately for reporting only.

That wrinkle is the kind of thing that survives exactly as long as the person who wrote it. It looks like an oversight. A reasonable engineer refactoring this code in a year would "fix" it.

Making the rule executable

object DistanceValidator {
    private const val TOLERANCE_METRES = 0.1

    fun validate(m: DistanceMetrics): ValidationResult {
        val errors = mutableListOf<ValidationError>()
        val warnings = mutableListOf<ValidationWarning>()

        // negatives are always a bug
        listOf("total" to m.total, "cleaned" to m.cleaned, "mock" to m.mock,
               "abnormal" to m.abnormal, "spike" to m.spike)
            .forEach { (field, v) -> if (v < 0) errors += NegativeDistance(field, v) }

        // CRITICAL: spike is excluded here. It is never folded into `total`,
        // so it isn't subtracted back out of `cleaned` either.
        val expectedCleaned = m.total - (m.mock + m.abnormal)
        val diff = abs(m.cleaned - expectedCleaned)
        if (diff > TOLERANCE_METRES) errors += ComponentMismatch(expectedCleaned, m.cleaned, diff)

        if (m.cleaned > m.total) errors += CleanedExceedsTotal(m.cleaned, m.total)
        // ...ratio warnings
        return ValidationResult(errors, warnings)
    }
}

The loud comment is deliberate. The invariant now enforces the wrinkle, and the comment explains it, so the refactor that would have broken it fails a test instead.

The tolerance matters too. These are accumulated floating point sums, so exact equality would produce false failures. 0.1 metres is well below anything meaningful and well above accumulated error.

Two tiers, doing different jobs

Errors block submission. Negative distances, component mismatch, cleaned exceeding total. These mean the code is wrong. The numbers contradict each other and nothing downstream should trust them.

Warnings surface without blocking. This tier is the one that earns its keep:

if (m.mock / m.total > 0.5)                      warn(UnusualRatio("mock", ratio))
if (m.abnormal / m.total > 0.5)                  warn(UnusualRatio("abnormal", ratio))
if (m.spike / (m.total + m.spike) > 0.3)         warn(UnusualRatio("spike", ratio))
if (abs(odometer - m.total) / odometer > 0.3)    warn(LargeDiscrepancy("odometer-vs-gps", ratio))

These are a fundamentally different kind of statement. The strict invariant catches code bugs. The ratios catch something internal consistency never can: a threshold being wrong.

Every one of those five numbers can be perfectly consistent with the others while the filter quietly discards half of a genuine journey. The arithmetic is fine. The classification is not. If most of a real trip is landing in the abnormal bucket, our detection is wrong, not the driver, and only a ratio check will ever tell you.

The odometer check goes further still. It compares against a genuinely independent measurement, so it can catch errors that no amount of self-consistency would reveal.

Why this is cheap

The whole validator is around eighty lines of pure Kotlin. No Android dependency, no I/O, no mocking. It is a function from data to a result, which means it is trivially unit testable and runs in milliseconds before submission.

Against that: distance figures that quietly drift wrong for weeks, discovered through a support queue, with no way to identify which journeys were affected.

The takeaway

Every system has relationships it assumes and never asserts. Components summing to a total. A subset never exceeding its parent. Two independent measurements staying within range. A count matching between client and server.

Those rules usually live in a comment, or a Slack message, or nowhere at all. Write them as code that runs. Errors for what must never happen, warnings for what is merely suspicious. Both before the data leaves.

🌀 Iteration 13 of **The Loopdown. field notes from an engineer who writes.
Series: **Chain Of Custody* · Featuring: The Archivist*
← Previously in this series: Never silently change a number someone gets paid on
📚 The full series: Chain Of Custody

Follow the loop → LinkedIn · dev.to · GitHub

Every filter needs a documented exception

Siddharth Pandalai — Wed, 23 Sep 2026 04:01:24 +0000

A filter with no exceptions has not met production yet. It has only met your test data.

The rule that was right nearly always

GPS reports an accuracy radius with every reading. Ours had a straightforward rule: anything worse than 50 metres is persisted, but does not count toward trusted distance.

val accuracyGated = fix.accuracyM > MAX_ACCURACY_THRESHOLD_M   // 50.0

This is sound. A 200 metre radius means the phone is in a basement, an underground car park, or surrounded by reflective towers. Counting that as travelled distance produces nonsense.

Then we hit the legitimate case: a vehicle parked for a long stop. Genuinely stationary. Decent fix. But occasionally reporting a slightly wider radius, and under the blanket rule those readings stopped contributing. The stop appeared as a hole in the journey.

Two bad fixes and one good one

Loosen the threshold. Move 50 to 70 and the complaint disappears, along with the protection you added the rule for. This is not solving the edge case, it is deleting the rule.

Special-case it silently. Add || fix.speedMps < 0.1 somewhere in the condition, move on. It works today and becomes archaeology within a quarter.

Name the exception. What we actually did:

// Stationary after recent movement with still-reasonable accuracy: the device is
// reliably placed, not drifting, so it may contribute despite the accuracy gate.
val exceptionalStationary =
    fix.speedMps <= EXCEPTIONAL_STATIONARY_SPEED_MPS &&   // 0.1 m/s: genuinely not moving
        fix.accuracyM < EXCEPTIONAL_STATIONARY_ACCURACY_M && // 20m: still a good fix
        hasMovementHistory()                                  // and it was moving recently

val accuracyGated = fix.accuracyM > maxAccuracyThreshold && !exceptionalStationary

Three conditions, all required, describing exactly one situation. The rule stays strict for everyone else.

Why the name is the important part

The logic here is four lines. Any competent engineer would arrive at something similar. What makes it survive is that it is called exceptionalStationary and carries a comment explaining the reasoning.

An unnamed condition is indistinguishable from a bug, and it gets treated like one. Somebody tidying up in six months sees speedMps <= 0.1f && accuracyM < 20f && hasMovementHistory() inline in a boolean, cannot work out why anyone would want that, and removes it. Tests pass, because nobody wrote a test for a case nobody could explain. The regression surfaces in support tickets a month later.

A named exception with a comment and a test is a different object entirely. It says: this is deliberate, here is the situation, here is why the general rule is wrong for it.

The other exception in the same pipeline

There is a second one worth showing, because it demonstrates the discipline that keeps exceptions from metastasising. After a pause and resume, the user may genuinely have moved while tracking was off, so the teleport-spike gate is relaxed for a grace window.

But the accuracy gate stays fully enforced during that window.

Relaxing one rule must never silently relax the neighbours. An exception should be as narrow as the situation that justifies it, and no narrower than the code makes explicit.

The takeaway

Write the rule. Then go looking for the legitimate case it breaks, because there is always one. When you find it, do not weaken the rule and do not smuggle in a condition. Give it a name, a comment with your reasoning, and a test.

Undocumented special cases do not survive contact with a refactor. Documented ones do.

🌀 Iteration 12 of **The Loopdown. field notes from an engineer who writes.
Series: **Sensors Who Lie* · Featuring: The Concussed Witness*
← Previously in this series: When two sensors disagree, rank them
📚 The full series: Sensors Who Lie

Follow the loop → LinkedIn · dev.to · GitHub

Never silently change a number someone gets paid on

Siddharth Pandalai — Sun, 20 Sep 2026 04:01:23 +0000

Our algorithm decided a driver had travelled 4km less than they thought. It was probably right. Shipping that silently would still have been wrong.

When cleaning stops being technical

Mileage tracking ends in an expense claim. Someone drives, the app measures, and the measurement becomes money.

Our pipeline removes several categories of distance: readings from mock location apps, physically implausible jumps, and teleport spikes over five kilometres in a single step. The resulting number is more accurate than the raw sum. That is not in dispute.

The mistake is thinking accuracy is the whole job.

A driver who believes they did 40km and sees 36km cannot distinguish between a careful algorithm and an employer quietly trimming expenses. From where they sit, both look identical: a smaller number, no explanation. Being right is invisible.

Explicit degradation, in the UI

The fix is not better filtering. It is showing the work.

Show the original next to the cleaned figure. Both numbers exist in the data model, so both can appear. The user sees that we measured 40km and are proposing 36km, rather than discovering 36km as if it were the only thing that ever happened.

Name what was removed. Not "adjusted for accuracy" but the actual categories: abnormal distance, mock distance, spike distance. Categories are checkable. Vagueness is not.

Let them overrule it. The trip screen has a toggle for whether the abnormal segment is subtracted:

var final = smartAnalysis.originalDistance
if (removeAbnormal) final -= smartAnalysis.abnormalDistance

This one gets pushback in review. Why let a user override the algorithm? Two reasons. They were physically present and we were not, so on a genuinely ambiguous journey they hold information the pipeline does not. And if our threshold is wrong, this is the cheapest possible feedback channel. The alternative is not that they accept it. The alternative is that they escalate to their finance team, who escalate to us, three weeks later, with no data attached.

Do not cry wolf. In the irregularities view, an app-killed event is explicitly classified as not an irregularity, because the tracker recovers on its own. If every notice is alarming, users learn to dismiss all of them, and the disclosure you worked on becomes noise.

This needs the data model to cooperate

None of this is possible if the pipeline throws away what it rejects. Showing original versus cleaned requires both numbers to survive. Naming the categories requires per-category accumulators. Letting the user toggle removal requires the removed distance to still be sitting there, addable back.

Transparency is a data-modelling decision made months before it becomes a UI decision.

The general case

The pattern applies well beyond mileage. Fraud scores that block a transaction. Ranking systems that decide what a seller earns. Time trackers. Usage-based billing. Anything where a system's output becomes an input to somebody's money, time, or reputation.

For those systems, correctness is table stakes. The actual product is: show the original, show the delta, show the reason, and let them push back.

Being right in silence is just asking to be trusted. Showing the working is how you earn it.

🌀 Iteration 11 of **The Loopdown. field notes from an engineer who writes.
Series: **Chain Of Custody* · Featuring: The Archivist*
← Previously in this series: Your data model is where uncertainty goes to die
📚 The full series: Chain Of Custody

Follow the loop → LinkedIn · dev.to · GitHub

When two sensors disagree, rank them

Siddharth Pandalai — Fri, 18 Sep 2026 06:44:33 +0000

GPS said the car was moving. The accelerometer said the phone had not twitched in ten minutes.

Asking a sensor to check itself

Stationary drift is the oldest problem in location tracking. A parked phone reports small movements in every direction. Sum the gaps and a car that never left the car park has quietly driven three kilometres.

The obvious defence is to look at reported speed and ignore readings that claim you are not moving. It half works, because the drift sometimes carries a plausible speed with it. The fix looks internally consistent while being completely wrong.

That is the structural problem. A single sensor can only be validated against itself, so every threshold you add is really a guess about your own input. You can spend months making that guess more elaborate without making it more correct.

A second signal that fails differently

The accelerometer is a poor navigator. Integrate it for position and the error compounds within seconds. But ask it a narrower question, "did this device physically move", and it is far better than GPS, because its failure modes have nothing to do with satellites, tunnels, or reflections off a glass tower.

Two sensors that fail differently can check each other. So the pipeline takes both:

fun process(
    fix: GpsFix,
    motionStill: Boolean = false,   // IMU says physically still
    harshAccel: Boolean = false,    // hard accel or braking this fix
): ProcessResult?

and when they disagree about movement, the IMU wins:

// drop the sub-gate wander when GPS history shows no movement,
// OR the IMU says still, OR this coincides with a harsh accel/brake event
if ((displacement < gate || stationaryMicroJitter) &&
    (!hasMovementHistory() || motionStill || harshAccel)) {
    return null
}

The comment in our source is blunt about why: accelerometer stillness is authoritative.

Harsh braking gets the same authority for a subtle reason. A hard stop can momentarily look like stillness to a speed gate, and the jitter it produces should not be counted as travelled distance either.

Rank per question, not overall

The important nuance is that we did not decide the accelerometer is "better". It is useless for position. GPS remains the only thing that knows where you are.

What we ranked was authority on one specific question:

Question	Authority
Where are we?	GPS
Did we move at all?	Accelerometer
How fast, over a distance?	GPS, checked against implied speed
Is the environment trustworthy?	Neither, the quality score

A sensor is not good or bad. It is good at some questions and bad at others, and a fusion strategy is mostly a table like that one made explicit.

The cheap version of the same idea

You do not need an IMU to use this principle. At journey level we compare total GPS distance against the vehicle odometer reading:

val discrepancyRatio = abs(odometer - total) / odometer
if (discrepancyRatio > 0.3) warn(LargeDiscrepancy("odometer-vs-gps", discrepancyRatio))

Two completely independent measurements of one number. When they disagree by more than 30 percent, something is wrong and a human should look, rather than the system silently trusting whichever number it happened to compute.

Most systems have a signal like this lying around unused. A client-side event count against a server-side one. A cache's view of a value against the source of truth. A model's confidence against observed outcomes.

The takeaway

Do not build a smarter threshold on a lonely sensor. Find a second signal that fails differently, and give it authority over the specific question it is genuinely good at.

Correlation beats cleverness.

🌀 Iteration 10 of **The Loopdown. field notes from an engineer who writes.
Series: **Sensors Who Lie* · Featuring: The Concussed Witness, The Second Witness*
← Previously in this series: One global threshold is how you delete valid data
📚 The full series: Sensors Who Lie

Follow the loop → LinkedIn · dev.to · GitHub

Your data model is where uncertainty goes to die

Siddharth Pandalai — Wed, 16 Sep 2026 05:08:31 +0000

The sensor knew how much to trust itself. Our database column did not have room for that, so we threw it away on the way in.

Everything arrives with an error bar

Android hands you a Location object that is far richer than a point on a map. It has an accuracy radius in metres, the provider that produced it, a bearing, an altitude, a timestamp, sometimes a speed. The platform is telling you, on every reading, how much to believe it.

This is not special to GPS. Models return confidences. Caches know their staleness. Upstream APIs know whether they served fresh data or a fallback. Almost every value in a real system arrives with a companion signal about its own reliability.

And almost every schema drops it.

Our first table was the obvious one:

@Entity
data class LocationData(
    val lat: Double,
    val lng: Double,
    val speed: Float,
    val date: Long,
)

Reasonable. Minimal. Exactly what the feature needed. It also silently discarded the accuracy radius on every write, which meant the discard was permanent.

The questions you cannot answer later

Within a few months, the real questions arrived:

Was this journey tracked well, or was the phone in a basement the whole time?
Which readings were fused location and which were raw GPS?
Was the device physically moving when this arrived, or just drifting?
Why is this trip's distance different from the driver's odometer?

Not one of those is an algorithm problem. They are all schema problems. The information existed at write time, in memory, for free, and we did not have a column for it.

You cannot backfill confidence. Once the row is written without it, that uncertainty is gone for every future reader, forever.

The row that carries its papers

What we persist now:

data class LocationData(
    // the reading
    val lat: Double, val lng: Double, val speed: Float,
    val bearing: Float, val altitude: Double, val locationTime: Long,
    // how much to trust it, and where it came from
    val accuracy: Float,
    val provider: String,
    // what the device was doing
    val gyroscopeX: Float, val gyroscopeY: Float, val gyroscopeZ: Float,
    val accelerometerX: Float, val accelerometerY: Float, val accelerometerZ: Float,
    val batteryPercentage: Double,
    val deviceModel: String, val appVersionName: String,
    // what we already decided about it
    val isMock: Boolean,
    val isAbnormal: Boolean,
    val isPaused: Boolean,
    val displacement: Double,
)

Three groups, and the third is the one people skip. Storing our own verdict next to the evidence means a later reader can see both what we concluded and what we concluded it from. When a threshold turns out to be wrong, every affected row is identifiable rather than theoretical.

Confidence as a number, not a vibe

Raw columns let you reconstruct the truth. They do not make it usable in a UI. So the environment collapses into a single score, 0 to 100:

Condition	Penalty
Permission missing	30
Mock location	25
Process killed	20
Restarted	20
GPS off	20
Battery optimised	15
Power saver	15

plus an accuracy tier (15m or better is free, worse than 75m costs 20) and a small bonus when the fix stream is stable. Now the app can say "this journey was tracked in poor conditions" instead of presenting a number with false confidence.

The rule

Design the row to carry doubt. If a value can be wrong, the schema needs somewhere to record how wrong, where it came from, and what you already decided about it.

Those columns are cheap. Add them before you need them, because the alternative is discovering at query time that the answer was thrown away at write time, months ago, by a schema that assumed confidence did not matter.

🌀 Iteration 9 of **The Loopdown. field notes from an engineer who writes.
Series: **Chain Of Custody* · Featuring: The Archivist*
📚 The full series: Chain Of Custody

Follow the loop → LinkedIn · dev.to · GitHub

One global threshold is how you delete valid data

Siddharth Pandalai — Mon, 14 Sep 2026 04:01:20 +0000

Our GPS filter worked perfectly, right up until someone sat in Bangalore traffic.

The phantom distance problem

A parked phone does not sit still in the data. The reported position wanders a few metres in every direction, and if you naively sum the gaps between readings, a stationary car quietly accumulates kilometres. On a mileage app that becomes an expense claim, so it matters.

The fix looks trivial. Ignore any step below some minimum:

if (displacement < MIN_DISPLACEMENT_M) return   // 5.0, felt about right

Parked cars stop drifting. Tests pass. Ship.

Then a driver crawls through traffic at walking pace for forty minutes. Every genuine step they take is under five metres. We deleted the whole journey and told them they had not moved.

The constant was making a claim

MIN_DISPLACEMENT_M = 5.0 looks like a number. It is really an assertion: five metres means the same thing whether you are parked, walking, cycling, on a motorway, or crawling in first gear. That assertion is false, and the filter had no way to know it.

Every threshold worth keeping is a function of context. Ours ended up depending on three things.

1. Speed bands

private fun minDisplacementForSpeed(speedMps: Double): Double = when {
    speedMps < 2.5 -> 2.0    // walking
    speedMps < 7.0 -> 3.0    // cycling
    else           -> 5.0    // driving
}

Faster travel expects and tolerates bigger steps between readings. Slower travel needs a tighter gate or you erase it.

2. Time-gap tiers

An implausible speed is only implausible relative to how long you were not looking. Under normal sampling, a 5km jump is a teleport. After a six hour gap it is a commute.

private fun isAbnormal(displacement: Double, impliedSpeed: Double, dtSec: Long) = when {
    dtSec < 30      -> displacement > 5_000 || impliedSpeed > 70.0  // ~252 km/h
    dtSec <= 300    -> impliedSpeed > 150.0
    dtSec <= 3_600  -> impliedSpeed > 100.0
    dtSec <= 21_600 -> impliedSpeed > 60.0
    else            -> displacement > 10_000   // stop testing speed entirely
}

Tunnels, killed processes, flights and genuine glitches all produce a jump. The gap is what tells them apart, so the gap has to be an input.

3. Recent history

This is the one that actually rescued the traffic case. A small step is only treated as jitter when the recent window agrees that nothing is happening:

private fun hasMovementHistory(): Boolean =
    recentSpeedHistory.isNotEmpty() && recentSpeedHistory.average() >= 1.5   // last 5 fixes

Sustained slow movement is movement. Genuine stillness is stillness. A rolling window can tell them apart. A constant never could.

Make them configurable while you are at it

Once thresholds are context-dependent there are more of them, and you will get some wrong. Ours live in one serialisable object rather than scattered const vals:

@Serializable
data class AbnormalDetectionConfig(
    val walkingMaxMps: Double = 2.5,
    val walkingJitterM: Double = 2.0,
    val cyclingJitterM: Double = 3.0,
    val drivingJitterM: Double = 5.0,
    val speedHistorySize: Int = 5,
    val movementHistoryMps: Double = 1.5,
    val spikeHardGateM: Double = 5_000.0,
    // ...gap tiers
)

Tuning becomes a config change rather than a release. The defaults reproduce the old hardcoded values exactly, so extracting them proved nothing changed.

The takeaway

When you write a constant into a filter, ask what context you are assuming. If the answer is "all of them", you have not written a threshold. You have written a bug with a delay on it.

🌀 Iteration 8 of **The Loopdown. field notes from an engineer who writes.
Series: **Sensors Who Lie* · Featuring: The Concussed Witness*
← Previously in this series: Filtered should never mean deleted
📚 The full series: Sensors Who Lie

Follow the loop → LinkedIn · dev.to · GitHub

Filtered should never mean deleted

Siddharth Pandalai — Sun, 06 Sep 2026 06:33:33 +0000

We shipped a filter that threw away bad GPS readings. Months later somebody asked whether it was working, and I could not answer. The evidence was gone.

That question changed how I build anything that rejects data.

The obvious version, and why it rots

Mileage tracking depends on trustworthy distance, and GPS lies constantly. So the first version of our cleanup did what everyone's first version does:

if (!fix.isPlausible(previous)) return   // drop it, move on
accumulateDistance(fix)

Clean data comes out the other end. It feels responsible. It is also a trap, because that return destroys the only record that could ever tell you whether the rejection was correct.

Six months in, someone asked the reasonable question: is the filter right? I could not say how many readings we had dropped, on which journeys, or whether any of them had been a genuine drive through a tunnel rather than a glitch. We had built a thing that made a judgement call thousands of times a day and kept no record of any of it.

Persist, then classify

The rebuild flipped the default. Rejection stopped being a return and became a label.

Only two cases are still deleted, because they cannot physically be real:

// impossible coordinates
if (fix.lat !in -90.0..90.0 || fix.lng !in -180.0..180.0) return null
// impossible accuracy: too precise to be true, or useless
if (fix.accuracyM <= 0.1f || fix.accuracyM >= 250f) return null

That is the entire delete list. Everything else is persisted and sorted into named accumulators:

originalDistanceM += displacement          // every metre we ever saw
when {
    fix.isMock  -> mockDistanceM += displacement
    abnormal    -> {
        abnormalDistanceM += displacement
        if (isHardSpike) spikeDistanceM += displacement
    }
    accuracyGated -> { /* recorded, deliberately not counted */ }
    else -> cleanedDistanceM += displacement
}

Five numbers instead of one. The UI shows cleaned. The rest live beside it.

And the row itself keeps its provenance: accuracy, provider, bearing, altitude, the gyroscope and accelerometer snapshot, battery level, device model, plus isMock and isAbnormal. A filtered reading is still sitting there, labelled with exactly why it did not count.

What that bought

Disputes became answerable. This is mileage that turns into an expense claim. When a driver says the distance is wrong, we pull the journey and show what was removed and why. Before, the honest answer was "the algorithm decided", which is not an answer.

Thresholds became tunable. We could replay real journeys and ask how much distance we were classifying as abnormal. Twice the data said our threshold was too aggressive on genuine motorway driving, and we moved it. That correction is impossible if the rejected steps are gone.

Ratios became alarms. Because we keep both numbers, we can assert on their relationship:

cleaned = total - (mock + abnormal)

and warn when abnormal exceeds half a journey, or when GPS distance and the vehicle odometer diverge by more than 30 percent. If most of a real trip is landing in the abnormal bucket, the threshold is wrong, not the driver.

The general rule

This is not a GPS lesson. Any system that rejects data has the same shape: fraud scoring, metrics pipelines, log sampling, feature stores, anything with a validation step. The moment you drop a record, you have made an irreversible claim about it and destroyed the ability to check.

Keeping it is usually cheap. A boolean column and a second accumulator is nothing next to permanently losing the answer to "is this working?"

So: filtered should never mean deleted. Filtered means "not counted, and here is the reason, written down, next to the thing itself."

If your pipeline drops rows, find out where they go. "Nowhere" is an answer, just not a good one.

🌀 Iteration 7 of **The Loopdown. field notes from an engineer who writes.
Series: **Sensors Who Lie* · Featuring: The Concussed Witness*
← Previously in this series: Teaching a phone to disbelieve its own GPS
📚 The full series: Sensors Who Lie

Follow the loop → LinkedIn · dev.to · GitHub

collectAsState is quietly leaking your work

Siddharth Pandalai — Fri, 04 Sep 2026 04:43:46 +0000

Here is a line you have written a hundred times:

val state by viewModel.data.collectAsState()

It looks harmless. It is the standard way to turn a Flow into Compose state. And on a screen with live data, it quietly keeps working long after the user has walked away.

The screen nobody is watching

collectAsState starts collecting the flow and stops only when the composable leaves the composition. That sounds right until you remember what backgrounding an app does, which is almost nothing to the composition. Send the app to the background and the composable is still composed. So the collection keeps running. The flow keeps emitting. Your ViewModel keeps fetching, computing, and pushing new state to a screen that is not on screen.

On a live price ticker or a location feed, that is real battery and mobile data spent in the dark. It is also a quiet source of bugs, because off-screen updates can pile up and then hit the user with a jump of stale changes the moment they return.

The one-word fix

val state by viewModel.data.collectAsStateWithLifecycle()

collectAsStateWithLifecycle ties the collection to the lifecycle owner. When the lifecycle drops below STARTED, it stops collecting. When it comes back, it resumes. If your upstream is a stateIn or shareIn flow with WhileSubscribed, the whole pipeline can now go cold while hidden and warm back up on return. Nobody pays for a screen that is not visible.

It lives in androidx.lifecycle.runtime.compose. Add the dependency, change the one call, and the leak is gone.

Make it the default

Treat collectAsStateWithLifecycle as the default and plain collectAsState as the exception. The rare time you actually want collection to continue while the screen is hidden should be a deliberate, commented decision, not something you got by reaching for the shorter name out of habit.

The takeaway

Lifecycle awareness is not a nice-to-have you bolt on later. It is the line between work that stops when the user leaves and work that runs forever in the background. The Recomposer does not mind an empty room. It will happily keep redrawing it. Your job is to turn the lights off when everyone goes home.

🌀 Iteration 6 of **The Loopdown. field notes from an engineer who writes.
Series: **Ghosts In The Recomposition* · Featuring: The Recomposer*
← Previously in this series: Your LazyColumn recomposes on every scroll
📚 The full series: Ghosts In The Recomposition

Follow the loop → LinkedIn · dev.to · GitHub