DEV Community: Puneet Khandelwal

Building a Zero-Knowledge Ballot Checker for Local Elections

Puneet Khandelwal — Tue, 13 Oct 2026 18:35:34 +0000

Trust in elections breaks the second you can't check your own vote. Last month, auditing an open-source voting tool, my engineering team hit a wall. We needed voters to prove their ballots counted without letting anyone—not even root admins—link a vote to a person. Better database encryption didn't fix it. Zero-knowledge cryptography did.

Most civic tech uses centralized ledgers or simple hashes. A hash proves a record existed, but brute-force attacks expose the underlying data. Zero-knowledge proofs let a prover convince a verifier a statement is true without leaking anything else. For our voting app, the statement is straightforward. My ballot sits in the final tally and my token is valid, but my identity and candidate choice stay hidden.

We tested this with an arithmetic circuit in a mock verification loop. The workflow splits into two parts. First, the voter's local device generates a witness with secret inputs like their private key and candidate ID. The client runs a proving algorithm, spitting out a cryptographic proof file that acts as a math-backed guarantee. Second, the public bulletin board verifies that proof. If the math checks out, the system counts the vote without ever looking at the ballot itself.

Here is what the verification logic looks like in Node.js using a standard verification key. We pass the public inputs, like the election root hash and nullifiers, alongside the proof payload.

const { verify } = require('snarkjs');
const fs = require('fs');

async function verifyBallotProof() {
 const vKey = JSON.parse(fs.readFileSync('verification_key.json'));
 const proof = JSON.parse(fs.readFileSync('ballot_proof.json'));
 const publicSignals = JSON.parse(fs.readFileSync('public_signals.json'));

 const isValid = await verify(vKey, publicSignals, proof);

 if (isValid) {
 console.log('Ballot verified successfully without exposing voter identity.');
 } else {
 console.error('Verification failed. Invalid proof or tampered signal.');
 }
}

verifyBallotProof();

The math looks great on paper, but municipal deployments are brutal. Circuit compilation takes forever, low-end mobile browsers run out of memory generating proofs, and a botched trusted setup ceremony wrecks the whole security model. If we want cryptography to replace blind trust in election hardware, these circuits need to run smoothly on cheap phones.

Code alone won't fix institutional distrust. Still, verifiable systems give communities the tools to demand accountability. When citizens can mathematically prove their election works, transparency stops being a campaign slogan and becomes an engineering standard.

The Feature Flag Technical Debt Trap Nobody Talks About

Puneet Khandelwal — Mon, 12 Oct 2026 17:21:00 +0000

We traded branch hell for boolean hell, and somehow convinced ourselves it was progress.

Feature flags promised continuous delivery utopia. Push to main whenever you want, wrap the unfinished work in an if (isFeatureEnabled('cool-new-ui')) block, and ship it safely. On paper, it sounds like the ultimate developer tool. In practice, codebases everywhere are rotting from the inside out with abandoned conditionals that nobody dares to touch.

Here is the reality eighteen months after adopting aggressive flagging. You stop writing modular code and start writing branched monoliths. A single route handler ends up looking like a nested maze of legacy checks. You have flags checking for flags. You have flags checking for user tiers, region rollouts, beta cohorts, and ancient experiments from two product managers who left the company last year. The compiler cannot help you here because these are runtime strings. Static analysis tools choke on the combinatorial explosion of states.

Look at a standard piece of modern backend drift. It usually starts clean:

async function renderDashboard(user: User) {
 if (await flags.isEnabled('v2-dashboard', user)) {
 return renderNewDashboard(user);
 }
 return renderLegacyDashboard(user);
}

Fast forward two years. The feature won, but nobody removed the flag. Now it looks like this:

async function renderDashboard(user: User) {
 const isV2 = await flags.isEnabled('v2-dashboard', user);
 const isDark = await flags.isEnabled('dark-mode-beta', user);

 if (isV2) {
 if (isDark && user.tier === 'enterprise') {
 return renderEnterpriseV2Dark(user);
 }
 return renderNewDashboard(user, isDark);
 }

 if (await flags.isEnabled('legacy-patch-99', user)) {
 return renderPatchedLegacy(user);
 }

 return renderLegacyDashboard(user);
}

With traditional branch-based deployments, dead code gets pruned naturally during code review or caught by git history when a feature branch goes stale. When a pull request sits around too long, you close it. The dead code vanishes with it. But with feature flags, dead code lives proudly in your main branch, executing checks against a remote service on every single request, introducing network latency and invisible logic paths.

The industry solved continuous integration by building robust tooling for merging code. We need to treat feature flag hygiene with the exact same rigor. If a flag doesn't have an automated removal ticket scheduled for the day after full rollout, don't merge it. Better yet, we need language-level semantics that treat flags as ephemeral compilation targets rather than permanent runtime configurations.

Until we treat a boolean toggle with the same disdain as a messy merge conflict, our codebases will continue to drown in conditional debt.

Why Your Productivity System Keeps Crashing

Puneet Khandelwal — Sun, 11 Oct 2026 16:19:02 +0000

If you spend all day debugging async race conditions, why do you design your personal routine like a monolithic shell script that fails silently at the first error? We've all done it. You spend a Sunday evening building the ultimate task management setup, color-coding tags, and nesting projects seven levels deep. By Tuesday afternoon, an unexpected meeting throws off the execution stack, and the whole system collapses into a guilt-ridden mess.

The shift happening right now in personal productivity points away from rigid optimization and toward fault-tolerant living. Instead of aiming for 100 percent uptime, people build routines that handle exceptions gracefully. When you code for production, you assume the network will drop and memory leaks will happen. Yet when we plan our weeks, we assume zero friction, endless energy, and absolute mental clarity.

This matters because hyper-optimized systems create brittle habits. When every hour is tightly coupled to a specific output, a small interruption causes cascading failures across your entire day. You miss a morning workout, skip meal prep, work late, and ruin your sleep. The system lacks circuit breakers.

Here's a non-obvious implication of this shift. The best productivity hack is reducing your dependency graph. Look at your daily stack. How many third-party apps, habit trackers, and review loops do you run concurrently? Every extra tool is a point of failure. When your routine requires five different services just to figure out what to do next, you spend more time maintaining infrastructure than writing code.

Software engineers understand that the most reliable code is the code you delete. The same rule applies to how you live. Strip out unnecessary dependencies in your daily schedule. Build fallback behaviors for when your energy crashes. If you can't run your full morning routine, what's the single fallback action that keeps the core process alive? Treat your lifestyle like a resilient distributed system, not a fragile script that panics when the database goes offline.

Building Secure Pipelines for Civic Technology

Puneet Khandelwal — Sat, 10 Oct 2026 15:28:10 +0000

When a municipal water valve fails, the root cause is rarely malicious. More often, it traces back to an unverified config push that bypassed staging because the deployment pipeline dragged. As engineers building software for public institutions, our mandate goes way beyond clean code or snappy UI components. We are managing the digital nervous system of our communities, and our testing pipelines need to reflect that weight.

Most commercial software can afford to break things and patch them later. Civic tech operates under a different social contract. If a voter registration database drops packets or a public transit routing API leaks location data, the damage strikes at the core of civic participation. Yet, many municipal contractors still treat security testing as an afterthought, bolted on right before production release when fixing vulnerabilities costs ten times more.

Designing a resilient testing pipeline starts with treating infrastructure as code and enforcing zero trust principles at every commit. Practically, this means moving past basic unit tests and running automated policy checks locally before any code touches a shared repository. A solid local check scans container images for known CVEs, verifies that database migration scripts don't expose personally identifiable information, and validates API schemas against strict public sector compliance standards.

Look at how we handle state management in public portals. A standard e-commerce site stores session tokens with minimal scrutiny. A municipal portal handling housing assistance applications must sanitize every incoming payload against hyper-specific regulatory frameworks. By embedding static analysis tools directly into pre-commit hooks, we catch boundary condition failures before code hits a pull request.

Staging environments deserve the exact same governance as production systems. Developers often use degraded production dumps for testing without proper anonymization, which creates massive honeypots for bad actors. Synthetic data generation isn't just convenient; it's an obligation when dealing with citizen records. Build generators that mimic the statistical shape of real municipal traffic without containing actual citizen identifiers so staging becomes a genuinely safe proving ground.

Technology is only as trustworthy as the systems we use to build it. If we want resilient public digital infrastructure, we have to start by making our engineering workflows transparent, accountable, and stubbornly secure.

Stop Building Microservices For A Monolith Problem

Puneet Khandelwal — Fri, 09 Oct 2026 13:17:16 +0000

We love complexity because it feels like progress. Three weeks into a new SaaS build, half the engineering team is arguing about service meshes, gRPC contracts, and eventual consistency models for a database that currently holds zero paying users.

This is a classic trap. Distributed systems solve organizational scaling friction. They aren't a badge of engineering maturity. When you slice a monolithic codebase into a dozen microservices before finding product-market fit, you pay a massive tax on your velocity for zero functional return.

Look at what actually happens when a three-person team deploys a distributed architecture early. Instead of shipping features, they spend days debugging network partitions between an auth service and a user service. They write custom service discovery logic. They spend hours configuring CI pipelines to orchestrate deployments across multiple repositories. The cognitive load shifts away from solving the user problem and toward keeping a fragile web of internal APIs alive.

At Kluvex, we learned this the hard way. Early on, we entertained the idea of splitting our core engine and our ingestion pipeline into separate services. It sounded clean. It sounded enterprise-ready. But every time we wanted to change a database schema, we had to coordinate deployments across three separate repos with overlapping dependency trees. We moved at a crawl.

So we did the unglamorous thing. We collapsed the system back into a modular monolith. We kept domain boundaries strict inside a single codebase, using internal packages instead of network boundaries. If a function needed data from another domain, it called a method, not an HTTP endpoint. Serialization overhead vanished. Debugging became a matter of stepping through a single stack trace instead of tailing logs across five different container instances.

The irony is that a well-structured monolith scales further than most people realize. You can run massive database instances, throw more memory at your single deployment, and optimize hot paths without worrying about distributed transactions or split-brain scenarios. You preserve the ability to move fast because your blast radius stays confined to a single process.

When the time finally comes to extract a service, you will know. It won't be because a blog post told you to use event sourcing. It will happen because a specific bottleneck hammers your CPU or a distinct team needs independent deployment cycles to stop stepping on each other's toes. Until then, keep your code in one place, push features to production, and let your users tell you when you actually need the extra moving parts.

Building a Custom Grafana Panel for Circadian Biometrics

Puneet Khandelwal — Thu, 08 Oct 2026 12:54:03 +0000

Most health dashboards treat time as a straight line. That works fine for server uptime, but it completely breaks down when you're trying to visualize human biology. Circadian rhythms are cyclical, messy, and personal. If you pull your sleep, heart rate variability, and continuous glucose data into standard time-series charts, you miss the actual phase shifts. You can't see the rhythm.

I spent last weekend getting tired of default visualizations and built a custom Grafana panel plugin specifically for circadian rhythm anomalies. If you're a developer tracking your own biometrics or building internal tools for a health tech team, moving beyond standard line graphs is the only way to spot real patterns.

Grafana supports custom panel development using React and the @grafana/data and @grafana/ui packages. The core architectural challenge here isn't fetching the data, though dealing with asynchronous time zones and daylight saving time transitions is its own headache. The real challenge is transformation. Standard dashboards expect metrics to increase or decrease over linear time. Circadian data needs a polar or 24-hour circular coordinate system to make sense of phase delays.

To build this, you start by scaffolding a plugin using the Grafana tool frontend plugin command. Your panel component receives a data frame containing your timestamped metrics, such as body temperature troughs or resting heart rate spikes. Instead of plotting these against a standard X-axis, you map the timestamp modulo twenty-four hours to an angle on a 360-degree SVG canvas. The radius can represent the intensity of the metric or the date index, letting you see if your sleep onset creeps later across the weeks.

Here's a snippet of the core time-to-angle transformation logic inside the React component:

import { DataFrame } from '@grafana/data';

interface CircadianProps {
 data: DataFrame[];
 width: number;
 height: number;
}

export const CircadianPanel: React.FC<CircadianProps> = ({ data, width, height }) => {
 const frame = data[0];
 if (!frame) return <div>No data found</div>;

 const timeField = frame.fields.find(f => f.type === 'time');
 const valueField = frame.fields.find(f => f.type === 'number');

 const points = timeField?.values.buffer.map((time, index) => {
 const date = new Date(time);
 const hours = date.getHours() + date.getMinutes() / 60;
 const angle = (hours / 24) * 2 * Math.PI;
 const value = valueField?.values.buffer[index] || 0;

 return { angle, value, timestamp: time };
 });

 return (
 <svg width={width} height={height} viewBox="-100 -100 200 200">
 <circle cx="0" cy="0" r="80" fill="none" stroke="#333" strokeWidth="1" />
 {points?.map((p, i) => {
 const r = 50 + (p.value * 5);
 const x = r * Math.cos(p.angle - Math.PI / 2);
 const y = r * Math.sin(p.angle - Math.PI / 2);
 return <circle key={i} cx={x} cy={y} r="2" fill="#5794F2" />;
 })}
 </svg>
 );
};

Why does this technical detour matter for wellness and lifestyle engineering? Because the tooling you use shapes what you notice. When you view health data through a cyclic lens, you stop blaming yourself for daily fluctuations and start seeing systemic environmental friction. You spot the exact Wednesday night routine that triggers a phase delay, or the subtle seasonal shift in your core temperature minimum.

The non-obvious implication of building your own biometric dashboards is ownership. Off-the-shelf fitness apps smooth out data into polite daily scores designed to keep you engaged through gamification. By writing your own panel plugins, you reclaim the raw telemetry. You stop consuming wellness as a finished product and start treating it as an open-source engineering problem.

Parsing Public Data: What Happens When Municipal APIs Break

Puneet Khandelwal — Wed, 07 Oct 2026 10:03:10 +0000

When a municipal transit API drops coordinate fields at 4 AM, your ingestion script shouldn't kill the notification service. We spend weeks building streaming setups for commercial APIs, yet we treat government feeds like static CSVs. Public data powers everything from housing trackers to air quality monitors, but the infrastructure behind it is fragile. As engineers in civic tech, our job is to absorb that friction so users get reliable data.

Working with city council voting records or spending databases brings specific headaches. Schemas shift because someone renamed a field in an old backend. Rate limits pop up out of nowhere because a server sits in a basement closet on decade-old hardware. Standard retries fail when endpoints return HTML error pages instead of JSON during a routine reboot. If your pipeline assumes consistency, it breaks.

A robust civic data pipeline treats every payload as untrusted input. We run strict schema validation at the boundary and drop malformed records into a dead-letter queue instead of letting bad data poison downstream models. We write idempotent parsers so running an extraction script twice for a delayed tax assessment batch doesn't duplicate entries in the database. Defensive programming here is a basic requirement.

Here is a Python fetch-and-parse routine that handles unexpected payload shifts without crashing your workers:

import requests
import json
import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

def fetch_municipal_feed(endpoint_url):
 try:
 response = requests.get(endpoint_url, timeout=10)
 response.raise_for_status()
 except requests.exceptions.RequestException as e:
 logger.error(f"Network failure reaching civic endpoint: {e}")
 return None

 content_type = response.headers.get("content-type", "")
 if "application/json" not in content_type:
 logger.error("Upstream server returned non-JSON payload, likely an error page.")
 return None

 try:
 data = response.json()
 except json.JSONDecodeError:
 logger.error("Malformed JSON received from municipal server.")
 return None

 return data

This check stops common pipeline crashes caused by unexpected maintenance screens. Beyond code, our job involves transparency. When public data pipelines break, public trust in open government erodes. We monitor these ingestion jobs with the same rigor we apply to core telemetry and alert ourselves to stale datasets before anyone else notices the gap.

Building better civic tech means accepting messy municipal infrastructure and writing code that bridges the gap between old systems and modern expectations.

Why Isolated Vector Database Benchmarks Are Useless

Puneet Khandelwal — Mon, 05 Oct 2026 18:38:27 +0000

If you judge a vector database by its raw QPS on a static HNSW index, you are buying a sports car based entirely on how fast it rolls down a hill. Most published benchmarks test storage engines in sterile isolation. They load a million synthetic embeddings, run single-threaded nearest neighbor queries, and declare a winner.

Real production systems do not work like this.

In a real SaaS app backed by an LLM, vector search is rarely a solitary operation. It competes with write transactions, metadata filters, and connection pooling from multiple microservices. When a user queries your RAG pipeline, the system hits the database while concurrently updating user session states and ingesting new document chunks. Static benchmarks ignore the garbage collection pause that happens when memory pressure spikes during concurrent writes.

Let us look at what actually breaks when you move from a benchmark script to production.

First, filtering destroys index locality. Vector search engines love pure vector math. Add a basic metadata filter, such as restricting results to a specific tenant ID or date range, and the query planner has to choose between pre-filtering and post-filtering. Pre-filtering shreds recall if the subset is too small. Post-filtering fetches too few candidates to compute accurate similarities. Most benchmarks skip complex metadata filters entirely because it ruins their clean latency numbers.

Second, memory mapping is not free. When your working set of vectors exceeds RAM, performance degrades rapidly. Benchmark scripts often fit entirely within cache. Real production workloads hit disk, triggering page faults that expose the underlying storage layer to latency spikes.

Here is a simple python snippet demonstrating how a naive retrieval setup hides the cost of metadata filtering under load:

import time
import numpy as np

def simulated_rag_query(index, vectors, metadata, query_vector, tenant_id):
 start = time.time()
 # Naive post-filtering: fetching top-k then filtering in application space
 raw_results = index.search(query_vector, k=100)
 filtered = [r for r in raw_results if metadata[r['id']]['tenant'] == tenant_id]
 latency = time.time() - start
 return filtered[:5], latency

This code looks harmless until concurrent threads flood the index. The database has to do extra work that the micro-benchmark never measured. If you rely on the database's internal filtering, the execution path shifts dramatically depending on selectivity.

The non-obvious implication here is that your vector database choice matters less than your data layout and chunking strategy. A mediocre database with a well-partitioned namespace and aggressive caching will easily outperform an enterprise grade engine configured with default settings and a monolithic index.

Stop reading synthetic benchmarks. Build a staging environment that mirrors your exact peak write-to-read ratio. Throw concurrent metadata updates at it, inject noisy background jobs, and measure the p99 latency under actual stress. That is the only benchmark that tells you whether your infrastructure will survive production.

Stop buying kitchen gadgets. Fix your vertical storage instead.

Puneet Khandelwal — Sun, 04 Oct 2026 16:01:49 +0000

If your kitchen counters look like a battleground of coffee grinders and blenders, you don't need a bigger kitchen. You need a better layout.

Counter space is prime real estate. When gadgets take over, your actual workspace shrinks, forcing you to prep meals around clumsy obstacles. Instead of blowing your budget on another countertop appliance, look up. Moving your daily essentials to the walls changes how a kitchen functions.

Think about your kitchen like a code repository. If your main branch—your prep surface—is constantly blocked by bulky toasters and knife blocks, nothing gets built efficiently. Vertical storage acts like a refactoring for your cooking routine. When you mount spice racks, utensil rails, and shelving directly to the wall, you pull clutter off the horizontal plane and open up square footage.

Modular units work well here. Grab a narrow shelf or cube storage unit, slide it into an awkward gap near your counter, and gain a dedicated coffee station. The trick is keeping footprints minimal while maximizing vertical height. Stack items logically based on how often you reach for them instead of letting things spread out horizontally.

Mounting things in a kitchen carries risks. If you rent, or just hate patching drywall, drilling holes for every small spice rack is a headache. Pressure-sensitive adhesive strips let you hang lightweight organizers securely without tearing up tiles. Check weight ratings carefully. Steam, grease, and heat near the stove weaken adhesives over time, so keep heavy cast iron pans on sturdy wall-mounted brackets and reserve strips for lighter tools.

Stop treating counters as permanent storage. Use your walls, and meal prep stops feeling like an obstacle course.

Hosting Open-Source Civic Tech Without Compromising Public Trust

Puneet Khandelwal — Sat, 03 Oct 2026 15:23:35 +0000

Municipalities run public portals, voting tools, and budget trackers on open-source repos. Building government software for years taught me one thing: standard dev workflows clash hard with public infrastructure security. We treat civic code like SaaS apps and forget public-sector data has entirely different threat vectors.

Open civic frameworks enable transparent auditing. Watchdogs and citizens check the source code for fairness in algorithms or municipal spending. Publishing that code also hands attackers an exploit blueprint. When a vulnerability drops in a public repo, automated scanners hit exposed servers in hours instead of months.

Securing these deployments means ditching perimeter defense for zero-trust. If you maintain a civic repo, code quality isn't your only worry. You need to ensure stolen contributor creds can't push malicious payloads into production builds.

Here is what that looks like in a standard CI pipeline for municipal apps. We isolate build runners, enforce container image signing, and lock down network egress. Check this baseline GitHub Actions workflow that signs build artifacts before they hit public registries:

name: Secure Civic Build
on:
 push:
 branches: [main]

jobs:
 build-and-sign:
 runs-on: ubuntu-latest
 permissions:
 contents: read
 id-token: write

 steps:
 - name: Checkout Source
 uses: actions/checkout@v4

 - name: Set up Docker Buildx
 uses: docker/setup-buildx-action@v3

 - name: Build and Push Container
 uses: docker/setup-buildx-action@v5
 with:
 push: true
 tags: registry.civic.gov/app:latest
 outputs: type=image,name=target,annotation-index.org.opencontainers.image.source=${{ github.repository }}

 - name: Sign Artifact
 run: |
 cosign sign --yes registry.civic.gov/app:latest

Crypto doesn't fix the human side of civic tech. Municipal IT departments run on tight budgets and thin staff. They use third-party hosts or cloud services that often leave storage buckets misconfigured with sensitive citizen data. A public budgeting tool leaks personal info the second database permissions match standard dev defaults.

This friction causes shadow civic IT. When official channels move too slowly, well-meaning devs spin up unvetted open-source instances on personal cloud accounts. That leaves fragmented data silos outside public records laws and oversight boards.

Our job goes beyond clean functions and passing tests. When we write civic infrastructure, we build the digital public square. Treat deployment pipelines as critical public utilities that demand strict auditing and threat modeling.

Why Dev Teams Still Fight Over Semicolons

Puneet Khandelwal — Fri, 02 Oct 2026 12:40:31 +0000

Every engineering team loses an afternoon to a religious war over code style. Tabs versus spaces. Trailing commas. Brace placement. Seniors roll their eyes while juniors take it personally, and leadership wonders why ticket velocity drops during refactoring sprints.

We pretend these arguments are about readability. We cite studies and compiler performance, but we lie to ourselves. Code style debates are proxy wars for cognitive load.

Here is what happens under the hood. Human memory is associative, and reading code requires holding a mental model of execution state in working memory. When you read code written in a style that diverges from your internal heuristics, your brain spends cycles on translation. It reads like a foreign language written in a familiar alphabet. You parse the words, but you cannot skim for meaning without friction.

Now layer artificial intelligence into this dynamic. Modern codebases are hybrid creations born half from human intent and half from autocomplete models trained on the public internet. Every time an LLM injects a block of TypeScript into your editor, it brings its own stylistic consensus. Sometimes it writes idiomatic modern syntax. Sometimes it dredges up patterns from ancient repositories.

This introduces a new source of cognitive noise. We used to fight because humans have distinct signatures. Now we fight because our tools have zero taste, churning out code that lacks internal stylistic consistency.

When a pull request triggers a massive debate about naming conventions or structural layout, the team tries to impose order on chaos. They attempt to lower the cognitive barrier to entry for whoever debugs that function at three in the morning.

Smart teams stopped fighting about this manually years ago. They automated the fix with strict linters and formatters running on every commit. If the machine enforces a single standard, human variance drops to zero, and emotional energy goes back where it belongs.

// A linter does not care about your feelings, only your syntax tree.
{
 "semi": true,
 "trailingComma": "es5",
 "singleQuote": true,
 "printWidth": 80
}

Even with formatters, style wars persist because formatting is only skin-deep. Deeper disagreements involve architectural patterns, modularity, and error handling philosophy. These resist config files.

When an AI assistant suggests a monolithic function spanning two hundred lines with deeply nested conditionals, it violates maintainable software engineering. Accepting that code saves time today and costs sanity tomorrow. Pushing back feels like pedantry, but it is self-defense.

We fight over code style because code is our primary medium of thought. If the syntax is messy, our thinking becomes messy. As tooling grows powerful, keeping codebases clean is the only way to remain the authors of our systems instead of their editors.

Local-First Heart Rate Tracking: A Database Schema Story

Puneet Khandelwal — Thu, 01 Oct 2026 09:30:10 +0000

Your wearable is leaking your pulse to a server you don't own.

Every time you run, a Bluetooth Low Energy peripheral broadcasts your cardiac rhythm into a proprietary black box. I got tired of export buttons that take three business days to email a zip file of CSVs, so I spent a weekend building a local-first pipeline for my fitness data.

Heart rate telemetry is just numbers over time. Yet, standard relational databases choke on the write amplification when you ingest one sample every single second. Treat a continuous stream like a standard CRUD table and your disk IO spikes while queries crawl.

The fix borrows a page from industrial IoT engineers. Instead of appending raw rows indefinitely, I designed a SQLite schema that relies on rigid time-bucket partitioning. Active workout data stays in a rolling in-memory buffer before flushing fixed five-minute chunks into compressed binary blobs on disk. SQLite handles the file management, and a lightweight custom parser runs the delta compression.

Here's the core schema design that keeps my local dashboard snappy, even with millions of raw samples:

CREATE TABLE workouts (
 id TEXT PRIMARY KEY,
 started_at INTEGER NOT NULL,
 sport_type TEXT NOT NULL
);

CREATE TABLE heart_rate_chunks (
 workout_id TEXT NOT NULL,
 bucket_epoch INTEGER NOT NULL,
 sample_count INTEGER NOT NULL,
 compressed_deltas BLOB NOT NULL,
 PRIMARY KEY (workout_id, bucket_epoch),
 FOREIGN KEY (workout_id) REFERENCES workouts(id)
);

Storing small integer deltas instead of absolute timestamps and raw beats-per-minute integers shrinks the storage footprint hard. A two-hour marathon trace fits comfortably inside a file smaller than a high-resolution JPEG. More importantly, I can run local analytics without my laptop fan spinning up to jet-engine speeds.

Building your own health stack forces you to confront how little control we have over our most intimate metrics. When your wellness data lives on your own hardware, you stop looking at fitness as a streak to maintain for an app badge and start treating it as a personal dataset to actually understand.