DEV Community: Devico Solutions

How to Modernize a Medical Imaging Platform Without Compromising Availability

Vic Tekys — Tue, 03 Nov 2026 10:31:00 +0000

A radiologist mid-read is a hard stop on your maintenance window. Not "please schedule around it" — a hard stop. And the old trick of deploying at 2 a.m. dies the moment your platform serves teleradiology across several time zones, because it's always working hours somewhere.

The reflex answer is more redundancy: another replica, blue-green deploys, chase another nine. I've watched teams spend quarters on that and still get paged, because the real availability risk on imaging platforms usually isn't in the deploy pipeline. It's in not knowing how the system is assembled — and in touching the wrong layer first.

After several years of working on radiology and imaging systems at Devico, my rule is: modernize from the outside in. The layers furthest from the reading path — infrastructure, backup and DR, monitoring, device fleets, analytics — absorb change safely, and that's where most of the actual risk lives. The viewing and reporting path changes last, incrementally, and only once you can observe it and restore it.

The stakes aren't abstract. In the 2025 Ponemon/Proofpoint study of 677 US healthcare IT and security professionals, 72% of organizations hit by common cyberattacks reported disruption to patient care, and the average cost of the most significant attack was $3.9 million. On an imaging platform, "disruption to patient care" means someone's diagnosis is late.

What "availability" means when a radiologist is mid-read

Before you modernize anything, get precise about what failure looks like. Imaging platforms fail in ways uptime checks are blind to.

Downtime, degraded reads, and lost studies are three different failures

They have different blast radii and different fixes:

Downtime — the worklist won't load, nobody can read. Loud, obvious, and honestly the rarest of the three.
Degraded reads — the worklist responds, but studies arrive late, images render slowly, prior exams take 30 seconds to hydrate. The radiologist works slower and trusts the system less. Your HTTP 200 monitoring says everything is fine.
Lost or stalled studies — a routing device drops a series, a study sits in a queue nobody watches. This is the worst one, because nothing looks broken until a clinician asks where the scan went.

We saw the second failure mode play out at medneo, a radiology-as-a-service provider founded in 2012 that runs MRI and CT sites across Germany and the UK on a bespoke RIS. There was no outage to point at. Patient volumes grew, features kept shipping, and load times quietly degraded while regressions started to slip through. Availability eroded without a single incident ticket.

So the first modernization deliverable isn't a new architecture. It's a definition:

Split the platform into paths: ingest, storage, routing, viewing, reporting.
Give each path its own SLO.
Measure what clinicians feel — study-arrival latency, time-to-first-image, worklist refresh time — not HTTP status codes.

The honest limitation: per-path SLOs need instrumentation that legacy imaging stacks don't have. That's not a reason to skip it. That's your first project.

And the load is only going up. Signify Research counted 5.9 billion diagnostic imaging procedures globally in 2024 across CT, MRI, X-ray and ultrasound, growing at a projected 4.5% a year through 2029. Meanwhile the Harvey L. Neiman Health Policy Institute projects that imaging demand will grow up to 26.9% by 2055 against a 25.7% growth in radiologist supply — the shortage persists. Every radiologist-minute your platform wastes is getting more expensive, not less.

Map the system before you change it

Here's the unglamorous truth: on most legacy imaging estates, the biggest availability risk is that nobody can tell you how the thing is put together.

Our engagement with OpenRad started exactly there. OpenRad — formed in November 2022 from the merger of Biotronics3D and Visbion — runs a cloud radiology reporting platform with a teleradiology module, an enterprise worklist, and a thin-client architecture that streams pixels instead of transferring image data. The brief wasn't "rearchitect the platform." It was: enterprise-standard IT, minimum downtime, and critical-system availability in a disaster event.

What we found first was an infrastructure with no documentation and a recovery path nobody had ever tested. Until that's fixed, every change is a gamble — you can't reason about blast radius on a system you can't describe.

This isn't an imaging-specific quirk. In the Uptime Institute's 2025 outage analysis, 40% of organizations reported a major outage caused by human error in the past three years — and 85% of those incidents traced back to staff not following procedures, or the procedures themselves being broken. Undocumented systems don't just fail more; they fail in ways you can't rehearse.

So the sequence before any migration is:

Inventory the critical services. Every VM, every dependency, every "oh right, that box" that turns out to route half the studies.
Write the runbook. On OpenRad we kept documentation in Trilium Notes — the tool matters far less than the habit.
Test the restore. Not "backups ran green." Actually rebuild a critical service from backup and time it.

A team that can't rebuild a node deterministically has no availability story, whatever its deploy strategy diagram says.

The cost is real, though: discovery produces no user-visible change for weeks, which makes it the first thing stakeholders want to cut. Defend it anyway.

Layer one — infrastructure you can rebuild and restore

Once the map exists, the highest availability-per-effort work is the layer that touches zero clinical code: virtualization, provisioning, backup, monitoring.

On OpenRad, a two-person Devico team — one sysadmin, one BI developer, engaged since 2023 — standardized the estate on:

Proxmox VE for virtualization, replacing an unoptimized, inconsistent setup
Proxmox Backup Server for backups that get restore-tested, not just taken
Ansible so a node is a playbook run, not an artisanal snowflake
Prometheus + Grafana so degradation shows up on a graph before it shows up in a radiologist's complaint

Deliberately boring choices, and mostly open source — which also cut vendor licence cost, a real consideration when the platform runs on infrastructure margins.

Why backups-you-can-actually-restore leads the list: Veeam's 2024 Ransomware Trends report (1,200 organizations that had been breached) found victims were unable to recover 43% of their affected data, and one in three organizations that paid the ransom still couldn't recover. For a hospital-facing platform, "we lost 43% of the studies" is not an IT incident, it's a clinical one. Ransomware pressure on the sector isn't easing either — Comparitech tracked 445 ransomware attacksagainst healthcare providers in 2025 alone.

The monitoring half of this layer is what makes the per-path SLOs from earlier real. The shape of the check matters more than the stack — what you want is an alert on the clinical symptom:

Alert on the thing the radiologist feels, not on the box

alert: StudyArrivalLatencyHigh

expr: histogram_quantile(0.95, rate(study_arrival_seconds_bucket[10m])) > 300

for: 15m

annotations:

summary: "p95 study arrival exceeds 5 minutes — reads are degrading"

(That's the shape, not OpenRad's config — the point is the metric: seconds from modality push to worklist-visible, not CPU on the ingest box.)

Be honest about what this layer is: infrastructure hardening, not application modernization. The result on OpenRad was a stable, enterprise-level infrastructure with documentation and a tested critical-service recovery plan. Nobody's reading experience changed — and that's exactly the point. You just removed the biggest class of catastrophic risk without touching the reading path.

Layer two — the transport edge: DICOM routing and device fleets

Engineers who haven't worked in imaging consistently forget this layer exists. A big share of an imaging service's reliability lives in hardware sitting at the hospital edge — devices that route, translate, compress and encrypt DICOM between modalities, trailers, and the platform.

Visbion is the sharpest example I've got. Their Image Cube is a DICOM 3.0 routing, translation, compression and encryption device used by NHS Breast Screening and Diagnostic Imaging Services — including managing mobile scanning trailers — and it's the UK's largest installed base of dedicated imaging routing hardware. When investment-funded growth pushed the fleet into the hundreds of devices, the constraint wasn't application code at all. It was fleet management.

The work — three Devico DevOps consultants, engaged since 2023 — was classic outside-in:

Infrastructure automation for fleet deployment, so provisioning hundreds of edge boxes stops being manual toil
DICOM compression and encryption optimization, because at fleet scale, transport efficiency is a reliability property
A modular redesign using cloud components, so the architecture can absorb the next few hundred devices

The reported outcome: lower operational costs and a marked increase in the performance and reliability of the entire imaging service. I'll flag the limitation myself — those results are qualitative; no before/after numbers are published.

Two more constraints make this layer slower than it looks:

Edge changes are gated by hospital network policy — you ship on the hospital's terms, not your sprint cadence.
Some of it needs physical access. You can't ansible-playbook your way into a scanning trailer that's parked at a different site today.

But notice what this layer shares with layer one: massive availability upside, zero changes to the viewing and reporting code that a radiologist depends on mid-read.

Layer three — the application: refactor under load, don't rewrite under load

Eventually you do have to touch the reading path. The question is how.

Under live clinical load, my strong preference is refactoring, framework upgrades and pipeline work over structural rewrites. A rewrite concentrates regression risk, and on an imaging platform regression risk is availability risk — a bug in study routing isn't a bad release, it's a delayed diagnosis.

medneo is the case I keep coming back to. The RIS wasn't down; it was accreting features, and every release made it a little slower and a little more regression-prone. Over three years (2019–2022), a team of eight Devico engineers worked on it like this:

Code optimization on the hot paths — including hunting down and removing unnecessary database calls that had accumulated over years of feature work.
Framework upgrades — moving legacy frameworks to current, more efficient versions in increments, instead of a big-bang platform swap.
CI/CD introduction — so every change to the reading path ships through the same tested pipeline.

The published results are directional, not numeric, and I'll keep them that way: significantly reduced load times, fewer outages, reduced impact from new feature rollouts, and headroom to scale with patient growth without performance loss.

The CI/CD piece deserves emphasis. On a system where you can't afford regressions, the pipeline isn't developer comfort — it's an availability control, the mechanism that lets you change the most sensitive layer in small, reversible steps.

And the limitation, stated plainly: refactoring cannot raise an architectural ceiling. If the reading path's fundamental design can't carry your growth, incremental optimization is buying time, not solving the problem. More on that at the end.

Layer four — move analytics off the transactional path

The last layer is about adding capability without adding risk: reporting and decision support belong beside the transactional system, on separate stores — never as heavy queries against the database your worklist runs on.

The failure mode you're preventing is mundane: someone builds a management dashboard against the production database, a quarterly report kicks off mid-morning, and suddenly image load times spike for every reading radiologist. Analytics traffic and clinical traffic have opposite shapes, and they should never compete for the same I/O.

On OpenRad this was the second workstream: BI across multiple databases, built on Power BI, Azure and MS SQL Server, shipped as standardized dashboard templates that OpenRad's clients drive themselves — slicing their own data without anyone touching the clinical path to answer a question.

For the architectural pattern at its cleanest, look at GazeHealth — with a caveat I'll make explicit: it's a mental-healthcare analytics platform and EMR, not an imaging system, so take it as evidence for the separation pattern only. Six Devico full-stack developers and two AI data scientists have been building it since 2019 as two components — an EMR and an analytics platform — designed to work independently or together, with role- and company-based routing enforced through AWS Cognito so users never even get routes to data they can't access. Either half can evolve, degrade, or be rebuilt without taking the other down. That's the property you want between your imaging platform and its analytics.

The costs of decoupling, honestly:

Your analytics are now behind the transactional truth by whatever your pipeline latency is.
You own a second consistency model, and someone will eventually ask why the dashboard and the worklist disagree.

Both are acceptable prices for the guarantee that a heavy query can never slow a read.

What this sequencing costs you

Outside-in sequencing is not free, and pretending otherwise is how the plan dies in month three. Going in, know that:

User-visible progress arrives slowly. Documentation, backup, monitoring and fleet work are invisible to clinicians for months. If leadership expects a new viewer by Q2, this ordering will read as stalling — set the expectation before you start.
The legacy core lives longer. You are deliberately keeping the old reading path in production while you harden everything around it. That's a feature of the plan, but it means carrying legacy operational cost longer than a rewrite roadmap promises (and roughly as long as a rewrite roadmap actually delivers).
It needs a whole-system owner. Without someone accountable for the full sequence, you end up with a permanently half-modernized estate: nice dashboards, solid backups, and a reading path nobody ever got to. These are multi-year engagements — medneo ran three years, OpenRad and Visbion have been running since 2023.

The ordering rule underneath all of it: you can't safely change what you can't restore or observe. So documentation, DR and monitoring come first. The transport edge and analytics come next, because they're separable from the reading path. The reading path itself comes last, and moves in small, pipeline-guarded steps.

And one test decides whether this playbook applies to you at all. Ask: can the current reading path, refactored but not rearchitected, carry the next two years of study volume? If yes, work outside-in and let the core evolve incrementally. If no, the infrastructure work is still worth doing — but be honest that it's buying time for a deeper change on the core, and start planning that change now, while your restores still work and your graphs still tell you the truth.

How to Take a Fintech Product from Web to Mobile

Vic Tekys — Tue, 27 Oct 2026 10:27:00 +0000

Every web-to-mobile project I've been on had the same scoping meeting. Someone opens with "we already have the API, so mobile is mostly UI," the room nods, and a quarter gets pencilled in.

That sentence hides four things: the read contract your web app grew up with, the auth model that lives in a cookie, the OS plumbing a browser gave you for free, and a release cycle where "ship the fix" meant a redeploy, not a store review.

Most of the work lands behind the screens. The mobile client is the cheap part; what costs money is everything the web app was allowed to assume. This is about re-negotiating those assumptions, with shipped projects as evidence.

Decide what parity means before you scope

"Parity" is the default nobody chooses. It arrives because nobody wrote down a narrower target, and it quietly becomes the acceptance criterion.

There are three defensible targets, and you should pick one out loud:

Companion app. Read everything, do one core action (approve a payment, sign a transaction, check a balance). Cheapest, and often the right first release.
Functional parity. Every flow on web works on mobile. This is what most stakeholders mean when they say "the app."
Mobile-first rebuild. Mobile is a net-new surface with its own flows, and web follows later.

Two projects show the ends of that spectrum. On Moneygate, a payment provider regulated by the Central Bank of Cyprus, the brief was explicitly a mobile app replicating all the functionality of the web platform: payments, multi-currency, fraud prevention, reporting, analytics.

Parity was a stated requirement, and a five-person team delivered it: two React Native devs, a QA engineer, a designer and a business analyst. The BA mattered; the case page counts more than 22 business flows and features that had to be documented before they could be scoped.

Ctrl, the wallet formerly known as XDEFI, went the other way. The product existed only as a web version and a browser extension, and "no mobile access" was the first problem on the list. Mobile there was new surface, not a port.

One limitation to accept up front: parity is measured in flows, not features. A web screen with six optional filters and a CSV export is not one mobile screen; it's three, plus a decision about which filter combinations deserve a bottom sheet.

Your web API is the first thing to break

Web APIs evolve under assumptions mobile invalidates:

Many small round trips are fine, because the tab sits on Wi-Fi.
Large payloads are fine, because the page renders whatever arrives.
Offline is a non-case; the user reloads.
Old API versions die the moment you redeploy, because every client reloads on the next visit.

Mobile invalidates all four, and the last one hurts most. The moment you ship a binary, you own a population of clients you cannot force-upgrade.

In June 2025 Apple's own numbers put iOS 18 on 82% of all active iPhones, nine months after release: one in five devices still on an older OS. Android is worse; April 2026 Statcounter data compiled by apilevels shows Android 11 or newer on 86.9% of devices, with the newest release at 22.3%.

Your app's update curve follows the same shape, so a /v1 endpoint you'd love to delete will get traffic for a year.

Chatty endpoints, cold starts and the unified read

The practical fix is server-side aggregation and deliberate versioning. Android vitals flags a cold start of 5 seconds or longer as excessive, and a home screen that fans out to eight REST calls on a cellular connection eats that budget before the first balance renders.

DEFI Wallet, a non-custodial multichain wallet spanning 20 blockchains, is the clearest example of building the read contract once. Its backend is a set of microservices behind a GraphQL router that combines their schemas: blockchain indexers for real-time transactions, a Python service for token metadata, and Node.js/Nest.js Asset and NFT balance services.

That single graph serves the browser extensions (Chrome, Edge, Opera, Brave), the web app and the iOS/Android apps. Mode, a UK-listed crypto company with GBP, EUR and BTC accounts, used microservices for the same reason across its mobile backend and admin portal, exposing GraphQL to both.

What that looks like from the client side is one query per screen, not one per widget:

One round trip for the portfolio screen; the router fans out server-side

query PortfolioScreen($account: ID!, $fiat: Currency!) {

account(id: $account) {

balances(fiat: $fiat) { asset { symbol chain } amount fiatValue }

nfts(first: 20) { edges { node { id name imageUrl } } }

recentTransactions(first: 10) { hash status timestamp }

}

The thing to look at is the first: arguments. Pagination limits that were optional on web become mandatory on mobile, because the alternative is a 4 MB response on a train.

The limitation is real, though. A unified API is a coupling decision: one aggregation layer serving three clients becomes the bottleneck every team queues behind, and a schema change for the extension now has a mobile review attached to it.

In the 2024 State of React Native survey, 59.3% of respondents use TanStack Query and 34.2% use Apollo Client. Whichever you pick, the server contract decides how many requests a screen makes, not the client library.

State management doesn't survive the trip

Redux-shaped web state is tuned for a tab that reloads on demand. A mobile app backgrounds, loses the network in a lift, and resumes four hours later with a store full of stale balances and an expired token. The state model has to separate two things the web app blurred:

Server state: cache, refetch, invalidation, background sync. Belongs to a query library.
Client state: navigation, form input, UI toggles. Belongs to a small store, persisted deliberately.

Moneygate used React Query for server state under a server-state pattern, with React Hook Form for input. Ctrl runs Redux Toolkit with Redux Persist in the extension but Zustand in the React Native app. DEFI Wallet made the same split: Redux on web, Zustand on mobile.

That is three teams independently accepting two state libraries in one product. They're not alone: Redux is still used by 71% of React Native respondents and Zustand by 46.7%, with Redux complexity the third most-cited pain point. Two stores is a maintenance tax; the case evidence shows teams paying it, not that it's free.

The part that bit us was hydration ordering. A persisted store that rehydrates after the first render shows a logged-out screen for one frame, then flips, and on a payments app that flash is a support ticket. Gate rendering on hydration explicitly:

// Zustand + persist: don't render auth-dependent screens until the store is back

export const useSession = create()(

persist(

(set) => ({ token: null, hydrated: false, setToken: (t) => set({ token: t }) }),

{

name: 'session',

storage: createJSONStorage(() => secureStorage), // not AsyncStorage for tokens

onRehydrateStorage: () => (state) => state && (state.hydrated = true),

);

The comment on storage is the point. AsyncStorage is used by 83.8% of respondents in the same survey, and it is plaintext. Session material goes in the keychain or keystore; everything else can go in AsyncStorage or MMKV.

Platform plumbing is the backlog nobody writes down

A browser handed the web app downloads, PDF viewing, printing and camera access, and none of it appeared in a ticket. On mobile each is an integration with its own permission model and OS-version behaviour. Budget them as features, not tasks.

Files, PDFs and the Android storage rules

Moneygate's challenge list included secure file downloads on Android and PDF preview with actions. Neither is exotic on web. On Android, apps targeting Android 11 or later run under scoped storage; the system ignores the requestLegacyExternalStorage flag, WRITE_EXTERNAL_STORAGE grants nothing extra, the Storage Access Framework refuses ACTION_OPEN_DOCUMENT_TREE on the Download directory, and declaring All Files Access "can affect your ability to publish and update your app on Google Play."

So the web app's became, on Moneygate:

expo-file-system with the Storage Access Framework for user-directed saves.
rn-pdf-reader-js for in-app PDF preview with actions.
react-native-gesture-handler plus bottom-sheet for full-screen modals replacing traditional buttons.
Reanimated for animation performance, because a statement list that stutters reads as a broken app.

Soapbox, a fulfilment-tech company's Android warehouse app, hit the other classic: React Native library compatibility problems implementing barcode scanning, settled with vision-camera-code-scanner and react-native-print.

Every one of those libraries carries native code, narrows your upgrade path and becomes a dependency you own through every SDK bump. The 2024 survey put New Architecture adoption at almost 50% with library compatibility the standing concern; the PDF viewer you picked in year one is exactly what blocks you in year two.

Biometrics are an unlock, not an authentication

This is the argument I have most often with product owners. Face ID does not log a user in. It gates something on the device, and the server should never learn or care that a biometric happened.

Treat it as a local factor:

The server issues a session or refresh token after real authentication (password, passkey, OTP, whatever your risk model requires).
The token sits in the keychain or keystore, wrapped by a key that requires user presence.
Biometrics unlock that key. Server-side authorization is unchanged and still expires and revokes on its own schedule.

Android's docs separate BIOMETRIC_STRONG (Class 3) from BIOMETRIC_WEAK (Class 2) and provide CryptoObject so a successful prompt unlocks a Cipher or Signature rather than returning a boolean. OWASP lists cryptographic binding as a best practice for exactly that reason: a boolean callback can be bypassed or replayed; a keystore key with setUserAuthenticationRequired(true) cannot be used without the prompt succeeding.

Moneygate used expo-local-authentication for biometric authorization. That library gives you the prompt; the key-wrapping around it is your design.

Users want this. In the FIDO Alliance's 2024 barometer, biometrics ranked as the login method consumers consider both the best experience and the most secure for the second year running, and 42% had abandoned a purchase in the previous month over a forgotten password. That demand is exactly why teams are tempted to let the prompt stand in for auth.

Custody and secrets change shape on mobile

Custodial fintech has the easy version: the secret is a session token, and the worst case is re-login. Non-custodial products have the hard version. The key material lives on a device you don't control, next to a screenshot tool, a clipboard and whatever the user side-loaded.

Ctrl and DEFI Wallet are both non-custodial. DEFI Wallet's case page states it does not store users' seed phrases or passwords, and it ships the same key operations across extensions, web and mobile, with Ledger hardware-wallet support as an alternative to on-device keys.

What I can offer as engineering practice, kept separate from what those clients published:

Generate and use keys inside the platform's hardware-backed store; export only for recovery flows, never for convenience.
Bind signing to Class 3 biometrics or a device credential, with explicit confirmation per transaction, not per session.
Invalidate biometric-bound keys when enrolment changes, another OWASP best practice for the same reason.
Design recovery as a first-class flow. A wallet whose recovery path is "the user wrote the phrase down" has outsourced its most dangerous UX to a napkin.

No source describes Ctrl's or DEFI Wallet's key storage, recovery or attestation design, so don't read that list as their architecture. It's the checklist the shipped surface implies you'll need.

Reuse what's shared, fork what isn't

The shared-code fantasy: one monorepo, one business-logic package, two thin UI shells. In practice the reuse band is narrower:

Design tokens. Colours, spacing, typography. Cheap to share, high payoff.
Components. Worth sharing when the primitive is the same on both platforms; the layout around it usually isn't.
Validation schemas. Zod or equivalent; IBAN rules don't change with screen size.
Business logic. Partial at best. The moment a flow touches storage, auth or navigation, it diverges.

Ctrl's problem list named component inconsistency across platforms as something slowing development, and the answer was a unified component library for web and mobile, with the extension on Tamagui and Zod. DEFI Wallet listed accelerating web development through component reusability as an explicit challenge.

Both teams treated the component layer as the reuse target. Neither claims they shared the business logic.

The cost is coupling. A shared library forces one release cadence on two platforms with different constraints: the extension can ship a fix the same day; the app cannot. That's the price of consistency, and it's worth paying only if you also set up the release side.

Shipping to a store is not shipping to a URL

This is the shift teams underrate. Store review, staged rollout and users on months-old binaries change how you branch, feature-flag and hotfix.

The numbers are better than they were, but they're not a redeploy:

Apple says 90% of submissions are reviewed in under 24 hours. The other 10% is where your Friday incident lives.
Google Play's staged rollouts let you halt a release, and the percentage never increases on its own; you go up, you don't go down.
Apple's guideline 2.5.2 says apps may not download or execute code that introduces or changes features. OTA JavaScript updates live inside the interpreted-code exception and the platforms' content rules; Expo's own docs are blunt that EAS Update handles non-native pieces such as JS, styling and images and anything native, any permission change, any SDK bump needs a new build.

Soapbox is what it looks like when none of this is set up. The app arrived with no cloud-based JS update system, no automated code review or deployment, no separation between development, staging and production, frequent push conflicts, and manual Play Store updates required from users.

The fix was environment stratification via GitLab and CodePush, ESLint, Datadog and full-cycle testing, followed by more than 15 successful releases. Fifteen is a count, not an improvement measure, but it's fifteen more than a team with no pipeline ships. Mode's case page lists 23.

Two more things that shape release engineering, both easy to miss at scoping:

Your OTA vendor is a dependency. Microsoft retired App Center and CodePush on March 31, 2025 and published a standalone server to self-host. Adopt CodePush in 2023, inherit a migration in 2025. In the 2025 State of React Native survey, EAS Build is at 68.2% and GitHub Actions at 61.9%; the ecosystem consolidated, and that's a bet on someone else's roadmap.
The web side doesn't escape platform-mandated rewrites. Ctrl's extension had to migrate from Manifest V2 to V3. Chrome disabled MV2 for all users on all channels by March 2025, removed the re-enable switch with Chrome 138, and deletes the remaining MV2 extensions from the Web Store on August 31, 2026. An extension is a store-distributed client with its own review; teams that file it under "web" get surprised.

Set up environments and OTA before the first store submission, not after the first incident. After the first incident you'll do it anyway, under worse conditions.

What the move actually costs

Stated plainly, going web-to-mobile buys you:

Two more runtimes with their own crash signatures.
Two more release channels, each with review and rollout mechanics.
A device and OS-version test matrix that grows every September.
Permanent version skew, so every API change is a compatibility decision.
A native-dependency surface (files, PDF, camera, biometrics) you upgrade by hand.

None of that is a reason not to do it. Moneygate's five-person team and Ctrl's twelve-person dev and QA team show the headcount range between parity and net-new surface. Ctrl reports a user base expansion of 5–7% and a revenue increase of 10–25% as ranges on its case page, but that engagement also covered the extension redesign, the MV3 migration and backend work, so none of it isolates the mobile app.

The counter-case: if the mobile use case is read-heavy and occasional, a responsive web app plus one narrow native capability (push, or a biometric-gated approval companion) is often the better trade. Apple's guideline 4.2 rejects apps that don't rise above a repackaged website anyway.

Sequencing it

If I were scoping this again tomorrow:

Pick a parity target and write it down in flows, not features.
Fix the read contract first: aggregate server-side, paginate everything, version deliberately, and plan to keep old versions alive for a year.
Decide the auth and custody model before the first screen. Biometrics unlock a local key; the server never trusts them.
Budget the platform plumbing (files, PDFs, camera, permissions) as features with owners.
Set up dev/staging/prod separation and OTA delivery before the first store submission.
Share tokens, components and schemas. Fork the logic and stop feeling bad about it.

The uncomfortable part is the last one. Once the app is live, the web product has a permanent second client that can't be hotfixed, running on binaries you shipped months ago, against an API you can no longer change on a whim. That constraint outlives the launch, and it's the real thing you signed up for in that scoping meeting.

How to Design an Offline-First Web App with IndexedDB and Conflict Resolution

Vic Tekys — Tue, 20 Oct 2026 09:16:00 +0000

An inspector finishes a site audit from a van with no signal. The form validates, the app says "saved," she drives off. Two hours later the device reconnects, the queue flushes, and one of two things happens: a colleague's older edit to the same record — synced first, because his van hit coverage earlier — silently overwrites hers, or the server rejects the audit on a rule the form never checked, and she finds out a week later, when the context is gone.

Both failures come from the same misunderstanding. The moment your client accepts a write without the server, it stops being a cache. It is a replica with its own history, and the design problem is what happens when two histories meet.

So my thesis: offline-first is a data-modeling decision, not a caching layer. The change log, the replay contract and the merge rule are the work. IndexedDB — the part most tutorials spend all their time on — is the easy part.

Decide what "offline" means per screen, not per app

The most expensive sentence in an offline project is "the app has to work offline." All of it? Every screen I've built lands in one of three classes:

Read-only offline. Catalogues, price lists, 3D models, reference data. A service worker and a cache solve this — nobody writes, so nothing conflicts.
Write-offline. Inspections, orders, audits, stock counts. The only class that needs a change log, replay and a merge rule.
Online-only. Payments, identity changes, anything with an external side effect — an email, a printed label, a card charge. Making these offline creates a duplicate-side-effect problem you cannot undo.

Only the middle class costs real money, so scoping it down is the biggest cost lever you have.

For scale, the industry mostly hasn't got past class one. The 2025 Web Almanac reports service workers on 20.5% of desktop sites, up roughly tenfold since 2022 — but only 12% of PWA sites' service workers even handle fetch. Most are installed for lifecycle events and install prompts, not offline behaviour of any kind.

The example from our work is a PWA for Castrol that authorizes and monitors the partner workshops selling its products, with both offline and online modes as a hard requirement. Authorizing a workshop is an audit workflow — the user records findings in the field. That is a class-two write path by definition, and it dictated the whole architecture below.

The limitation: the boundary moves. A read-only price list is class one until someone edits a quantity on it. Re-classify on every feature, not once at kickoff.

Store intent, not state

The hack persists final state: the user edits an inspection, you write the new object to IndexedDB, and on reconnect you PUT it. A snapshot cannot be merged — the server sees a whole object and can't tell which fields were touched, from what version, or in what order.

The architecture persists intent: an append-only log of mutations, with local state derived from replaying it.

What belongs in the change log

Each entry carries enough to be replayed, rebased, partially rejected, and shown to a user:

interface ChangeLogEntry {

opId: string; // client-generated UUID — the idempotency key

seq: number; // local monotonic sequence, for ordering

entity: 'inspection' | 'workshop' | 'finding';

entityId: string; // client-generated for creates, too

op: 'create' | 'update' | 'delete';

baseVersion: number | null; // server version this was made against

payload: Record; // only the fields touched

createdAt: number; // for display only — never for ordering

status: 'pending' | 'sent' | 'acked' | 'rejected' | 'conflict';

}

Three fields do the heavy lifting:

opId lets the server dedupe a replayed operation.
baseVersion lets the server detect that the record moved while the client was offline — the basis of conflict detection.
payload holding only touched fields is what makes per-field merge possible later.

On Castrol, the architecture kept all data in sync with local IndexedDB on the frontend, with the ability to log changes and new entries so the system could be updated as soon as the user reconnected, plus a merge-conflict resolution mechanism on top. The stack was React and Redux, Dexie.js over IndexedDB, Koa on Node.js, and CosmosDB on Azure. The case study doesn't say why Dexie or CosmosDB were picked, so I won't invent a rationale — the log-plus-merge shape is the point.

Two costs you sign up for:

Logs grow. Once an entry is acknowledged, fold it into the local snapshot and drop it — but keep the snapshot's version.
Replaying old entries against a schema that has since changed is a migration problem. The log format needs a version field too.

IndexedDB is a database — treat it like one

Most IndexedDB pain comes from treating it as a key-value bag. What I hold to:

Versioned schemas and upgrade transactions. Dexie's design doc is blunt: a version with an upgrader attached must never be altered — further changes go in a new version declaration, and the upgrade function for version 3 transforms data as it existed in version 2.
Real indexes for the queries you actually run. Dexie stores any property but only queries indexed ones. statusand seq on the change log matter, because "pending ops in order" is the query on every reconnect.
One transaction per unit of work. A create that touches inspections and appends to changelog is one transaction, or you will eventually get a logged intent with no entity, or the reverse.
A wrapper over the raw event API. Dexie auto-commits a transaction once you stop doing database work in it, and awaiting a non-Dexie promise inside one ends it — the failure everyone hits once.

import Dexie, { type Table } from 'dexie';

class FieldDB extends Dexie {

inspections!: Table;

changelog!: Table;

constructor() {

super('field');

this.version(1).stores({

inspections: 'id, workshopId, version',

changelog: 'opId, seq, status, [status+seq]',

});

}

export async function recordUpdate(db: FieldDB, id: string, patch: Partial) {

return db.transaction('rw', db.inspections, db.changelog, async () => {

const current = await db.inspections.get(id);

if (!current) throw new Error(unknown inspection ${id});

await db.inspections.update(id, patch);

await db.changelog.add({

opId: crypto.randomUUID(),

seq: await nextSeq(db),

entity: 'inspection',

entityId: id,

op: 'update',

baseVersion: current.version,

payload: patch,

createdAt: Date.now(),

status: 'pending',

});

}

Two things to look at: the compound index [status+seq] is the reconnect query, and baseVersion is captured at write time, not sync time.

Quota, eviction and the cache you thought was durable

Client storage is a budget, not a disk, and someone else sets the budget.

Per MDN's quota documentation, Chromium gives an origin up to 60% of total disk, Firefox's best-effort mode caps a site group at the smaller of 10% of disk or 10 GiB, and Safari on macOS 14 / iOS 17 allows roughly 60% of disk for browser apps but only about 15% for embedded web views.
Under storage pressure, browsers evict best-effort origins least-recently-used first, and eviction is all-or-nothing: IndexedDB, Cache API and OPFS for that origin go together.
Safari's tracking prevention deletes script-written storage from any origin the user hasn't touched in seven days of browser use — though web.dev notes the cap doesn't apply to installed PWAs.
A committed transaction isn't necessarily on disk. IndexedDB's durability option treats "relaxed" data as committed once handed to the OS, while "strict" waits for the storage medium. For a change log, ask for strict.

Two of our projects show storage discipline with large payloads. Vitus, a Danish constructability platform, stores 3D models locally in IndexedDB as part of a package that also moved to SVF2, parallel loading, web workers and list virtualization; the team reports models 5x smaller and loading roughly 2.5x faster — for the bundle, not IndexedDB alone, and Vitus is not offline-first. On a fashion-tech sales platform for in-store kiosks, the previous app cached so much media locally it ran out of memory; the rebuild moved to a central media service with caching and preloading plus Workbox, and reports a 95% smaller local cache, memory crashes eliminated, and 40% faster startup. That's storage hygiene, not offline sync.

The rule that follows: local writes are durable only until acknowledged, never the system of record. Losing the entire local store should lose at most the unsynced tail — and the user should be able to see how long that tail is.

Replay is a distributed-systems problem

Reconnection is not "retry the failed POSTs." It is a batch write from a stale replica, with all the failure modes of one. The contract I use:

The client sends pending operations in seq order, each with its opId and baseVersion.
The server processes each operation independently and returns a per-operation verdict: accepted (with the new version), rejected (with a reason), or conflict (with current server state).
The client marks each entry by verdict and never discards the batch because one entry failed.

type Verdict =

| { opId: string; status: 'ok'; version: number }

| { opId: string; status: 'rejected'; code: string; message: string }

| { opId: string; status: 'conflict'; serverState: unknown; serverVersion: number };

async function replay(db: FieldDB, api: SyncApi) {

const pending = await db.changelog

.where('[status+seq]').between(['pending', Dexie.minKey], ['pending', Dexie.maxKey])

.toArray();

if (!pending.length) return;

const verdicts = await api.push(pending); // same opIds every attempt

for (const v of verdicts) {

await db.changelog.update(v.opId, { status: v.status === 'ok' ? 'acked' : v.status });

if (v.status === 'conflict') await parkForResolution(db, v);

}

The batch is re-sent with the same opIds on every attempt. That is what makes three scenarios survivable:

Partial-batch failure. Five of twelve operations applied before the connection dropped. With per-op verdicts, the client knows which seven to resend; without them, it resends all twelve and the server must dedupe.
The tab that reconnects twice. Two tabs, both detect online, both flush. Same opId, so the server applies each once. Take a navigator.locks lock around replay anyway, so they don't also fight over the local log.
Two devices on one account. Phone and laptop, both with pending edits to one record. That isn't a replay bug, it's a merge — the merge-rule section below.

Idempotency and the duplicate-record bug

The naive replay outcome is the duplicate: a retried create that produces two records. Stripe's design is the reference I copy: the server saves the status code and body of the first request for each idempotency key — success or failure — returns the same result on every replay, errors if the parameters differ, and prunes keys after at least 24 hours. Your sync endpoint needs exactly that table, keyed on opId.

This bug class doesn't need an offline app to appear. Sway, a US logistics company running an online system, hit race conditions in trip creation where processing delays assigned parcels to the wrong trips; the fix was queue functionality that eliminated the unnecessary trip creation, with 20% less stored data reported as a result. Sway never had an offline client — I cite it because if concurrent creation misbehaves with milliseconds between requests, it will with hours of buffered writes arriving at once.

The limitation of strict ordering: one poisoned operation — a create the server will never accept — blocks everything behind it, so you need to park an entry (status rejected, out of the replay path, shown to the user) and let the rest drain. And background delivery is uneven: the Background Sync API sits at 76.7% global support with no Firefox or Safari implementation, so on iOS replay only runs while the app is open.

Validation has to run twice

If the client only checks required fields and the server enforces the real rules, offline users fill forms that fail on replay — when they can no longer see the site, the workshop or the part number they were describing. The client must apply the server's rules at write time.

The way to do that without maintaining two rule sets is one schema, two runtimes. On Castrol that was JSON Schema with the same validation applied on the client and the backend, using AJV on both sides.

// shared/schemas/inspection.json is imported by both the React app and the Koa API

import Ajv from 'ajv';

import inspectionSchema from '../shared/schemas/inspection.json';

const ajv = new Ajv({ allErrors: true });

export const validateInspection = ajv.compile(inspectionSchema);

// client, at write time — before anything hits the change log

if (!validateInspection(draft)) {

return showErrors(validateInspection.errors);

}

What travels and what doesn't:

Travels: types, ranges, required fields, enums, cross-field rules within one record.
Stays server-side: uniqueness, quotas, permissions — anything depending on state the client doesn't have.

So rejections still happen. Design the rejection UI — the parked entry, the reason, the "edit and resubmit" path — instead of pretending the shared schema made them impossible.

Auth when the token expires offline

The access token expires at 09:00, the device is offline from 08:30 to 16:00, and the user is locked out of data already on the device. There is nothing to refresh against. Two decisions have to be made explicitly:

What may an expired session still do locally? My default: read everything cached, write to the change log, block replay until re-authentication. Offline work continues; nothing reaches the server under a dead session.
How are credentials held on the device? Castrol's approach was to encrypt credentials on the fly using dynamically generated RSA keys, store the encrypted credentials in the user's JWT with the public key on the backend, and generate a fresh key pair on every token refresh, on top of OAuth2. The case study doesn't say why this design over alternatives, so treat it as one working shape, not a recommendation.

Whatever you hold locally, follow RFC 9700, the January 2025 OAuth 2.0 Security Best Current Practice: refresh tokens for public clients must be sender-constrained or rotated on every use.

The honest limitation: any credential usable offline weakens revocation. A lost device stays useful until it reconnects and the server gets to say no. You can shorten the window — local PIN or biometrics, wipe the store after N days offline — but you cannot close it. Say so in the design review instead of presenting it as solved.

Choosing a merge rule you can defend

The rule most teams start with is last-write-wins on a client timestamp. It is wrong for offline in a specific way: the device that was offline longest often carries the newest wall-clock time, because it was still being edited after the others synced. LWW then rewards the stalest replica. And the clocks aren't trustworthy anyway — Airship's analysis of its mobile event stream (from 2016, so older data) found about 2% of devices with clocks three or more minutes off. A merge rule built on those clocks is arbitrary with extra steps.

Ranked by cost, the options I'd actually consider:

Server-authoritative rejection. If baseVersion doesn't match the current version, reject and return the server state. Cheap, correct, annoying — the user re-does the edit. The right default for anything money- or compliance-shaped.
Per-field merge on disjoint fields. The change log carries only touched fields, so if two edits touched different fields, apply both; conflict only on the same field. This resolves most real conflicts on form-shaped data for very little code.
Domain-specific merge. Append-only collections merge trivially (findings on an inspection are a set, not a document). Counters travel as deltas, never absolutes — "+3 units" merges, "quantity = 12" doesn't.
CRDTs. Real convergence without coordination, at real cost. Kleppmann and colleagues' local-first paper (2019) is upfront that CRDTs store all history, that performance and memory became a problem in their own prototypes, and that the one change they cannot resolve is concurrent updates to the same property. The engineering has improved: Automerge 2.0 (2023) stores a ~260k-keystroke editing trace in 129 KB against 107 KB of plain text, Automerge 3.0 (July 2025) reports a 10x memory cut, and the crdt-benchmarks suite puts the same trace at 160 KB for Yjs with a 39 ms parse against 1.8 s for Automerge 2.1. Those are text-editing numbers; for a form with forty fields, the library is a dependency you carry for a problem you may not have.
User-mediated resolution. Show both versions, let the human choose. Most correct, most expensive — the UI has to exist for every conflicting entity type.

LWW is defensible in exactly one case: single-user-per-record data, ordered by server-assigned timestamps on arrival. Then it's just "the server applies operations in the order it receives them," which is fine.

The limitation that applies to every option, CRDTs included: convergence is about data, not business rules. Two replicas can converge perfectly on an order whose delivery date precedes its creation date. Invariants still get checked on the server after the merge, and you're back to the rejection UI.

None of the case studies above published which merge rule they used — Castrol's mechanism is described only as merge-conflict resolution. The ranking is mine, from building and rebuilding this a few times.

What offline-first costs

I'd rather you hear this from me than from your on-call rotation:

The write path roughly doubles: local apply plus log, then server apply plus verdict.
Schema migrations happen on two sides, and the client has to migrate a log written by an older version of itself.
Every bug becomes a distributed one. Reproducing it means reconstructing a device's log, in order, against the server state it saw. Ship an "export my change log" button on day one.
Storage is a budget you don't control, and eviction is silent.
Revocation has a hole the size of the offline window.

If your users don't have real intermittent connectivity — a warehouse with one dead aisle, a clinic basement, a van on a rural route — you don't need this. An optimistic UI with a retry queue covers flaky office Wi-Fi at a tenth of the work.

If they do, this is the order of effort:

Scope offline per screen, and fight every flow out of the write-offline class that you can.
Log intent, not state, with client-generated IDs and base versions.
Make replay idempotent with per-operation verdicts, and give a poisoned entry somewhere to go.
Share validation as one schema in both runtimes, then design the rejection UI for what doesn't travel.
Decide the merge rule per entity, write it down, and reach for CRDTs only where convergence without a server is a requirement rather than a preference.

And the unglamorous half: conflicts you don't design for still get resolved. Just arbitrarily, by whichever device reconnected last — and the user finds out weeks later, from a colleague, that their inspection isn't there.

How to Scale a SaaS Platform 10 Without a Full Rewrite

Vic Tekys — Tue, 13 Oct 2026 09:13:00 +0000

A barcode scan took 3–5 seconds. One scan per package. Hundreds of thousands of packages.

That was the state of Sway, a US logistics startup handling returns and deliveries for online retailers, when our team started on it. The company had processed more than $10 million worth of returns and was expanding city by city, and every package still cost an operator 3–5 seconds of waiting before it could move.

Nobody called that a scaling problem. It looked like UX. But at 4 seconds a scan, 100,000 packages is 111 hours of a human standing still. That was the throughput ceiling, and it had nothing to do with the architecture.

The reflex when volume grows an order of magnitude is to reach for a rewrite, a sharding plan, or a bigger instance class. All three would have missed the scan. And the first one has a documented failure rate: in a 2022 Wakefield Research survey of 250 senior IT leaders at large enterprises, 79% of application modernization projects failed, at an average cost of $1.5 million and a 16-month timeline. That's the thesis of this piece: 10× volume is rarely absorbed by new architecture. It's absorbed by finding the two or three places where you pay a cost per unit of work — a scan, a query, a stored row, a message — cutting each, and then redistributing what's left across roles, services, and teams.The architecture usually survives. The hot paths don't.

Define the unit before you name the bottleneck

"We need to scale 10×" is not a requirement until you say 10× of what. Every platform I've worked on had one or two units that everything else hangs off, and the unit is where the per-unit cost lives.

Across the projects I'll use as evidence here, the unit was different every time:

Sway: the package. Every package gets scanned, routed, stored, and reported on.
CYDEF, a Canadian cybersecurity company whose analysts review 100% of endpoint activity: the telemetry message, arriving at hundreds of gigabytes per day.
medneo, a radiology-as-a-service business running MRI and CT centers across Germany and the UK: the patient, multiplied by a feature set that never stopped growing.
MINT, an Italian AdTech platform managing media investment across 16 channels and 350+ vendor integrations: the release.

Before you touch code, put a price on the unit. For Sway that price was 3–5 seconds of scan latency, plus whatever a package cost in query time and stored bytes. For CYDEF it was the cost of ingesting, storing, and querying one message. You can't optimize a number you haven't measured.

Why a 3-second delay counts as a scaling cost and not a cosmetic one: Jakob Nielsen's response-time research puts 1 second as the limit for a user's flow of thought to stay uninterrupted and 10 seconds as the limit for keeping their attention on the task at all. A 3–5 second scan sits past the first limit on every single package, which means the operator context-switches hundreds of times per shift. Getting Sway's scan to 1–2 seconds moved it back under the flow-of-thought threshold. That's a throughput change, not a polish change.

The trap is multi-tenant platforms with several units at once — tenants, users, records, API calls. Pick the wrong one and you'll spend a quarter making the wrong path fast. If revenue and the ops team's pain both point at the same unit, that's your unit.

Profile the hot path, then change architecture around it

Sequencing matters more than technique. Profiling tells you which architectural change is worth making. Guessing produces re-architecture that doesn't move the number.

On CYDEF, the platform had serious production performance issues and slow database request processing. The path we took was not "microservices" or "move to a new database." It was: run profilers against the running system, find the actual weak points, then make architectural modifications around what the profiler said — and build a dedicated memory-leak tracking service with alerting and process blocking, because leaks were one of the things the profiler kept surfacing.

The mental model I use is Brendan Gregg's USE method: for every resource, check utilization, saturation, and errors. Gregg's own claim is that the checklist finds roughly 80% of server issues with a small fraction of the effort, and saturation — work queued because the resource can't service it — is the one that usually exposes a per-unit cost.

If your SaaS runs on PostgreSQL you're in the majority: in the 2025 Stack Overflow Developer Survey, 58.2% of professional developers reported working with PostgreSQL in the past year, against 39.6% for MySQL. For a PostgreSQL-backed SaaS, the cheapest first profiler is pg_stat_statements, which the PostgreSQL docs describe as tracking planning and execution statistics of all SQL statements executed by a server. It needs shared_preload_libraries = 'pg_stat_statements' and a restart, and then the question "what is our per-unit cost?" becomes one query:

SELECT

calls,

round(total_exec_time::numeric, 0) AS total_ms,

round(mean_exec_time::numeric, 2) AS mean_ms,

rows,

left(query, 80) AS query

FROM pg_stat_statements

ORDER BY total_exec_time DESC

LIMIT 15;

Sort by total_exec_time, not mean_exec_time. A 2 ms query called 40 million times a day costs more than a 4-second report run hourly, and it's the one that grows linearly with your unit.

The limitation: synthetic load lies. Lock contention, the race condition Sway had in trip creation, a cache that thrashes at one specific working-set size — these appear only at real concurrency. Profile production, or a replay of it, before you trust the plan.

The data layer carries most of the 10×

If I had to bet where a growth-stage SaaS is paying its per-unit cost, I'd bet on the database, and I'd usually win. Most "we need to scale" is three things: unindexed reads, repeated aggregation, and data you chose to keep.

Indexes and materialized views vs. more instances

Sway is the primary evidence. The codebase had query inefficiencies and missing indexes, excessive data in API requests, redundant entries created during parcel creation, and routing errors under high load. None of that is fixed by a bigger RDS instance: a bigger instance speeds an unindexed scan in proportion to the hardware, while an index speeds it in proportion to the table size.

The "just buy more instances" reflex also has a bill attached that most engineering teams never see. In Flexera's 2025 State of the Cloud report, 84% of the 750+ respondents named managing cloud spend as their top cloud challenge, and organizations reported cloud budgets already exceeded by 17%. Scaling by instance class is a linear cost curve; an index is a one-off.

What we shipped on the data side of Sway:

PostgreSQL indexes on the columns the hot queries actually filtered and joined on.
Materialized views for the aggregations that reports and dashboards recomputed on every request.
Query rewriting to stop pulling columns and rows the API never used.
Parcel-creation refactoring that removed redundant writes, which the Sway case study reports reduced database load by at least 5%.

The Sway case study reports query speed up by at least 40% from the PostgreSQL optimization. Treat that as a floor, not a mean — it's the client's published figure, and it wasn't measured per query.

The materialized view pattern is worth spelling out, because it's the one people get wrong. Precompute the aggregate, index it, and refresh it on a schedule:

CREATE MATERIALIZED VIEW daily_parcel_stats AS

SELECT

date_trunc('day', created_at) AS day,

hub_id,

status,

count(*) AS parcels

FROM parcels

GROUP BY 1, 2, 3;

- CONCURRENTLY requires a unique index covering all rows

CREATE UNIQUE INDEX ON daily_parcel_stats (day, hub_id, status);

- On a schedule (cron, pg_cron, a Lambda):

REFRESH MATERIALIZED VIEW CONCURRENTLY daily_parcel_stats;

The PostgreSQL docs state that CONCURRENTLY refreshes without locking out concurrent selects but is only allowed with at least one unique index that uses only column names and includes all rows — and that even with it, only one REFRESH at a time may run against a given view.

That last sentence is the trade-off. Materialized views trade freshness for speed, and the refresh cost doesn't disappear — it moves. If the refresh takes longer than its schedule interval, you have relocated the bottleneck, not removed it. Size the refresh against the growth of the underlying table, not against today's row count.

CYDEF ran the same play on a different stack: database requests were optimized through indexation, normalization and denormalization where each was warranted, and improved caching. Same three levers, .NET and PostgreSQL instead of Node.js and PostgreSQL.

Store less: queues, retention, and the cost of keeping everything

The second data-layer lever is the one teams skip because it feels like deleting value: stop storing everything.

On Sway, introducing queue functionality — moving transient work through SQS instead of persisting every intermediate state — reduced stored data by 20% and cut storage costs, per the case study. Twenty percent less data is 20% less to index, back up, vacuum, and scan.

The related move was getting analytics off the transactional database entirely. Reporting on Sway runs through a Node.js pipeline into Google Cloud Storage, then Snowflake, then Sigma; the operational PostgreSQL never sees a dashboard query. Reporting is the workload that grows fastest with volume, because every new customer wants a new cut of the data.

A checklist I now run on any table over a few hundred million rows:

Which columns does nothing read after 30 days? Those are candidates for a cold store.
Which rows are intermediate states that only exist because a workflow persisted every step? Those belong in a queue.
Which aggregates get recomputed on read? Those become materialized views or a warehouse table.
Which retention policy is "forever" because nobody wrote one? Write one.

Get the firehose off the request path

Continuous high-volume input is a different problem from a slow query, and it needs a different answer: a transport built for throughput, plus cold storage for history, rather than a bigger API tier.

CYDEF's telemetry was the firehose — hundreds of gigabytes of endpoint data daily, all of which had to be processed and kept, because the product's promise is that experts review 100% of endpoint activity. Putting that through a request/response API and into the transactional database was the per-unit cost that would have scaled linearly with every new endpoint.

The changes:

Ingestion moved to Azure IoT Hub, which the case study reports receiving, processing, and storing millions of messages. For scale, the IoT Hub pricing tiers are quoted per unit per day: 400,000 messages on S1, 6 million on S2, 300 million on S3 — the throughput is a provisioning decision, not a code change.
The threat-processing workflow was completely redefined so that workflow history lives in the database and AWS Blob Storage. The original workflow had no way to step back to a previous stage; long-running investigations needed that, and keeping history in cheap storage rather than in the hot tables is what made it affordable.
An internal device-monitoring tool replaced the manual, labor-intensive process of watching devices and resolving issues.

The pattern generalizes past IoT: any continuous input — events, webhooks, logs — should land in a partitioned transport first and be consumed at the rate the system can afford, not the rate producers send it.

The limitation: managed ingestion moves cost to storage and egress, and gives your architecture a vendor shape — IoT Hub semantics, Blob Storage tiers, region constraints — you'll live with for years. On CYDEF the trade was correct because the alternative was linear growth of a database that was already slow.

Split the work, not just the code

The lever engineers underrate is the non-code one: throughput also rises when the same work is divided across roles, environments, and teams.

Sway's drivers and operators were doing manual tasks with no mobile app, and the two roles were working through the same interfaces. We built React Native apps that split driver logic from operator logic into separate applications. The case study reports delivery speed and efficiency up 50% and package processing roughly 2× faster. I won't attribute those numbers to the app alone — the same period included the scanner and API optimization, the indexes, an admin audit page for routing errors, AWS Location Service and Google Maps, and Zebra printer and carrier API (FedEx, ClearJet) integrations. The case study lists changes and outcomes without mapping one to the other, so neither will I.

What I will say: splitting the work by role removed contention no query optimization could touch — two roles no longer waited on each other's screens.

MINT is the team-level version. The goal was to scale development while keeping QA discipline, and the shortcoming was process: getting a feature approved for production was slow because the business kept changing requirements mid-release. The fix was structural:

Development split into independent teams, each with its own sandbox.
A dedicated UAT environment, so business sign-off stopped blocking the pipeline.
At least one QA professional embedded in every team.
Daily rallies, feature demos, and retrospectives to keep 14 teams aligned.

The result, per the case study, is 14 active development teams shipping several releases per day, on a 40-engineer team of developers, QA, and BAs.

For context on what "several releases per day" means: in the 2024 DORA report, as summarized by Octopus Deploy, the elite cluster — under 20% of respondents — deploys on demand with a 5% change failure rate and recovers from a failed deployment in under an hour, while the low cluster deploys monthly to biannually with a 40% change failure rate and takes a week to a month to recover. The gap between the two is 182× in deployment frequency. Team structure is what moves an organization between those clusters; database indexes don't.

Be precise about what this buys. Team splitting raises release throughput, not runtime throughput — MINT's platform doesn't handle more ad spend per second because there are 14 teams; it ships faster. And the rallies and demos aren't free; they're the price of independence.

Keeping the ceiling from coming back

Every optimization above is a regression waiting to happen: a column added to the hot query, a new endpoint that skips the index, a workflow step persisted that was meant to be transient. The per-unit cost creeps back at exactly the rate you stop measuring it. The only durable defense is making regression detection cheaper than regression.

The price of not doing that is measurable. In the Uptime Institute's 2024 Annual Outage Analysis, 54% of respondents said their most recent significant outage cost more than $100,000 and 20% said more than $1 million, and IT and networking issues — change management and misconfiguration, mostly — made up 53% of all reported causes. A performance regression that ships under load is exactly that category of outage.

On CYDEF, that meant automated regression suites integrated into the CI/CD pipeline, parallel execution on cloud-based test environments, AWS virtual machines for dynamic test data, and CI/CD extended to components that previously had none. The case study reports over 1,500 test cases, over 700 automated tests, and 98% test coverage of the desktop agents. Note the scope: 98% is the desktop agents, not the platform.

On medneo, the RIS had been degrading under patient growth while every new feature introduced regressions. The work was refactoring and removing unnecessary database calls, upgrading legacy frameworks and libraries, and a CI/CD pipeline specifically to reduce the impact of feature rollouts. The published results are qualitative — significantly reduced load times, fewer outages, headroom to scale — and I'd rather say that than invent a percentage. Eight engineers, 2019 to 2022.

Coverage numbers describe effort, not defect escape rate; 700 automated tests that never exercise the hot path won't catch a dropped index. Put a performance assertion in CI for each unit cost you optimized — scan latency, p95 of the top five pg_stat_statements queries, storage growth per unit — and fail the build when it moves.

When incremental scaling stops being honest

Everything above is an argument against the rewrite reflex. It's also an argument against its mirror image: the belief that tuning is infinite.

Incremental scaling ends in two places I've seen:

Where the infrastructure is manual, undocumented, and per-client. CYDEF's deployment for each new client was time-consuming, hand-built, and undocumented. No index fixes that. What fixed it was automating the majority of infrastructure creation and writing the deployment documentation — structural work, not tuning.
Where ingestion can't be batched. If every unit needs a synchronous answer and the units arrive faster than the synchronous path can serve them, you're out of per-unit tricks and into a topology change.

The diagnostic is simple to state and hard to fake. After you've optimized, plot cost per unit against volume:

Flat means you can keep growing on the platform you have. Sway's stack — React and Next.js on the front, Node.js and Serverless on the back, PostgreSQL alongside DynamoDB and Aurora, GraphQL, AWS — is the same stack it started with. It absorbed a 10× increase in package volume, per the case study, and a seven-engineer team has been running it since 2020.
Rising after optimization means the ceiling is structural. That is the only case where a rewrite is the cheaper answer, and it's the case you should be able to prove with the same per-unit measurement you started with — before you join the 79% from the Wakefield survey who started one without it.

One note on timelines: none of this was fast. Sway and CYDEF have run since 2020, MINT since 2019, medneo took three years. Absorbing 10× on your existing stack is cheaper than a rewrite. It is not quick.

Measure the unit. Cut its cost where you're paying it. Split the work. Then check the curve — and only rewrite if it's still rising.

How to Build White-Label SaaS Without Creating a Code Fork for Every Customer

Vic Tekys — Tue, 06 Oct 2026 09:28:00 +0000

I still remember the pull request that created release/acme-corp. One enterprise customer needed a different label format on their shipping documents. The deadline was tight, the config system didn't cover it, and branching felt like the pragmatic move. Everyone agreed it was temporary.

Eleven months later we were fixing the same date-parsing bug twice, in two codebases, with two QA passes — and the second fix shipped three weeks after the first because nobody remembered the fork existed until Acme filed the ticket.

Here's the position I'll argue: a fork is a code decision, not a deployment decision. Teams fork because they never sat down and named which layers actually vary per customer. Name them — branding, data model, workflow, integrations, infrastructure — give each one an explicit extension mechanism, and per-customer work collapses into configuration against one codebase. Even when a customer demands their own servers.

What "white-label" means to the customer

"We need white-labeling" is one of the least precise sentences a prospect can say. In my experience it decodes into four different requests, at four wildly different costs:

A logo. Their brand on your product: colors, name, domain. Cheap if you built for it, a fork factory if you didn't.
A data boundary. "Our data can't sit next to anyone else's." An isolation requirement, not a code requirement.
Their workflow. Approval steps, field validation, document formats that differ from your default. This is the expensive one.
Their integrations. "We use carrier X and SSO provider Y." Every enterprise deal comes with this rider.

Real customers ask for all four. On the E2EE mobile messenger we've worked on since 2020 — a Geneva-based product for privacy-focused enterprises, 6 people on dev and QA — the product ships customizable branding options to align with each client's unique identity. On the carbon management platform we build for a German client, the product has to integrate with the ESG, GRC, compliance, and supply-chain software its customers already run.

Both of those are published demand, not published mechanism — the case pages tell you what customers ask for, not how to implement it. The implementation opinions below are mine.

The point of the taxonomy: classify the ask before estimating it. A logo request priced like a workflow request loses the deal; a workflow request priced like a logo request loses your margin, and eventually your architecture.

Branding is the cheap layer — keep it that way

Branding stays cheap under exactly one condition: it lives in runtime configuration, not build-time flags. The moment someone writes #ifdef ACME — or its JS equivalent, if (BUILD_TARGET === 'acme') — you've started a fork, just distributed across the codebase instead of across branches.

The mechanics that have worked for me:

Resolve the tenant from the host header, once, in middleware. acme.yourapp.com or app.acme.com maps to a tenant ID; everything downstream reads config, never hostnames.
Ship theme as data: design tokens, logo URLs, product name, email sender, legal footer — one JSON document per tenant.
Always render a default. Missing config must produce your stock brand, not a crash. Tenant onboarding will forget a field; that can't be a sev-2.

// theme resolution: config, not conditionals

async function resolveTheme(hostname: string): Promise {

const tenant = await tenants.byHostname(hostname); // cached, obviously

return { ...DEFAULT_THEME, ...(tenant?.theme ?? {}) };

}

The spread against DEFAULT_THEME is the load-bearing line. Every tenant override is partial; the default absorbs whatever onboarding missed.

One warning from the trenches: branding stops being cheap the moment a customer asks to move things — "can the sidebar go on the right," "we want approvals before the summary screen." That's not theming. That's a workflow request wearing a theming costume, and it needs to be priced and architected as one.

Embedded surfaces are a tenant boundary

There's a surface everyone forgets when listing what varies per customer: the part of your product that runs inside someone else's website.

We've been on Xola, a booking platform for tour and activity operators, since 2021 with 11 engineers across the USA and India. One of its core products is a checkout module — an app embedded into a reseller's website so that their clients can purchase tours and activities right on that website. And per-customer integrations there include Google Analytics, Facebook pixel, MailChimp, and Zoom.

An embedded widget inherits everything about its host page:

the host's CSS resets and global styles,
the host's Content-Security-Policy,
the host's analytics and consent tooling,
whatever ancient jQuery the host still loads.

If your widget's styling or script execution depends on assumptions about the surrounding page, every new reseller is a new integration project — and integration projects that touch core code are how forks get born. Isolate the surface instead: an iframe when you can afford the postMessage tax, shadow DOM when you need to live in the host's document. (That's my reasoning, to be clear — the Xola case study documents the embedded checkout and the integrations, not the isolation technique.)

Treat the embed like a public API with a version, not like a page you own. The reseller's site is production you don't control.

Integrations are where forks are born

If I had to name the single largest source of per-customer branches across projects I've seen, it's this sentence: "Customer X uses carrier Y."

The naive implementation adds if (tenant === 'x') { fedexClient.ship(...) } somewhere in the order flow. The second carrier adds an else if. The fifth carrier — with its slightly different retry semantics and label format — is when someone proposes "a dedicated build for the logistics-heavy customers," and now you're forking.

The scale of the problem is real. Sway, a US logistics company (formerly Returnmates) where a dedicated team of 7 Devico engineers has worked since 2020, serves more than 250 brands — Rent The Runway among them — through one unified tracking system spanning FedEx, ClearJet, Million, T-Force, SmartKargo, and UDS, plus Stripe for payments, Google Maps and AWS Location Service for geo, and Zebra ZPL label generation. Six carriers, 250+ brands, one system. (The case study lists the integrations; the adapter design below is my prescription, not a description of Sway's internals.)

The pattern that keeps this from forking is old and boring, which is why it works:

Define one internal interface per vendor category — Carrier, PaymentProvider, SsoProvider.
Implement each vendor as an adapter behind that interface.
Select the adapter per tenant from config, at one composition point.
Contract-test every adapter against the same suite, so "works for tenant X" is provable without tenant X.

interface Carrier {

createShipment(o: Order): Promise;

track(id: string): Promise;

renderLabel(s: Shipment): Promise; // ZPL, PDF, whatever

}

const carriers: Record = {

fedex: new FedExAdapter(),

smartkargo: new SmartKargoAdapter(),

};

function carrierFor(tenant: TenantConfig): Carrier {

return carriers[tenant.carrier] ?? carriers[DEFAULT_CARRIER];

}

The discipline is in what the interface excludes. The moment Carrier grows a fedexSpecificOptions field, tenant knowledge has leaked into core code and the boundary is decorative. Vendor quirks live inside the adapter or they live everywhere.

Budget honestly for this: each adapter is real work — auth, rate limits, sandbox environments, webhook shapes. But it's work that lands in an isolated, contract-tested module instead of in if statements sprayed through your order pipeline.

Per-tenant infrastructure, not per-tenant code

Now the assumption I most want to kill: "separate infrastructure per customer means a separate build." It doesn't. Isolation is an infrastructure-as-code concern, and IaC takes parameters.

Compliance is the usual driver of these demands, and it's not exotic. Arklign, a US dental technology platform where a 7-person Devico team has worked since 2016, runs lab fabrication, logistics, and data workflows across distributed dental practices in full HIPAA compliance — and regulated verticals like that are exactly where a customer's counsel eventually asks where, physically, the data lives. (That last inference is mine; the case study documents the compliance, not the question.)

The carbon management platform I mentioned earlier is the cleanest example I've shipped. The client — Germany, working with 3 Devico full-stack devs since 2023 — required white-labeling with deployment on a separate host and superfast creation of distinct infrastructure per client, while moving from a basic app reliant on numerous third-party services to an independent solution with proprietary formulas and algorithms. The stack: TypeScript, React, NestJS, Prisma, PostgreSQL, and AWS Lambda, SNS, SQS, DynamoDB, and Cognito, defined with the AWS TypeScript CDK (plus SAM).

The CDK is what makes "separate host" compatible with "one repo": infrastructure configurations deploy on whatever cluster fits the client's location. A tenant is a stack instantiation, not a branch:

// one stack definition, N instantiations — the whole trick

for (const tenant of tenants) {

new PlatformStack(app, platform-${tenant.id}, {

env: { account: tenant.awsAccount, region: tenant.region },

dataResidency: tenant.region, // eu-central-1 for the German clients

cognitoPool: tenant.isolatedAuth, // per-tenant user pool if required

});

}

Same application code, same serverless pipeline — on this platform, clients upload bulk CSV data that gets validated and processed through staged serverless services — but each customer gets their own host, their own data boundary, their own region. Residency, compliance, and noisy-neighbor requirements: satisfied. Fork: avoided. The client secured investment in the project's early stages; I'll claim no engineering metric beyond that, because none is published.

Full disclosure on the cost, because this is where the model hurts:

N stacks means N migration runs, and migration N-3 will fail differently than the others.
N monitoring targets, N alert routes, N dashboards.
N versions in the wild unless you enforce a deploy train — version drift between tenant stacks is the fork problem sneaking back in through the ops door.

Parameterized infrastructure is still a fleet. You've traded a code-merge problem for an ops problem. I take that trade every time, but it's a trade.

When a fork is right

After all that: sometimes a fork is the correct call. I can defend exactly three cases.

The domain model diverges permanently. Not a field or a workflow — the actual entities and their relationships stop being the same product. Config can't paper over a different domain.
Contractual isolation forbids shared code paths. Some regulated deals require it in writing. You can't architect around a signature.
The account funds its own roadmap. If one customer pays for a dedicated team and a divergent feature stream indefinitely, you don't have a customer — you have a second product. Staff it like one.

Even then, two conditions or you're just deferring the Acme story: the fork gets a named owner whose job includes syncing shared fixes, and a sunset date — a scheduled decision point to merge back, spin off, or kill. A fork without an owner and a date isn't a strategy. It's a slow leak with a branch name.

The cost of configurability

Fair's fair: I've spent this whole article arguing against forks, so let me argue against my own medicine for a minute. Configuration isn't free — it just bills differently.

Every switch you add multiplies the test matrix. Ten boolean tenant options is a theoretical 1,024 configurations; nobody tests that, and nobody should pretend to. Worse, tenant-specific bugs are miserable to repro: the bug lives in one tenant's combination of theme, workflow config, and adapter selection, and your local environment has none of it.

No vendor I know of publishes defect rates tied to configuration complexity, so I can't hand you a stat here — this is argued from scars, not data. What's kept the matrix survivable for me:

Test the default tenant plus the two most divergent ones. Not every tenant — the default and the extremes. The extremes are where the spread operators and fallbacks actually get exercised.
Contract tests per adapter, so every vendor implementation proves the same behavioral guarantees without a live tenant in the loop.
Version tenant config as a reviewable artifact. Config in a repo, changed by PR, deployed like code. A tenant's config diff should be as reviewable as a code diff, because it changes production behavior exactly like one.

That last one matters more than it looks. If config lives in a database row edited by hand, you have all the risk of a fork with none of the git history.

Order of operations

If you're staring at your first "we need white-label" deal, this is the sequence I'd run:

Enumerate the variation layers for this customer: branding, data boundary, workflow, integrations, infrastructure. Write them down; price them separately.
Give branding runtime tokens with a default fallback — and refuse to let layout changes hide in the theming budget.
Put every vendor behind one internal interface with per-tenant adapter selection, contract-tested.
Parameterize infrastructure so isolation and residency are stack parameters, not repo branches.
Only then, if the domain model itself is diverging, consider a fork — with a named owner and a sunset date attached before the branch is created.

Reach for this when you're a B2B SaaS with a growing roster of enterprise or reseller customers who each arrive with a logo, a compliance rider, and a favorite vendor. Skip the machinery when you have two customers and no divergence yet — speculative extension points rot as fast as forks do.

And one operational habit, because config that nobody exercises is config that's silently broken: the tenant that breaks in production is always the one whose configuration nobody has rendered since onboarding. Boot every tenant's config in CI — default theme, adapter wiring, stack synth — even the quiet tenants. Especially the quiet tenants.

How to Cut Regression Testing from Weeks to Days

Vic Tekys — Tue, 29 Sep 2026 09:15:00 +0000

Here's the fact that reframed how I think about regression speed. On an E2EE mobile messenger our team has worked on since 2020, the test suite grew from effectively nothing to over 700 automation scripts on iOS alone, with almost the same count on Android and up to 98% of test cases automated.

The suite got dramatically bigger. And the regression cycle fell from about 20 days to up to 2.

That combination breaks the usual explanation — "regression is slow because we have too many tests, so cut scope or automate harder." Test count went up sharply while duration collapsed by an order of magnitude. What changed wasn't how many tests existed. It was how they ran.

Regression duration is dominated by serialization and waiting: environments that take days to get, single-threaded execution, manual re-verification across every browser and locale, and triage on failures nobody trusts. Attack those, and the suite can keep growing while the calendar shrinks.

Measure the cycle before optimizing it

Before touching anything, time one full regression cycle end to end and split it into four buckets:

Pure test execution
Waiting for environments
Triage of ambiguous failures
Re-runs

Most teams have never done this. They know the cycle is "three weeks" but not where the three weeks live. In my experience the middle two buckets — environment wait and triage — are where most of it sits.

This matters because the fix depends on the diagnosis. A suite that executes in six hours but blocks for four days on a shared staging environment is an infrastructure problem. Making the tests 30% faster fixes nothing anyone will notice.

Triage and re-runs deserve their own line item, because they compound. Every ambiguous failure means an engineer reproducing it by hand, and every "just re-run it" doubles the execution cost of whatever shard it lives in. A cycle with a 15% re-run rate is quietly carrying a sixth of its execution time as pure waste.

We saw this shape clearly at CYDEF, a Canadian cybersecurity company we've worked with since 2020. Infrastructure deployment for new clients took a lot of time and effort, and the process wasn't documented at all. No test report will ever show you that cost — it lives in Slack threads and calendar gaps, not in CI logs.

Don't wait for proper instrumentation to start. Rough timestamps over two or three cycles are enough to find the dominant bucket. Precision comes later; direction comes now.

Automate the environment before the suite

If environment wait dominates, the highest-value automation work isn't a single test script. It's making environment provisioning boring.

The pattern that works:

Declarative environment definitions, versioned next to the code
Seeded test data, so a fresh environment is usable immediately
Setup owned by the CI/CD pipeline, not by a person with tribal knowledge
One environment per run instead of a queue for one shared staging box

Shared-staging queueing is invisible in test reports and dominant in calendars — five teams waiting for the same environment is regression time, even though no test is running.

At CYDEF that's exactly the order we worked in. The team automated infrastructure creation for new clients, wrote comprehensive deployment documentation, extended CI/CD coverage to components it didn't reach, and wired automated regression suites directly into the pipeline. Alongside that, database indexing and caching work removed a slow-request bottleneck that was quietly stretching every test run.

The shape of a declarative environment doesn't need to be exotic:

ephemeral env per regression run, torn down after

jobs:

provision:

steps:

run: terraform apply -auto-approve -var="env_id=reg-${{ github.run_id }}"
run: ./scripts/seed-data.sh reg-${{ github.run_id }}

regression:

needs: provision

suite runs against its own env — nobody queues, nobody's data collides

The point isn't the tool. It's that "get me a working environment with known data" becomes a pipeline step measured in minutes, not a favor measured in days.

The honest counterargument: ephemeral environments cost real money and real engineering time. A small team on a stable monolith with one deploy target may get more from fixing and disciplining one shared environment than from building a provisioning system. Do the math on your own environment-wait bucket first.

Parallelism is the biggest lever — and it demands a deterministic suite

Once environments stop being the bottleneck, execution time is next, and parallelism is the mechanism behind the largest reductions I've seen. Wall-clock time falls near-linearly with worker count — but only if the tests are actually independent.

On the E2EE messenger, iOS and Android automation runs in parallel on real devices, on at least 3 threads. That's what makes 700+ scripts per platform compatible with a regression cycle of up to 2 days instead of 20. On Abbott's LibreView, the suite is over 1700 tests supporting 4 web browsers — and the browser matrix itself is the natural sharding axis.

Independence is the price of entry. For every test in the parallel lane, that means:

It owns its data — per-test setup, not shared fixtures
It assumes nothing about execution order
It cleans up after itself

Shared state is how a green suite turns red the moment you add a second worker.

Concretely: a test that needs a user creates its own user, with a unique identifier, through an API call in its setup — it doesn't reach for test_user_1 that forty other tests also mutate. That setup cost feels wasteful right up until you try running the suite on 16 shards.

strategy:

matrix:

browser: [chrome, firefox, edge, safari]

shard: [1, 2, 3, 4]

steps:

run: npx playwright test --shard=${{ matrix.shard }}/4 --project=${{ matrix.browser }}

Keep an explicit serial lane for the handful of flows that genuinely can't parallelize — schema migrations, payment flows, anything mutating global state. Naming that lane out loud is better than pretending everything parallelizes and debugging the lie later.

And know what you're signing up for: parallelism turns flakiness from an irritation into a blocker. Order-dependent and time-dependent tests that limped along in serial runs fail loudly under parallel execution. Parallel runs also multiply infrastructure spend. An unstable suite on 16 workers doesn't give you answers faster — it gives you wrong answers faster.

Tier the suite instead of running one monolith

A single "run everything" gate is what makes regression a calendar event. Nobody needs the full suite's verdict on every commit; everybody needs some verdict fast.

The tiering that works in practice:

Smoke — a minutes-to-hours gate on every build
Change-scoped selection — full depth on what the change plausibly touches, mid-cycle
Full suite — nightly or pre-release, as the safety net

The numbers make the case. On the E2EE messenger, automated smoke testing takes up to 2 hours against a full regression of up to 2 days — a fast honest signal roughly 24x cheaper than the full answer. On Abbott, the time to perform a smoke test dropped from 7 days to 1 day, and smoke testing of individual countries got over 85% faster. (To be precise: those Abbott figures are smoke durations — the case doesn't publish a full-regression before/after.)

Mid-cycle, risk-based selection earns its keep: run what the change plausibly touches, at full depth, immediately. But be honest about its blind spot — regressions love showing up in "unrelated" areas, because the coupling in real systems is never fully mapped. The periodic full run is the safety net that makes aggressive mid-cycle selection defensible. Tier the suite; don't quietly shrink it.

Kill the multiplier — configs, locales, platforms

Here's the hidden cause behind most "regression takes weeks" stories: cycle length isn't test count. It's test count × configurations. Four browsers, a dozen locales, three platforms — a modest suite becomes a monster through multiplication alone, and every new market multiplies again.

You attack the multiplier structurally. On Abbott, the team built the suite on a keyword-driven testing methodology with a focus on performance and flexibility — one scenario definition executes across the whole 4-browser matrix instead of existing as four hand-maintained copies. On a platform serving over 4 million users across many country configurations, that's the difference between linear and multiplicative maintenance, and it's part of how per-country smoke time came down by over 85%.

Two more cuts at the same multiplier:

Verify translations as their own concern — string presence, formatting, layout — instead of re-running full behavioral suites per locale; behavior doesn't change because the button label is in Portuguese.
Add devices or browsers to the matrix only where failure modes genuinely differ, not for symmetry.

Which is also the counterargument: not every configuration deserves parity. Give every locale and every device the full-matrix treatment "to be safe" and you've rebuilt the original problem with more YAML.

Speed you can't maintain isn't speed

Everything above decays without ownership. Test automation is a development effort — it needs code review, refactoring, and debt management like any codebase. An unowned suite rots into a distrusted one, and a distrusted suite is worse than slow: people re-verify its results manually, and you're back to weeks with extra steps.

The structural fixes are unglamorous:

Named ownership for the suite — someone's job, not everyone's hobby
Stable locators — dedicated test IDs, not brittle XPath — so UI churn doesn't shred half the tests
A real quarantine policy — a flaky test gets pulled from the gate immediately, then fixed or deleted on a deadline

"Retry until green" isn't a policy; it's a slow-motion way of teaching the team to ignore red.

And QA embedded where the work happens, so regression is continuous rather than a phase. On MINT, an advertising resource management platform we've worked on since 2019, 14 development teams each include at least one quality assurance professional, teams run independently with their own sandboxes plus a dedicated UAT environment — and the platform ships several releases per day. Quality isn't a gate at the end of that pipeline; it's inside every team.

Tipalti, a payments platform we've supported since 2011, shows the same discipline at a different cadence: 16 QA engineers (up to 20 at peak), end-to-end test cases first, then sanity plus automated regression as the second control layer, releasing 25 times a year. That cadence is deliberately conservative — it's regulated payments. Fast regression enables frequent releases; it doesn't require them. The choice of cadence stays a product decision once regression stops making it for you.

To sum up

If your regression cycle is measured in weeks, here's the diagnostic order I'd follow:

Measure where the cycle time actually sits
Automate provisioning before adding tests
Make the suite parallel-safe and shard it
Tier by risk, with a periodic full run as the net
Collapse the config multiplier
Give the suite a named owner

Notice what's not on the list: "automate more" as a goal in itself, and any particular tool. Every project above changed several variables at once — process, infrastructure, suite architecture, team shape. No single framework caused any of these results, and none of these numbers is a benchmark you can order off a menu.

Reach for this when your gate is slow because of structure — queues, serial runs, config sprawl. Skip the heavy machinery when a small suite on a stable monolith just needs one shared environment fixed. And go in with both trade-offs priced: shorter feedback loops cost infrastructure spend and permanent maintenance, and a fast suite nobody trusts is worse than a slow one people do.

How to Build a Resilient Integration Layer for Dozens of External APIs

Vic Tekys — Tue, 22 Sep 2026 09:49:00 +0000

A carrier's label API times out at ten seconds. Your HTTP client throws, your retry wrapper fires, the second request lands — and the vendor processed both. The customer now has two shipping labels on one parcel. Swap the carrier for a payment provider and it's two charges. Nothing in your dashboards looks wrong, because from your side every request eventually "succeeded."

That incident is what a uniform retry policy buys you once third-party API integration becomes the product.

Because integrations don't fail the same way:

Some gate a flow and can't degrade — KYC checks, custody, payments.
Some can serve stale data for an hour and nobody notices — prices, metadata, enrichment.
Some cause things to happen in the physical world — labels print, locks open, drivers dispatch.

My thesis for this whole article: resilience in an integration layer comes from classifying each vendor by what breaks when it fails, then isolating it — its own adapter, its own data model, its own timeout budget, its own cached state — so the failure stays local. The thing that scales worst is the thing most teams build first: one canonical interface with one retry-and-circuit-breaker policy applied to everything.

Count integrations by blast radius, not by vendor

When someone says "we have 30 integrations," that number tells me nothing. The useful inventory is by blast radius, and in my experience it always sorts into three classes.

Critical-path, can't degrade. On Mode, a UK digital finance app dealing in GBP, EUR and BTC, Onfido gates onboarding and Chainalysis and Sardine.ai gate compliance checks — if those calls fail, the flow stops, full stop. Coinbase and Modulr sit in the money movement path. There is no "serve stale" option for any of them.
Degradable. Mode's Twilio integration sends notifications; a delayed SMS is an annoyance, not an incident. Token prices, asset metadata, enrichment data — all of this can be served from cache while the vendor is down.
Side-effecting on the physical world. A shipping label, a door lock, a dispatch call. These are the scariest, because a duplicated call isn't a wasted request — it's a duplicated event you can't un-happen.

Each class gets different rules. In practice that looks like config, not philosophy:

const policies = {

critical: { timeoutMs: 8000, retries: 2, fallback: 'fail-closed' },

degradable: { timeoutMs: 1500, retries: 0, fallback: 'serve-cached' },

sideEffect: { timeoutMs: 15000, retries: 0, fallback: 'park-and-reconcile' },

} satisfies Record;

Note the retries on the last two: zero. More on that below.

The honest limitation: classes shift under your feet. A price feed is degradable while it renders a chart and critical the moment it prices a trade. The classification is per call site, not per vendor — and it needs re-checking every time a new feature reuses an old integration.

One adapter per provider, one owner per data class

The structural rule I hold to: every provider — or every data class, where that's the better seam — gets its own adapter that owns:

the translation to and from vendor types,
the auth and credential handling,
the rate limits,
the error taxonomy.

Nothing downstream ever sees a vendor's types. This is the anti-corruption layer pattern, and Microsoft's own writeup is upfront that it costs you latency and an extra service to maintain — it's still the cheapest line item in this article.

DEFI Wallet, a multichain Web3 wallet a 9-person Devico team has been building since 2021, is split exactly along data-class seams:

a Python service for token metadata,
a Node.js/Nest.js service for asset data,
a separate service for NFT balances,
a GraphQL router on top, combining the service schemas.

The vendor list underneath is long — Infura, RPC nodes, DefiLlama, THORNode, Moralis, Covalent, CoinGecko, TrustWallet API — and no consumer of the graph touches any of them directly.

GoodShape, a UK wellness platform, supports 16 different fitness-tracker brands — an integration surface that became tractable after a migration out of a roughly 15-year-old monolith into microservices, where tracker data collection lives as its own service on RabbitMQ and AWS SQS.

Per-vendor adapters also give you bulkheads for free: one slow vendor exhausts its own connection pool and its own worker budget, not the shared one your payments path is standing in.

The cost is real, though. Dozens of adapters means dozens of deploy units, credential sets and dashboards. Don't pretend otherwise when you pitch this to your team.

Where a gateway helps — and where it hides failure

A router or gateway in front of the adapters earns its place by deciding what the caller sees: which fields compose into one response, what happens when a sub-service is down.

That's also its failure mode. A GraphQL router combining schemas is a coupling point where a partial outage flattens into a null field — the NFT service is down, the wallet renders an empty tab, and nobody pages, because every layer "handled" the error.

A gateway decides what the caller sees. It does not make the underlying call resilient. If your resilience story is "we have a gateway," you have a single point of failure with good manners.

Reads: get vendors off the request path

The highest-leverage change I know for read traffic: stop proxying. Every synchronous pass-through call donates your vendor's latency and downtime directly to your user.

The alternative is to ingest — pull vendor data into your own store on your own schedule, serve every user request from that store, refresh asynchronously. Cache, materialized view, indexer, full pipeline: the mechanism varies, the principle doesn't. The vendor's availability stops being your availability.

DEFI Wallet takes this to the logical end: rather than reading chains per request, the team built blockchain indexers for real-time transaction processing, so the wallet reads its own indexed state.

Sway, a US reverse-logistics company (ex-Returnmates), does the same for reporting: a pipeline from Node.js through Google Cloud Storage into Snowflake, surfaced in Sigma — not live queries against operational systems. On the operational side, PostgreSQL indexing and materialized views were part of a rework that Sway reports cut scan times from 3–5 seconds to 1–2 seconds and improved query speed by at least 40%. Those figures come from the case study, and they cover a bundle of work — the mobile app, DB indexing, refactored parcel creation, queues — so I won't pretend they isolate the read-path change. The architecture, though, is exactly the shape I'm describing.

What you inherit in exchange:

staleness windows you now have to define per data class,
backfills for whenever ingestion falls behind,
a reconciliation job that never quite finishes.

For prices and metadata, stale is fine. For balances, compliance states and anything money-shaped, stale isn't "old data" — it's wrong data, and this pattern doesn't apply. That's the classification from section one doing its job.

Writes: idempotency before retries

Here's the assumption I most want to break: that retry logic is where write-path resilience starts. It's where it ends.

Order of construction on any side-effecting integration:

An idempotency key on every outbound write.
A persisted intent record.
A reconciliation job.
Then retries.
Only then a circuit breaker.

Without the first three, a retry converts a lost response into a duplicate action.

Stripe's API is the reference design worth copying: the client sends an idempotency key, the server saves the status code and body of the first request for that key — success or failure — and returns the same result on every replay, pruning keys after at least 24 hours. That contract is what makes a retry safe: the vendor guarantees the second request can't act twice.

Your side of the contract is the intent record — persist what you're about to do before you do it:

INSERT INTO outbound_intents (idempotency_key, vendor, action, payload, status)

VALUES ('parcel-19f3-label', 'carrier-x', 'create_label', $1, 'pending');

- call the vendor with the same key; mark 'confirmed' only on response.
- A reconciliation job sweeps 'pending' rows and ASKS the vendor
- what happened before anything retries.

The row that stays pending after a timeout is precisely the case from this article's first paragraph: you don't know what happened. Reconciliation — query the vendor, then decide — is the only honest answer. Blind resend is a coin flip.

Webhooks force the same discipline from the other direction. Stripe's docs state plainly that:

endpoints might receive the same event more than once,
event order isn't guaranteed,
delivery retries run for up to three days with exponential backoff.

Their recommended defense is tracking processed event IDs — idempotency again, inbound this time. And with a three-day retry window, "we'll have processed the earlier event by then" is not a safe assumption at any point.

The stakes scale with how physical the side effect is. Sway generates carrier labels and dynamic ZPL for Zebra printers across FedEx, ClearJet, Million, T-Force, SmartKargo and UDS — six carriers, each a place where a duplicated call is a real label on a real box. Space Plus, a staffless self-storage operator, issues digital keys through Sensorberg's IoT platform — a retried unlock is a door opening. To be clear about attribution: those case studies document the integrations, not the idempotency machinery; the machinery is my recommendation for anyone operating that class of integration.

The gap you'll hit: plenty of vendors offer no idempotency key at all. Then your dedupe needs a natural key — order ID plus carrier plus day, say — and natural keys are ambiguous in exactly the edge cases that matter, like a legitimate second label for a split shipment. Budget design time for that key; it's harder than the plumbing.

When the retry is the bug

Retries also fail at the aggregate level, and there's good data on how. Google's SRE book works through the arithmetic: a backend at capacity starts rejecting requests, clients retry, and the retries themselves push the backend further past capacity — in their worked example a 10,000 QPS service ends up receiving 10,200 QPS precisely because it's failing. Their guidance:

Randomized exponential backoff, always.
A hard cap on retries per request.
A server-wide retry budget — their example is 60 retries per minute, after which requests just fail.

The multiplier that catches microservices teams: retries stack per layer. Three attempts at each of four layers is up to 64 attempts hitting the bottom, per the same chapter. If your adapter retries, your gateway must not.

That's why retry policy is a per-write-path decision, not a framework default. A blanket "3 retries with backoff" in your HTTP client is the single config line most likely to charge a customer twice.

Aggregators vs. direct integrations

At some vendor count, someone proposes an aggregator: one contract, one API, many providers behind it. It's usually decided by reflex — either "obviously yes, less code" or "obviously no, we control our stack." Both reflexes skip the actual trade.

What an aggregator buys you:

one contract and one integration surface,
someone else maintaining the long tail of providers.

What it costs you:

one failure domain where you used to have several,
one shared rate limit,
someone else's semantics for everything behind it — their error taxonomy, their timeout behavior, their idea of what a "declined" means.

Space Plus is a reasonable picture of the pragmatic middle: payments route through Adyen, alongside PayPal, AmazonPay and Sofort, plus a Store365 CRM integration — a setup the case study describes as a hybrid of bespoke build and off-the-shelf integration. Aggregate where the aggregator's domain is mature; go direct where the vendor is load-bearing enough to deserve its own adapter.

The overlapping-provider version of this problem is subtler. DEFI Wallet pulls asset and market data from several providers whose coverage overlaps — Moralis, Covalent, CoinGecko, DefiLlama and others. I'll flag what the source does and doesn't say: it lists the integrations; it doesn't describe a failover strategy, and I won't invent one. But anyone who's run overlapping providers knows the operational truth: two providers of "the same" data return different numbers, and deciding which one your product displays is a product decision wearing an engineering costume.

Vendor drift is an operational problem

Everything so far is architecture. The failures that actually page you a year in are operational: integrations rot silently.

A vendor tightens a rate limit without an announcement.
A deprecated field starts returning null.
A schema gains a field your parser chokes on.
Semantics move under a stable version number.

None of this throws a 5xx, so none of it trips a circuit breaker — your integration is "up" and wrong.

What catches it:

per-vendor contract tests running against recorded responses,
alerts on response shape and volume anomalies, not only on error rates,
a habit of treating a vendor changelog like a dependency upgrade.

A breaker knows the vendor is down. It cannot know the vendor is lying.

Sandboxes, UAT and release cadence at hundreds of integrations

Past a certain integration count, keeping validation running is a team-topology problem, not a tooling one. MINT, an adtech resource-management platform, operates 16 channels and 350+ vendor integrations; the working setup there was organizational:

14 independent teams, each with its own sandboxes,
QA embedded in the teams rather than centralized,
a dedicated UAT environment.

That replaced release-approval bottlenecks with multiple releases a day. The relevant lesson is about process: at that vendor count, a centralized release gate is itself the outage.

One warning from hard experience: sandboxes lie. Plenty of vendor test environments differ from production in latency, rate limits, and occasionally behavior. A green sandbox suite tells you your code matches the vendor's documentation — production tells you whether the documentation was true.

What this costs

None of this is free, and I'd rather you hear it from me than from your own on-call rotation.

Per-vendor isolation multiplies code, deploy units, credentials, dashboards and alert routes.
Caching reads buys you a reconciliation problem in perpetuity.
Idempotency adds a storage table, a dedupe path and a sweeper job to everything that writes.
Contract tests are a suite someone has to own forever.

Below roughly five integrations, most of this is overhead — a well-tested direct integration with sane timeouts beats a prematurely general integration layer. The flip point is easy to name because you'll feel it: the first time one vendor's outage takes down a feature that had nothing to do with that vendor, you've outgrown the shared plumbing.

So the order of effort, when you get there:

Classify every integration by blast radius.
Isolate each vendor behind its own adapter.
Move reads off the request path.
Make writes idempotent.
Only then add the retries and circuit breakers everyone starts with.

Each step makes the next one safe; run in reverse order, the list is a duplicate-payment generator.

And keep the last dollar for operations. The integration that breaks in production is rarely the one that went down — it's the one that changed on the vendor's side six weeks ago, while nobody was watching the response shape.

How to Build a Scalable File-Processing Pipeline: Validation, Metadata Extraction, Queues, and Workers

Vic Tekys — Tue, 15 Sep 2026 09:35:00 +0000

Here's a failure I've seen more than once on upload-heavy products: the endpoint returns 200, the user closes the tab, and twenty minutes later there's no order in the system because a worker choked on a vendor's proprietary container format. Nobody told the user. Nobody told the worker's author either, because the retry policy quietly redelivered the file until it hit a dead-letter queue nobody monitored.

The standard advice — "put a queue in front of it" — doesn't prevent any of this. The queue was never the hard part.

After building ingestion for dental CAD files, construction models, bulk carbon data, and business documents, I've settled on a different frame: a file pipeline is four stages with different latency budgets and different failure semantics. Throughput is decided by how early you reject bad input and how much derived work leaves the request path — not by which broker you pick.

Four stages, not one queue

The four stages are gate, ingest, extract, derive.

Gate is synchronous and cheap: auth, quota, size, extension, magic-byte sniffing. It runs while the user is still watching and its budget is milliseconds.
Ingest moves bytes into object storage and writes the metadata record. It should touch your API process as little as possible.
Extract is asynchronous, per-format parsing: opening the container, pulling metadata, validating structure. This is where the real engineering lives.
Derive produces the artifacts your product actually reads — thumbnails, previews, normalized records, optimized model formats. Also async, and unlike extract, usually cacheable and re-runnable.

The reason to name the stages is that the boundaries between them are where you retry, cache, and shed load. A parse failure shouldn't re-run the upload. A thumbnail failure shouldn't re-run the parse. When everything is one "process file" job, every retry repeats all the work and every failure is opaque.

On the Carbon Management Solution project — bulk emissions data, where a user uploads a single file containing any amount of data that then gets validated and processed — we ran this on serverless AWS with Lambda, SNS, SQS, and DynamoDB, structured so that a certain service is responsible for a certain stage of the process. Stage isolation was the design, not an accident of the stack.

Honest caveat: service-per-stage multiplies your operational surface. Four services means four sets of alarms, four deploy pipelines, four things to reason about at 2 a.m. At low volume, one worker pool consuming from a couple of queues is the right call. Draw the stage boundaries in code first; split the infrastructure when a stage actually needs independent scaling.

Validation is a two-tier problem

The all-async reflex — "accept everything, validate in the worker" — is how you get the silent-failure story from the intro. But the opposite reflex, deep validation in the request handler, is how upload endpoints time out. Validation is two tiers, and the split is not negotiable.

What has to fail while the user is still watching

Anything a user can fix by picking a different file must fail synchronously: wrong extension, oversized file, empty file, exhausted quota, a MIME type that doesn't match the bytes. If the user learns about these hours later from an email, your gate failed at its one job.

The gate checks are cheap because they read a few bytes, not the structure:

import { fileTypeFromBuffer } from "file-type";

const ALLOWED = new Set(["model/stl", "application/zip", "application/pdf"]);

const MAX_BYTES = 500 * 1024 * 1024;

export async function gate(head: Buffer, declaredSize: number, user: User) {

if (declaredSize === 0 || declaredSize > MAX_BYTES) {

throw new UploadError("size_rejected");

}

if (await quotaExceeded(user, declaredSize)) {

throw new UploadError("quota_exceeded");

}

// Sniff real content type from magic bytes — never trust the filename

const sniffed = await fileTypeFromBuffer(head);

if (!sniffed || !ALLOWED.has(sniffed.mime)) {

throw new UploadError("type_rejected");

}

The line to look at is the sniffing: content type comes from the bytes, not the filename. Renamed .exe files and mislabeled archives are a when, not an if.

On BEGO, a German dental CAD/CAM company, we went further and put scenario detection in the order wizard itself: the upload step auto-detects whether the user dropped 3Shape output, exocad output, standalone STLs, or a mixed set, and routes the flow accordingly. That's still gate-tier work — it reads signatures and file lists, not full structure — and it means the user is corrected before the order exists, not after.

What can only fail inside a worker

Structural correctness of a real-world format is unknowable until you parse it. Whether a proprietary archive decrypts, whether the XML inside references parts that exist, whether an STL mesh is watertight — none of that fits a request budget, and none of it should.

So the contract with the user changes shape: the synchronous response means "your file is accepted for processing," never "your file is valid." The pipeline then needs a way to deliver bad news — a per-file status the UI polls or subscribes to, flipping from processing to failed with a reason a human can act on. On BEGO, the order platform tracks per-file status through queue processing for exactly this reason.

The failure mode to design against: sync checks can be spoofed and deep checks can't be rushed. Any team that tries to collapse the two tiers into one ends up with either a slow gate or a lying one.

Metadata extraction is format work disguised as infrastructure

Generic pipeline tutorials treat "extract metadata" as one box in the diagram. In practice it's the majority of the engineering, because it's per-format parsing, and formats are hostile.

On BEGO the extract stage deep-parses laboratory formats — .3ox, .dentalProject, .constructionInfo, .modelInfo— plus encrypted .bego/.begostl archives with decryption and retry logic, pulling out patient and order metadata, materials, colors, and tooth mapping. None of that generalizes. Every one of those parsers is bespoke work against a vendor format that ships no public spec and changes without notice.

On MemoMeister, a document-management SaaS we built from scratch for the German market, extraction went a step further into classification: workers analyzed the keywords, data formats, and other characteristics of an upload to determine the document type and route it to the right user or group. The file's type came from its content, not its name.

Two lessons from shipping that kind of extraction. First, content-derived classification is probabilistic — you will guess wrong, so the design must include a manual-correction path, not just a confidence score. Second, budget for vendor formats changing silently underneath you; a parser that worked for a year is not a parser that works.

This is why "which queue should I use" is usually the wrong first question. The broker is a commodity. The .dentalProject parser is not.

Get bytes out of your API process

One flat infrastructure rule: file bytes should not flow through your API. The client uploads directly to object storage; the API's job is to issue the upload authorization, write the metadata record, and enqueue the job.

BEGO uploads via pre-signed URLs to cloud storage. MemoMeister stored documents on Amazon S3, with workers picking files up after a successful upload. The pattern is the same either way: the API hands out a scoped, expiring write permission and steps out of the data path.

The shape of it, with a commit step:

// 1. API issues the upload — after gate checks, including quota

const key = uploads/${user.id}/${randomUUID()};

const url = await getSignedUrl(

s3,

new PutObjectCommand({

Bucket: BUCKET,

Key: key,

ContentLength: declaredSize, // cap what the URL can write

}),

{ expiresIn: 300 },

);

await db.file.create({ key, userId: user.id, status: "pending_upload" });

// 2. Client PUTs bytes straight to storage

// 3. Client confirms; API verifies the object exists, then enqueues

export async function commit(key: string) {

const head = await s3.send(new HeadObjectCommand({ Bucket: BUCKET, Key: key }));

await db.file.update({ key }, { status: "uploaded", etag: head.ETag });

await queue.send({ key, etag: head.ETag });

}

The part to look at is step 3. Without an explicit commit, you have no moment at which the upload is known-complete, and your extract stage starts firing on half-written objects.

Be honest about what this pattern costs: validation now happens after the write. A client can obtain a URL and never call commit, so you need a sweeper for orphaned objects. Quota has to be enforced before the URL is issued, on the declared size, and re-checked at commit. Direct-to-storage upload is still the right default — it just isn't free.

The queue decisions that actually change throughput

Now the section everyone expects, argued honestly: broker choice matters far less than the policies around it.

BEGO runs its workflow processing on Redis-backed queues under NestJS and Kubernetes. MemoMeister ran RabbitMQ, with some workers calling third-party services like Google Vision and others monitoring database changes to update document state. Carbon Management Solution used SNS and SQS. Three stacks, one set of decisions that mattered:

Queue per stage, not one queue for everything. Extraction backing up shouldn't delay thumbnail generation for files already parsed. Separate queues give you separate backpressure and separate scaling knobs — and keep CPU-bound parsing workers apart from IO-bound workers waiting on third-party APIs, which want completely different concurrency settings.

Visibility timeout longer than your worst file. On SQS-style brokers, a message picked up but not deleted becomes visible again after the timeout. Set it shorter than your slowest legitimate parse and you get two workers processing the same giant file, doubling load exactly when the system is already struggling.

A dead-letter queue you actually watch. Bounded retries, then park the message. An unmonitored DLQ is the silent-failure story again, one hop removed.

One warning on MemoMeister's DB-watching workers: they worked, but they couple the pipeline to your schema — every migration becomes a pipeline change. I'd reach for explicit events first and treat DB-watching as a legacy-integration tool.

Idempotency, retries, and poison files

Standard queues are at-least-once delivery: the broker guarantees a message arrives, not that it arrives once. Operationally that means every worker you write will eventually run twice for the same file. Plan for it or debug it.

The fix is idempotency keyed on the stored object — bucket, key, and ETag — checked before the expensive work:

async function handle(msg: { key: string; etag: string }) {

const claimed = await db.extraction.tryClaim(msg.key, msg.etag); // unique index

if (!claimed) return; // duplicate delivery or a lost race — done

const meta = await parse(msg.key); // the expensive part

await db.extraction.complete(msg.key, msg.etag, meta);

}

Keying on the ETag rather than just the key means a re-uploaded file with new bytes is new work, while a redelivered message for the same bytes is a no-op.

Then there are poison files — the input that kills the worker itself. The classic is the model that OOMs the process: the worker dies before it can nack or delete, the visibility timeout expires, the message comes back, and the next worker dies too. Your fleet is now a crash loop with one file as the ignition. Defenses are boring and essential: a max-receive count routing to the DLQ, memory limits that fail the job instead of the pod, and a size ceiling in the gate so the 9 GB file never enters the pipeline at all.

Derived artifacts are the product

Here's the second assumption worth breaking: the pipeline doesn't exist to store files. It exists to produce cheap-to-read outputs. If the artifact your product serves is still expensive to consume, you optimized the wrong half.

BEGO generates STL thumbnails at extract time so the order UI never touches a mesh. And the clearest evidence I have is Vitus, a Danish constructability platform handling .ifc, .nwd, .rvt, .3dm, and other CAD formats for construction companies like Munck, Femern A/S, VINCI, and COWI. Their viewer struggled not because ingestion was slow, but because the format being loaded was heavy.

The fix was a bundle of read-side changes — migrating models to SVF2, parallel loading, web workers off the main thread, list virtualization, IndexedDB caching. Together they cut model size by 5x and loading time by roughly 2.5x, with multi-model loading 5x faster. To be precise about what those numbers are: client-side model-loading gains from the whole bundle of changes, not a server-side pipeline benchmark, and no single change gets the credit. But that's exactly the point — the wins came from changing the data format and the loading strategy. Worker count had nothing to do with it.

The tax on derived artifacts: they need versioning, and a format migration is a backfill, not a deploy. When Vitus moved to SVF2, every existing model needed re-derivation. Stamp every artifact with the version of the deriver that produced it from day one, so "re-derive everything older than v3" is a query and a queue-fill instead of a forensic project.

Constraints that beat the ideal design

Everything above assumes you get to choose where processing runs. Sometimes you don't, and residency, encryption, and hardware outrank performance.

MemoMeister was restricted to third-party services that stored and processed data within Germany. That single compliance line forced a self-hosted OnlyOffice deployment instead of a managed document service — an architectural decision made by a regulation, not a benchmark.

Visbion is the sharper example: their Image Cube compresses and encrypts DICOM 3.0 medical images on dedicated routing hardware inside mobile scanning trailers, serving NHS Breast Screening Services as the UK's largest installed base of dedicated imaging routing hardware. The heaviest processing happens at the edge, on a device in a trailer, before any cloud pipeline sees a byte — because bandwidth from a mobile trailer and encryption requirements for medical imaging say so. Our work there was consulting on fleet-deployment automation, compression and encryption refinement, and a modular redesign — tuning that constraint-driven architecture, not replacing it with a textbook one.

The trade is real: edge and self-hosted processing give up elasticity and easy observability for compliance and bandwidth. Check these constraints before drawing the architecture, because they don't negotiate afterwards.

Where to spend the effort, in order

If I'm starting an ingestion system today, the order is: a cheap synchronous gate that rejects everything a user can fix on the spot; bytes going straight to object storage with an explicit commit; idempotent workers with a DLQ someone watches; and only then worker scaling and broker tuning. The per-format parsers will take longer than all of the infrastructure combined — budget accordingly.

Reach for the full four-stage split when files are big, formats are hostile, and derived artifacts are what users actually consume. Skip the ceremony when you're thumbnailing avatars — a gate check and one background job is a complete architecture at that size.

And check the read side last, every time. If the artifact coming out of the pipeline is still expensive to consume, no amount of queue work will save you — you've optimized the half of the system your users never touch.

How to Modernize a Legacy SaaS Without Freezing Feature Delivery: Refactor, Replatform, or Rewrite?

Vic Tekys — Wed, 09 Sep 2026 14:30:07 +0000

The moment arrives on every aging SaaS. Sales has commitments on the roadmap, and the codebase is telling you it can't carry them — every feature ships with a regression tax, the framework stopped getting patches years ago, and nobody fully understands the billing module anymore.

The reflex answer is always the same: freeze the roadmap for two quarters, build "v2" on the side, migrate everyone over, resume shipping. I've sat in that meeting more than once.

That reflex is wrong twice over. First, no revenue-generating SaaS actually gets a two-quarter freeze — customers keep filing tickets, competitors keep shipping, and the freeze quietly becomes four quarters. Second, it treats "modernize" as one decision you make once, for the whole system.

It isn't. Refactor, replatform, or rewrite is a per-component verdict. And the way you avoid the freeze is to stop running modernization as a separate project and make it one backlog with feature delivery: build new capabilities as new services, refactor the code you're already touching, and rewrite only what genuinely can't be saved.

The stakes of getting this wrong are documented. In a 2022 Wakefield Research survey of 250 senior IT leaders at large enterprises, 79% of app modernization projects failed, at an average cost of $1.5 million and a 16-month timeline. Those numbers describe big-bang programs at big companies — but the failure mode they capture is exactly the "pause everything, modernize, resume" shape most teams reach for by default.

The false trilemma: you don't choose once for the whole system

Most refactor-vs-rewrite articles frame this as a single fork in the road. Pick a verb, apply it to the platform, done.

Every real modernization I've been close to mixed the verbs — sometimes all three, over years.

On GoodShape, a wellness platform that had been in production for about 15 years, we ran a microservices migration and heavy in-place refactoring at the same time: new capabilities went out as new services while hundreds of bugs got fixed and thousands of code smells got eliminated in the monolith that was still serving traffic.

On Sunlayar, the engagement started around the legacy QMPV solar-design platform — and only after working with it did the honest assessment land: continued maintenance would cost more than a rewrite, so a rewrite is what happened.

Same company doing the work, opposite verdicts. Because the verdict belongs to the component and its situation, not to a philosophy.

One warning before you adopt per-component thinking, though: it needs a whole-system owner. Somebody has to hold the target architecture in their head and decide which extraction happens next. Without that, "decide per component" degrades into a half-migrated estate that stays half-migrated forever — the worst of both worlds, permanently.

Diagnose what's actually failing before picking a verb

"Legacy" is not a diagnosis. It's a bucket for at least four different diseases that get treated very differently:

An obsolete stack — frameworks and runtimes past end-of-life.
Missing documentation — the system's behavior lives in people's heads, some of whom left.
Inconsistent code — written by different developers across different eras, no common standards.
An architecture at its scalability ceiling — the structure itself can't take more load or more change.

The first three are painful, but they're curable in place. Only the fourth justifies structural change — replatforming or rewriting.

This distinction matters because tech debt pain is nearly universal and therefore nearly useless as a signal. In the 2024 Stack Overflow survey, 62.4% of professional developers named technical debt as their top frustration — the number one answer among both ICs and managers. And McKinsey estimated back in 2020 that tech debt amounts to 20–40% of the value of a company's entire technology estate. If frustration were the trigger for a rewrite, everything would get rewritten. The trigger has to be sharper than that.

Architecture at its ceiling vs. code that's merely ugly

The question I actually ask: can this architecture carry the next two years of roadmap? Not "is this code nice." Not "would we build it this way today." Nobody would build any five-year-old system the way they'd build it today; that's not a finding.

GoodShape had all four diseases at once — no documentation, no common coding standards, an obsolete stack, and a monolith that had nearly reached peak performance, with further scaling becoming impossible. That last item is what earned structural change. The other three would have been refactor-shaped on their own.

medneo, a radiology-as-a-service provider operating across Germany and the UK, looked superficially similar — a Radiology Information System degrading under a growing patient load, with regressions creeping in as features accreted. But the architecture could still carry the roadmap. The diseases were slow code, inefficient database access, and outdated frameworks. Refactor-shaped, all of it.

One more bias to name: inside teams overwhelmingly want the rewrite. It's more fun, and it promises escape from code they resent. But fifteen years of production code encodes fifteen years of edge cases, regulatory quirks, and 3 a.m. fixes that nobody remembers making. Joel Spolsky called the full rewrite "the single worst strategic mistake" back in 2000, and the reason he gave — old code has been used, and using code shakes bugs out of it — hasn't aged a day.

Refactor in place — when the foundation still holds

Refactoring is the least glamorous verdict and the one I end up defending most often.

On medneo, the work was exactly as unglamorous as it sounds: optimize hot paths, eliminate unnecessary database calls, upgrade legacy frameworks and libraries to supported versions, and stand up a CI/CD pipeline. Eight engineers, from 2019 to 2022.

The outcomes — significantly reduced average load times, fewer outages, and new rollouts that stopped degrading performance — are qualitative, because the client doesn't publish before/after numbers. I'd rather tell you that plainly than invent a percentage. But the strategic point is quantitative enough: three years of platform improvement happened while the RIS kept serving hospitals, with zero freeze, because none of that work required stopping feature delivery. Refactoring almost never does.

Soapbox, an e-commerce fulfillment platform, ran the same play at a much earlier stage: a React Native warehouse app that had shipped as a prototype — poorly structured code, plenty of bugs, no separation between dev, staging, and production. The team did a global refactoring of the core logic, put a linter in place, set up GitLab and CodePush, and split the environments — and shipped more than 15 releases along the way, including entirely new features like stocktaking, warehouse audit, and barcode scanning. Cleanup and delivery, one backlog, four people.

The limitation is the one from the diagnosis section, and it's hard: refactoring cannot raise an architectural ceiling. If the structure itself is the disease, no amount of in-place cleanup cures it. You'll just have beautiful code that still can't scale.

Replatform incrementally — migration under live traffic

The middle verdict is the one that actually answers this article's title, so it gets the most detail.

GoodShape is the fullest version of it I've worked near. The platform had been in production about 15 years, serving 200+ UK employers — including NHS organizations, Philips, and Heathrow Express — with 24/7 nurse support riding on it. You do not take that offline for a rebuild. The verdict on the monolith was replatform: migrate toward microservices, incrementally, under live traffic. A team of 7, working since 2018, on Java 17 and Spring Boot with React/Redux up front.

The mechanism is what matters, because it's the anti-freeze tactic in concrete form:

New capabilities ship as new services. Push notifications and fitness-tracker data collection — integrating 16 different tracker brands — were built as standalone microservices from day one. The roadmap item was the migration step. Nobody had to choose between them.
Code you're already touching gets refactored. A few hundred bugs fixed and thousands of code smells eliminated — not as a dedicated cleanup phase, but woven through feature work in the monolith.
Documentation gets written as you go, capability by capability, instead of as a doomed up-front archaeology project.
The frontend got split into subprojects and refactored with updated React components — same incremental logic, applied to the UI.

Even purely manual pain got folded in: monthly client reporting had been a hand-built 50-slide PowerPoint; it became an automated reporting feature. Roadmap value and modernization, again the same line item.

If you want the underlying pattern with a name, it's Martin Fowler's strangler fig: new growth wraps the old system until the old system is no longer load-bearing. Stripped to its skeleton, the runtime side is just routing — carve one capability's traffic away from the monolith at the edge:

Everything still goes to the monolith...

location / {

proxy_pass http://legacy-monolith;

}

...except the capabilities that have been extracted.

location /api/notifications/ {

proxy_pass http://notification-service;

}

location /api/tracker-data/ {

proxy_pass http://tracker-service;

}

The routing is the easy part. The discipline is the hard part, and it's the limitation to be honest about: this is a multi-year posture, not a project with an end date — GoodShape's has been running since 2018. You need that whole-system owner deciding which extractions the roadmap actually justifies, or you'll extract whatever's fun instead of whatever's next.

One more honesty note, since it's tempting to oversell: GoodShape won gold at the 2020 ECCCSA awards for application of technology while all this was mid-flight. I'm not claiming the migration caused that — the sources don't, so I won't. The claim I am making is narrower and more useful: the business never had to stop operating, competing, or shipping while its 15-year-old core was being restructured underneath it. And the scalability gains came from the migration plus*the refactoring *plus the process changes — not from microservices as a magic word.

Rewrite — the expensive verdict, and what earns it

Sometimes the honest answer is that the patient doesn't recover in place.

The threshold I use: rewrite when maintaining the old platform costs more than replacing it, and little of it is worth preserving. Both conditions. The first alone describes half the software in production; it's the second that earns the verdict.

Sunlayar met both. The QMPV platform for planning rooftop solar projects ran on obsolete technologies, and as complexity grew it became impossible to add new business functions quickly — the assessment concluded a rewrite would beat continued maintenance. So it was rebuilt as a new platform on Node.js, TypeScript, React, and MongoDB, by a team of 2 developers and 2 QA engineers, in an engagement running since 2014. The rewritten platform outlived its original owner: QMPV was later acquired by Esdec, and the platform is still in production today as EcoFasten's Design Assistant Project Calculator. (Again — the acquisition is a fact of the timeline, not a result I'm attributing to the rewrite.)

BEGO, a German dental company, is the other flavor: not one obsolete platform but several independent portals, with an architecture where every new feature had to work around structural limitations. The verdict was rebuild-as-one-platform — React, TypeScript, NestJS, PostgreSQL, Kubernetes — delivered by a team of about six, with 14 major features, cooperation running since 2025, and a high daily order volume from launch.

But here's the detail that matters most for this article: even BEGO wasn't a pure greenfield rewrite. The proven backend file-processing components were carefully reused. "Rewrite" as a verdict doesn't mean "salvage nothing" — it means salvage selectively, keeping the parts that have earned their production scars.

The limitation: rewrites only look fast in retrospect. Sunlayar's engagement spans over a decade; nothing about these timelines is quick. If someone pitches you a rewrite with a confident six-month schedule, that schedule is the first bug.

The process layer that keeps features shipping

None of the above works without an unglamorous substrate, and it shows up in every one of these cases.

Environment separation, so modernization work and feature work can land in the same codebase without stepping on each other — Soapbox literally could not do both safely until dev, staging, and prod were split. CI/CD, so the cost of shipping small stays near zero — medneo's pipeline is what let optimization work roll out continuously instead of batching into risky releases. Automated linting, so a refactored module and a legacy module stop diverging in style the day after cleanup. Documentation as a habit, which is what GoodShape used to stop the no-docs disease from re-infecting every new service.

The trap: process work reads as slowdown first. It's the line item that gets cut when the roadmap pressure is highest — which is exactly the moment you need it, because it's the only thing that lets one team serve two masters in one codebase.

The question to write down first

The three verdicts aren't symmetric, and choosing between them gets easier once you say the asymmetry out loud.

Refactoring is cheap and reversible — if it turns out to be insufficient, you've lost little and the code is better for whatever comes next. Incremental replatforming trades speed for continuity — slower than the big-bang fantasy, but the platform never stops earning. Rewriting is a one-way door — occasionally still the correct door, as Sunlayar and BEGO show, but you don't walk through it because the old code is annoying. You walk through it when maintenance costs more than replacement and almost nothing is worth saving — and even then, you carry the proven parts with you.

So before picking a verb, write down the answer to one question, per component: can this architecture carry the next two years of roadmap?

Where yes — refactor what you touch, and keep shipping. Where no, but the capability boundaries are extractable — replatform incrementally, and let the roadmap decide the extraction order. Where no, and nothing's worth saving — rewrite, salvage selectively, and be honest with everyone about the years, not months, you just signed up for.

The one answer that's never right is the one that sounds safest in the meeting: freeze everything and build v2 on the side. That's not a modernization strategy. That's the 79%.

How to Keep a Data-Heavy React App Fast

Vic Tekys — Mon, 31 Aug 2026 13:18:24 +0000

A React dashboard that feels perfectly responsive with 200 rows may become almost unusable at 20,000. Memoization is often the first proposed fix, but React.memo cannot help much when the browser is laying out thousands of DOM nodes or a large data transformation is blocking the main thread.

Those problems may produce the same visible symptom—a slow interface—but they have different causes. Rendering too much calls for virtualization. Expensive JavaScript may belong in a Web Worker. Oversized or repeatedly downloaded datasets point toward pagination, caching, or IndexedDB. Media introduces another set of costs.

Getting the diagnosis right matters because users experience far more than the initial page load. Google estimates that roughly 90% of a user's time on a page is spent after it has loaded, when responsiveness depends on how quickly the application handles input, updates data, and paints the next frame.

Find the bottleneck before choosing the optimization

Most performance problems in data-heavy React applications fall into four categories.

Rendering cost grows with the number and complexity of DOM nodes React creates, reconciles, and updates. A table containing 10,000 visible rows is expensive even if every component is neatly organized.

Computation cost comes from JavaScript occupying the main thread. Sorting large arrays, parsing CSV files, filtering records, aggregating chart data, and transforming API responses can all delay keyboard input, scrolling, and rendering.

Network cost usually appears as an oversized response or a chain of dependent requests. A 4 MB JSON payload is one kind of problem; twelve sequential API calls are another.

Media cost becomes significant when an application loads large images, videos, or 3D assets before they are needed—or serves them at a higher resolution than the interface can display.

Memoization has a narrower role than it is often given. React.memo and useCallback can prevent avoidable component updates, while useMemo can prevent the same calculation from being repeated when its dependencies have not changed. None of them makes an unavoidable 400 ms transformation cheap, reduces a large API response, or removes thousands of DOM nodes.

Start with the React Profiler

Record the interaction that feels slow: changing a filter, sorting a table, opening a tab, or selecting a chart range.

The React DevTools Profiler shows which components rendered, how long the commit took, and what triggered the update. A wide flame graph containing hundreds of component renders usually points to a rendering problem. Depending on what the trace shows, the answer may be virtualization, more stable props, narrower state subscriptions, or selective memoization.

The same profiling model applies in React 18 and 19, so upgrading React does not change the basic diagnostic workflow.

Check what is blocking the main thread

Record the same interaction in the Chrome DevTools Performance panel. Look for long tasks—work that occupies the main thread for 50 ms or more.

A long JavaScript block during a calculation, without a corresponding burst of React rendering, suggests a computation bottleneck. Virtualizing a list will not fix it. The calculation may need to be reduced, scheduled differently, or moved to a Web Worker.

Chrome's Long Animation Frames API, available since Chrome 123, adds more detail by attributing slow frames to the scripts responsible for them. That makes it useful when investigating poor Interaction to Next Paint results.

Read the request waterfall

The Network panel answers two different questions:

How much data is being transferred?
Are requests happening in parallel or waiting unnecessarily for one another?

A single 3 MB response calls for pagination, compression, or a smaller field selection. A diagonal sequence of independent requests suggests a waterfall that could be flattened. Client-side caching may improve repeat navigation, but it does not make the first oversized response smaller.

The central distinction is simple: are there too many elements to render, or is one piece of JavaScript preventing the browser from rendering at all? The React Profiler is best suited to the first question; the Performance panel helps answer the second.

Match the symptom to the fix

The techniques in this article are complementary, but they are not interchangeable.

If the page janks while scrolling through a long table or list, the likely cause is rendering overload created by thousands of DOM nodes. Start with list or table virtualization using react-window, TanStack Virtual, or React Virtuoso. Keep in mind that off-screen content will not be present in the DOM, so accessibility, find-in-page, and SEO may require additional work.
If the interface freezes during sorting, filtering, parsing, or aggregation, heavy computation is probably blocking the main thread. Try reducing the amount of work or moving the calculation to a Web Worker. Workers add serialization overhead, cannot access the DOM, and require additional coordination code.
If a large dataset is repeatedly downloaded or kept in memory, consider persisting suitable data in IndexedDB with Dexie.js, idb, or RxDB. This reduces repeated loading and memory pressure, but the application must handle browser eviction, synchronization, schema changes, and stale data.
If the same requests run repeatedly or form a network waterfall, use TanStack Query or SWR together with data prefetching and parallel requests. The trade-off is that caching introduces questions around invalidation, staleness, and memory consumption.
If images, videos, or 3D assets delay interaction, start with compression, right-sizing, lazy loading, and intent-based preloading. Placeholders require additional UX work, while excessive prefetching can waste bandwidth.

A complex application may need several of these changes. A large 3D product viewer, for instance, might use a Worker for calculations, IndexedDB for model data, virtualization for property lists, and preloading for media. The important part is that each technique addresses a measured cost.

Virtualization, pagination, or both?

Virtualization and pagination both limit how much work the browser performs, but they change the product in different ways.

Virtualization maintains the experience of one continuous dataset. It renders only the rows inside or near the viewport and reuses DOM nodes as the user scrolls. A table may contain 50,000 records while keeping only a few dozen row elements mounted.

That makes virtualization a natural fit for authenticated dashboards, chat histories, log viewers, data grids, and infinite feeds. The compromise is that off-screen items do not exist in the DOM. Browser find-in-page cannot discover them, crawlers cannot index them, and some assistive-technology workflows require deliberate focus and navigation handling.

Pagination places a fixed number of items on each page and provides controls for moving between pages. It is usually easier to make crawlable, linkable, and accessible. Public search results, product categories, article indexes, and other SEO-relevant listings generally benefit from this model.

A hybrid approach is often the practical answer. The server returns a bounded page of records, and the client virtualizes that page if it is still large. Payload sizes remain controlled, while the browser avoids mounting more rows than the user can see.

Remember that grids can require two forms of virtualization. Row virtualization helps when there are thousands of records. Column virtualization matters when a financial or analytical grid contains dozens of columns. A library that handles simple lists well may not be equally capable with two-dimensional grids, sticky columns, variable row heights, and keyboard navigation.

Virtualization lets the user scroll through one continuous list while keeping only the visible rows in the DOM. It works well for dashboards, feeds, logs, chat histories, and authenticated applications. The downside is that off-screen content may not be indexed by search engines, while accessibility and browser find-in-page require additional work.
Pagination divides results into separate pages and keeps the number of DOM elements under control. Pages can be indexed, linked, and shared individually. It is usually the better choice for public listings, product directories, article indexes, and search results.
Server pagination combined with client-side virtualization provides a middle ground. The server returns a limited page of data, while the client renders only the visible part of that page. This works well for large authenticated datasets and complex data grids, although it requires managing both pagination and virtualization.

For libraries, react-window remains a focused option for straightforward lists and is actively maintained. Its older sibling, react-virtualized, is rarely the first choice for a new implementation.

TanStack Virtual is headless, which suits teams that want complete control over markup and styling or already use the TanStack ecosystem. React Virtuoso provides more behavior out of the box and is particularly useful for variable-height content.

The library decision should follow the interface requirements. A basic fixed-height list does not need the same machinery as an editable grid with pinned columns, dynamic heights, grouped rows, and accessible keyboard navigation.

Move expensive calculations to Web Workers

When a trace shows a long calculation without a corresponding rendering spike, moving that work off the main thread can restore responsiveness. A Web Worker can sort, parse, aggregate, or transform data while the browser continues to process input and paint frames.

Workers have firm boundaries. They cannot access the DOM, window, or localStorage. They can use fetch, WebSockets, and IndexedDB, which means a worker can retrieve, transform, and persist a large dataset without routing every step through the UI thread.

Messages between the main thread and a worker normally use the structured-clone algorithm. Large values must be copied, and that serialization cost can erase part of the expected performance gain.
Transferable objects avoid that copy for resources such as ArrayBuffer. Ownership moves from one thread to the other, leaving the original buffer unusable on the sending side. For binary files, large typed arrays, and some data-processing pipelines, this can be considerably cheaper than cloning.

SharedArrayBuffer allows genuine shared memory, but it introduces stricter deployment requirements. The page must be cross-origin isolated using COOP and COEP headers, which may conflict with embedded third-party content. Most applications can use transferable objects without taking on that constraint.

The native postMessage interface is manageable for one or two worker operations but becomes tedious as the API grows. Comlink, a small library from Google Chrome Labs, wraps worker communication in a proxy so asynchronous worker functions can be called more like ordinary methods.

A worker pool can help when the application runs many independent jobs. Instead of creating a worker for every task, the application maintains a small number of workers and feeds them work from a queue. The pool size still needs restraint: saturating every available core may speed up the calculation while competing with the browser for resources.

Workers do not reduce layout or painting cost. A page that mounts 10,000 rows will remain expensive after its data transformation moves off-thread. Workers address computation; virtualization addresses rendering.

Use IndexedDB when the browser needs a real local data store

React state is not persistent storage, and localStorage is poorly suited to large datasets. It stores strings, operates synchronously, and commonly offers only a few megabytes per origin. Reading or parsing a large value can block the main thread before the application renders anything useful.

IndexedDB is asynchronous and transactional, with a practical capacity far beyond localStorage, depending on the browser, device, and available disk space. It can store objects, typed arrays, Blob values, and files without manually converting everything to JSON.

That makes it useful for offline-first and local-first products, large working sets that must survive a reload, and expensive data that can be reconstructed but should not be fetched on every visit.

The native IndexedDB API is verbose and event-driven, so most React applications use a wrapper:

Dexie.js provides a promise-based database and query layer.
idb stays close to the native API while replacing the event-based interface with promises.
RxDB adds reactive queries and replication for applications that need live synchronization.

Choosing IndexedDB changes more than storage. Once a working copy lives in the browser, the application needs rules for refreshing it, resolving conflicts, migrating schemas, and distinguishing fresh data from stale data. A local-first architecture is a data-model decision, not a larger version of a cache.

Eviction also has to be expected. Browsers may remove storage under pressure. Safari applies a particularly strict tracking-prevention policy under which script-writable storage, including IndexedDB, can be deleted after seven days without user interaction with the site.

Unless the product explicitly guarantees durable browser storage, treat IndexedDB as a local copy that can be rebuilt. The server—or another controlled persistence layer—should remain the source of truth.

Cache repeat work and remove network waterfalls

After rendering and computation have been addressed, repeated network work often becomes the next visible delay.

TanStack Query and SWR solve several problems that otherwise end up scattered through components: request deduplication, cached results, background refreshes, retry behavior, and stale-data handling. They can return an existing result immediately during repeat navigation instead of showing another loading state for data the user has already seen.

TanStack Query's defaults deserve attention. It considers fetched data stale immediately by default, while unused cache entries are garbage-collected after five minutes. Those settings are safe but can produce unnecessary refetches when the underlying data changes infrequently. staleTime should reflect how quickly each resource can become outdated, not one global guess for the entire application.

Both libraries support the stale-while-revalidate pattern: return cached data first, verify it in the background, and update the interface if the response has changed. It improves perceived performance because repeat visits do not begin with an empty screen.
Prefetching can remove the remaining wait from predictable navigation. Route data can be requested when the user hovers over a link, focuses a control, or reaches a point where the next action is likely. Prefetch too aggressively, however, and the application downloads data the user never requests.

Request structure matters just as much as caching. Independent calls should begin together through Promise.all, parallel queries, or a backend endpoint designed to aggregate them. A five-request waterfall cannot be corrected by memoizing the component that waits for it.
Media benefits from the same discipline. A gallery, 3D viewer, or image-heavy customer interface can preload the next likely asset, retain previously fetched media, and lazy-load everything outside the active view. Service-worker caching through Workbox can extend the approach to unreliable connections and offline use.

Measure whether the change helped

A performance change is only useful if it improves the interaction users actually perform.

Interaction to Next Paint (INP) replaced First Input Delay as a Core Web Vital in March 2024. Unlike FID, which evaluated only the first interaction, INP considers interactions across the visit. A result of 200 ms or less at the 75th percentile is considered good.

That makes INP relevant to data-heavy React products, where roughly 90% of a user's time may occur after the initial load. A dashboard can achieve an excellent Largest Contentful Paint score and still feel slow every time someone filters a table.

Core Web Vitals will not capture every product-specific operation. “Time to render 10,000 rows,” “time to display the first interactive chart,” and “duration of a CSV import” require custom instrumentation.

The Performance API provides the necessary primitives:
performance.mark("filter-start");

// Run the operation.

performance.mark("filter-end");
performance.measure("filter-duration", "filter-start", "filter-end");

Benchmark before and after under the same conditions: identical dataset, device profile, browser version, and network throttling. Comparing a developer laptop with a production user's older machine produces an impressive number, but not a useful one.

Lab and field measurements answer different questions. Lighthouse and local traces provide controlled, repeatable feedback while engineers iterate. Real-user monitoring, CrUX, and the web-vitals library show what happens across the hardware and network conditions users actually have. Lab data helps isolate the change; field data confirms that the improvement survived production.

How these techniques combine in production

The strongest implementations rarely depend on one optimization.

Vitus, a construction-intelligence platform that works with several large 3D models, combined Web Workers, list virtualization, and IndexedDB. Model calculations moved off the main thread. React Virtuoso handled long property and data lists because pagination did not fit the product requirements. Dexie.js stored large model data locally instead of keeping the entire working set in React state.

A separate change—from the previous model format to SVF2 with parallel loading—cut model size roughly 5× and load time about 2.5×. That result came from addressing several layers: computation, rendering, storage, and network transfer.

Datasport, a sports-event platform with chart- and table-heavy dashboards, needed a different combination. Large tables were virtualized because pagination was not suitable, GET requests were cached, and component memoization reduced unnecessary rendering. React Context replaced Redux in areas where a smaller state layer was sufficient. Web Workers were not part of the solution because main-thread computation was not the defining bottleneck.

The Sales Platform, a media-heavy fashion-tech interface, paired virtualization with a dedicated media layer. Virtualized rendering reduced the number of mounted elements by about 70%, while preloading and caching reduced wait times by 50%. SWR managed data caching, and Workbox supported service-worker caching and offline behavior.

The same constraints appear in analytics products. GazeHealth, a mental-health analytics platform, renders a custom D3 Sankey diagram alongside roughly fifteen data visualizations over large datasets. GoodShape, an employee-wellness platform, combines complex charts with AG Grid tables and optimized large-data processing.

The projects use related tools, but not identical stacks. That is the useful lesson: performance work begins with the bottleneck, not with a preferred library.

When specialized performance engineering makes sense

One slow table or blocking filter is usually a bounded problem. An experienced engineer can profile the interaction, identify the expensive work, and implement a focused fix without redesigning the application.

The decision changes when performance problems cross architectural boundaries. Worker APIs, client-side databases, synchronization rules, data-fetching layers, and two-dimensional virtualization all create decisions that other parts of the product will depend on. Those changes need ongoing ownership.

Capacity matters as well. A team may understand the problem but lack time to investigate it without delaying product work. Engineers who have already implemented virtualized grids, worker pipelines, and IndexedDB-backed data layers can often recognize the performance signature faster than a team encountering it for the first time.

External help is most useful when the scope is explicit. A short engagement can address a measured bottleneck. A dedicated team is more appropriate when performance work affects the application's data architecture and will continue across multiple releases.