DEV Community

Cover image for Edge Read Slices: Take Read Load Off Your Central Database
dbslice
dbslice

Posted on Originally published at dbslice.net

Edge Read Slices: Take Read Load Off Your Central Database

#sqlite #postgres #mariadb #readreplicas


In many companies a single SQL Server or PostgreSQL database answers every read: microservices, web and
mobile apps, branches and point-of-sale terminals. As usage grows, reads compete with writes, and the central
database becomes the bottleneck for everyone.

This video shows a practical pattern: keep writes on the central database and move read-only slices of
the data to where they are used, in SQLite, PostgreSQL or MariaDB, refreshed by dbSlice on a schedule.

Who it is for

  • Architects looking for a way to cut read traffic on the central database without rewriting the apps.
  • Teams running branches or terminals that need local data, even when the link to headquarters is slow.
  • DBAs who want the copy to be a scheduled job with a log, not a script someone has to remember.

What the video shows

  • One synchronization per slice. A WHERE filter per branch, a key column, Upsert, and an incremental column such as updated_at that brings only the rows changed since the last run.
  • Reference tables with the Replicator. Tables every branch needs in full are copied in foreign-key order.
  • Automation. The same job.json runs from the Windows command line (with Task Scheduler) or on a cron schedule in the Linux Server Edition (Docker), with run history and webhooks on success or failure.
  • Checks. Structure compare and key-by-key Record Diff show whether the slice matches the source.
  • Dev and test. Copies for development can have sensitive columns masked while they are written.

Honest notes

This is not real-time replication: local data is as fresh as your schedule. The incremental column brings
inserts and updates, not deletes, so plan a periodic full reload of each slice. And measure performance on
your own data before you size the schedule.

Read more

The written guide, with the setup step by step, is in
SQLite at the edge. For scheduling jobs from the command line, see
database sync from the CLI. The free version is on the
Microsoft Store.

Originally published on the dbSlice blog. dbSlice is a Windows app for data migration, replication and sync between the leading databases.

Top comments (0)