DEV Community

Cover image for Thirteen Routines, One Keeper
Anton Brilliantov
Anton Brilliantov

Posted on

Thirteen Routines, One Keeper

Every pen in the zoo is fed by the same routine; the pen only carries a name tag. Thirteen routines of an entity are written once in the library, and a service binds names and keys to them.


👋 Hi, I'm Anton - a software engineer. These days I work on a platform: contracts, generated code, data paths and the tooling around them. Running notes live on my GitHub: github.com/brilliant-almazov.

The thirteen routines are built into the shared library as one runtime; four services import it so far. Everything below describes the runtime and its shape; I did not count how much copied code it replaced.

This is part 6 of the series on what a shared platform does for every service, told as a zoo. Maybe you already do it better, maybe you'd do it another way - this is how it is going, and where it stops.


In real life

Nora carries a bag of rule books. One per pen. "Rules for pen seven, version thirteen..." Every book says almost the same thing - when to feed, how to count, what to do when the bowl is empty - and every book is a little different, because each was copied from the previous one and then fixed in its own way.

Pip takes the bag away. "One book. Every pen." Nora's answer is the real joy of the change: "Finally! Light bag."

A four-panel comic: Nora with a pile of rule books, version thirteen; Pip says one book, every pen, and Nora says finally, light bag; Miso next to its name tag says my part is just the tag; Pickle fixes a typo in the book and Pip says every pen got it

What stays at each pen is a name tag. Miso's tag says "MISO, table: cats", and that is Miso's whole part in the routine. And when Pickle finds a typo in the book and fixes it, Pip delivers the punchline: every pen got the fix, at once, because there is only one book.

Thirteen things every entity does

Look at enough services and the same thirteen things show up for almost every entity. Read one by id, read a list, check that one exists, count, read many by ids. Create, find-or-create, update, upsert, soft-delete. Consume a message once, recount the table through the bus, answer a gRPC method.

The library's scenario runtime holds the logic of each of those thirteen exactly once, as a generic. The file of an entity in a service is a thin binding: which table, which key, which routines this entity uses. The library's description says it in one line - the logic of each scenario lives here once, the entity file in a service is a thin binding.

A waffle of thirteen cells in three colours: five read routines, five write routines, three bus and job routines; Pip says thirteen, written once; a pill reads built

Kind Routines
read by id, list, exists, count, by ids
write create, find-or-create, update, upsert, soft delete
bus and jobs consume once, backfill through the bus, a gRPC method

Two of these routines already had parts of their own in this series: consume-once (One Ticket, Punched Once) and the backfill (Feeding By Sectors). The other eleven are the same idea, smaller.

Three tiles with faces: a thin binding, happy; the generic, proud; the database, happy; labels read names and keys, one SQL shape; pills read built, and used by a few services

A name tag, and the routine it borrows

What a binding carries, as a form - the names are illustrative, the shape is the point:

# an entity in a service - a thin binding
entity: <name>
table: <schema.table>
key: id
scenarios: [byid, list, exists, count, create, update, softdelete]
Enter fullscreen mode Exit fullscreen mode

And what a routine does with it. Soft delete, for example, is one statement the generic writes for every entity that asks for it:

UPDATE <table> SET <mark> = TRUE WHERE <key> = $1 AND <mark> IS NOT TRUE;
-- zero rows -> a probe: is the row missing, or already deleted?
Enter fullscreen mode Exit fullscreen mode

A dark card: the binding form - entity, table, key, scenarios - and the soft-delete statement; the UPDATE line is highlighted; a pill reads built

That one statement shows why a routine is worth writing once. "Delete" sounds trivial until you ask what zero updated rows mean. The row may not exist - that is a not-found for the caller. The row may already be deleted - that is fine, deleting twice is the same as deleting once. A hand-written delete in each service answers that question each time, slightly differently, and some forget to ask it. The generic asks it once, with a probe after zero rows, for every entity.

So the list of what a binding does not carry is longer than the binding itself:

  • its own SQL for any of the thirteen routines;
  • its own answer to "zero rows" after a write;
  • its own transaction rules for consuming a message or a backfill window;
  • its own copy of a fix someone made elsewhere.

One fix, every pen

The benefit people notice first is less code. The benefit that matters is the fix. When a routine is wrong, it is wrong in one place, and the fix reaches every service that binds to it with the next version of the library - not "every service whose author remembers to copy the fix".

An animation: a bug in the routine makes the generic and three services worried; the generic is fixed once and turns proud; services A, B and C turn happy one by one; a pill reads every pen has it

Before and after: copies - 13 times N, the same logic per service; drift, one copy fixed, others not - and one keeper - 13 generics written once; thin, a binding per entity

Who uses it

Columns: files about 120, commits since October about 15, services 4; Miso, bored, says only four use it yet; pills read counted, rounded, and used by a few services

The runtime is young and still moving - around fifteen commits since the start of October and around a hundred and twenty files. Four services import it: the identity service, the token service and two more. The rest of the zoo still feeds its pens the older way, and that is the honest size of this part: one book, a handful of pens using it, more to move.

What went wrong

The reason for this runtime is the oldest failure in any codebase with many services: the same logic copied, then fixed in some copies and not in others. Thirteen routines times every entity of every service is a lot of copies, and a soft delete that treats "already deleted" as an error in one service and as success in another would be exactly the kind of difference nobody finds until a caller trips on it.

The second lesson is in the binding. A binding that grows logic - a special case here, an extra query there - is the first page of a new rule book. The binding stays a name tag on purpose; when an entity needs something truly its own, that is written as its own code, not smuggled into the tag.

Service tiles with faces: identity, tokens, project data and processing are happy; the generic proud; a copy by hand bored; a new pen happy; the reviewer happy; pills read built, and used by a few services

What I do not claim

  • Not that every service uses the runtime. Four do.
  • Not how many lines of copied code it replaced. I did not count.
  • Not that thirteen routines cover every entity's needs. They cover the common thirteen.
  • Not the exact binding format. The forms above show the shape.
  • Not that the runtime is finished. It is young and still changing.

What of this is method

  • Write a routine once, bind it many times. The service carries a name tag, not a rule book.
  • Ask the awkward question once. Zero rows after a delete is answered in the generic, for everyone.
  • Keep the binding thin on purpose. Logic in a tag is the first page of a new copy.

Zoo keeping, built in - Part 6. Next: A New Species Is One Line In The Register.

If your services share generic data routines, where does the binding stop and the code start? If you copy CRUD between services, how do you find the copy that missed a fix? If you tried generics for this, which of the thirteen was hardest to make generic? How many rule books does your keeper carry?

Top comments (0)