DEV Community: Federico Sordillo

Effective communication between services (and people)

Federico Sordillo — Tue, 24 Aug 2021 08:54:39 +0000

As programmers we known full well that code without side-effects is both perfect and perfectly useless, which should be enough to convince you of how important communication and boundaries (or I/O, using software jargon) are.

But they are not just important, they are ubiquitous: there is a boundary between databases and servers, a boundary between servers and clients, a boundary between clients and the final user, but also boundaries between servers and servers, between clients and third-party softwares, and the list goes on..

At the end of the road, any software we can think of is just implementation detail over the mapping it creates between its domain and codomain.

If that wasn't enough, with the rising of micro-services the problem of communication has become even more apparent as there is an ever increasing number of actors and communications to juggle.

Communication is facing the same scaling problems that it was trying to solve in the first place.

But what is a communication? How can we achieve success when communicating?

I argue that you need two things (plus an additional one that I will mention at the end):

a shared format: in software engineering this is usually represented by a protocol like HTTP or SOAP, in verbal communication it is sound.
a shared meaning: in verbal communication it is created by validating decoded input against a more or less shared "common sense" while in software engineering it usually corresponds to input parsing.

Shared format

As applications are loosely coupled, there is no guarantee that the same context will apply to both ends of the communication, which is why you usually have to translate your communication using a "shared format" to make sure that what is being sent can be identified as communication by the receiving party.

When a communication takes place four things happen:

The sender "serializes" a communication intent into the shared format.
the sender then sends the communication.
The receiver receives the communication.
The receiver "deserialize" the communication using the same shared format used by the sender, then process it accordingly.

Summarizing, we have:

2 actors talking in their own dialects
1 shared communication protocol

And that is all apparently. Except it is not.

Agreeing on a protocol and knowing how to translate to/from such shared protocol is in fact not enough to guarantee a communication is taking place: it only guarantees the syntactical correctness of the content but there are still zero guarantees about its contextual coherence and its "purpose".

As an example, is this a conversation?

Person_A: "hi person_B! How are you? Everything fine?"
Person_B: "I own five dogs"

Sure both persons were able to "decode" each other's sentences, but I have difficulties marking it as a "successful conversation"... Person_A was probably able to "deserialize" the message sent by Person_B but it doesn't make any sense.. It could mean that life is a blessing because he/she is enjoying the company of five (five!) dogs or it could mean that he/she is on the verge of death because Person_B is very allergic to dogs fur.

There is likely a context missing or a a problem with how Person_B communicate "meaning".

Shared meaning

There is indeed another level of abstraction that we have to secure to deem our conversation successful: we have to agree on a set of intelligible communication intents. In other words, we have to make sure that the receiver can interpret the messages forwarded by the sender in a meaningful way and viceversa.

How do we do it? Well, in human interactions we (usually) have a second "shared asset" other than the communication protocol which is common sense: both the sender and the receiver just assume that they both have the "common-sense" resource available and validate communications against it.
Nonetheless, this is not a perfect guarantee that the conversation will be successful: if the versions of "common-sense" held by the two actors differ in substantial ways the conversation intent will be lost in translation.

Software development is a whole other story. We have to deal with extremely limited resources compared to our brains and even if someone ever succeded in encoding a digital "common sense" (something very far from happening, if ever so) we would still not have a guarantee that our softwares are able to communicate effectively.

The poor man solution to this dilemma is to always encode all valid communication intents and validate every deserialized content against it. If we can decipher the communication intent it is all good, otherwise the best thing we can do is to fail fast, break everything (maybe with a gracious error message) and wait for someone to fix it.

In real life the set of valid communication intents is huge, but in software development it is usually much smaller, often just a singleton set, so this encoding effort, albeit sometimes expensive, is at least doable.

Ergonomicity

That would be all if it wasn't for one last thing that we always do (usually going unnoticed in human communication), which is the translation from the sender dialect to the receiver dialect.

When we talk with someone we always unconsciously map the content of its communication intent to the set of our own internal meanings before analyzing it and creating our response. This is particularly evident when we talk with someone in a different language: we receive a communication that we are able to deserialize through our shared format (sound), validate through our shared meaning (common sense) then, before processing it and continue the conversation with a response that we deem reasonable, we parse it to translate to our own language.

This parsing phase is merely for the sake of ergonomics: we already known that the message is well-formatted and make sense but we transform it into something that our brain can handle faster and better in order to "smoothen the gears".

Conclusions

Each of the three steps depicted are of fundamental importance for a conversation to be successful, yet, especially in software using dynamic languages like JavaScript or PHP, one or more of these steps is often overlooked or even skipped altogether.

My passionate suggestion is to never try to skip the un-skippable, there are tons of great libraries that helps you not only validating but parsing your payloads like io-ts or yup (if you are not willing to give functional programming a try :))

Refactoring absolute paths to relative ones

Federico Sordillo — Mon, 29 Jun 2020 10:03:36 +0000

This is hardly a post, more like simple advice to anyone willing to move his/her codebase to relative paths.

I came to despise magic paths as I believe they solve no problem and usually introduce several non-trivial compilation bugs as soon as you move things around a little.

I did this kind of refactoring several times in the past and noticed that often people just go brute force at the problem, rewriting all the absolute paths one by one and wasting hours of their time.

A simple way to critically reduce the time spent on the task is to use VSCode regexp pattern matching to only target files located at a specific depth level. Assuming you have a magic path pointing to your components folder and a project structure like this:



...
├── package.json
└── src
    └── components

You can refactor it by a simple find and replace:

And then you just go recursively:

Hope this will save someone some minutes :)

What should and shouldn't be handled with Reader

Federico Sordillo — Mon, 24 Feb 2020 10:45:54 +0000

Recently I had the chance to discuss with a colleague (friend?) of mine about the Reader monad and why all the examples found in literature seem to always be about implicitly threading a Config object through your application.

I was frustrated to see every article resort to the same old example and felt there would surely be dozens of other use cases where you could fruitfully put Reader to use. This frustration led me to a few thoughts that I want to share with everyone in the hope that it may be helpful to other people naively approaching the subject (as I was).

First of all a little intro: the Reader monad in fp-ts: 2.0 is nothing else than an alias of the function profunctor, its signature is:

interface Reader<R, A> {
  (r: R): A
}

In fact, this very simple (and powerful) structure allows us to define not only profunctor instances for it but also monadic ones! So feel free to chain on it like there's no tomorrow.

The way this abstraction is usually put into use in codebases is by creating a context that can be accessed at any level of your application without the need to thread it to all the places where it needs to be used:

// src/index.ts
type C = {
  i18n: [[k:string]: string],
  threads: 2,
  dbHost: string
}

const MyApp: Reader<C, unknown>= (c: C) => {...}

// src/repository/entities/arbitraryNesting/../index.ts
import { ask } from 'fp-ts/lib/Reader'

const getById = (id: string) => {
  const { dbHost } = ask<C>()
  ...
}

so what I was thinking is: why just configuration? Why don't we use Reader every time there is unnecessary chaining of parameters?

In this example (from which I stripped everything not strictly related to the point I am trying to make) I only pass roles to the function getDocsByRoles and filter is retrieved autonomously by getDocsByRole from the Reader context, thus avoiding defining it explicitly on getDocsByRoles API:

import { reader, Reader } from "fp-ts/lib/Reader";
import { array } from "fp-ts/lib/Array";


const getDocsByRole = (role: string) => (filter: string): any => {
  ...
}

const getDocsByRoles = (roles: string[]): Reader<string, any> => {
  const docs: Reader<string, any[]> = array.traverse(reader)(roles, getDocsByRole)
  return reader.chain(docs, fancyTransformation)
}

const getUserDocs = (request: DocsRequest) => {
  const roles = ...
  const filter = ...

  return getDocsByRoles(roles)(filter);
}

What I did in this example may look right but there is fundamental (and subtle IMHO) flaw: while we gained a little in conciseness by avoiding to chain down the filter parameter, we lost a lot in terms of readability of the API. In fact, there would be no working implementation of getDocsByRoles without a filter and its return value strictly depends on it: although the function does not use it, it is a fundamental part of its ergonomics.

This, in my opinion, is the reason why all the Reader examples you see around are ones involving App configuration and global contexts.

While db location, secrets, and even i18n are usually details of an app that can be safely hidden from the surface of your internal APIs, other runtime values are usually strictly tied to the behavior of your implementations and, therefore, should not be hidden. It is a matter of ergonomics.