DEV Community: jericirenej

Chaining callbacks in TypeScript using the monad design pattern

jericirenej — Sun, 18 Feb 2024 19:23:47 +0000

A piece of information often results from linked operations where any of them can potentially return a nullish, invalid value that makes further operations meaningless or might even cause them to throw. This requires thorough checking of returns and parameters at each step in order to achieve consistent results.

Instead of having to do this verification manually - not only very tedious, but also very error-prone - the monad design pattern allows chaining of an arbitrary number of operations in a pipe-like fashion, while preserving type information and encapsulating the handling of nullish values within its implementation.¹

TLDR

To chain operations with built-in nullish checks:

Replicate the code (and the imported functions and types) from either the monad or monadAsync file.
- Using the async monad version is preferred, since it can handle both sync and async callbacks.
Use monadSetup | monadAsyncSetup functions, pass the nullish type list that will form the basis for nullish checks and assign this to a variable.
Use this variable for chaining operations.

Example

// Suppose undefined and null should be treated as "nullish types"
// that should prevent any further evaluation.
const monadFactory = monadAsyncSetup(null, undefined);

// Chain operations as necessary. Note that all of the arguments are supplied in the
// first invocation, but that the first argument is omitted in subsequent pipe calls
// since the previous pipe operation supplies it.
const result = await monadFactory(cb, firstArgument, ...rest).pipe(cb2 ...rest).value

Setting the stage

Before deriving an example of a monad pattern in TypeScript, let's first illustrate how we would deal with a sequence of potentially nullish values manually. Suppose we want to suggest which item a user should choose based on their hobby. The following functions are available:

const getUser: (userId: string) => User | undefined;
const getUserPreference: (user: User) => string | undefined;
const getPreferredItemId = (preference: string) => bigint | undefined;
const getItemDetails = (itemId: bigint) => Item | undefined;

If the user's id is available, the following steps are required:

1. Get the user data
2. Extract the preference
3. Retrieve the item id based on the preference
4. Return the item information based on the supplied id.

Each of these operations relies on the "truthy" output of the previous one, but each of them can also potentially return undefined.²

Manual checking

The most direct approach to deal with optional returns would rely on early returns for each intermediate step.

const suggestItem = (userId: string): Item | undefined => {
  const user = getUser(userId);
  // Here, the falsy value could also be returned directly
  // in each of the checks, for example: if (!user) return user;
  if (!user) return undefined;
  const preference = getUserPreference(user);
  if (!preference) return undefined;
  const itemId = getPreferredItemId(preference);
  if (!itemId) return undefined;
  return getItemDetails(itemId);
};

While this function has a mere 7 lines, 3 of them consist of checking for undefined values. If curly braces for the if statements were used, then checks would consume 9 lines out of 12!

This repetition exhibits a clear pattern that should be automated and abstracted away: we want to be able to perform a piping operation that has a built-in short-circuit logic in which valid return values can trigger further evaluation, while nullish ones simply echo down the line as far as necessary, with strict typing of results and arguments at each stage. This is just what monads can do.

The monad design pattern, as it will be implemented here, consists of a function³ that takes an initial value and a list of "nullish types" (think values such as undefined, null, perhaps 0 or -1). It returns a potentially nullish value property and a pipe operator. The pipe operator will take the previous result and conditionally invoke the supplied callback (along with any additional arguments that the callback might require) before returning the same thing as the monad did: a value property containing the potentially nullish result and ... itself, a new pipe operator which allows further chaining operations to take place.

Monad internals

For the sake of brevity, not all of the helper functions or types are defined here, but their meaning and functionality are often self-evident from their namings. On the other hand, more verbose types are sometimes used (for example for some function signatures) to better illustrate the type relationships. The reader is of course welcome to check out the entire code and typings contained within the repository's src folder.

Sync monads

At the heart of the monad is the conditional invocation - the bind function. It receives the callback, a potentially nullish value (corresponding to the the first argument of the callback), a list of arguments that count as being nullish, and any other additional callback arguments. The bind function encapsulates the nullish check that would otherwise have to be performed manually and conditionally runs the callback.

const bind = <Args extends unknown[], FirstArgument, Result, Nullish>(
  // Callback takes in a list of arguments and returns a potentially nullish result.
  callback: (...args: Args) => Result | Nullish,
  // Specifies a list of values under which an argument is deemed to be nullish or invalid.
  nullishTypes: Nullish[],
  // The potentially nullish first argument that is either 
  // the initial value or the result of a previous callback call.
  firstArgument: FirstArgument | Nullish,
  // DropFirst is a generic type that returns all of the array elements, except the first.
  ...rest: DropFirst<Args>
):Result | Nullish => {
  // isNullish is a type guard that evaluates whether the firstArgument 
  // can be found within the nullishTypes.
  if (isNullish(firstArgument, nullishTypes)) {
    // Value is nullish. Echo it down the pipeline, do not invoke callback.
    return firstArgument;
  }
  // Value is valid. Combine it with the rest parameters and return the callback result.
  return callback(...([firstArgument, ...rest] as Parameters<typeof callback>));
};

When using monads, bind never has to be explicitly invoked. Instead, the pipe function which facilitates the actual chaining operations will be the one calling it.

Pipe is a higher order function which takes in a potentially nullish value (first callback argument) and a list of "nullish value types". It returns another function to which the desired callback and any additional (rest) arguments can be passed as arguments:

const pipe =
  // The outer function is called either by the monad initializer function or
  // by the pipe itself in its return. It should never have to be invoked manually.
    <FirstArgument, Nullish>(
      prev: FirstArgument | Nullish,
      nullishTypes: Nullish[],
    ) =>
    // The argument list is bound to the FirstArgument type to ensure
    // that the compiler complains if an invalid argument is passed during
    // chaining. This safeguards against potential mismatching of functions.
    <Args extends [FirstArgument, ...unknown[]], Result>(
      callback: (...args: Args) => Result | Nullish,
      ...rest: DropFirst<Args>
    ) => {
      // To arrive at the inner function return, the bind function
      // which takes care of the nullish check is first called...
      const value = bind(callback, nullishTypes, prev, ...rest);
      return {
        // ...the potentially nullish result is returned,
        value,
        // along with a new pipe that allows additional chaining.
        // Note that the pipe itself provides the context to its outer invocation.
        pipe: pipe(value, nullishTypes),
      };
    };

The setup is almost complete, all that is left is to start it with an initializer function.

Granted, this extra step is optional, since the pipe itself has everything needed and could simply be called with the initial argument and a list of nullish values, then invoked again with the actual callback and optional rest parameters.

const result = pipe("userId", [undefined])(getUser)

However, that double invocation seems clumsy. The outer call can be refactored to a separate function which gives access to the pipe and value operators.

const monad = <FirstArgument, Nullish>(
  firstArgument: FirstArgument | Nullish,
  ...nullishTypes: Nullish[]
) => ({ value: firstArgument, pipe: pipe(firstArgument, nullishTypes) });

It is now possible to execute an arbitrary chain of type-safe piping operations that take care of the nullish checks automatically and allow adding of any additional arguments that each passed callback might require or allow. Furthermore, it is possible to access the current result or state of the pipe operation at any step of the chaining process.

Async monads

The monad just defined works perfectly fine as long as all of the piped operations are synchronous. However, often calls to (a mix of sync and) async functions are required. This presents additional considerations:

Even though pipes will accept async functions, they should not do any awaiting of the results themselves. If they did, each one of them would need to be awaited in turn (e. g. (await pipe(asyncCb).(await pip(asyncCb2))...) which would substantially hinder the pipe's usability. Instead, chaining should work just the same as in the sync monad. Only the values themselves should need to be unwrapped with a single await call.
Callback functions expect a resolved value, not Promises that async functions produce. As these results cannot be awaited in the pipe, they need to be resolved in an augmented bind function.

To achieve this, first the bind function is modified slightly. It should be async in order to await the potential Promise-like arguments before conditionally invoking the callback:

const bindAsync = async <Args extends unknown[], FirstArgument, Result, Nullish>(
  // The CallbackSyncOrAsync is just a shorthand for a function that can
  // be sync or async and that returns or resolves to a Result | Nullish type.
  callback: CallbackSyncOrAsync<Args, Result, Nullish>,
  nullishTypes: Nullish[],
  // The first argument can be an ordinary or Promise-like value from a previous
  // pipe operation.
  firstArgument: SyncOrPromise<FirstArgument | Nullish>,
  ...rest: DropFirst<Args>
): Promise<Result | Nullish> => {
  let unwrappedFirst = firstArgument;
  // If passed value is an instance of a Promise it is awaited before
  // performing the nullish check and conditionally invoking the callback.
  // Only the argument needs to be awaited, not the callback invocation itself.
  if (unwrappedFirst instanceof Promise) {
    unwrappedFirst = await unwrappedFirst;
  }
  /*...
    Same code as in the sync function, except the unwrapped variable is used
    instead of the firstArgument.
    ...*/
};

The modifications to the pipeAsync function are equally subtle, concerning mostly the changed parameter types, the call to the bindAsync function, and the result promisification.

export const pipeAsync = <FirstArgument, Nullish>(
  prev: SyncOrPromise<FirstArgument | Nullish>,
  nullishTypes: Nullish[],
) => {
  // Notice that the type of the FirstArgument needs to be "unwrapped"
  // as the argument might be async.
  return <Args extends [Awaited<FirstArgument>, ...unknown[]], Result>(
    callback: CallbackSyncOrAsync<Args, Result, Nullish>,
    ...rest: DropFirst<Args>
  ) => {
    const value = bindAsync(callback, nullishTypes, prev, ...rest);
    return {
      // Promise.resolve() flattens the nested promise
      // signature that results when an async callback
      // is passed to the bindAsync helper.
      value: Promise.resolve(value),
      pipe: pipeAsync(value, nullishTypes),
    };
  };
};

Finally, the initializer function will promisify the initial value and call the pipeAsync.

const monadAsync = <FirstArgument, Nullish>(
  firstArgument: FirstArgument | Nullish,
  ...nullishTypes: Nullish[]
) => ({
  // Since the first argument is most likely not a promise
  // but the value should return one, we promisify it.
  value: Promise.resolve(firstArgument),
  pipe: pipeAsync(firstArgument, nullishTypes),
});

The cherry on the cake: reducing boilerplate

The current monad implementation tries to be as clear and flexible as possible, but this might come at a cost to usability.

The monad initiator doesn't do anything except prepare the context for the pipe operator which still needs to be invoked manually.
Oftentimes nullish values are predictable. They will be undefined or undefined | null. As it stands currently, these values have to be specified each time a monad is created, which can lead to a lot of repetition. And repetition is something we've been trying to avoid.

Both of these issues can be addressed in several ways without touching the original implementations with the help of higher order functions. Let's look at some of them by using the example of the sync monad (the async implementations are available in the src folder).

Executing the callback with a default nullish type

If undefined is the default nullish value and the callback should be executed while defining a monad, this can be achieved in the following way:

type DefaultNullType = undefined;
type DefaultMaybe<T> = T | DefaultNullType;

export const monadRun = <
  FirstArgument,
  Args extends [FirstArgument, ...unknown[]],
  Result,
>(
  callback:  (...args: Args) => DefaultMaybe<Result>,
  firstArgument: DefaultMaybe<FirstArgument>,
  ...rest: DropFirst<Args>
) => {
  const nullishTypes: DefaultNullType[] = [undefined];
  return monad(firstArgument, ...nullishTypes).pipe(callback, ...rest);
};

// Use case:
const result = monadExec(callback, ...argumentList)
  .pipe(secondCallback, ...argumentList)
  .pipe(thirdCallback, ...argumentList).value;

Defining nullish conditions, but deferring execution until needed

On the other hand, it might be more desirable to just setup the "nullish types" context, but then invoke the monad again with some arbitrary callback and arguments at various points in our code.

export const monadSetup =
  // First invocation will only take in the nullish list and do nothing
  // until invoked for the second time.
    <Nullish>(...nullishTypes: Nullish[]) =>
    // Here the procedure is similar as in the monadRun implementation
    <FirstArgument, Args extends [FirstArgument, ...unknown[]], Result>(
      callback: (...args: Args) => DefaultMaybe<Result>,
      firstArgument: FirstArgument | Nullish,
      ...rest: DropFirst<Args>
    ) =>
      monad(firstArgument, ...nullishTypes).pipe(callback, ...rest);

// Use case:
const monad = monadSetup(undefined, null, -1);

const result = monad(callback, firstArgument, ...rest).pipe(
  secondCallback,
  ...rest,
).value;

Conclusion

Repetition of the same logical checks and conditional invocations while performing successive operations is something that is required, but at the same time this requirement should not be done manually in order to prevent errors, improve readability and reduce code bloat.

The monad design pattern offers a solution by abstracting the logical principles behind this repetition and encapsulating them within its implementation, while offering a straightforward configuration setup and an intuitive interface for chaining operations and extracting their results. Hopefully, the structures derived in this article and the associated repository are a useful example of that.

This article will not go into great detail about the theory behind monads and its various implementations. Instead, it will only focus on the explanations and the implementation of a type-safe pipe-like operation in TypeScript which also just happens to be particular application of a monad pattern, because it seems to satisfy its conditions. ↩

The conditions satisfied here are: 1. the monadic type ("nullish types" list) with which the original value types are augmented in a type union via type assertions, function parameters and return typings; the 2. the unit operator, where the input and output values are always implicitly augmented as being potentially nullish, and 3. the bind operator, as these augmented values are always unwrapped before being passed as a parameter to a function that expects the original value type and which in turn also returns a potentially nullish result. For more information about monads, see the Wikipedia entry or these instructive videos: What is a monad? (Design Pattern), What is a Monad? - The Last Monad Intro You'll Ever Need.

Note the use of the "seems to satisfy" phrase in the first paragraph. As no particular expertise in the theoretical foundations of monads is claimed here, there is a possibility that the structures derived here may not satisfy the definition of a monad in the strict sense. The hope is, instead, that they are "close enough" to being a monad - in other words, that they follow the monad design pattern.
Although here the undefined return is typed explicitly, often these nullish returns remain implicit in non-strict contexts, making it much easier to forget to double check that the result actually resolves to something rather than nothing. ↩
Although functions will be used here, the same functionality could be achieved with classes as well. ↩

Emoji string lengths

jericirenej — Mon, 07 Aug 2023 17:49:29 +0000

A modest contribution on how to count what is seen, not what is composed

TLDR

To produce counts of Unicode strings that correspond to observed distinct graphical symbols, several reductive operations need to be performed on the original string.

We need to remove surrogate and variant encodings, as well as modifiers where appropriate. We also need to account for the Zero Width Joiner (ZWJ) connector. The final result can be seen in the index.ts file or at the bottom of this document.

We count what we see

But JavaScript does not

A perceived piece of text is as long as the sum of its discrete parts.

We expect that anything that we recognize as a single unit - a letter, punctuation mark, or other distinct graphical signs - should also be taken as indivisible and counted, one by one, until we get to the end.

Intuitively, this seems clear. Just as the word Hello has 5 distinct letters, each of the following emojis: 💩, ❤️, 👋🏻, 🤽🏿‍♀️ or 👨‍👩‍👧‍👧 are perceived as distinct, single separate units. Consequently, counting the parts of the string Hello 👋🏻 should amount to a length of 7.

Except that's not the way this goes in JavaScript.

"👋🏻".length; // => Expected 1, got 4.
"👨‍👩‍👧‍👧".length; // => Expected 1, got 11.
"🤽🏿‍♀️".length; // => Expected 1, got 7.
"Hello 👋🏻".length; // => Expected 7, got 10.
"Family 👨‍👩‍👧‍👧".length; // => Expected 8, got 18.

Why does this discrepancy occur?

From encoding to appearance

The many are one

The original length results above actually represent a correct assessment of the required Unicode character combinations that produce the observed symbols. The length operation does not count what we immediately expect - the final visual units result - but rather all the bits and pieces that are combined together to compose that final appearance: a pale hand, a family, a woman playing waterpolo.

That simple symbols are actually composed from more primitive ones should be very familiar idea. When we learned to write, we saw that every letter was composed from different lines. The single letter I is drawn with a single line, the single letter H by a combination of three different lines |, -, |.

Outputting other symbols, such as emojis, can also require composition. A waving hand emoji can have a skin color modifier. A waterpolo player can be gendered (a woman) with a medium dark skin tone. A heart can have a red color variant. And so on...

To help us understand how to count what we see and perceive-as-distinct, different explanations, proposals, and strategies have been made, with varying degrees of success and flexibility.¹ This article attempts to build on that and offer a relatively compact function that will allow for counting the length-as-perceived of many different Unicode strings, particularly when they contain emoji characters.²

Obviously, it isn't perfect ☝️

Obviously, any suggestions are welcome 😀

Count rules

Ignore that which will not be seen

In our - admittedly anecdotal - tests we have always observed that string counts are at least as long as the number of symbols that we expect to see, but that they can sometimes overshoot. We have not observed counts that would be lower than the number of final symbols.

The extra counts are due to characters which modify or connect characters, which have an effect on the final appearance, but do not appear by themselves as separate symbols.

Therefore, there are two main sets of rules that will guide our code structure:

Modifiers should generally be ignored

Surrogate pairs (combination of two characters to generate a single symbol with the aim of expanding the Unicode space)³ should be ignored, as the pair express a single visual entity.
Variant encodings (for example the encoding for the red heart emoji) should be fused together with the character whose variation they represent and should have no visual meaning apart from them.
Similarly, skin tone selectors augment another body part emoji and are fused with that appearance.
- Exception: For modifiers with its own graphical representation (like the skin tone modifier), these should count as distinct if used on their own.

Connector sequences should suspend the count

Zero width joiner (ZWJ) indicates that the previous and subsequent standalone symbols should be treated as a single unit.
A connector sequence is defined by a chain of single graphics connected linked by the ZWJ.
- For example, a variant of the family emoji (👨‍👩‍👧‍👧) is composed of four standalone symbols ('👨', '👩','👧', '👧'), connected by the ZWJ. All of these distinct elements are combined together visually and count-as-one due to their connection with a ZWJ.

Code implementation

The parts

Remove surrogate pairs: Spreading the string into an array ([...str]) will remove any surrogate pairs (the infamous "💩".length equals 2 issue).
Remove variant selectors: The spread will not remove the variant encodings (the encoding that makes the 🤍 emoji into a red ❤️ symbol), however, so these still return a count of 2. To address that, we split the string on a regular expression (regex) which captures these encodings (/[\u{fe00}-\u{fe0f}]/gu). After splitting string and then joining it again, the variants will be removed (str.split(regex).join("")).
Remove modifiers: Same splitting approach, with a twist. We still want to count the modifiers, if they only represent themselves - and thus appear - and do not modify anything else. Our splitter is therefore a composite:
- Modifier capture: here we are limiting ourselves to skin modifiers, but it's easy to extrapolate to other cases: [\u{1f3fb}-\u{1f3ff}].
- Negative lookbehind: We presuppose that a modifier comes after the thing it modifies. Therefore, it should not be preceded by a space, or be placed on the beginning of the line. We also presuppose that modifiers do not modify ordinary script letters. So the lookbehind assertion, that condition whether or not a modifier gets captured, will be: (?<!(\p{L}|^|\s|\p{Punctuation}))
- Final regex: /(?<!(\p{L}|^|\s|\p{Punctuation}))[\u{1f3fb}-\u{1f3ff}]/gu
Account for ZWJ:
- After removing surrogates, variants, and modifiers, we lastly split the string on the ZWJ capture regex: /\u{200d}/gu
- If the split length is 1, we have no ZWJ and can safely join the filtered string, spread it and count its length.
- Otherwise we calculate the length of the array by reducing it in the following way:
- For the first element, we take its length.
- For subsequent element, we add its length, then subtract 1 to adjust for the fact that the current element forms a single unit with the previous one via the ZWJ.

The whole deal

export const characterCount = (str: string) => {
  // Not strictly needed for the count, but why not normalize, if we can 😀
  const normalized = str.normalize();

  // Define regex selectors
  const variantsSelector = /[\u{fe00}-\u{fe0f}]/gu;
  const skinModifiers = /(?<!(\p{L}|^|\s|\p{Punctuation}))[\u{1f3fb}-\u{1f3ff}]/gu;
  const zeroJoinRegEx = /\u{200d}/gu;

  // Remove variants and modifiers.
  const purifiedStr = normalized
    .split(variantsSelector)
    .join("")
    .split(skinModifiers)
    .join("");

  //
  const splitWithZero = purifiedStr.split(zeroJoinRegEx);

  if (splitWithZero.length === 1) {
    return [...splitWithZero.join("")].length;
  }

  // Because an emoji that contains ZWJ can contain other text left and right from it
  // we need to count the entire text length from each part, then subtract one.
  // For example: "A 👩‍❤️‍👨 is two people and a heart" splits into  [ 'A 👩', '❤️', '👨 is two people and a heart' ]
  const total = splitWithZero.reduce((sum, curr, currIndex) => {
    if (currIndex === 0) return (sum += [...curr].length);
    sum += [...curr].length - 1;
    return sum;
  }, 0);

  return total;
};

For example, the well known "💩".length === 2 webpage does a terrific job of explaining the different peculiarities of trying to count the length of Unicode encoded text. ↩

However, when it comes to resolving the issue of the ZWJ, it will not count correctly whenever the inspected string has more than one character. In those cases it will even return fractional values! It will also fail for cases in which modifiers are used, for example the skin modifier.

To observe these discrepancies, take a look at the example test suite.
There are a number of great sources that deal with the intersection of Unicode and JavaScript. Besides the already referenced "💩".length === 2, you're invited to also take a look at The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!), What every JavaScript developer should know about Unicode, and JavaScript has a Unicode problem. ↩
See Surrogate pairs and variation selectors ↩