ecyrbe

Posted on Jun 11, 2022

How to use advanced Typescript to define a `pipe` function

#typescript #programming #tutorial #functional

Typescript is awesome but some functionnal libraries have limited implementation for Typescript definitions of :

pipe
compose
flow

To remind you what these functions do, let take an example. Imagine you want to chain multiple functions calls :

const n = -19;
const result = Math.floor(Math.sqrt(Math.abs(n))));

This can be hard to read hence the pipe function :

const n = -19;
const result = pipe(n, Math.abs, Math.sqrt, Math.floor);

This is easier to read, because the read order is the same as the execution order. This becomes clearer as the number of composed functions grow.

The problem

But unfortunately that's where most libraries have issues. Ideed, if you look closely at fp-ts or lodash pipe implementations you'll see that typescript support has a limited function count support.

For fp-ts, after 19 composed functions, you'll have no more type checking.
For lodash, it's even worse, only 9 functions are supported.
This is because these libraries are using Typescript function overload definition instead of recursive Typescript definitions.

So here we'll take a look at some advanced Typescript to define unlimited function parameters for pipe definition.

Javascript pipe implementation

First we are implementing a version of the pipe function without type annotations :

function pipe(arg, firstFn, ...fns) {
  return fns.reduce((acc, fn) => fn(acc), firstFn(arg));
}

It's pretty simple. An alternative for loop implementation would be :

function pipe(arg, firstFn, ...fns) {
  let result = firstFn(arg);
  for (let fn of fns) {
    result = fn(result);
  }
  return result;
}

How is pipe function constrained ?

When we say we want to add Typescript definition it's because if one of the functions result don't match the next function parameter type, we are likely to encounter a runtime error.
And it would be nice to catch this error at compile time, so the programmer knows early that there is a type missmatch.

To illustrate the constraints of the pipe function parameters, we can take the fp-ts implementation for 2 functions:

function pipe<A, B, C>(a: A, ab: (a: A) => B, bc: (b: B) => C): C

what we can observe is that :

The first function parameter type should match the first pipe parameter type
The result type of intermediary functions should match the parameter type of the next function
The result type of the last function is the result type of the pipe function

we'll now translate these contraint in Typescript.

Typescript pipe definition

Let's add types with the final function definition. Don't be affraid, we'll explain everything :

function pipe<FirstFn extends AnyFunc, F extends AnyFunc[]>(
  arg: Parameters<FirstFn>[0],
  firstFn: FirstFn,
  ...fns: PipeArgs<F> extends F ? F : PipeArgs<F>
): LastFnReturnType<F, ReturnType<FirstFn>> {
  return (fns as AnyFunc[]).reduce((acc, fn) => fn(acc), firstFn(arg));
}

This implementation is the same as the javascript one, but with some type annotations.

Let's decrypt everything :

first line

function pipe<FirstFn extends AnyFunc, F extends AnyFunc[]>(

This mean that pipe is a generic function with two generic parameters unknown when defining this function. But we know at least that all generic parameters should be functions. that's the meaning of FirstFn extends AnyFunc and F extends AnyFunc[].

And AsyncFn is defined as :

type AnyFunc = (...arg: any) => any;

first parameter

  arg: Parameters<FirstFn>[0],

Typescript comes with some utility type. Here we are using Parameters that extracts all parameters from the supplied function.
Here we only care about the first parameter of the first function passed as parameter, hence the [0] type array access.
This first arg is validating our first constraint :

The first function parameter type should match the first pipe parameter type

second parameter

This one is pretty straightforward. It's the first function passed to the pipe one.

rest of the parameters

  ...fns: PipeArgs<F> extends F ? F : PipeArgs<F>

So here is the real deal. What is PipeArgs ? why not just telling Typescript that fns is of type F?

This is where we are applying our second constraint :

The result type of intermediary functions should match the parameter type of the next function

PipeArgs definition :

type PipeArgs<F extends AnyFunc[], Acc extends AnyFunc[] = []> = F extends [
  (...args: infer A) => infer B
]
  ? [...Acc, (...args: A) => B]
  : F extends [(...args: infer A) => any, ...infer Tail]
  ? Tail extends [(arg: infer B) => any, ...any[]]
    ? PipeArgs<Tail, [...Acc, (...args: A) => B]>
    : Acc
  : Acc;

So PipeArgs is running through all function parameters and returning a new type with function definition where the return type of a function is the first parameter of the next function. It's a recursive type definition, and we are using Typescript Tail recursive optimization to be allowed to have around 1000 possible functions passed as pipe parameters.

For example, if we have this type definition (invalid pipe arguments since D is not of type B :

type Input<A,B,C,D> = [(a: A) => D, (b: B) => C]

then we have in output :

type Output<A,B,C,D> = PipeArgs<Input<A,B,C,D>>
// Output is [(a: A) => B, (b: B) => C]

The first function is now a valid pipe parameter, since we satisfy our second constraint.

Now all we have to do is check if this PipeArgs<F> is equal to F.

If so, we have a valid definition. Else it's invalid. If it's invalid we return the valid definition so Typescript will point exactly where is the error.

That's what this does :

  ...fns: PipeArgs<F> extends F ? F : PipeArgs<F>

result type constraint

): LastFnReturnType<F, ReturnType<FirstFn>> {

And we are getting back the last function return type
and if it's not defined we use ReturnType utility type to return the first function result type.

Here is the definition of LastFnReturnType by using Typescript leading spread operator to match the last function :

type LastFnReturnType<F extends Array<AnyFunc>, Else = never> = F extends [
  ...any[],
  (...arg: any) => infer R
] ? R : Else;

Conclusion

This is advanced Typescript for library authors. By adding this kind of type definition you can improve your type definitions for advanced functionnal code.

I'd like to emphasize that Typescript type system is Turing complete, so if you think some advanced typing is impossible to do with Typescript, you should think twice.

Because chances are that you can. It might not be easy and i hope Typescript will improve on this part.

For those that want to check the whole code, it's here on the playground

If you this article was helpfull to you, don't forget to add a thumbs up.

Top comments (7)

Paul Elliott • Nov 30 '23 • Edited

Tweeked your pipe to take "rest" argument as second parameter:

export function pipe<F extends AnyFunc[]>(
  arg: Parameters<F[0]>[0],
  ...fns: PipeArgs<F> extends F ? F : PipeArgs<F>
): LastFnReturnType<F, ReturnType<F[0]>> {
  return (fns.slice(1) as AnyFunc[]).reduce((acc, fn) => fn(acc), fns[0](arg));
}

To accept array of functions

const transforms = [addOne, toString];
const s = pipe(1, ...transforms)

Not sure what it breaks yet.

Michal Grňo • Mar 19 • Edited

~~If you replace fns[0](arg) with just `arg`, then it's solid! Otherwise, pipe(value) would pass the type checker but throw at runtime.~~

EDIT: My bad, it's solid! The code that would error at runtime also produces a compile time error.

It even fixes a bug in the OP where the first function's return type doesn't have to match the second function's input:

pipe(
    42,
    (x: number) => x.toString(),
    (x: number) => x + 2
);

This code correctly errors with your definition, but incorrectly passes in OP.

To fix the bug directly in OP, one would have to replace the

function pipe<FirstFn extends AnyFunc, F extends AnyFunc[]>

in the declaration with something like

function pipe<
    F extends AnyFunc[],
    FirstFn extends (v: any) => [] extends F ? any : Parameters<F[0]>[0],
>

Alexandre BODIN • Jun 6 '23

Nice article

One thing I'm unsure about is it doesn't look like you enforce that output of the firstFn should be the input of the second one with pipeArgs as you only pass in the rest of the functions types.

It might be infered thanks to the code but I'm unsure. Did I miss something?

Michal Grňo • Mar 19

You are correct, the OP does not correctly check the first function.

To fix it, you'd have to replace the first line with something like the following:

function pipe<
    F extends AnyFunc[],
    FirstFn extends (v: any) => [] extends F ? any : Parameters<F[0]>[0],
>

James Saget • Jul 19 '23 • Edited

Great article, I was implementing this in a recent project and tried to make a simple passThrough() function to log some analytics/metrics <T>(inputOutput: T) => T and this was previously working with pipe on fp-ts but wanted to utilise your functionality of using a pipe without a limit on arguments.

It looks like when combined with a generic input and output (like the above) that is not mapped to the type from the previous function

Example:

pipe(
      input,
      (input: string) =>
        TE.tryCatch(
          async () => Buffer.from(input),
          async (reason) => toError(reason),
        ),
      TE.map(<T>(inputOutput: T): T => inputOutput),
      <T>(inputOutput: T): T => inputOutput,
      TE.map((buffer: Buffer) => 'string'),
    );

Produces the following error: TS2345: Argument of type '(inputOutput: T) => T' is not assignable to parameter of type '(inputOutput: unknown) => TaskEither'. Type 'unknown' is not assignable to type 'TaskEither'.

I appreciate the vagueness of the question & maybe I have made some false assumptions as I am new to fp-ts but is there any advice you could offer?