DEV Community: Michael Arnaldi

A bright future for Effect

Michael Arnaldi — Sat, 18 Feb 2023 15:58:50 +0000

With this post, we are announcing that the fp-ts project is officially merging with the Effect-TS ecosystem, and that the author of the project, Giulio Canti (@gcanti), is being welcomed into the Effect organization!

Acknowledging the Past

Giulio Canti certainly requires no introduction. He pioneered (and continues to lead) exploration of how the TypeScript type system could be used to build powerful, composable, and type-safe libraries for the JavaScript ecosystem. His work has brought us several notable projects, such as fp-ts and io-ts, which have been instrumental in making functional programming in TypeScript more accessible and practical for developers.

Through his work, he has shown that types can be used not only for simple type-checking, but also for expressing complex relationships and building high-level abstractions. Giulio's passion for functional programming and his dedication to open-source software inspired a whole new generation of influential projects, such as zod, and has caused many developers to explore new ways of building applications. His impact on the TypeScript community is undeniable.

Where Are We Now?

As a functional programming community, we have learned a lot since the initial inception of fp-ts@1.0.0. It became clear that plain functional programming, as it is found in other languages, cannot be directly adapted to work with TypeScript. We realized that we should be working to embrace the strengths of TypeScript as a language while avoiding its weaknesses. This has been the focus of the Effect organization for the past few years.

When @gcanti decided to begin working on new versions of the core fp-ts libraries, we all saw it as an opportunity to increase collaboration between Effect and fp-ts. We wanted to center the focus of fp-ts by clarifying core design principles, unify the fp-ts and Effect ecosystems to provide a more seamless developer experience and make our libraries accessible to a wider audience.

As the target user group shifted from functional programming experts to mainstream TypeScript users, we had to adapt our libraries to be cohesive, useful, and easy to adopt. We want to ensure that our ecosystem of libraries is accessible to developers who may not have extensive experience with functional programming. It became increasingly necessary to integrate fp-ts and Effect and the lines between the two were becoming less and less distinct.

The Future

For this reason, we have made the decision to unite and collaborate on effect as a generational project that aims to serve a wide audience. The goal is to provide a comprehensive suite of solutions that cover everything from data modeling and serialization to effect execution, query optimization, cluster management, workflows, and much more. By combining our expertise and resources, we can create a more cohesive and powerful ecosystem of libraries that are accessible and useful to developers of all levels.

The Effect community is committed to building high-quality, type-safe, and composable libraries for building applications, and this merge will only serve to strengthen that mission. We can't wait to see what kind of innovations this merger will bring!

You can follow our all of our progress on GitHub.

We also have a thriving community on Discord - come join us!

Also, don't forget to sponsor Giulio's work on GitHub!

Immutability: what a monster...

Michael Arnaldi — Sun, 01 Jan 2023 14:31:54 +0000

So, you heard about immutability and all the sweet that comes from it, but what is it actually?

We'll be using TypeScript in this article, but the same principles apply to different languages.

In simple words, immutability means that a value can never be changed or, in a deeper sense, that the observable behaviour of a value can never be changed.

When you deal with immutable values, you can always safely "capture" them because you know that they will never change – doing the same with mutable values requires making a copy.

Let's have a look at some usage of immutable code:

let arr: ReadonlyArray<number> = []

while (arr.length < 10) {
  const newArr = [...arr, arr.length]
  setTimeout(() => {
    console.log(newArr.length)
  }, 0)
  arr = newArr
}

When executed, this code will produce on the console every number between 1 and 10 (included).

If we were to do the same with mutable code, we would get to a totally different result. Let's have a look:

const arr: Array<number> = []

while (arr.length < 10) {
  arr.push(arr.length)
  setTimeout(() => {
    console.log(arr.length)
  }, 0)
}

This code will instead print 10 times the number 10.

What we've done here is showcase what "capture" means, the call to setTimeout takes a reference from the local scope and carries it across to a different scope; setTimeout will always execute after all the sync operations in our program are done.

At that point, in the first example, we get effectively 10 different arrays, each with different lengths, while in the second example, we get a single array containing all the numbers from 0 to 9 (having a length of 10).

Copy On Write

The technique we used in the previous example is commonly known as "copy on write", meaning whenever we intend to "mutate" something, we create a new copy of it, which allows the initial structure (pre-mutation) to remain unchanged (aka frozen).

We can see that no changes happen to the original structure by using Object.freeze that will raise an error anytime something is mutated:

let arr: ReadonlyArray<number> = Object.freeze([])

while (arr.length < 10) {
  const newArr = Object.freeze([...arr, arr.length])
  setTimeout(() => {
    console.log(newArr.length)
  }, 0)
  arr = newArr
}

The same technique can be applied to objects and generally any structure that can be "cloned".

All good, isn't it? Well, all good, except made for performance, cloning becomes progressively expensive as things grow.

const copyOnWrite = (n: number) => {
  console.time(`copy-on-write-${n}`)
  let arr: ReadonlyArray<number> = []
  while (arr.length < n) {
    const newArr = [...arr, arr.length]
    arr = newArr
  }
  console.timeEnd(`copy-on-write-${n}`)
}

copyOnWrite(10)
copyOnWrite(100)
copyOnWrite(1000)
copyOnWrite(10000)
copyOnWrite(100000)

This program will produce:

copy-on-write-10: 0.074ms
copy-on-write-100: 0.093ms
copy-on-write-1000: 1.034ms
copy-on-write-10000: 82.26ms
copy-on-write-100000: 52.010s

Clearly, this becomes prohibitive very quickly, especially in problems of unbounded size (where you don't know the size of the structures you'll need to clone).

So, is immutability screwed for good? No... but copy on write is.

Persistent Data Structures

I won't go into all the details and techniques around Persistent Data Structures. If you're interested in the topic in more details, I can recommend following the amazing course by Erik Demaine entitled "Advanced Data Structures" consumable, for free, from the MIT opencourseware platform: https://ocw.mit.edu/courses/6-851-advanced-data-structures-spring-2012/.

For the sake of this article, we'll say that persistent data structures are data structures that remember their history and each new "copy" shares as much as possible with the original.

In the case of the example seen earlier, a persistent array structure should remember and share all the elements with the previous one instead of having to clone everything every time.

Let's do the same example using a persistent array. We will use the "Chunk" data structure from the new @fp-ts/data that is based on the following paper: http://aleksandar-prokopec.com/resources/docs/lcpc-conc-trees.pdf.

import * as Chunk from "@fp-ts/data/Chunk"

const copyOnWrite = (n: number) => {
  console.time(`persistent-${n}`)
  let arr: Chunk.Chunk<number> = Chunk.empty()
  while (arr.length < n) {
    const newArr = Chunk.append(Chunk.size(arr))(arr)
    arr = newArr
  }
  console.timeEnd(`persistent-${n}`)
}

copyOnWrite(10)
copyOnWrite(100)
copyOnWrite(1000)
copyOnWrite(10000)
copyOnWrite(100000)

As we can see the code looks almost the same as the previous one but its execution doesn't even compare:

persistent-10: 0.534ms
persistent-100: 0.728ms
persistent-1000: 1.493ms
persistent-10000: 14.425ms
persistent-100000: 10.705ms

It may seem surprising that 100000 is even faster than 10000, that's because V8(the JS engine behind Node.js) was able to JIT optimize the code.

The point is that the complexity doesn't grow with size and all the previous references can be captured safely as they are still valid.

Magic? feels like it sometimes.

Conclusion

If you like those kind of things don't forget to check out all the work that is undergoing in https://github.com/fp-ts and https://github.com/effect-TS. And, if you can, donate a few $$ to support the amazing work of the contributors.

Corollary

Why the Record & Tuple proposal https://github.com/tc39/proposal-record-tuple is bad?

It conflates immutability with copy on write & structural freeze, and its implementation doesn't allow for custom types to be mixed in, so when used it's an all-in.

Overall, if accepted, I feel it would slow the adoption of proper immutable data structures in a close to non-reversible manner, and it will be ignored by the functional community that immutability targets in the first place and where the concept of immutability comes from.

What would be better?

As you can see from this article, it is already possible to do everything we need to do, but some custom syntax is nice; namely the proposal would allow records and tuples to be compared using === and to be created using #{ a: 0 } / #[0, 1, 2], which is arguably a nice to have.

An example of it would be the operator overloading proposal https://github.com/tc39/proposal-operator-overloading that would allow overriding ===, and it would also solve in one shot many other problems (like numeric types extension, pipeable operator, and all the ones listed in the proposal).

In addition to that, a custom # operator could be added to create objects & arrays using a prototype that overloads === making it value based.

The Case for TS+

Michael Arnaldi — Sun, 10 Apr 2022 15:00:51 +0000

Note from 2023:

TS+ is an experimental fork of TS, don't use it in prod.

Hi!

So, what's this post gonna be about?

Well, with the Effect community, we've been hard at work for the last ~6 months on something new which I am now truly proud to introduce in detail for the first time.

Why TS+
- The Development of Fluent APIs
- The Development of Pipeable APIs
Introducting TS+
Installing TS+
Fluent and Static Extensions
Call Extensions
Binary Operators
Lazy Arguments
Type Unification
Global Imports
Runtime Types and Derivation
Module Structure
Compiled Output
What's Next

Why TS+?

Let's begin with a little bit of a back story. For the last four years (yeah, it's already been that long...), we have been working hard on the development of Effect.

Originally, Effect started as a small project inspired by the ideas brought forth by the developers of ZIO in Scala. However, it progressively evolved to a full blown port of all the core modules of ZIO, together with a very rich set of extension modules that bridges the gap between the data structures available in Scala and in TypeScript.

During all this work we have grown to love much of the TypeScript language and we were very surprised since the very first days how much we could do, and how safe and pleasant from a developer experience perspective the language was to use.

However, we also encountered a decent amount of limitations - not really limitations on what we could do in the language, but rather what we could do efficiently in the language, both in terms of verbosity and in terms of tree-shakeability of the produced code.

Let's start with an example:

The Development of Fluent APIs

A fluent API in TypeScript might look something like:

export class IO<A> {
  static succeed = <A>(a: A) => new IO(() => a);
  static succeedWith = <A>(a: () => A) => new IO(a);

  private constructor(readonly io: () => A) {}

  map<A, B>(this: IO<A>, f: (a: A) => B): IO<B> {
    return new IO(() => f(this.io()));
  }

  flatMap<A, B>(this: IO<A>, f: (a: A) => IO<B>): IO<B> {
    return new IO(() => f(this.io()).io());
  }

  zip<A, B>(this: IO<A>, b: IO<B>): IO<[A, B]> {
    return new IO(() => [this.io(), b.io()]);
  }

  run<A>(this: IO<A>) {
    return this.io();
  }
}

which can be used like:

export const program = IO.succeed(0)
  .map((n) => n + 1)
  .flatMap((r) =>
    IO.succeedWith(() => {
      console.log(`result: ${r}`);
    })
  );

program.run();

In this approach we have a data-type IO<A> together with a "companion object" IO. This approach lends itself to a certain organization of code:

Constructors like succeed and succeedWith are placed in the "companion object" as static members
Methods like map, flatMap, and zip are placed in the class instance as members of the type IO<A>

This approach is very common in object-oriented languages and its advantages lie in the ease of use. In fact, for the user of such a module, the editor experience is beautiful - if you want to construct an IO you type IO. and get a full list of suggestions on ways you can construct an IO. In addition, when you want to do something with a value of type IO<A>, you type value. and you are again prompted with a nice list of options.

This fluent type of API is "a dream", but also has a few drawbacks that have led the biggest part of the FP community away from it.

The limitations are staggering:

All methods have to be inside the class (or inside a common abstract class for ADTs)
Once a class is defined adding methods to it from the outside requires module augmentation and unsafe mutation of object prototypes
Worst of all, none of it can be optimized from a tree shaking perspective, so you'll end up with a huge bundle size that becomes prohibitive at a certain point

The current "solution" that the FP community has adopted is the use of pipeable APIs.

The Development of Pipeable APIs

In a pipeable API we would rewrite the prior example to:

// file: IO.ts
export class IO<A> {
  constructor(readonly io: () => A) {}
}

export const succeed = <A>(a: A) => new IO(() => a);

export const succeedWith = <A>(a: () => A) => new IO(a);

export function map<A, B>(f: (a: A) => B): (self: IO<A>) => IO<B> {
  return (self: IO<A>) => new IO(() => f(self.io()));
}

export function flatMap<A, B>(f: (a: A) => IO<B>): (self: IO<A>) => IO<B> {
  return (self) => new IO(() => f(self.io()).io());
}

export function zip<A, B>(b: IO<B>): (self: IO<A>) => IO<[A, B]> {
  return (self) => new IO(() => [self.io(), b.io()]);
}

export function run<A>(self: IO<A>) {
  return self.io();
}

// file: Function.ts

// omitted definition of pipe due to its length
// For the curious, take a look at https://github.com/Effect-TS/core/blob/master/packages/system/src/Function/pipe.ts

// file: Program.ts
import * as IO from "./IO";
import { pipe } from "./Function";

export const program = pipe(
  IO.succeed(0),
  IO.map((n) => n + 1),
  IO.flatMap((r) =>
    IO.succeedWith(() => {
      console.log(`result: ${r}`);
    })
  )
);

IO.run(program);

What we have essentially done is extracted all the constructors and methods outside of the class, and moved the this parameter to be a normal curried function parameter that the pipe function will carry through each function call.

The resulting API is still quite nice visually, but it does have a few drawbacks.

First of all, auto-imports do not work well with namespaced imports (recently folks at Unsplash have been working on language service plugins for this). Even if you do get auto-imports to work, you'll end up with tons of imports.

Additionally, we have lost the meaning and categorization of IO - namely we no longer have a data-type IO<A> and a "companion object" IO. We now only have a data-type IO<A> and a module of functions that include both constructors, such as IO.succeed, and pipeable functions (which are also called aspects), such as IO.flatMap. So when programming a user needs to know exactly which module to import, exactly which functions are constructors of the datatype and which are methods for the datatype, and exactly how to use them.

Finally, having had the experience of teaching those concepts to other developers, it seems fairly common that folks sometimes have issues reading pipeable function signatures.

Introducing TS+

TS+ is a new language developed as a super-set of TypeScript and maintained as a fork of the original TypeScript compiler that is rebased daily.

In order to guarantee full support with the TypeScript ecosystem, we limit what can be extended. TS+ emits standard declaration files consumable from plain TypeScript and consumes plain definition files. TS+ also does not add any new syntax.

TS+ diverges from TypeScript in that we do not limit ourselves from emitting code using type information. Instead, we leverage the type information as much as possible to produce both highly optimized code and improve the developer experience.

With those decisions comes a set of trade-offs. TS+ can be compiled only with the TS+ compiler (a fork of tsc) and cannot be compiled by tools like babel which "simply" remove the types.

We suggest using a compilation pipeline where tsc emits ES2022 modules, followed by another tool such as esbuild/swc/babel that takes over from there leveraging the powerful infrastructure of project references to optimize compilation.

Installing TS+

To install TS+, you must first add it as a dev dependency with:

yarn add -D @tsplus/installer
# or
npm i -D @tsplus/installer

Then, you can add a postinstall script to your package.json. For example:

{
  "scripts": {
    "postinstall": "tsplus-install"
  }
}

This will replace the TypeScript that is installed in your node_modules with the TS+ version.

Note: this install process is temporary until a better mechanism for installation is determined.

After installing, you will want to also ensure that your IDE uses the TypeScript language service from the workspace and not the default one.

If you want to start playing around with a pre-configured repository, you can open up https://github.com/ts-plus/starter-lib in GitPod.

Using TS+ (Fluent Methods)

Using our initial example, we will start by extracting all methods from the main IO class:

/**
 * @tsplus type article.IO
 */
export class IO<A> {
  static succeed = <A>(a: A) => new IO(() => a);
  static succeedWith = <A>(a: () => A) => new IO(a);

  constructor(readonly io: () => A) {}
}

/**
 * @tsplus fluent article.IO map
 */
export function map<A, B>(self: IO<A>, f: (a: A) => B): IO<B> {
  return new IO(() => f(self.io()));
}

/**
 * @tsplus fluent article.IO flatMap
 */
export function flatMap<A, B>(self: IO<A>, f: (a: A) => IO<B>): IO<B> {
  return new IO(() => f(self.io()).io());
}

/**
 * @tsplus fluent article.IO zip
 */
export function zip<A, B>(self: IO<A>, b: IO<B>): IO<[A, B]> {
  return new IO(() => [self.io(), b.io()]);
}

/**
 * @tsplus fluent article.IO run
 */
export function run<A>(self: IO<A>) {
  return self.io();
}

One thing that you may notice in the example above is that we have added a JSDoc annotation to our IO class. This annotation identifies the IO type in TS+ with a "type tag" of article.IO. The "type tag" for each type you define with TS+ should be globally unique. Prefixing type tags with your package's name is common, since non-prefixed tags should be reserved for the TS+ standard library.

Now that we have added our type tag to the IO class, we can begin by extracting each method from inside the class to a function, renaming this to something else (in this case we have chosen to use self).

We further added a JSDoc annotation above each function, which associates the function with a type in TS+. For example, to associate the flatMap function to the IO datatype, we annotate flatMap with @tsplus fluent article.IO flatMap. This essentially tells the compiler "put this fluent method in any type that matches the tag article.IO and name it flatMap".

Having done that we can already use IO exactly as before:

export const program = IO.succeed(0)
  .map((n) => n + 1)
  .flatMap((r) =>
    IO.succeedWith(() => {
      console.log(`result: ${r}`);
    })
  );

program.run();

But we can already note from the IDE the following:

We see from the IDE quick documentation that this is a "fluent" extension and not a classical "method".

That's about it for fluent extensions, they are declared as normal functions that you can even call directly like map(a, f). They can be declared anywhere you want and the path to the method will be resolved during compilation.

In order to support the folks who prefer a pipeable API, we also have a macro to derive a pipeable function from a data first one. This can be used as follows:

export function map<A, B>(self: IO<A>, f: (a: A) => B): IO<B> {
  return new IO(() => f(self.io()));
}

const mapPipeable = Pipeable(map)

Note: this is a compiler extension that will deal with the generics of the signature in a correct way, it doesn't use conditional types.

Additionally fluent methods can be defined directly using pipeable functions like:

/**
 * @tsplus pipeable article.IO flatMap
 */
export function flatMap<A, B>(f: (a: A) => IO<B>): (self: IO<A>) => IO<B> {
  return self => new IO(() => f(self.io()).io());
}

This only solves half of the problem though. We managed to extract methods from a datatype, but constructors are still static members of the "companion object" (non extensible and non tree-shakable).

Let's solve that next:

/**
 * @tsplus type article.IO
 * @tsplus companion article.IO/Ops
 */
export class IO<A> {
  constructor(readonly io: () => A) {}
}

/**
 * @tsplus static article.IO/Ops succeed
 */
export const succeed = <A>(a: A) => new IO(() => a);

/**
 * @tsplus static article.IO/Ops succeedWith
 */
export const succeedWith = <A>(a: () => A) => new IO(a);

Here we identified a "companion" object for the IO dataype by annotation the IO class with with @tsplus companion article.IO/Ops. Then, to extract the constructors from the class, we used @tsplus static article.IO/Ops succeed to say "put this value as a static member of any type tagged as article.IO/Ops"

Note: there is no difference between a type tag and one defined with companion, but a class has 2 types the instance type and the constructor type so we need a way of distinguishing which tag links to what.

An alternative pattern for those who prefer to avoid using a class is the following:

/**
 * @tsplus type article.IO
 */
export interface IO<A> {
  readonly io: () => A;
}

/**
 * @tsplus type article.IO/Ops
 */
export interface IOOps {
  <A>(io: () => A): IO<A>;
}

export const IO: IOOps = io => ({ io });

/**
 * @tsplus static article.IO/Ops succeed
 */
export const succeed = <A>(a: A) => IO(() => a);

/**
 * @tsplus static article.IO/Ops succeedWith
 */
export const succeedWith = <A>(a: () => A) => IO(a);

/**
 * @tsplus fluent article.IO map
 */
export function map<A, B>(self: IO<A>, f: (a: A) => B): IO<B> {
  return IO(() => f(self.io()));
}

/**
 * @tsplus pipeable article.IO flatMap
 */
export function flatMap<A, B>(f: (a: A) => IO<B>): (self: IO<A>) => IO<B> {
  return self => IO(() => f(self.io()).io());
}

/**
 * @tsplus fluent article.IO zip
 */
export function zip<A, B>(self: IO<A>, b: IO<B>): IO<[A, B]> {
  return IO(() => [self.io(), b.io()]);
}

/**
 * @tsplus fluent article.IO run
 */
export function run<A>(self: IO<A>) {
  return self.io();
}

//
// Usage
//

export const program = IO.succeed(0)
  .map((n) => n + 1)
  .flatMap((r) =>
    IO.succeedWith(() => {
      console.log(`result: ${r}`);
    })
  );

program.run();

Another thing we usually do is to put related types to IO like for example a type like ExtractResult in a namespace named IO like:

/**
 * @tsplus type article.IO
 */
export interface IO<A> {
  readonly io: () => A;
}

export declare namespace IO {
  export type ExtractResult<I extends IO<any>> = [I] extends [IO<infer A>] ? A : never;
}

/**
 * @tsplus type article.IO/Ops
 */
export interface IOOps {
  <A>(io: () => A): IO<A>;
}

export const IO: IOOps = io => ({ io });

Basically giving IO 3 meanings:

As an interface/type (i.e. for methods)
As a term/value (i.e. for constructors)
As a namespace (i.e. for related types)

All of this together provides an extensible way of developing easily to usable and discoverable APIs that are fully tree-shakable and optimized. In fact, the program that we have written here is a valid program that utilizes the IO module that we have defined.

Using TS+ (Call Extension)

There are cases where we would like to add a call signature to something that isn't a function, for example we could refactor the above constructor like:

/**
 * @tsplus type article.IO/Ops
 */
export interface IOOps {}
export const IO: IOOps = {};

/**
 * @tsplus static article.IO/Ops __call
 */
export function make<A>(io: () => A): IO<A> {
  return { io };
}

This allows us to construct values for a datatype using the __call expression that we define. For example, we can create an IO<string> using the __call expression for the IO datatype above:

const io: IO<string> = IO(() => "Hello, World!")

The name __call is a special name that basically says "use the function as the call signature for a type". In the example above, TS+ will resolve the __call expression for the IO datatype to the make function that we defined for IO.

In some extreme cases you may want access to the "this" and for that you can use a "fluent" variant of a __call expression instead of a "static" one.

/**
 * @tsplus fluent article.IO/Ops __call
 */
export function make<A>(self: IOOps, io: () => A): IO<A> {
  return { io };
}

You may think the set of features we have described thus far is enticing enough... ohh but we have only just started.

Using TS+ (Binary Operators)

There are many binary operators in JS - it's a shame there isn't a way of extending those (and the proposal to do so is inefficient in terms of tree shaking, limited, and may never gonna come to fruition)... Yeah, we can do that too :)

Looking at the IO type that we defined above, the zip combinator that we defined for IO looks quite a bit like a binary operator. Given that the "+" symbol doesn't make any sense when used between two IO types in plain JavaScript/TypeScript, we can override it in TS+:

/**
 * @tsplus fluent article.IO zip
 * @tsplus operator article.IO +
 */
export function zip<A, B>(self: IO<A>, b: IO<B>): IO<[A, B]> {
  return IO(() => [self.io(), b.io()]);
}

With just that additional JSDoc annotation we can now call zip on two IO types using the + operator:

const zipped = IO.succeed(0) + IO.succeed(1);

And looking at the quick info from VSCode:

Furthermore, you can define multiple operators and multiple fluent extensions with the same target type by specify an operator precedence. This enables neat DSL's, such as the ones we have created for @tsplus/stdlib/collections. (by the way, if you are here you should also probably install @tsplus/stdlib).

You can take a more detailed look at how these operators are used in the test suite for the TS+ standard library. For example: List.test.ts (and the other tests).

One last thing - we've also implemented go-to-definition for custom operators, so you can click an operator and use your IDE's go-to-definition to quickly navigate to the implementation of that operator.

Using TS+ (Lazy Arguments)

In JavaScript/TypeScript, lazy evaluation of a computation is generally implemented via a thunk (i.e. () => A). Deferring evaluation of computations can be quite beneficial in a variety of circumstances, particularly when we want to avoid eager computation of a value. This paradigm of "lazy" programming becomes even more powerful when combined with effect systems.

However, if you attempt to write a truly lazy program in JavaScript/TypeScript, you'll quickly find yourself writing a lot of annoying arrow functions like T.succeed(() => console.log("A")). This is because we want the effect system to assume control over whether or not the console.log is actually called.

To avoid this, TS+ allows us to define function parameters as "lazy":

/**
* @tsplus type LazyArgument
*/
interface LazyArg<A> {
  (): A
}

/**
 * @tsplus static Effect/Ops succeed
 */
function succeed<A>(f: LazyArg<A>) {
  f()
}

When calling Effect.succeed(x), if x is not already evaluated lazily (i.e. if x is not already a thunk), the compiler will transform it to Effect.succeed(() => x) making it possible to trim annoying boilerplate.

So something like:

Effect.succeed(() => console.log("Hello, world!"))

becomes

Effect.succeed(console.log("Hello, world!"))

Note that the behavior is only added to function arguments of a type that has a type tag of value LazyArgument and not generally to any function arguments.

Using TS+ (Type Unification)

Ever ended up with something like Left<0> | Left<1> | Right<string> in something that should have been Either<number, string>?

That's because TypeScript assumes that the right-most type is always the strictest (which is a good assumption, if you manually type it, it compiles).

We can be explicit about type unification in TS+ though:

/**
 * @tsplus type Option
 */
export type Option<A> = None | Some<A>;

/**
 * @tsplus unify Option
 * @tsplus unify Option/Some
 * @tsplus unify Option/None
 */
export function unifyOption<X extends Option<any>>(
  self: X
): Option<[X] extends [Option<infer A>] ? A : never> {
  return self;
}

/**
 * @tsplus type Eval
 */
export interface Eval<A> {
  readonly _A: () => A;
  readonly _TypeId: unique symbol;
}

/**
 * @tsplus unify Eval
 */
export function unifyEval<X extends Eval<any>>(self: X): Eval<[X] extends [Eval<infer AX>] ? AX : never> {
  return self;
}

and TS+ will use the result of the unify function to unify any time a union is generated.

So that for example:

Using TS+ (Global Imports)

As mentioned before, having a lot of imports can be quite painful. To avoid this, we have found that many users of Effect, for example, define their own "prelude" or "utils" files that re-export commonly used modules. Unfortunately, this often leads to edge cases in tree-shaking algorithms used by bundlers that have only recently begin to improve shaking of deeply nested dependency trees.

With TS+, we solve this problem using the concept of global imports.

A global import is an import defined in a declaration file (.d.ts) using the following syntax:

/**
 * @tsplus global
 */
import { Chunk } from "@tsplus/stdlib/collections/Chunk/definition";

When defining a type as "global", TS+ will make the type and its associated constructors/methods available globally in the project. However, during compilation TS+ will resolve usage of the constructors/methods associated with a datatype and add relevant imports to each file using the datatype. However, note that imports will only be added to a file during compilation if the global datatype is actually used in that file.

It is a common practice to define a prelude.d.ts in your root and add it to the "files" portion of the "tsconfig.json". For example:

// tsconfig.json
{
  "files": [
    "./prelude.d.ts"
  ]
}

Also you can share your preludes across your projects like we do with @tsplus/stdlib-global that if you import in your prelude file it will give you access to the full blown standard library everywhere in your project.

Using TS+ (Runtime Types and Derivation)

Thought it couldn't get any better? close.

One of the biggest pain points we've ever experienced in app development is the definition of types that are safe at runtime. Many solutions have been attempted so far, including but not limited to: io-ts, morphic-ts, zod, @effect-ts/schema, @effect-ts/morphic, etc.

All of the mentioned libraries are lovely because they are all attempting to solve a huge problem - encoding, decoding, guarding (and potentially generating arbitraries) for types.

They all use the same trick, instead of defining a type you define some sort of value from which the type is derived. The difference between the libraries lies in the details of how that sort of value is modeled.

This leads to a set of trade-offs where the libraries can end up emitting unoptimized types, end up being verbose, hard to extend, and sometimes quite painful to use.

To be clear they are still better than the alternative which would require manual implementation of everything.

After months of work we think we finally have a solution to (limited/structural) custom typeclass derivation so that you can forget the problem!

Let's dig into it:

export interface Person {
  name: string;
  age: Option<number>;
  friends: Chunk<Person>;
}

export const PersonEncoder: Encoder<Person> = Derive();
export const PersonDecoder: Decoder<Person> = Derive();
export const PersonGuard: Guard<Person> = Derive();

That's it, you can now do:

const encoded = PersonEncoder.encodeJSON({
  name: "Mike",
  age: Option.some(30),
  friends: Chunk()
});

const decoded = PersonDecoder.decodeJSON(encoded);

if (decoded.isRight()) {
  //
}

const maybePerson = {};

if (PersonGuard.is(maybePerson)) {
  maybePerson.age;
}

You may ask, how?? well the compiler can tell you, add a "explain" parameter to a call to Derive like:

And the best thing is that there is nothing special to the 3 instances above in fact all the rules are custom and extensible:
Guard.ts
Encoder.ts
Decoder.ts

Each derivable type is defined as an interface with a tag like:

/**
 * A Guard<A> is a type representing the ability to identify when a value is of type A at runtime
 *
 * @tsplus type Guard
 */
export interface Guard<A> {
  readonly is: (u: unknown) => u is A;
}

Then implicit instances are defined as:

/**
 * Guard for a number
 *
 * @tsplus implicit
 */
export const number: Guard<number> = Guard((u): u is number => typeof u === "number");

And rules as:

/**
 * @tsplus derive Guard lazy
 */
export function deriveLazy<A>(
  fn: (_: Guard<A>) => Guard<A>
): Guard<A> {
  let cached: Guard<A> | undefined;
  const guard: Guard<A> = Guard((u): u is A => {
    if (!cached) {
      cached = fn(guard);
    }
    return cached.is(u);
  });
  return guard;
}

/**
 * @tsplus derive Guard<_> 10
 */
export function deriveLiteral<A extends string | number>(
  ...[value]: Check<Check.IsLiteral<A> & Check.Not<Check.IsUnion<A>>> extends Check.True ? [value: A] : never
): Guard<A> {
  return Guard((u): u is A => u === value);
}

/**
 * @tsplus derive Guard[Option]<_> 10
 */
export function deriveOption<A extends Option<any>>(
  ...[element]: [A] extends [Option<infer _A>] ? [element: Guard<_A>]
    : never
): Guard<A> {
  return Guard((u): u is A => {
    if (u instanceof None) {
      return true;
    }
    if (u instanceof Some) {
      return element.is(u.value);
    }
    return false;
  });
}

The only special rule here is the one called lazy which is used when the compiler encounters a recursive derivation, the remaining are all custom.

A rule is a function with a rule tag which has the following format:

@tsplus derive Guard[Option]<_> 10

Where:

Guard is the tag of the typeclass we want to derive
[Option] lets you (if not omitted) further scope when the rule is applied (in this case to types like Show<Option<A>>)
<_> tells the compiler how to call the function arguments and has to be specified per each generic of the typeclass we are deriving (for example Refinement<_,_>), this can be
- _ calls it with the type, this case "Option"
- | match a union and calls with a tuple of the members
- & match an intersection and calls with a tuple of the members
- [] match a tuple and calls with a tuple of the members
10 is a priority that defines how early this rule is applied

Using TS+ (Module Structure and Config)

We've been going trough a list of example and by now you probably noticed we tend to use fully qualified imports like "@tsplus/stdlib/collections/Chunk/definition" even for local references.

You are not forced to do the same. However, for global imports and extensions in general your files must also be importable via a fully qualified name, the mapping between "file => fully qualified import" together with a map of "file => tracing name" is defined in a dedicated config like the following: tsplus.config.json.

The trace map is needed because in TS+ you can support compile time tracing of your function calls, for example if you have a function like:

function hello(message: string, __tsPlusTrace?: string) {
}

when a call expression like hello("world") is found if a trace is not passed explicitly the compiler fill it up with hello("world", "file_trace_as_per_in_map:line:col").

While this is not something that can be relied upon as always present (because clearly a non TS+ user will not have a compiler that fill up the trace), when it is present this can be a powerful tool to build easily debuggable systems. For example, the tracing system inside Effect allows for rendering full stack traces, even for programs with async operations.

Using TS+ (Compiled Output)

So how does it compile in practice? When an exported value has a tag like static/fluent/implicit/derivation etc when we emit definition files .d.ts per each function we add a further jsdoc annotation called "location" like:

/**
 * @tsplus fluent Decoder decode
 * @tsplus location "@tsplus/stdlib/runtime/Decoder"
 */
export declare function decode<A>(decoder: Decoder<A>, value: unknown): import("../data/Either").Either<string, A>;

This is how we know how to import things and from where, in a JS file when something like that is used an import is introduced accordingly.

This design ensures modules are always imported as close to the declaration as possible and it helps preventing circular references, it may lead to a big number of imports in the compiled js files but tree-shaking can inline very well.

What's next for TS+

We think we are pretty much feature ready for the first decent release, we have to write tons of tests and finish extracting out the standard library from the effect codebase before this is considered usable at a decent level externally (internally effect is already written in TS+) but we feel very confident and the TS folks have been amazing in offering valuable advise.

Encoding HKTs in TypeScript (Once Again)

Michael Arnaldi — Sun, 19 Dec 2021 14:06:03 +0000

It was about a year ago when I was writing enthusiastically about a new way of encoding Higher Kinded Types (HKTs) in TypeScript in such a way that we could support transformers and reduce the need for verbose declarations.

The article mentioned can be found at: https://www.matechs.com/blog/encoding-hkts-in-ts4-1

A lot has happened in Effect this year, many new contributors and many new ecosystem projects and I couldn't be more excited, looking forward into the future I see a lot of good happening in this space!

Behind all the good there is a lot of effort, and sometimes to reach something that looks simple you have to iterate into multiple stages tackling complexity bit by bit.

This is the story of how we collectively missed something simple, for many years.

If you want to read the history read the article mentioned above, I will start here from zero.

Why do we need HKTs?

We want to implement common functionality across modules, let's for example take the following functions:



declare const mapArray: 
  <A>(self: Array<A>, f: (a: A) => B) => Array<B>

declare const mapTree: 
  <A>(self: Tree<A>, f: (a: A) => B) => Tree<B>

declare const mapOption: 
  <A>(self: Option<A>, f: (a: A) => B) => Option<B>

We can notice how much of the above signatures is in common, in fact they have everything equal except for the type they target Array|Tree|Option.

We would like to define an interface to describe this behaviour, unfortunately it isn't that obvious how.

Let's dream for a second and write:



interface Mappable<F<~>> {
  readonly map: 
    <A, B>(self: F<A>, f: (a: A) => B) => F<B>
}

with that we could say:



declare const mapArray: Mappable<Array>["map"]
declare const mapTree: Mappable<Tree>["map"]
declare const mapOption: Mappable<Option>["map"]

in fact we could also define:



declare const ArrayMappable: Mappable<Array>
declare const TreeMappable: Mappable<Tree>
declare const OptionMappable: Mappable<Option>

and we could even implement generic functions like:



const stringify = 
  <F>(T: Mappable<F>) => 
  (self: F<number>): F<string> => 
  T.map(self, (n) => `number: ${n}`)

and use it like:



const stringifiedArray: Array<string> = 
  stringify(MappableArray)([0, 1, 2])

To introduce some terminology F<~> is called a Higher Kinded Type and interface Mappable<F<~>> is called a TypeClass.

So far we only seen data types with a single parameter like Option<~> or Array<~>, there are also data types with multiple parameters like Either<~, ~> or Effect<~, ~, ~> and we would also like to include those in the same setup, ideally we could do so by defining:



interface Mappable<F<~, ~, ~>> {
  readonly map: <R, E, A, B>(
    self: F<R, E, A>, 
    f: (a: A) => B
  ) => F<R, E, B>
}

but we now have the problem of defining who's who, for example is A in Either the first or the second parameter? we could have some conventions where E is always the second and R is always the first and A always the last but that isn't too flexible too.

In fact there is no general solution to this, a lot of ideas exists though, for example in Scala you would be able to define type lambdas (sort of mappers between type parameters).

Let's now stop dreaming and realise that F<~> isn't even valid TypeScript, hopefully we got the idea of what we would like to have.

Let me tell you about

Lately I've been hearing more and more that a good design is a design that minimize the needs to tell you about unrelated things, in the previous encodings this list would be huge but I am now happy to say that I only have to tell you about a single "trick" (confirmed as reliable expected behaviour).

The this type unification logic, given:



interface MyInterface {
  readonly x?: unknown
}

type X = (MyInterface & { readonly x: number })["x"]

you would rightfully expect the type X to be number given that unknown & number = number.

Inside an interface you can also reference the this type so you can also expect:



interface MyInterface {
  readonly x?: unknown
  readonly y: this["x"]
}

type Y = (MyInterface & { readonly x: number })["y"]

that can be surprising, one would think that y would be always unknown but instead the this parameter is special because it always represent the current type even in an extension chain X extends Y extends Z, something defined as this in Z will appear as X. That's fairly logical if you think about it for the usage in classes and interfaces for plain OOP inheritance.

Single Parameter Encoding

Let's define something along the lines of the above, namely:



interface HKT {
  // will reference the A type
  readonly _A?: unknown

  // will represent the computed type
  readonly type?: unknown
}

now we'll need a way to reference a generic type using something like Kind<F, A> as a meaning for F<A>, doing that is a little tricky:



type Kind<F extends HKT, A> = 
  F extends {
    readonly type: unknown
  } ?
  // F has a type specified, it is concrete (like F = ArrayHKT)
  (F & {
    readonly _A: A
  })["type"] :
  // F is generic, we need to mention all of the type parameters
  // to guarantee that they are never excluded from type checking
  {
    readonly _F: F
    readonly _A: () => A
  }

and then we can say:



interface ArrayHKT extends HKT {
  readonly type: Array<this["_A"]>
}

type X = Kind<ArrayHKT, number>

and expect that X is number[].

Now we can define our Mappable from before:



interface Mappable<F extends HKT> {
  readonly map: 
    <A, B>(self: Kind<F, A>, f: (a: A) => B) => Kind<F, B>
}

and we can define:



const MappableArray: Mappable<ArrayHKT>

we can check the signature of MappableArray["map"] and it will appear as <A, B>(self: A[], f: (a: A) => B) => B[].

With the above we can also write:



const stringify = 
  <F extends HKT>(T: Mappable<F>) => 
  (self: Kind<F, number>): Kind<F, string> => 
  T.map(self, (n) => `number: ${n}`)

and the job is done.

Multi Parameter Encoding

The only thing to really know here is that the base case of Kind has to mention all of the type parameters with their respective variance, so for example let's say we want to support up to 3 params one input and two outputs called R, E and A.



interface HKT {

  readonly _R?: unknown

  readonly _E?: unknown

  readonly _A?: unknown

readonly type?: unknown

}

type Kind<F extends HKT, R, E, A> = 

  F extends {

    readonly type: unknown

  } ?

  (F & {

    readonly R: R

    readonly _E: E

    readonly _A: A

  })["type"] :

  {

    readonly _F: F

    readonly _R: (: R) => void

    readonly _E: () => E

    readonly _A: () => A

  }

interface Mappable<F extends HKT> {

  readonly map: 

    <R, E, A, B>(

      self: Kind<F, R, E, A>,

      f: (a: A) => B

    ) => Kind<F, R, E, B>

}

interface ArrayHKT extends HKT {

  readonly type: Array<this["_A"]>

}

const stringify = 

  <F extends HKT>(T: Mappable<F>) => 

  <R, E>(self: Kind<F, R, E, number>) => 

  T.map(self, (n) => number: </span><span class="p">${</span><span class="nx">n</span><span class="p">}</span><span class="s2">)

declare const ArrayMappable: Mappable<ArrayHKT>

const res = stringify(ArrayMappable)([0, 1, 2])

Inference with TypeClasses

We fully encoded Higher Kinded Types and we were able to define a TypeClass like Mappable, to improve inference it is necessary to mention the F parameter inside it otherwise it will be lost, we can do so by adding an optional parameter like:



interface TypeClass<F extends HKT> {

  readonly _F?: F

}

interface Mappable<F extends HKT> extends TypeClass<F> {

  readonly map: 

    <R, E, A, B>(

      self: Kind<F, R, E, A>,

      f: (a: A) => B

    ) => Kind<F, R, E, B>

}

The End

You can find the full prototype of this encoding at https://github.com/mikearnaldi/hkt-new including the usage with transformers (such as ValidationT, ReaderT, etc).

At the moment of writing this post Effect still uses the old encoding and progress to move to this one is tracked in https://github.com/Effect-TS/core/issues/1005.

Building custom DSLs in TypeScript

Michael Arnaldi — Sun, 16 May 2021 19:48:42 +0000

Why do we even need custom domain-specific languages?

First of all, we should define what do we mean by custom domain-specific languages.

In everyday programming, we have to encode real-world problems in code. We are limited by the power of expressivity of the language we are using. The programming languages are fairly low when it comes to model the real world.

Basically, we translate from a human language to a more basic form of language that a computer can understand.

While translating, and for future reference, we would like to keep the code as close to the real world as possible (easier to get feedback, easier to maintain, to assess).

Domain-specific languages are what is required to fill this gap, your application can be thought of as a series of modules each translating into progressively lower-level concepts so you encapsulate implementation details from business features and keep the code in a maintainable state.

Each of those modules exposes an API that other modules consume, the API is itself a domain-specific language.

Domain-specific languages are challenging, and there are multiple ways to do things (and trade-offs attached).

Usually, we think of a DSL to be tied to a single interpreter but that is not always the case, for example, we can think of a DSL to describe domain objects and out of the description we would like to get things like a serializer, a deserializer, an arbitrary, a comparator and many other things.

Generally, we can think of a DSL as an algebra of operations that can be interpreted in one or many ways.

So how can we build DSLs efficiently? In this blog post, we are going to discuss Final Encodings and Initial Encodings and look at many ways of implementing those.

Final Encoding

Starting from the easy one, Final Encoding means defining the language in terms of its direct interpretation.

Final Encodings are usually efficient but they come at a price, defining the language in terms of its interpretation means not having the possibility of multiple interpretations, they are harder to optimize (sometimes impossible) and many times unsafe (primarily due to stack safety in the js space).

Let's take a base example, we would like to model synchronous failable computations, we could encode the problem using a type like the following:

import { pipe } from "@effect-ts/core/Function";

//
// DSL
//

export class Failure<E> {
  readonly _tag = "Failure";
  constructor(readonly error: E) {}
}

export class Success<A> {
  readonly _tag = "Success";
  constructor(readonly result: A) {}
}

export type Result<E, A> = Failure<E> | Success<A>;

export class IO<E, A> {
  readonly _tag = "IO";
  constructor(readonly thunk: () => Result<E, A>) {}
}

export function succeed<A>(a: A): IO<never, A> {
  return new IO(() => new Success(a));
}

export function fail<E>(e: E): IO<E, never> {
  return new IO(() => new Failure(e));
}

export function succeedWith<A>(f: () => A): IO<never, A> {
  return new IO(() => new Success(f()));
}

export function failWith<E>(e: () => E): IO<E, never> {
  return new IO(() => new Failure(e()));
}

export function tryCatch<E, A>(
  f: () => A,
  onError: (_: unknown) => E
): IO<E, A> {
  return new IO(() => {
    try {
      return new Success(f());
    } catch (e) {
      return new Failure(onError(e));
    }
  });
}

export function chain<A, E1, B>(that: (a: A) => IO<E1, B>) {
  return <E>(self: IO<E, A>): IO<E | E1, B> =>
    new IO<E | E1, B>(() => {
      const a = self.thunk();
      if (a._tag === "Success") {
        return that(a.result).thunk();
      }
      return a;
    });
}

export function suspend<E, A>(f: () => IO<E, A>): IO<E, A> {
  return new IO(() => f().thunk());
}

export function run<E, A>(io: IO<E, A>): Result<E, A> {
  return io.thunk();
}

//
// Usage
//

const program = pipe(
  succeed(0),
  chain((n) => succeed(n + 1)),
  chain((n) => succeed(n * 2)),
  chain((n) =>
    succeedWith(() => {
      console.log(`computed: ${n}`);
    })
  )
);

You can find the code at the following sandbox: Link

In this case, we decided to encode a failable operation with a thunk of code and our language allows for composition of operations by composition of the thunks of each operation.

Separately we can execute the program by calling up the thunk via the run method and the operations contained in the program description (including side effects) will be executed to return a result.

Let's look at the trade-offs, in this case, perf can't get any better and we don't really need multiple interpretations of a program so everything looks good!

Except if we try large recursive procedures like:

import { pipe } from "@effect-ts/core/Function";
import * as IO from "./io";

function factorial(n: number): IO.IO<never, number> {
  if (n === 0) {
    return IO.succeed(1);
  }
  return pipe(
    IO.suspend(() => factorial(n - 1)),
    IO.chain((x) => IO.succeed(x * n))
  );
}

And try to compute factorial(100_000) you will get:

RangeError
Maximum call stack size exceeded

Now, that's a dumb way to compute a factorial but in real life, you might encounter frequently problems that will blow up the stack.

Initial Encoding

The idea behind initial encoding is to encode the DSL itself so in the previous example we would like to encode IO in terms of its operations (succeed, fail, suspend, chain, etc).

That is not the easiest thing, before being able to do so we need to take a few steps.

The first concepts we will see are plain and simple ADTs (Algebraic Data Types).

For general reference Algebraic Data Type is a broader term that can refer to types constructed as results of operations on types, those can be A | B, A & B, A ^ B, A / B, A x B, and many more.

For the purpose of this blog and in general, when you hear ADTs we mean unions of types, so things of the form A | B | C | D.

Let's try to encode the following problem: We would like to express basic algebraic operations (add, mul, div, etc) for numeric values.

We can do something like:

import { pipe } from "@effect-ts/core/Function";

//
// DSL
//

export type Expr = Val | Add | Mul | Sub | Div;

export class Val {
  readonly _tag = "Val";
  constructor(readonly value: number) {}
}

export function val(value: number): Expr {
  return new Val(value);
}

export class Add {
  readonly _tag = "Add";
  constructor(readonly left: Expr, readonly right: Expr) {}
}

export function add(right: Expr) {
  return (left: Expr): Expr => new Add(left, right);
}

export class Mul {
  readonly _tag = "Mul";
  constructor(readonly left: Expr, readonly right: Expr) {}
}

export function mul(right: Expr) {
  return (left: Expr): Expr => new Mul(left, right);
}

export class Sub {
  readonly _tag = "Sub";
  constructor(readonly left: Expr, readonly right: Expr) {}
}

export function sub(right: Expr) {
  return (left: Expr): Expr => new Sub(left, right);
}

export class Div {
  readonly _tag = "Div";
  constructor(readonly left: Expr, readonly right: Expr) {}
}

export function div(right: Expr) {
  return (left: Expr): Expr => new Div(left, right);
}

//
// Usage
//

export const operation = pipe(
  val(0),
  add(val(1)),
  mul(pipe(val(1), div(val(3)))),
  sub(val(2))
);

Sandbox at: Link

We've represented an Expr in a "free" way, meaning an Expr is the set of operations that is composed of.

We should note that we were able to write a program operation using multiple primitives without ever defining the meaning of the primitives themselves.

Lets now define an interpreter that computes the operation:

import * as Expr from "./expr";

export function compute(expr: Expr.Expr): number {
  switch (expr._tag) {
    case "Add": {
      return compute(expr.left) + compute(expr.right);
    }
    case "Mul": {
      return compute(expr.left) * compute(expr.right);
    }
    case "Div": {
      return compute(expr.left) / compute(expr.right);
    }
    case "Sub": {
      return compute(expr.left) - compute(expr.right);
    }
    case "Val": {
      return expr.value;
    }
  }
}

And running compute(operation) will result in the computed value.

We can see here that the compute function (the interpreter) has a recursive nature, which means large expressions will fail and blow up the stack.

But that recursion is contained in a single procedure that we could "easily" rewrite in a non-recursive manner making our language safe.

For the purpose of completeness, it is worth taking a look at the steps necessary to transform a general recursive program into an iterative one, the idea is that we will have a series of loops that progressively traverse the operations while preserving a stack of the remaining things to do.

What before was a nice and simple procedure now becomes an awful (but safe) piece of code:

import * as Expr from "./expr";

export function computeSafe(expr: Expr.Expr): number {
  let result: number | undefined;

  type Frame =
    | { _tag: "Add"; right: Expr.Expr }
    | { _tag: "AddValue"; value: number }
    | { _tag: "Mul"; right: Expr.Expr }
    | { _tag: "MulValue"; value: number }
    | { _tag: "Sub"; right: Expr.Expr }
    | { _tag: "SubValue"; value: number }
    | { _tag: "Div"; right: Expr.Expr }
    | { _tag: "DivValue"; value: number };

  const stack: Frame[] = [];

  recursing: while (1) {
    pushing: while (1) {
      switch (expr._tag) {
        case "Val": {
          result = expr.value;
          break pushing;
        }
        case "Add": {
          stack.push({ _tag: "Add", right: expr.right });
          expr = expr.left;
          continue pushing;
        }
        case "Mul": {
          stack.push({ _tag: "Mul", right: expr.right });
          expr = expr.left;
          continue pushing;
        }
        case "Sub": {
          stack.push({ _tag: "Sub", right: expr.right });
          expr = expr.left;
          continue pushing;
        }
        case "Div": {
          stack.push({ _tag: "Div", right: expr.right });
          expr = expr.left;
          continue pushing;
        }
      }
    }
    popping: while (1) {
      if (stack.length === 0) {
        return result!;
      }
      const frame = stack.pop()!;
      switch (frame._tag) {
        case "Add": {
          expr = frame.right;
          stack.push({ _tag: "AddValue", value: result! });
          result = undefined;
          continue recursing;
        }
        case "AddValue": {
          result = frame.value + result!;
          continue popping;
        }
        case "Mul": {
          expr = frame.right;
          stack.push({ _tag: "MulValue", value: result! });
          result = undefined;
          continue recursing;
        }
        case "MulValue": {
          result = frame.value * result!;
          continue popping;
        }
        case "Div": {
          expr = frame.right;
          stack.push({ _tag: "DivValue", value: result! });
          result = undefined;
          continue recursing;
        }
        case "DivValue": {
          result = frame.value / result!;
          continue popping;
        }
        case "Sub": {
          expr = frame.right;
          stack.push({ _tag: "SubValue", value: result! });
          result = undefined;
          continue recursing;
        }
        case "SubValue": {
          result = frame.value - result!;
          continue popping;
        }
      }
    }
  }
  // eslint-disable-next-line
  throw new Error("non reachable");
}

As we can see we are now in full control of how the interpreter b behaves and we can make it safe, with the same idea in mind we will be able to make the previous module (IO) safe to use.

Exercise to the reader: implement print(expression): string that renders the expression like (1 + (3 * (4 / 1)))

Phantom Types & GADTs

We are now able to encode a generic DSL but we haven't introduced generics into the equation, Expr from before was only representing a numeric expression, what if we want an expression that can be numeric or string-based?

Also what if some operations are only relevant to numeric expressions and literal expressions?

We would like to have some form of: type Expr<A> where A can be number | string depending on the type of the expression.

In languages like Scala, we are able to define closed "interfaces" (sealed traits, sealed abstract classes) meaning interfaces implementable only by a specific number of concrete classes (like only in the same file).

In TS we have union types, so we could say:

type Expr<A> = NumericValue | Add | Mul | ... | StringValue | Concat | Stringify

The problem is A is unused here (phantom), and also there is no information that given a value NumericValue that actually corresponds to Expr<number>.

To simulate this behavior we will add the A parameter to every primitive so NumericValue will become NumericValue<A> and in the definition of NumericValue we will not only require a value of type number but we will also require a function _A: (n: number) => A that converts a number into the generic A.

the function _A will be filled by the constructor function numericValue(value: number): Expr<number> as an identity function.

That ensures A is fixed to number in the context of a NumericValue and will be used in the interpreter to return a generic A when dealing with concrete values of type number.

Let's see the code:

import { identity, pipe } from "@effect-ts/core/Function";

export type Expr<A> =
  | NumberValue<A>
  | StringValue<A>
  | Add<A>
  | Mul<A>
  | Sub<A>
  | Div<A>
  | Stringify<A>
  | Concat<A>;

export class NumberValue<A> {
  readonly _tag = "NumberValue";
  constructor(readonly value: number, readonly _A: (_: number) => A) {}
}

export function numeric(value: number): Expr<number> {
  return new NumberValue(value, identity);
}

export class StringValue<A> {
  readonly _tag = "StringValue";
  constructor(readonly value: string, readonly _A: (_: string) => A) {}
}

export function string(value: string): Expr<string> {
  return new StringValue(value, identity);
}

export class Add<A> {
  readonly _tag = "Add";
  constructor(
    readonly left: Expr<number>,
    readonly right: Expr<number>,
    readonly _A: (_: number) => A
  ) {}
}

export function add(right: Expr<number>) {
  return (left: Expr<number>): Expr<number> => new Add(left, right, identity);
}

export class Mul<A> {
  readonly _tag = "Mul";
  constructor(
    readonly left: Expr<number>,
    readonly right: Expr<number>,
    readonly _A: (_: number) => A
  ) {}
}

export function mul(right: Expr<number>) {
  return (left: Expr<number>): Expr<number> => new Mul(left, right, identity);
}

export class Sub<A> {
  readonly _tag = "Sub";
  constructor(
    readonly left: Expr<number>,
    readonly right: Expr<number>,
    readonly _A: (_: number) => A
  ) {}
}

export function sub(right: Expr<number>) {
  return (left: Expr<number>): Expr<number> => new Sub(left, right, identity);
}

export class Div<A> {
  readonly _tag = "Div";
  constructor(
    readonly left: Expr<number>,
    readonly right: Expr<number>,
    readonly _A: (_: number) => A
  ) {}
}

export function div(right: Expr<number>) {
  return (left: Expr<number>): Expr<number> => new Div(left, right, identity);
}

export class Stringify<A> {
  readonly _tag = "Div";
  constructor(readonly child: Expr<number>, readonly _A: (_: string) => A) {}
}

export function stringify(child: Expr<number>): Expr<string> {
  return new Stringify(child, identity);
}

export class Concat<A> {
  readonly _tag = "Concat";
  constructor(
    readonly left: Expr<string>,
    readonly right: Expr<string>,
    readonly _A: (_: string) => A
  ) {}
}

export function concat(right: Expr<string>) {
  return (left: Expr<string>): Expr<string> =>
    new Concat(left, right, identity);
}

//
// Usage
//

// Expr<number>
export const operation = pipe(
  numeric(0),
  add(numeric(1)),
  mul(pipe(numeric(1), div(numeric(3)))),
  sub(numeric(2))
);

// Expr<string>
export const operationStr = pipe(
  string("operation:"),
  concat(stringify(operation))
);

Let's write for simplicity a recursive interpreter, we know from before how to eventually make it stack-safe.

import * as Expr from "./gadt-expr";

export function compute<A>(expr: Expr.Expr<A>): A {
  switch (expr._tag) {
    case "Add": {
      return expr._A(compute(expr.left) + compute(expr.right));
    }
    case "Mul": {
      return expr._A(compute(expr.left) * compute(expr.right));
    }
    case "Div": {
      return expr._A(compute(expr.left) / compute(expr.right));
    }
    case "Sub": {
      return expr._A(compute(expr.left) - compute(expr.right));
    }
    case "NumberValue": {
      return expr._A(expr.value);
    }
    case "StringValue": {
      return expr._A(expr.value);
    }
    case "Concat": {
      return expr._A(compute(expr.left) + compute(expr.right));
    }
    case "Stringify": {
      return expr._A(String(compute(expr.child)));
    }
  }
}

We can see that now compute is fully generic on A and how the identity function _A keeps track of the relationship concrete <> phantom.

Using the above compute(operationStr) will yield a string result, have a look at Link.

Existential Types

There is only one step left to do before being able to fully encode IO<E, A> using an initial representation, using the trick above we could implement almost all the primitives Success, Failure, etc except for Chain.

Chain should contain an operation of type IO<E, A> and a function f: (a: A) => IO<E1, B> and it should represent overall an IO<E | E1, B>.

If we think IO as type IO<E, A> = Success<E, A> | Failure<E, A> | ... | Chain we should have a Chain<E, A>.

There are a few types missing from the equation, those types are known as "Existential" meaning types that only exist within a bounded context in this case the Chain operation.

Existential types are very common when it comes to model continuations (like chain, operations that depend on the result of something else, and a set of functions that specify how to "continue", whatever "continue" means in context).

In order to model these additional parameters, we are going to model the behavior of using Chain instead of modeling Chain itself, meaning we will provide a use function that has to be used to access the content of Chain.

Let's get to code:

import { identity, pipe } from "@effect-ts/core/Function";

export type IO<E, A> =
  | Success<E, A>
  | Failure<E, A>
  | Suspend<E, A>
  | Chain<E, A>;

export class Success<E, A> {
  readonly _tag = "Success";
  constructor(readonly value: A, readonly _E: (_: never) => E) {}
}

export function succeed<A>(value: A): IO<never, A> {
  return new Success(value, identity);
}

export class Failure<E, A> {
  readonly _tag = "Failure";
  constructor(readonly error: E, readonly _A: (_: never) => A) {}
}

export function fail<E>(error: E): IO<E, never> {
  return new Failure(error, identity);
}

export class Suspend<E, A> {
  readonly _tag = "Suspend";
  constructor(readonly io: () => IO<E, A>) {}
}

export function suspend<E, A>(io: () => IO<E, A>): IO<E, A> {
  return new Suspend(io);
}

export class Chain<E, A> {
  readonly _tag = "Chain";
  constructor(
    readonly use: <X>(
      body: <E0, A0, E1, A1>(_: {
        readonly _E: (_: E0 | E1) => E;
        readonly _A: (_: A1) => A;

        readonly io: IO<E0, A0>;
        readonly f: (a: A0) => IO<E1, A1>;
      }) => X
    ) => X
  ) {}
}

export function chain<A0, E1, A1>(f: (a: A0) => IO<E1, A1>) {
  return <E0>(io: IO<E0, A0>): IO<E0 | E1, A1> =>
    new Chain((body) =>
      body({
        io,
        f,
        _E: identity,
        _A: identity
      })
    );
}

export function succeedWith<A>(f: () => A): IO<never, A> {
  return suspend(() => succeed(f()));
}

export function failWith<E>(e: () => E): IO<E, never> {
  return suspend(() => fail(e()));
}

export function tryCatch<E, A>(
  f: () => A,
  onError: (_: unknown) => E
): IO<E, A> {
  return suspend(() => {
    try {
      return succeed(f());
    } catch (e) {
      return fail(onError(e));
    }
  });
}

//
// Usage
//

export const program = pipe(
  succeed(0),
  chain((n) => succeed(n + 1)),
  chain((n) => succeed(n * 2)),
  chain((n) =>
    succeedWith(() => {
      console.log(`computed: ${n}`);
    })
  )
);

As we can see instead of having Chain contain io and f instead it contains a function use that takes a body thunk providing parameters to the body.

To interact with Chain we will use the use function by passing in use(({io, ...}) => do stuff).

Interpreter time, starting with the recursive one:

import * as E from "@effect-ts/core/Either";
import * as IO from "./existential-io";

export function run<E, A>(io: IO.IO<E, A>): E.Either<E, A> {
  switch (io._tag) {
    case "Success": {
      return E.right(io.value);
    }
    case "Failure": {
      return E.left(io.error);
    }
    case "Suspend": {
      return run(io.io());
    }
    case "Chain": {
      return io.use(({ io, f, _E, _A }) => {
        const res = run(io);
        if (res._tag === "Right") {
          const resB = run(f(res.right));
          if (resB._tag === "Left") {
            return E.left(_E(resB.left));
          } else {
            return E.right(_A(resB.right));
          }
        } else {
          return E.left(_E(res.left));
        }
      });
    }
  }
}

As we can see to deal with Chain we have to use a function (that is BTW not free perf-wise).

This simple recursive interpreter will have the same issue as the Final Encoding in running large recursive operations.

Also, the types are a bit verbose, on the plus side, everything is truly type-safe.

I personally think in most cases this encoding should be the first to be tried when building the first scratch of your modules and the perf compromise (calling _E, _A, use, etc) is negligible compared to the actual cost of the operations we model, leveraging the type-safety and the compiler help is definitely important in driving the correct implementation of the primitives and the interpreter.

When it comes to the conversion of the above to a non-recursive procedure we will, unfortunately, lose this nice safety.

Let's see how it would look like:

import * as E from "@effect-ts/core/Either";
import { pipe } from "@effect-ts/core/Function";
import * as IO from "./existential-io";

export function runSafe<E, A>(io: IO.IO<E, A>): E.Either<E, A> {
  let result = undefined;
  let isError = false;
  let current: IO.IO<any, any> = io;
  type Frame = { _tag: "ChainCont"; f: (_: any) => IO.IO<any, any> };
  const stack: Frame[] = [];

  recursing: while (1) {
    pushing: while (1) {
      switch (current._tag) {
        case "Success": {
          isError = false;
          result = current.value;
          break pushing;
        }
        case "Failure": {
          isError = true;
          result = current.error;
          break pushing;
        }
        case "Suspend": {
          current = current.io();
          continue pushing;
        }
        case "Chain": {
          current.use(({ io, f }) => {
            stack.push({ _tag: "ChainCont", f });
            current = io;
          });
          continue pushing;
        }
      }
    }
    popping: while (1) {
      if (stack.length === 0) {
        break recursing;
      }
      const frame = stack.pop()!;
      switch (frame._tag) {
        case "ChainCont": {
          if (!isError) {
            current = frame.f(result);
            continue recursing;
          } else {
            continue popping;
          }
        }
      }
    }
  }

  return isError ? E.left(result) : E.right(result);
}

//
// Recursive factorial
//

function factorial(n: bigint): IO.IO<never, bigint> {
  if (n === BigInt(0)) {
    return IO.succeed(BigInt(1));
  }
  return pipe(
    IO.suspend(() => factorial(n - BigInt(1))),
    IO.chain((x) => IO.succeed(x * n))
  );
}

Sandbox at Link

Running the above in sandbox will fail because browsers have limited memory but if you run it in node it will output an insanely large number.

Optimized Variant

Given at the end if we make stack safe interpreters we are bailing out of type safety at the interpreter level we could remove the need for calling _E, _A, use and relax the type safety of primitives using fake functions to instruct the type-system on the correct variance of parameters.

We will end up with something like:

/* eslint-disable @typescript-eslint/no-non-null-assertion */
/* eslint-disable no-constant-condition */
import * as E from "@effect-ts/core/Either"
import { pipe } from "@effect-ts/core/Function"

export type IO<E, A> = Success<E, A> | Failure<E, A> | Suspend<E, A> | Chain<E, A>

export class Success<E, A> {
  readonly _tag = "Success"
  readonly _E!: () => E
  readonly _A!: () => A
  constructor(readonly value: any) {}
}

export function succeed<A>(value: A): IO<never, A> {
  return new Success(value)
}

export class Failure<E, A> {
  readonly _tag = "Failure"
  readonly _E!: () => E
  readonly _A!: () => A
  constructor(readonly error: any) {}
}

export function fail<E>(error: E): IO<E, never> {
  return new Failure(error)
}

export class Suspend<E, A> {
  readonly _tag = "Suspend"
  readonly _E!: () => E
  readonly _A!: () => A
  constructor(readonly io: () => IO<any, any>) {}
}

export function suspend<E, A>(io: () => IO<E, A>): IO<E, A> {
  return new Suspend(io)
}

export class Chain<E, A> {
  readonly _E!: () => E
  readonly _A!: () => A
  readonly _tag = "Chain"
  constructor(readonly io: IO<any, any>, readonly f: (a: any) => IO<any, any>) {}
}

export function chain<A0, E1, A1>(f: (a: A0) => IO<E1, A1>) {
  return <E0>(io: IO<E0, A0>): IO<E0 | E1, A1> => new Chain(io, f)
}

export function succeedWith<A>(f: () => A): IO<never, A> {
  return suspend(() => succeed(f()))
}

export function failWith<E>(e: () => E): IO<E, never> {
  return suspend(() => fail(e()))
}

export function tryCatch<E, A>(f: () => A, onError: (_: unknown) => E): IO<E, A> {
  return suspend(() => {
    try {
      return succeed(f())
    } catch (e) {
      return fail(onError(e))
    }
  })
}

//
// Usage
//

export const program = pipe(
  succeed(0),
  chain((n) => succeed(n + 1)),
  chain((n) => succeed(n * 2)),
  chain((n) =>
    succeedWith(() => {
      console.log(`computed: ${n}`)
    })
  )
)

//
// Safe
//

export function runSafe<E, A>(io: IO<E, A>): E.Either<E, A> {
  let result = undefined
  let isError = false
  let current: IO<any, any> = io
  type Frame = { _tag: "ChainCont"; f: (_: any) => IO<any, any> }
  const stack: Frame[] = []

  recursing: while (1) {
    pushing: while (1) {
      switch (current._tag) {
        case "Success": {
          isError = false
          result = current.value
          break pushing
        }
        case "Failure": {
          isError = true
          result = current.error
          break pushing
        }
        case "Suspend": {
          current = current.io()
          continue pushing
        }
        case "Chain": {
          stack.push({ _tag: "ChainCont", f: current.f })
          current = current.io
          continue pushing
        }
      }
    }
    popping: while (1) {
      if (stack.length === 0) {
        break recursing
      }
      const frame = stack.pop()!
      switch (frame._tag) {
        case "ChainCont": {
          if (!isError) {
            current = frame.f(result)
            continue recursing
          } else {
            continue popping
          }
        }
      }
    }
  }

  return isError ? E.left(result) : E.right(result)
}

//
// Recursive factorial
//

export function factorial(n: bigint): IO<never, bigint> {
  if (n === BigInt(0)) {
    return succeed(BigInt(1))
  }
  return pipe(
    suspend(() => factorial(n - BigInt(1))),
    chain((x) => succeed(x * n))
  )
}

Note how we erased types from the content of primitives and how now Chain no longer needs use.

Note also how we use function types to control the variance of the IO type, in every primitive we inhabit the following:

  readonly _E!: () => E
  readonly _A!: () => A

In this case, both E and A represent outputs so they need to be covariant if we had an input, contravariant, parameter like a context R the computation needs we would inhabit that using:

  readonly _R!: (_: R) => void

End

The code in the article is available in github, try it out using gitpod

Abusing TypeScript Generators

Michael Arnaldi — Mon, 02 Nov 2020 18:31:25 +0000

In the previous chapters of the series we explored the @effect-ts/core ecosystem and some of its data-types like Effect, Managed, Layer and more.

We have explored, at length, the advantages of using statically typed functional programming in typescript to gain modularity, testability and safety of our code.

One thing is missing though, the API we have been using feels a bit awkward for people that come from non-functional languages, for the ones that come from functional languages there is a big missing, namely in scala for, in haskell do.

For the newcomers, let's first explore the reason behind this apis.

We have seen how Monads can represent the concept of sequential operations and we have been using chain to express this all over the place.

This style of programming highlight the function composition aspect, we would like to have a way to also express sequential steps in an imperative way.

For the people coming from javascript or typescript we would like to have something similar to what async/await does but generalised to every Monad.

Early approaches

Historically there have been a significant amount of attempts at approximating Do in the context of functional programming in typescript.

Namely we had 2 approaches:

the first one by Paul Grey detailed explained in his article at: https://paulgray.net/do-syntax-in-typescript/ using a fluent based api
the second one by Giulio Canti using a pipeable based api (that we also have as an alternative)

Let's look at the second one:

import * as T from "@effect-ts/core/Effect"
import { pipe } from "@effect-ts/core/Function"

const program = pipe(
  T.do,
  T.bind("a", () => T.succeed(1)),
  T.bind("b", () => T.succeed(2)),
  T.bind("c", ({ a, b }) => T.succeed(a + b)),
  T.map(({ c }) => c)
)

pipe(
  program,
  T.chain((c) =>
    T.effectTotal(() => {
      console.log(c)
    })
  ),
  T.runMain
)

Basically what happens here is T.do initializes an empty state {} to hold the scope of the computation and bind uses it to progressively fill up the variables step by step.

This way each step has access to the whole set of variables defined up to that point.

Both the fluent & the pipeable APIs are really nice to use but still doesn't feel native typescript and there is a good degree of repetition in accessing the scope explicitly at every bind operation.

Enter the world of generators

Before anything in order to use this API you will need to enable "downlevelIteration": true in your tsconfig.json.

Let's start to explore what are generators:

function* countTo(n: number) {
  let i = 0
  while (i < n) {
    yield i++
  }
}

const iterator = countTo(10)

let current = iterator.next()

while (!current.done) {
  console.log(current)

  current = iterator.next()
}

This will print out:

{ value: 0, done: false }
{ value: 1, done: false }
{ value: 2, done: false }
{ value: 3, done: false }
{ value: 4, done: false }
{ value: 5, done: false }
{ value: 6, done: false }
{ value: 7, done: false }
{ value: 8, done: false }
{ value: 9, done: false }

So basically every yield give us a value and execution is controlled by the caller by calling consuming the generator.

Let's now proceed with the second thing to know:

function* countTo(n: number) {
  let i = 0
  while (i < n) {
    const a = yield i++

    console.log(a)
  }
}

const iterator = countTo(10)

let input = "a"
let current = iterator.next(input)

while (!current.done) {
  console.log(current)

  input += "a"
  current = iterator.next(input)
}

This will print out:

{ value: 0, done: false }
aa
{ value: 1, done: false }
aaa
{ value: 2, done: false }
aaaa
{ value: 3, done: false }
aaaaa
{ value: 4, done: false }
aaaaaa
{ value: 5, done: false }
aaaaaaa
{ value: 6, done: false }
aaaaaaaa
{ value: 7, done: false }
aaaaaaaaa
{ value: 8, done: false }
aaaaaaaaaa
{ value: 9, done: false }
aaaaaaaaaaa

So basically with a generator we can push the return values of yields from the consumer to populate the variable scope.

The types are just horrible, const a in the generator body is any and there is really no inference working.

Another feature that we are going to exploit is yield* to yield a generator into a generator, surprisingly this works well type wise
:)

Let's see:

function* constant<A>(a: A): Generator<A, A, A> {
  return yield a
}

function* countTo(n: number) {
  let i = 0
  while (i < n) {
    const a = yield* constant(i++)

    console.log(a)
  }
}

const iterator = countTo(10)

let current = iterator.next()

while (!current.done) {
  current = iterator.next(current.value)
}

This will print out:

and all the types work well, in the signature of constant we read Generator<A, A, A>, the first A is the type of the yield, the second is the type of the yielded value and the third is the type required by the next step.

The idea here is to use a generator to declare the body of a computation in a monadic context and then, at runtime, starve the generator building up a set of chained computations (initial attempts by https://github.com/nythrox and some others, forgive me for not remembering all).

Generator based `do` for `Effect`

Let's put together what we seen so far, we would like to avoid modifying the types of Effect and in general any type so we won't directly add a generator inside them. That would also break variance of the type.

We are going to instead use a function to lift an effect into a generator of effect.

import * as T from "@effect-ts/core/Effect"
import { pipe } from "@effect-ts/core/Function"
import type { _E, _R } from "@effect-ts/core/Utils"

export class GenEffect<K, A> {
  constructor(readonly op: K) {}

  *[Symbol.iterator](): Generator<GenEffect<K, A>, A, any> {
    return yield this
  }
}

const adapter = (_: any) => {
  return new GenEffect(_)
}

export function gen<Eff extends GenEffect<any, any>, AEff>(
  f: (i: {
    <R, E, A>(_: T.Effect<R, E, A>): GenEffect<T.Effect<R, E, A>, A>
  }) => Generator<Eff, AEff, any>
): T.Effect<_R<Eff["op"]>, _E<Eff["op"]>, AEff> {
  return T.suspend(() => {
    const iterator = f(adapter as any)
    const state = iterator.next()

    function run(
      state: IteratorYieldResult<Eff> | IteratorReturnResult<AEff>
    ): T.Effect<any, any, AEff> {
      if (state.done) {
        return T.succeed(state.value)
      }
      return T.chain_(state.value["op"], (val) => {
        const next = iterator.next(val)
        return run(next)
      })
    }

    return run(state)
  })
}

const program = gen(function* (_) {
  const a = yield* _(T.succeed(1))
  const b = yield* _(T.succeed(2))

  return a + b
})

pipe(
  program,
  T.chain((n) =>
    T.effectTotal(() => {
      console.log(n)
    })
  ),
  T.runMain
)

And there we go, we have our nice imperative style of composing effects!

One thing I learned is: "if you have restrictions at the api level, find opportunities to exploit them"

In this case we have an unwanted _ function that we use to convert an effect to a generator of effects we can exploit that to lift a generic natural transformation from any type we want to effect, in the @effect-ts/core/Effect package the gen function can directly deal with:

f: (i: {
    <A>(_: Tag<A>): GenEffect<Has<A>, never, A>
    <E, A>(_: Option<A>, onNone: () => E): GenEffect<unknown, E, A>
    <A>(_: Option<A>): GenEffect<unknown, NoSuchElementException, A>
    <E, A>(_: Either<E, A>): GenEffect<unknown, E, A>
    <R, E, A>(_: Effect<R, E, A>): GenEffect<R, E, A>
    <R, E, A>(_: Managed<R, E, A>): GenEffect<R, E, A>
  }) => Generator<Eff, AEff, any>

So you can use it like:

import * as E from "@effect-ts/core/Classic/Either"
import * as O from "@effect-ts/core/Classic/Option"
import * as T from "@effect-ts/core/Effect"
import * as M from "@effect-ts/core/Effect/Managed"
import { pipe } from "@effect-ts/core/Function"

const result = T.gen(function* (_) {
  const a = yield* _(O.some(1))
  const b = yield* _(O.some(2))
  const c = yield* _(E.right(3))
  const d = yield* _(T.access((_: { n: number }) => _.n))
  const e = yield* _(
    M.makeExit_(
      T.effectTotal(() => {
        console.log("open")

        return 5
      }),
      () =>
        T.effectTotal(() => {
          console.log("release")
        })
    )
  )

  yield* _(
    T.effectTotal(() => {
      console.log(a + b + c + d + e)
    })
  )
})

pipe(result, T.provideAll({ n: 4 }), T.runMain)

That will produce:

open
15
release

Multi-Shot Monads

The trick explored so far works well for effects that produce a single value, like Effect, Managed and any io-like effect type.

There is a problem with multi-shot effects like Stream or Array, for those the approach taken doesn't work, as we work through the computation above we notice how we progress the iterator at every step of the execution, in a stream or in an array we would have to replay the same for many elements but we have only one iterator.

Iterators are mutable so there is no way we can "clone" the iterator or do any form of defensive copy.

That is not the end though, supposing the body of the generator is pure, as in none of the lines of code outside of the ones inside the yields can modify any external variable or perform any side-effect, we can solve the issue by keeping the stack of computations around and by replaying the iterator every time (idea of https://github.com/mattiamanzati).

This technique has a performance complexity of O(n^2) where n is the number of yields (not the number of elements produced) so this must be taken into account when building large generators.

Let's look at the code of the Stream generator:

import type { Effect } from "../../Effect"
import { fromEither, service } from "../../Effect"
import { die } from "../../Effect/die"
import type { Either } from "../../Either"
import { NoSuchElementException, PrematureGeneratorExit } from "../../GlobalExceptions"
import type { Has, Tag } from "../../Has"
import * as L from "../../List"
import type { Option } from "../../Option"
import type { _E, _R } from "../../Utils"
import { isEither, isOption, isTag } from "../../Utils"
import { chain_ } from "./chain"
import { Stream } from "./definitions"
import { fail } from "./fail"
import { fromEffect } from "./fromEffect"
import { succeed } from "./succeed"
import { suspend } from "./suspend"

export class GenStream<R, E, A> {
  readonly _R!: (_R: R) => void
  readonly _E!: () => E
  readonly _A!: () => A
  constructor(readonly effect: Stream<R, E, A>) {}
  *[Symbol.iterator](): Generator<GenStream<R, E, A>, A, any> {
    return yield this
  }
}

const adapter = (_: any, __?: any) => {
  if (isOption(_)) {
    return new GenStream(
      _._tag === "None"
        ? fail(__ ? __() : new NoSuchElementException())
        : succeed(_.value)
    )
  } else if (isEither(_)) {
    return new GenStream(fromEffect(fromEither(() => _)))
  } else if (_ instanceof Stream) {
    return new GenStream(_)
  } else if (isTag(_)) {
    return new GenStream(fromEffect(service(_)))
  }
  return new GenStream(fromEffect(_))
}

export function gen<RBase, EBase, AEff>(): <Eff extends GenStream<RBase, EBase, any>>(
  f: (i: {
    <A>(_: Tag<A>): GenStream<Has<A>, never, A>
    <E, A>(_: Option<A>, onNone: () => E): GenStream<unknown, E, A>
    <A>(_: Option<A>): GenStream<unknown, NoSuchElementException, A>
    <E, A>(_: Either<E, A>): GenStream<unknown, E, A>
    <R, E, A>(_: Effect<R, E, A>): GenStream<R, E, A>
    <R, E, A>(_: Stream<R, E, A>): GenStream<R, E, A>
  }) => Generator<Eff, AEff, any>
) => Stream<_R<Eff>, _E<Eff>, AEff>
export function gen<EBase, AEff>(): <Eff extends GenStream<any, EBase, any>>(
  f: (i: {
    <A>(_: Tag<A>): GenStream<Has<A>, never, A>
    <E, A>(_: Option<A>, onNone: () => E): GenStream<unknown, E, A>
    <A>(_: Option<A>): GenStream<unknown, NoSuchElementException, A>
    <E, A>(_: Either<E, A>): GenStream<unknown, E, A>
    <R, E, A>(_: Effect<R, E, A>): GenStream<R, E, A>
    <R, E, A>(_: Stream<R, E, A>): GenStream<R, E, A>
  }) => Generator<Eff, AEff, any>
) => Stream<_R<Eff>, _E<Eff>, AEff>
export function gen<AEff>(): <Eff extends GenStream<any, any, any>>(
  f: (i: {
    <A>(_: Tag<A>): GenStream<Has<A>, never, A>
    <E, A>(_: Option<A>, onNone: () => E): GenStream<unknown, E, A>
    <A>(_: Option<A>): GenStream<unknown, NoSuchElementException, A>
    <E, A>(_: Either<E, A>): GenStream<unknown, E, A>
    <R, E, A>(_: Effect<R, E, A>): GenStream<R, E, A>
    <R, E, A>(_: Stream<R, E, A>): GenStream<R, E, A>
  }) => Generator<Eff, AEff, any>
) => Stream<_R<Eff>, _E<Eff>, AEff>
export function gen<Eff extends GenStream<any, any, any>, AEff>(
  f: (i: {
    <A>(_: Tag<A>): GenStream<Has<A>, never, A>
    <E, A>(_: Option<A>, onNone: () => E): GenStream<unknown, E, A>
    <A>(_: Option<A>): GenStream<unknown, NoSuchElementException, A>
    <E, A>(_: Either<E, A>): GenStream<unknown, E, A>
    <R, E, A>(_: Effect<R, E, A>): GenStream<R, E, A>
    <R, E, A>(_: Stream<R, E, A>): GenStream<R, E, A>
  }) => Generator<Eff, AEff, any>
): Stream<_R<Eff>, _E<Eff>, AEff>
export function gen(...args: any[]): any {
  function gen_<Eff extends GenStream<any, any, any>, AEff>(
    f: (i: any) => Generator<Eff, AEff, any>
  ): Stream<_R<Eff>, _E<Eff>, AEff> {
    return suspend(() => {
      function run(replayStack: L.List<any>): Stream<any, any, AEff> {
        const iterator = f(adapter as any)
        let state = iterator.next()
        for (const a of replayStack) {
          if (state.done) {
            return fromEffect(die(new PrematureGeneratorExit()))
          }
          state = iterator.next(a)
        }
        if (state.done) {
          return succeed(state.value)
        }
        return chain_(state.value["effect"], (val) => {
          return run(L.append_(replayStack, val))
        })
      }
      return run(L.empty())
    })
  }

  if (args.length === 0) {
    return (f: any) => gen_(f)
  }
  return gen_(args[0])
}

From https://github.com/Matechs-Garage/matechs-effect/blob/master/packages/system/src/Stream/Stream/gen.ts

Apart from all the overloadings and the adapter code to support different monadic values the key part is:

suspend(() => {
  function run(replayStack: L.List<any>): Stream<any, any, AEff> {
    const iterator = f(adapter as any)
    let state = iterator.next()
    for (const a of replayStack) {
      if (state.done) {
        return fromEffect(die(new PrematureGeneratorExit()))
      }
      state = iterator.next(a)
    }
    if (state.done) {
      return succeed(state.value)
    }
    return chain_(state.value["effect"], (val) => {
      return run(L.append_(replayStack, val))
    })
  }
  return run(L.empty())
})

As we can see we carry around a stack of result and we reconstruct the local scope at each call.

This can be used just like the previous one:

import * as E from "@effect-ts/core/Classic/Either"
import * as O from "@effect-ts/core/Classic/Option"
import * as T from "@effect-ts/core/Effect"
import * as S from "@effect-ts/core/Effect/Stream"
import { pipe } from "@effect-ts/core/Function"

const result = S.gen(function* (_) {
  const a = yield* _(O.some(0))
  const b = yield* _(E.right(1))
  const c = yield* _(T.succeed(2))
  const d = yield* _(S.fromArray([a, b, c]))

  return d
})

pipe(
  result,
  S.runCollect,
  T.chain((res) =>
    T.effectTotal(() => {
      console.log(res)
    })
  ),
  T.runMain
)

will produce:

[0, 1, 2]

Generalisation

Thanks to the HKT structure and the prelude described in the previous chapters we have been able to generalise this approach to work with any monad, the generic code comes in 2 functions available at: https://github.com/Matechs-Garage/matechs-effect/blob/master/packages/core/src/Prelude/DSL/gen.ts namely genF (to be used in one-shot cases, very efficient) and genWithHistoryF (to be used in multi-shot cases, n^2 complexity for the n-th yield).

Bonus:

Let's use all we've seen so far and build up a simulation of a simple program using 2 services, we'll simulate a message broker that flushes the messages on completion and a database that keeps state.

import "@effect-ts/core/Operators"

import * as Array from "@effect-ts/core/Classic/Array"
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as L from "@effect-ts/core/Effect/Layer"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import type { _A } from "@effect-ts/core/Utils"
import { tag } from "@effect-ts/system/Has"

// make Database Live
export const makeDbLive = M.gen(function* (_) {
  const ref = yield* _(
    Ref.makeRef<Map.Map<string, string>>(Map.empty)["|>"](
      M.make((ref) => ref.set(Map.empty))
    )
  )

  return {
    get: (k: string) => ref.get["|>"](T.map(Map.lookup(k)))["|>"](T.chain(T.getOrFail)),
    put: (k: string, v: string) => ref["|>"](Ref.update(Map.insert(k, v)))
  }
})

// simulate a database connection to a key-value store
export interface DbConnection extends _A<typeof makeDbLive> {}

// Tag<DbConnection>
export const DbConnection = tag<DbConnection>()

// Database Live Layer
export const DbLive = L.fromManaged(DbConnection)(makeDbLive)

// make Broker Live
export const makeBrokerLive = M.gen(function* (_) {
  const ref = yield* _(
    Ref.makeRef<Array.Array<string>>(Array.empty)["|>"](
      M.make((ref) =>
        ref.get["|>"](
          T.chain((messages) =>
            T.effectTotal(() => {
              console.log(`Flush:`)
              messages.forEach((message) => {
                console.log("- " + message)
              })
            })
          )
        )
      )
    )
  )

  return {
    send: (message: string) => ref["|>"](Ref.update(Array.snoc(message)))
  }
})

// simulate a connection to a message broker
export interface BrokerConnection extends _A<typeof makeBrokerLive> {}

// Tag<BrokerConnection>
export const BrokerConnection = tag<BrokerConnection>()

// Broker Live Layer
export const BrokerLive = L.fromManaged(BrokerConnection)(makeBrokerLive)

// Main Live Layer
export const ProgramLive = L.all(DbLive, BrokerLive)

// Program Entry
export const main = T.gen(function* (_) {
  const { get, put } = yield* _(DbConnection)
  const { send } = yield* _(BrokerConnection)

  yield* _(put("ka", "a"))
  yield* _(put("kb", "b"))
  yield* _(put("kc", "c"))

  const a = yield* _(get("ka"))
  const b = yield* _(get("kb"))
  const c = yield* _(get("kc"))

  const s = `${a}-${b}-${c}`

  yield* _(send(s))

  return s
})

// run the program and print the output
main["|>"](T.provideSomeLayer(ProgramLive))["|>"](T.runMain)

We can see how easy it is to access services using this generator based approach:

const { get, put } = yield* _(DbConnection)

Directly yielding the Tag of a service will give you access to it's content.

Also if we highlight through the code we can see that the full types are correctly inferred and almost never explicitly specified.

Stay tuned

In the next article of the series, in 2 weeks time, we will continue to explore the data types available in the @effect-ts/core ecosystem!

The Effect Data Types: Managed & Layer

Michael Arnaldi — Mon, 19 Oct 2020 18:27:23 +0000

In the previous post of the series we started to explore the Effect data type in details.

The Effect type is itself used to construct a variety of data-types that are thought to cover many of the common patterns of software development.

This time we will explore Managed and Layer both constructed using Effect, the first uses to safely allocate and release resources and the second used to construct environments.

In the code snippets we will simulate data stores by making usage of a third data-type built on Effect the Ref data-type, Ref<T> encodes a mutable Reference to an immutable value T and can be used to keep track of State through a computation.

Bracket

Let's start by taking a look at a classic pattern we encounter many times: bracket

import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"

// simulate a database connection to a key-value store
export interface DbConnection {
  readonly put: (k: string, v: string) => T.UIO<void>
  readonly get: (k: string) => T.IO<NoSuchElementException, string>
  readonly clear: T.UIO<void>
}

// connect to the database
export const connectDb = pipe(
  // make a reference to an empty map
  Ref.makeRef(<Map.Map<string, string>>Map.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): DbConnection => ({
        // get the current map and lookup k
        get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
        // updates the reference to an updated Map containing k->v
        put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
        // clean up (for simulation purposes)
        clear: ref.set(Map.empty)
      })
    )
  )
)

// write a program that use the database
export const program = pipe(
  // acquire the database connection
  connectDb,
  T.bracket(
    // use the database connection
    (_) =>
      pipe(
        T.do,
        T.tap(() => _.put("ka", "a")),
        T.tap(() => _.put("kb", "b")),
        T.tap(() => _.put("kc", "c")),
        T.bind("a", () => _.get("ka")),
        T.bind("b", () => _.get("kb")),
        T.bind("c", () => _.get("kc")),
        T.map(({ a, b, c }) => `${a}-${b}-${c}`)
      ),
    // release the database connection
    (_) => _.clear
  )
)

// run the program and print the output
pipe(
  program,
  T.chain((s) =>
    T.effectTotal(() => {
      console.log(s)
    })
  ),
  T.runMain
)

Bracket is used to safely acquire Resources like DbConnection to be used and safely released post usage.

Managed

The idea behind Managed is to generalize the behaviour of bracket by isolating the declaration of acquire and release into a specific data-type.

The inner details of how this data-type works are out of the scope of this post but the design is a very interesting (more advanced) topic to explore. It is based on the implementation of ZManaged in ZIO that is itself based on the ResourceT designed by Michael Snoyman in haskell.

Let's take a look at the same code when isolated into a Managed:

import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"

// simulate a database connection to a key-value store
export interface DbConnection {
  readonly put: (k: string, v: string) => T.UIO<void>
  readonly get: (k: string) => T.IO<NoSuchElementException, string>
  readonly clear: T.UIO<void>
}

// connect to the database
export const managedDb = pipe(
  Ref.makeRef(<Map.Map<string, string>>Map.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): DbConnection => ({
        get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
        put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
        clear: ref.set(Map.empty)
      })
    )
  ),
  // release the connection via managed
  M.make((_) => _.clear)
)

// write a program that use the database
export const program = pipe(
  // use the managed DbConnection
  managedDb,
  M.use((_) =>
    pipe(
      T.do,
      T.tap(() => _.put("ka", "a")),
      T.tap(() => _.put("kb", "b")),
      T.tap(() => _.put("kc", "c")),
      T.bind("a", () => _.get("ka")),
      T.bind("b", () => _.get("kb")),
      T.bind("c", () => _.get("kc")),
      T.map(({ a, b, c }) => `${a}-${b}-${c}`)
    )
  )
)

// run the program and print the output
pipe(
  program,
  T.chain((s) =>
    T.effectTotal(() => {
      console.log(s)
    })
  ),
  T.runMain
)

We can see how we moved the release step from our program into the data-type of managedDb, now we can use the Resource without having to track its closing state.

Composing Managed Resources

One other advantage that we gained is we can now easily compose multiple Resources together, let's take a look at how we might do that:

import * as Array from "@effect-ts/core/Classic/Array"
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"

// simulate a database connection to a key-value store
export interface DbConnection {
  readonly put: (k: string, v: string) => T.UIO<void>
  readonly get: (k: string) => T.IO<NoSuchElementException, string>
  readonly clear: T.UIO<void>
}

// simulate a connection to a message broker
export interface BrokerConnection {
  readonly send: (message: string) => T.UIO<void>
  readonly clear: T.UIO<void>
}

// connect to the database
export const managedDb = pipe(
  Ref.makeRef(<Map.Map<string, string>>Map.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): DbConnection => ({
        get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
        put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
        clear: ref.set(Map.empty)
      })
    )
  ),
  // release the connection via managed
  M.make((_) => _.clear)
)

// connect to the database
export const managedBroker = pipe(
  Ref.makeRef(<Array.Array<string>>Array.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): BrokerConnection => ({
        send: (message) =>
          pipe(ref, Ref.update<Array.Array<string>>(Array.snoc(message))),
        clear: pipe(
          ref.get,
          T.chain((messages) =>
            T.effectTotal(() => {
              console.log(`Flush:`)
              messages.forEach((message) => {
                console.log("- " + message)
              })
            })
          )
        )
      })
    )
  ),
  // release the connection via managed
  M.make((_) => _.clear)
)

// write a program that use the database
export const program = pipe(
  // use the managed DbConnection
  managedDb,
  M.zip(managedBroker),
  M.use(([{ get, put }, { send }]) =>
    pipe(
      T.do,
      T.tap(() => put("ka", "a")),
      T.tap(() => put("kb", "b")),
      T.tap(() => put("kc", "c")),
      T.bind("a", () => get("ka")),
      T.bind("b", () => get("kb")),
      T.bind("c", () => get("kc")),
      T.map(({ a, b, c }) => `${a}-${b}-${c}`),
      T.tap(send)
    )
  )
)

// run the program and print the output
pipe(
  program,
  T.chain((s) =>
    T.effectTotal(() => {
      console.log(`Done: ${s}`)
    })
  ),
  T.runMain
)

We have added a second Managed called managedBroker to simulate a connection to a message broker and we composed the 2 using zip.

The result will be a Managed that acquires both Resources and release both Resources as you would expect.

There are many combinators in the Managed module, for example a simple change from zip to zipPar would change the behaviour of the composed Managed and will acquire and release both Resources in parallel.

Layer

The Layer data-type is closely related to Managed, in fact the idea for the design of Layer has its roots in a pattern commonly discovered in the early days of ZIO by early adopters.

We have looked at the Effect data-type already and we notice how the environment R can be used to embed services in order to gain great testability and code organization.

The question now is, how do I construct those environments?

It seems simple in principle but when you get to write an app this embedding of services becomes so pervasive that is completely normal to find yourself embedding hundreds of services in your "main" program.

Many of the services are going to be dependent on database connections and stuff like that, and as we saw before Managed is a good use case for those.

The community has started to construct their environments by using Managed so that services are constructed at bootstrap time and the result is used to provide the environment.

This pattern progressively evolved to an independent data-type that is based on Managed tailored to specifically to construct environments.

Let's start by simply transforming our code to use Layers, it will be straightforward given we have already designed it using Managed:

import * as Array from "@effect-ts/core/Classic/Array"
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as L from "@effect-ts/core/Effect/Layer"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"

// simulate a database connection to a key-value store
export interface DbConnection {
  readonly put: (k: string, v: string) => T.UIO<void>
  readonly get: (k: string) => T.IO<NoSuchElementException, string>
  readonly clear: T.UIO<void>
}

// simulate a connection to a message broker
export interface BrokerConnection {
  readonly send: (message: string) => T.UIO<void>
  readonly clear: T.UIO<void>
}

// connect to the database
export const DbLive = pipe(
  Ref.makeRef(<Map.Map<string, string>>Map.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): DbConnection => ({
        get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
        put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
        clear: ref.set(Map.empty)
      })
    )
  ),
  // release the connection via managed
  M.make((_) => _.clear),
  // construct the layer
  L.fromRawManaged
)

// connect to the database
export const BrokerLive = pipe(
  Ref.makeRef(<Array.Array<string>>Array.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): BrokerConnection => ({
        send: (message) =>
          pipe(ref, Ref.update<Array.Array<string>>(Array.snoc(message))),
        clear: pipe(
          ref.get,
          T.chain((messages) =>
            T.effectTotal(() => {
              console.log(`Flush:`)
              messages.forEach((message) => {
                console.log("- " + message)
              })
            })
          )
        )
      })
    )
  ),
  // release the connection via managed
  M.make((_) => _.clear),
  // construct the layer
  L.fromRawManaged
)

export const ProgramLive = L.all(DbLive, BrokerLive)

// write a program that use the database
export const program = pipe(
  // access Db from environment
  T.access(({ get, put }: DbConnection) => ({ get, put })),
  // access Broker from environment
  T.zip(T.access(({ send }: BrokerConnection) => ({ send }))),
  // use both
  T.chain(([{ get, put }, { send }]) =>
    pipe(
      T.do,
      T.tap(() => put("ka", "a")),
      T.tap(() => put("kb", "b")),
      T.tap(() => put("kc", "c")),
      T.bind("a", () => get("ka")),
      T.bind("b", () => get("kb")),
      T.bind("c", () => get("kc")),
      T.map(({ a, b, c }) => `${a}-${b}-${c}`),
      T.tap(send)
    )
  )
)

// run the program and print the output
pipe(
  program,
  T.chain((s) =>
    T.effectTotal(() => {
      console.log(`Done: ${s}`)
    })
  ),
  // provide the layer to program
  T.provideSomeLayer(ProgramLive),
  T.runMain
)

What we did is we turned our Managed into Layers by using the fromRawManaged constructor and we then plugged the results into a single ProgramLive layer.

We then transformed our program to consume the DbConnection and BrokerConnection services from enviroment.

At the very end we provided the ProgramLive to our program by using provideSomeLayer.

The behaviour itself remained the same but we have now fully separated program definition from resources and services location.

The Has & Tag utilities

In the last article of the series we scratched the surface of the Has & Tag apis, we are going to see now a bit more details about the inner working and how it integrates into Layers.

Up to now we haven't noticed issues because we took care of keeping function names like send and get separated but in reality it is not a good idea to intersect casual services into a big object because conflicts might easily arise.

One conflict we had without even noticing (because it's used only locally inside managed) is the "clear" effect of both DbConnection and BrokerConnection, that will get shadowed at runtime in the intersected result.

In order to solve this issue, and in order to improve the amount of types you end up writing in your app we designed two data-types that solve the issue.

We took inspiration from the work done in ZIO where the Has type is explicit and the Tag type is implicit in typescript we had to come up with an explicit solution.

The trick works like that:

import { tag} from "@effect-ts/core/Classic/Has"

/**
 * Any interface or type alias
 */
interface Anything {
  a: string
}

/**
 * Tag<Anything>
 */
const Anything = tag<Anything>()

/**
 * (r: Has<Anything>) => Anything
 */
const readFromEnv = Anything.read

/**
 * (_: Anything) => Has<Anything>
 */
const createEnv = Anything.of

const hasAnything = createEnv({ a: "foo" })

/**
 * Has<Anything> is fake, in reality we have:
 *
 * { [Symbol()]: { a: 'foo' } }
 */
console.log(hasAnything)

/**
 * The [Symbol()] is:
 */
console.log((hasAnything as any)[Anything.key])

/**
 * The same as:
 */
console.log(readFromEnv(hasAnything))

/**
 * In order to take ownership of the symbol used we can do:
 */
const mySymbol = Symbol()

const Anything_ = tag<Anything>().setKey(mySymbol)

console.log((Anything_.of({ a: "bar" }) as any)[mySymbol])

Fundamentally Tag<T> encodes the capability of reading T from a type Has<T> and encodes the capability of producing a value of type Has<T> given a T.

With the guarantee that given T0 and T1 the result Has<T0> & Has<T1> is always discriminated.

Let's see it practice by rewriting our code:

import * as Array from "@effect-ts/core/Classic/Array"
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as L from "@effect-ts/core/Effect/Layer"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"
import { tag } from "@effect-ts/system/Has"

// simulate a database connection to a key-value store
export interface DbConnection {
  readonly put: (k: string, v: string) => T.UIO<void>
  readonly get: (k: string) => T.IO<NoSuchElementException, string>
  readonly clear: T.UIO<void>
}

export const DbConnection = tag<DbConnection>()

// simulate a connection to a message broker
export interface BrokerConnection {
  readonly send: (message: string) => T.UIO<void>
  readonly clear: T.UIO<void>
}

export const BrokerConnection = tag<BrokerConnection>()

// connect to the database
export const DbLive = pipe(
  Ref.makeRef(<Map.Map<string, string>>Map.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): DbConnection => ({
        get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
        put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
        clear: ref.set(Map.empty)
      })
    )
  ),
  // release the connection via managed
  M.make((_) => _.clear),
  // construct the layer
  L.fromManaged(DbConnection)
)

// connect to the database
export const BrokerLive = pipe(
  Ref.makeRef(<Array.Array<string>>Array.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): BrokerConnection => ({
        send: (message) =>
          pipe(ref, Ref.update<Array.Array<string>>(Array.snoc(message))),
        clear: pipe(
          ref.get,
          T.chain((messages) =>
            T.effectTotal(() => {
              console.log(`Flush:`)
              messages.forEach((message) => {
                console.log("- " + message)
              })
            })
          )
        )
      })
    )
  ),
  // release the connection via managed
  M.make((_) => _.clear),
  // construct the layer
  L.fromManaged(BrokerConnection)
)

export const ProgramLive = L.all(DbLive, BrokerLive)

// write a program that use the database
export const program = pipe(
  // access Db from environment
  T.accessService(DbConnection)(({ get, put }) => ({ get, put })),
  // access Broker from environment
  T.zip(T.accessService(BrokerConnection)(({ send }) => ({ send }))),
  // use both
  T.chain(([{ get, put }, { send }]) =>
    pipe(
      T.do,
      T.tap(() => put("ka", "a")),
      T.tap(() => put("kb", "b")),
      T.tap(() => put("kc", "c")),
      T.bind("a", () => get("ka")),
      T.bind("b", () => get("kb")),
      T.bind("c", () => get("kc")),
      T.map(({ a, b, c }) => `${a}-${b}-${c}`),
      T.tap(send)
    )
  )
)

// run the program and print the output
pipe(
  program,
  T.chain((s) =>
    T.effectTotal(() => {
      console.log(`Done: ${s}`)
    })
  ),
  // provide the layer to program
  T.provideSomeLayer(ProgramLive),
  T.runMain
)

By this very simple change we are now safe, and as we can already see, instead of manually annotate the type DbConnection to access the service we can now simply use the Tag value.

We can go a bit further and derive some functions that we would normally like to write, the idea is:

T.accessService(DbConnection)(({ get, put }) => ({ get, put }))

You don't want that in code.

To avoid repeating this you end-up writing commodity functions like:

export function get(k: string) {
  return T.accessServiceM(DbConnection)((_) => _.get(k))
}

and use this instead of direct access.

We can automatically derive such implementations in many of the common function types except the case of functions containing generics.

Let's see it in practice:

import * as Array from "@effect-ts/core/Classic/Array"
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as L from "@effect-ts/core/Effect/Layer"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"
import { tag } from "@effect-ts/system/Has"

// simulate a database connection to a key-value store
export interface DbConnection {
  readonly put: (k: string, v: string) => T.UIO<void>
  readonly get: (k: string) => T.IO<NoSuchElementException, string>
  readonly clear: T.UIO<void>
}

export const DbConnection = tag<DbConnection>()

// simulate a connection to a message broker
export interface BrokerConnection {
  readonly send: (message: string) => T.UIO<void>
  readonly clear: T.UIO<void>
}

export const BrokerConnection = tag<BrokerConnection>()

// connect to the database
export const DbLive = pipe(
  Ref.makeRef(<Map.Map<string, string>>Map.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): DbConnection => ({
        get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
        put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
        clear: ref.set(Map.empty)
      })
    )
  ),
  // release the connection via managed
  M.make((_) => _.clear),
  // construct the layer
  L.fromManaged(DbConnection)
)

// connect to the database
export const BrokerLive = pipe(
  Ref.makeRef(<Array.Array<string>>Array.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): BrokerConnection => ({
        send: (message) =>
          pipe(ref, Ref.update<Array.Array<string>>(Array.snoc(message))),
        clear: pipe(
          ref.get,
          T.chain((messages) =>
            T.effectTotal(() => {
              console.log(`Flush:`)
              messages.forEach((message) => {
                console.log("- " + message)
              })
            })
          )
        )
      })
    )
  ),
  // release the connection via managed
  M.make((_) => _.clear),
  // construct the layer
  L.fromManaged(BrokerConnection)
)

export const ProgramLive = L.all(DbLive, BrokerLive)

export const { get, put } = T.deriveLifted(DbConnection)(["get", "put"], [], [])

export const { send } = T.deriveLifted(BrokerConnection)(["send"], [], [])

// write a program that use the database
export const program = pipe(
  T.do,
  T.tap(() => put("ka", "a")),
  T.tap(() => put("kb", "b")),
  T.tap(() => put("kc", "c")),
  T.bind("a", () => get("ka")),
  T.bind("b", () => get("kb")),
  T.bind("c", () => get("kc")),
  T.map(({ a, b, c }) => `${a}-${b}-${c}`),
  T.tap(send)
)

// run the program and print the output
pipe(
  program,
  T.chain((s) =>
    T.effectTotal(() => {
      console.log(`Done: ${s}`)
    })
  ),
  // provide the layer to program
  T.provideSomeLayer(ProgramLive),
  T.runMain
)

The other 2 arrays of deriveLifted are used to generate functions to access Constant Effects and Constants as Effects.

Like deriveLifted other 2 functions can be used to derive commonly used utilities and those are deriveAccess and deriveAccessM.

It is left as an exercise to the reader to try out those combinations and look at the result function types in order to better understand the context of each derivation.

Another way to smooth the usage of services without the need to actually write any of the utility functions is to effectively write your main program as a service and leverage the layer constructor utilities to automate the environment wiring.

Let's see how we might do that:

import * as Array from "@effect-ts/core/Classic/Array"
import * as Map from "@effect-ts/core/Classic/Map"
import * as T from "@effect-ts/core/Effect"
import * as L from "@effect-ts/core/Effect/Layer"
import * as M from "@effect-ts/core/Effect/Managed"
import * as Ref from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import type { NoSuchElementException } from "@effect-ts/system/GlobalExceptions"
import { tag } from "@effect-ts/system/Has"

// simulate a database connection to a key-value store
export interface DbConnection {
  readonly put: (k: string, v: string) => T.UIO<void>
  readonly get: (k: string) => T.IO<NoSuchElementException, string>
  readonly clear: T.UIO<void>
}

export const DbConnection = tag<DbConnection>()

// simulate a connection to a message broker
export interface BrokerConnection {
  readonly send: (message: string) => T.UIO<void>
  readonly clear: T.UIO<void>
}

export const BrokerConnection = tag<BrokerConnection>()

// connect to the database
export const DbLive = pipe(
  Ref.makeRef(<Map.Map<string, string>>Map.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): DbConnection => ({
        get: (k) => pipe(ref.get, T.map(Map.lookup(k)), T.chain(T.getOrFail)),
        put: (k, v) => pipe(ref, Ref.update(Map.insert(k, v))),
        clear: ref.set(Map.empty)
      })
    )
  ),
  // release the connection via managed
  M.make((_) => _.clear),
  // construct the layer
  L.fromManaged(DbConnection)
)

// connect to the database
export const BrokerLive = pipe(
  Ref.makeRef(<Array.Array<string>>Array.empty),
  T.chain((ref) =>
    T.effectTotal(
      (): BrokerConnection => ({
        send: (message) =>
          pipe(ref, Ref.update<Array.Array<string>>(Array.snoc(message))),
        clear: pipe(
          ref.get,
          T.chain((messages) =>
            T.effectTotal(() => {
              console.log(`Flush:`)
              messages.forEach((message) => {
                console.log("- " + message)
              })
            })
          )
        )
      })
    )
  ),
  // release the connection via managed
  M.make((_) => _.clear),
  // construct the layer
  L.fromManaged(BrokerConnection)
)

export function makeProgram({ get, put }: DbConnection, { send }: BrokerConnection) {
  return {
    main: pipe(
      T.do,
      T.tap(() => put("ka", "a")),
      T.tap(() => put("kb", "b")),
      T.tap(() => put("kc", "c")),
      T.bind("a", () => get("ka")),
      T.bind("b", () => get("kb")),
      T.bind("c", () => get("kc")),
      T.map(({ a, b, c }) => `${a}-${b}-${c}`),
      T.tap(send)
    )
  }
}

export interface Program extends ReturnType<typeof makeProgram> {}

export const Program = tag<Program>()

export const ProgramLive = L.fromConstructor(Program)(makeProgram)(
  DbConnection,
  BrokerConnection
)

export const MainLive = pipe(ProgramLive, L.using(L.all(DbLive, BrokerLive)))

export const { main } = T.deriveLifted(Program)([], ["main"], [])

// run the program and print the output
pipe(
  main,
  T.chain((s) =>
    T.effectTotal(() => {
      console.log(`Done: ${s}`)
    })
  ),
  // provide the layer to program
  T.provideSomeLayer(MainLive),
  T.runMain
)

We basically introduced a new service Program as a constructor makeProgram that simply takes dependencies as arguments and we used the Layer constructor fromConstructor to glue everything.

We had to add the ProgramLive layer to our final MainLive cake and in the main function we called the effect from the program service (that can be easily derived as before).

Stay Tuned

In the next post of the series we will be discussing about generators and how to get back to an imperative paradigm by using generators to simulate a monadic do that will look very similar to the dear old async/await but with much better typing.

The Effect Data Types: Effect

Michael Arnaldi — Mon, 12 Oct 2020 17:57:55 +0000

In the first 2 parts of the series we have looked at the overall module structure of @effect-ts/core and discussed the general aspects of how modules are organised.

We will now take a deep dive into the specific of the @effect-ts/core/Effect segment, the primary target for the modules that we will discuss today is node.js and in general backend development.

While the technologies in question are perfectly usable in the frontend space we believe most of the features are overkill if used on top of pre-existing frameworks like react and that within the @effect-ts/core system there are more appropriate data types like Async & Sync and in general the ones available in the @effect-ts/core/Classic segment that better fit the requirements.

In general the advise will be to use Effect and its derivative in frontend development only if major usage is involved and the bundle-size vs feature-set make sense for you (Effect and its derivatives tend to add a minimum of 30-80k), for example we have seen very successful integrations of Stream with a redux epic-inspired design but we have also seen adding 50k to the bundle to simply have interruption on a Promise (that would be something to avoid).

What we will discuss today (and in the next few articles of the series) in most parts is directly usable across all the data types that implements instances of Environmental thanks to the approach discussed in the previous posts.

The Effect Type

Let's start with the basic and let's take a look at the type definition.



type Effect<R, E, A> = omitted.

the real definition of Effect is omitted from the purpose of this discussion because the reality is you have no reasons in the world to ever look inside of it.

The logical definition that we can think of is in the lines of:



type Effect<R, E, A> = (r: R) => () => Promise<Either<E, A>>

Basically the Effect type describes a computation that, in order to run requires an argument of type R to be provided and when the computation runs and completes will produce a result that can either be a failure of type E or a success of type A.

In reality the implementation uses a different internal representation and makes usage of fibers to represent concurrent computations and additionally has an unchecked exception channel to signal defects of code or unexpected exceptions, the Effect type uses Cause to represent the error case of the Exit and Cause allows for tracking multiple exit causes with different methods (like parallel/sequential cause of failures, interruptions, etc).

Let's take a look at a basic usage of all the parameters:



import * as T from "@effect-ts/core/Effect"
import { pipe } from "@effect-ts/core/Function"
import { runMain } from "@effect-ts/node/Runtime"

interface Input {
  x: number
  y: number
}

const division = pipe(
  T.environment<Input>(),
  T.chain(({ x, y }) => (y === 0 ? T.fail("division by zero") : T.succeed(x / y))),
  T.chain((result) =>
    T.effectTotal(() => {
      console.log(`Final result: ${result}`)
    })
  )
)

pipe(division, T.provideAll({ x: 1, y: 2 }), runMain)

If we highlight on the type of division:

We can easily read that in order to run this computation we will be required to provide an argument of type Input and we can expect either an error of type string or a success of type void (nothing basically).

In the last line of the snippet we proceed to run the program with:



pipe(division, T.provideAll({ x: 1, y: 2 }), runMain)

If we remove T.provideAll({ x: 1, y: 2 }) we will be alerted with a compiler error that state Input is not assignable to DefaultEnv that translates to: "you need to give Input otherwise I have no idea what to run".

If we run the computation with y = 0 we will get:



A checked error was not handled.
"division by zero"

Using runMain we have automatic reporting of errors in a prettified format (we will take a look at a larger report later), additionally runMain returns a function that when called will cancel the running computation.

Snippets Location

All the snippets in this article are available at: https://github.com/Matechs-Garage/effect-ts-bootstrap/tree/master/articles/effect

In the repository you find also a playground environment that bootstrap a postgresql, runs migrations and implements a basic persistence module with detailed integration tests some of which using arbitrary generated models (property-based), we will cover those aspects in future articles.

Variance

Like we discussed in the previous articles of the series variance is a key component in the design of the modules of @effect-ts/core in the case of Effect we discussed 3 type parameters R, E, A representing one input R one possible error (output) E and one possible output A.

In mostly all of the available combinators E is treated as co-variant and R is treated as contra-variant, this means that, by default, mixing different Effects the result will be an Effect that requires R1 & R2 and can fail for E1 | E2.

Let's see it in practice:



import * as T from "@effect-ts/core/Effect"
import { pipe } from "@effect-ts/core/Function"
import { runMain } from "@effect-ts/node/Runtime"

interface InputA {
  x: number
}
interface InputB {
  y: number
}
interface ErrorA {
  _tag: "ErrorA"
  value: string
}
interface ErrorB {
  _tag: "ErrorB"
  value: string
}

// T.Effect<InputA & InputB, ErrorA | ErrorB, void>
const program = pipe(
  T.access((_: InputA) => _.x),
  T.chain((n) =>
    n === 0 ? T.fail<ErrorA>({ _tag: "ErrorA", value: "n === 0" }) : T.succeed(n)
  ),
  T.chain((n) => T.access((_: InputB) => _.y + n)),
  T.chain((n) =>
    n === 10 ? T.fail<ErrorB>({ _tag: "ErrorB", value: "n === 10" }) : T.succeed(n)
  ),
  T.chain((n) =>
    T.effectTotal(() => {
      console.log(n)
    })
  )
)

pipe(program, T.provideAll({ x: 1, y: 1 }), runMain)

Handling Checked Errors

In the previous example we used a tagged union to represent the error type, that choice was purposely made because of typescript's ability to automatically discriminate over the members of such union.

Let's see for example how we could handle the ErrorA case of the program above:



import * as T from "@effect-ts/core/Effect"
import { pipe } from "@effect-ts/core/Function"
import { runMain } from "@effect-ts/node/Runtime"

interface InputA {
  x: number
}
interface InputB {
  y: number
}
interface ErrorA {
  _tag: "ErrorA"
  value: string
}
interface ErrorB {
  _tag: "ErrorB"
  value: string
}

// T.Effect<InputA & InputB, ErrorA | ErrorB, void>
const program = pipe(
  T.access((_: InputA) => _.x),
  T.chain((n) =>
    n === 0 ? T.fail<ErrorA>({ _tag: "ErrorA", value: "n === 0" }) : T.succeed(n)
  ),
  T.chain((n) => T.access((_: InputB) => _.y + n)),
  T.chain((n) =>
    n === 10 ? T.fail<ErrorB>({ _tag: "ErrorB", value: "n === 10" }) : T.succeed(n)
  ),
  T.chain((n) =>
    T.effectTotal(() => {
      console.log(n)
    })
  )
)

// T.Effect<InputA & InputB, ErrorB, void>
const programAfterErrorHandling = pipe(
  program,
  T.catchAll((error) => {
    switch (error._tag) {
      case "ErrorA": {
        return T.effectTotal(() => {
          console.log(`handling ErrorA: ${error.value}`)
        })
      }
      default:
        return T.fail(error)
    }
  })
)

pipe(programAfterErrorHandling, T.provideAll({ x: 1, y: 1 }), runMain)

If you use this standard for errors and tagged unions in general you can leverage the pattern & matchTag utilities of @effect-ts/core/Utils and rewrite the above as:



import * as T from "@effect-ts/core/Effect"
import { pipe } from "@effect-ts/core/Function"
import { matchTag } from "@effect-ts/core/Utils"
import { runMain } from "@effect-ts/node/Runtime"

interface InputA {
  x: number
}
interface InputB {
  y: number
}
interface ErrorA {
  _tag: "ErrorA"
  value: string
}
interface ErrorB {
  _tag: "ErrorB"
  value: string
}

// T.Effect<InputA & InputB, ErrorA | ErrorB, void>
const program = pipe(
  T.access((_: InputA) => _.x),
  T.chain((n) =>
    n === 0 ? T.fail<ErrorA>({ _tag: "ErrorA", value: "n === 0" }) : T.succeed(n)
  ),
  T.chain((n) => T.access((_: InputB) => _.y + n)),
  T.chain((n) =>
    n === 10 ? T.fail<ErrorB>({ _tag: "ErrorB", value: "n === 10" }) : T.succeed(n)
  ),
  T.chain((n) =>
    T.effectTotal(() => {
      console.log(n)
    })
  )
)

// T.Effect<InputA & InputB, ErrorB, void>
const programAfterErrorHandling = pipe(
  program,
  T.catchAll(
    matchTag(
      {
        ErrorA: ({ value }) =>
          T.effectTotal(() => {
            console.log(`handling ErrorA: ${value}`)
          })
      },
      T.fail
    )
  )
)

pipe(programAfterErrorHandling, T.provideAll({ x: 1, y: 1 }), runMain)

Unchecked Exceptions

So far we've seen how to raise checked errors that have specific types and are safely represented for the type-checker to process, there are cases where instead we want to raise some stronger form of error, for example to handle cases like a code defect or cases where unknown system errors have been observed, to do so Effect provides a dedicated Error channel.

Note that raising on this channel will skip any checked error management process like the above T.catchAll.

Let's make an example:



import * as T from "@effect-ts/core/Effect"
import { pipe } from "@effect-ts/core/Function"
import { matchTag } from "@effect-ts/core/Utils"
import { runMain } from "@effect-ts/node/Runtime"

interface InputA {
  x: number
}
interface InputB {
  y: number
}
interface ErrorA {
  _tag: "ErrorA"
  value: string
}

// T.Effect<InputA & InputB, ErrorA, void>
const program = pipe(
  T.access((_: InputA) => _.x),
  T.chain((n) =>
    n === 0 ? T.fail<ErrorA>({ _tag: "ErrorA", value: "n === 0" }) : T.succeed(n)
  ),
  T.chain((n) => T.access((_: InputB) => _.y + n)),
  T.chain((n) => (n === 10 ? T.die("something very wrong happened") : T.succeed(n))),
  T.chain((n) =>
    T.effectTotal(() => {
      console.log(n)
    })
  )
)

// T.Effect<InputA & InputB, never, void>
const programAfterErrorHandling = pipe(
  program,
  T.catchAll(
    matchTag(
      {
        ErrorA: ({ value }) =>
          T.effectTotal(() => {
            console.log(`handling ErrorA: ${value}`)
          })
      },
      (e) =>
        pipe(
          T.effectTotal(() => {
            console.log(`Default Handler`)
          }),
          T.andThen(T.fail(e))
        )
    )
  )
)

pipe(programAfterErrorHandling, T.provideAll({ x: 1, y: 9 }), runMain)

By running the code above we will get:



yarn ts-node articles/effect/04.ts 
yarn run v1.22.4
$ /home/ma/os/articles/node_modules/.bin/ts-node articles/effect/04.ts
An unchecked error was produced.
"something very wrong happened"
error Command failed with exit code 2.

We can notice none of the error handling logic have been accounted for, in fact neither console.log cases of the T.catchAll logic have been produced.

Handling full Cause

In order to take control over the full exit cause of a program it is necessary to access lower level combinators like foldCause or a combination of sandbox/catchAll.

Let's take a look at a possibility:



import * as A from "@effect-ts/core/Classic/Array"
import * as T from "@effect-ts/core/Effect"
import * as Cause from "@effect-ts/core/Effect/Cause"
import { pipe } from "@effect-ts/core/Function"
import { matchTag } from "@effect-ts/core/Utils"
import { runMain } from "@effect-ts/node/Runtime"

interface InputA {
  x: number
}
interface InputB {
  y: number
}
interface ErrorA {
  _tag: "ErrorA"
  value: string
}

// T.Effect<InputA & InputB, ErrorA, void>
const program = pipe(
  T.access((_: InputA) => _.x),
  T.chain((n) =>
    n === 0 ? T.fail<ErrorA>({ _tag: "ErrorA", value: "n === 0" }) : T.succeed(n)
  ),
  T.chain((n) => T.access((_: InputB) => _.y + n)),
  T.chain((n) => (n === 10 ? T.die("something very wrong happened") : T.succeed(n))),
  T.chain((n) =>
    T.effectTotal(() => {
      console.log(n)
    })
  )
)

// T.Effect<InputA & InputB, never, void>
const programAfterErrorHandling = pipe(
  program,
  T.catchAll(
    matchTag(
      {
        ErrorA: ({ value }) =>
          T.effectTotal(() => {
            console.log(`handling ErrorA: ${value}`)
          })
      },
      (e) =>
        pipe(
          T.effectTotal(() => {
            console.log(`Default Handler`)
          }),
          T.andThen(T.fail(e))
        )
    )
  )
)

const handleFullCause = pipe(
  programAfterErrorHandling,
  T.sandbox,
  T.catchAll((cause) => {
    const defects = Cause.defects(cause)

    if (A.isNonEmpty(defects)) {
      return T.effectTotal(() => {
        console.log("Handle:", ...defects)
      })
    }

    return T.halt(cause)
  })
)

pipe(handleFullCause, T.provideAll({ x: 1, y: 9 }), runMain)

With the combinator Cause.defects we can extract a list of defects that might be present on the cause, there are many more combinators you can use to encode the many possible scenarios you might need to handle.

The type definition Cause is as follows:



export type Cause<E> = Empty | Fail<E> | Die | Interrupt | Then<E> | Both<E>

// empty cause, no cause at all, used for monoidal composition
export interface Empty {
  readonly _tag: "Empty"
}

// represent a checked error
export interface Fail<E> {
  readonly _tag: "Fail"
  readonly value: E
}

// represent an unchecked error
export interface Die {
  readonly _tag: "Die"
  readonly value: unknown
}

// represent an interruption triggered by fiber with id FiberID
export interface Interrupt {
  readonly _tag: "Interrupt"
  readonly fiberId: FiberID
}

// represent two causes that happened in sequence
export interface Then<E> {
  readonly _tag: "Then"
  readonly left: Cause<E>
  readonly right: Cause<E>
}

// represent two causes that happened in parallel
export interface Both<E> {
  readonly _tag: "Both"
  readonly left: Cause<E>
  readonly right: Cause<E>
}

Let's take a look at a more complex code where we might trigger multiple failures and let's take a look at how it gets reported:



import * as T from "@effect-ts/core/Effect"
import * as Rand from "@effect-ts/core/Effect/Random"
import { pipe } from "@effect-ts/core/Function"
import { runMain } from "@effect-ts/node/Runtime"

export class ProcessError {
  readonly _tag = "ProcessError"
  constructor(readonly message: string) {}
}

// this effect sleeps for a random period between 100ms and 1000ms and randomly suceed or fail
export const randomFailure = (n: number) =>
  pipe(
    Rand.nextIntBetween(100, 1000),
    T.chain((delay) =>
      pipe(
        Rand.nextBoolean,
        T.chain((b) => (b ? T.unit : T.fail(new ProcessError(`failed at: ${n}`)))),
        T.delay(delay)
      )
    )
  )

// build up a n-tuple of computations
export const program = T.tuplePar(
  randomFailure(0),
  randomFailure(1),
  randomFailure(2),
  randomFailure(3),
  randomFailure(4),
  randomFailure(5)
)

// runs the program
pipe(program, runMain)

Running this various times will produce different outputs given the random nature of the program but an average result will look like:



$ /home/ma/os/articles/node_modules/.bin/ts-node articles/effect/06.ts
╥
╠══╦══╦══╦══╦══╗
║  ║  ║  ║  ║  ║
║  ║  ║  ║  ║  ╠─An interrupt was produced by #6.
║  ║  ║  ║  ║  ▼
║  ║  ║  ║  ║
║  ║  ║  ║  ╠─An interrupt was produced by #5.
║  ║  ║  ║  ▼
║  ║  ║  ║
║  ║  ║  ╠─An interrupt was produced by #4.
║  ║  ║  ▼
║  ║  ║
║  ║  ╠─An interrupt was produced by #1.
║  ║  ▼
║  ║
║  ╠─A checked error was not handled.
║  ║ {
║  ║   "message": "failed at: 1",
║  ║   "_tag": "ProcessError"
║  ║ }
║  ▼
▼
error Command failed with exit code 2.

We can see that at the first failure the tuplePar combinator takes care of interrupting the other running computations and we can see by the horizontal nature of the report that there were running 6 fibers in total.

Let's take a look at how error handling applies in this scenarios:



import * as T from "@effect-ts/core/Effect"
import * as Rand from "@effect-ts/core/Effect/Random"
import { pipe } from "@effect-ts/core/Function"
import { runMain } from "@effect-ts/node/Runtime"

export class ProcessError {
  readonly _tag = "ProcessError"
  constructor(readonly message: string) {}
}

// this effect sleeps for a random period between 100ms and 1000ms and randomly suceed or fail
export const randomFailure = (n: number) =>
  pipe(
    Rand.nextIntBetween(100, 1000),
    T.chain((delay) =>
      pipe(
        Rand.nextBoolean,
        T.chain((b) => (b ? T.unit : T.fail(new ProcessError(`failed at: ${n}`)))),
        T.delay(delay)
      )
    )
  )

// build up a n-tuple of computations
export const program = T.tuplePar(
  randomFailure(0),
  randomFailure(1),
  randomFailure(2),
  randomFailure(3),
  randomFailure(4),
  randomFailure(5)
)

// runs the program
pipe(
  program,
  T.catchAll((_) =>
    T.effectTotal(() => {
      console.log(`Process error: ${_.message}`)
    })
  ),
  runMain
)

This will now emit something like:



$ /home/ma/os/articles/node_modules/.bin/ts-node articles/effect/07.ts
Process error: failed at: 2
Done in 2.12s.

We can see that catchAll peaks at the first available checked error in the cause tree and handles that.

In order to handle multiple causes of failures we can, as outlined before, jump in a sandbox and traverse the cause with available combinators like collect and similar.

Environment

We have covered up to now the usage of E and A, let's now take a deep look at how R works.

The crucial idea we will discuss here is I believe the single 10x improvement ZIO had to offer to the scala ecosystem and it applies perfectly (even more compared to scala) to the structural variance of typescript.

The main idea is to embed dynamic services that contains business logic as part of the environment having as a consequence a type-level representation of what modules a specific computation needs in order to run, for example in previous snippets we have interacted directly with console.log, that code will easily become non testable without weird tricks like monkey-patching modules.

Let's start with the basic:



import * as T from "@effect-ts/core/Effect"
import { pipe } from "@effect-ts/core/Function"
import { runMain } from "@effect-ts/node/Runtime"

// define a module
export interface ConsoleModule {
  log: (message: string) => T.UIO<void>
}

// access the module from environment
export function log(message: string) {
  return T.accessM((_: ConsoleModule) => _.log(message))
}

// T.Effect<ConsoleModule, never, void>
export const program = pipe(
  log("hello"),
  T.andThen(log("world")),
  T.andThen(log("hello")),
  T.andThen(log("eatrh"))
)

// Run the program providing a concrete implementation
pipe(
  program,
  T.provide<ConsoleModule>({
    log: (message) =>
      T.effectTotal(() => {
        console.log(message)
      })
  }),
  runMain
)

When executed this program will output:



$ /home/ma/os/articles/node_modules/.bin/ts-node articles/effect/08.ts
hello
world
hello
eatrh
Done in 1.68s.

We can now provide a different implementation that we might use to test the correct implementation of program without output to console, for example:



import * as T from "@effect-ts/core/Effect"
import * as R from "@effect-ts/core/Effect/Ref"
import { pipe } from "@effect-ts/core/Function"
import { runMain } from "@effect-ts/node/Runtime"

// define a module
export interface ConsoleModule {
  log: (message: string) => T.UIO<void>
}

// access the module from environment
export function log(message: string) {
  return T.accessM((_: ConsoleModule) => _.log(message))
}

// T.Effect<ConsoleModule, never, void>
export const program = pipe(
  log("hello"),
  T.andThen(log("world")),
  T.andThen(log("hello")),
  T.andThen(log("eatrh"))
)

// Run the program providing a concrete implementation
pipe(
  T.do,
  T.bind("ref", () => R.makeRef(<string[]>[])),
  T.bind("prog", ({ ref }) =>
    pipe(
      program,
      T.provide<ConsoleModule>({
        log: (message) =>
          pipe(
            ref,
            R.update((messages) => [...messages, message])
          )
      })
    )
  ),
  T.chain(({ ref }) => ref.get),
  T.tap((messages) =>
    messages.length === 4 ? T.succeed(messages) : T.fail("wrong number of messages")
  ),
  runMain
)

In this way we can run the program without ever interacting with the actual console.

Has & Tag

It's all nice but at some point we will encounter strange situations, the intersection of 2 types is not generally commutative and can lead to very strange behaviours.

One example would be having 2 services implement the log function with different signatures, those will be collapsed to never when ServiceA & ServiceB are present, additionally inference cannot always discern those members from the mangled result, we need to remember, at the end, that the typescript type-system is structural.

Here comes Has and Tag to solve the issue, Tag<T> represent an object that has the ability to read and write from and to and environment of type Has<T>, Has is designed to be safe for intersection.

Let's rewrite the above using Has and Tag:



import { tag } from "@effect-ts/core/Classic/Has"
import * as T from "@effect-ts/core/Effect"
import { pipe } from "@effect-ts/core/Function"
import { runMain } from "@effect-ts/node/Runtime"

// define a module
export interface ConsoleModule {
  log: (message: string) => T.UIO<void>
}

// define a tag for the service
export const ConsoleModule = tag<ConsoleModule>()

// access the module from environment
export function log(message: string) {
  return T.accessServiceM(ConsoleModule)((_) => _.log(message))
}

// T.Effect<Has<ConsoleModule>, never, void>
export const program = pipe(
  log("hello"),
  T.andThen(log("world")),
  T.andThen(log("hello")),
  T.andThen(log("eatrh"))
)

// Run the program providing a concrete implementation
pipe(
  program,
  T.provideService(ConsoleModule)({
    log: (message) =>
      T.effectTotal(() => {
        console.log(message)
      })
  }),
  runMain
)

There are many combinators to work with services (provide, access, replace, etc).

Services are also highly used in structuring modular environments constructed dynamically using Layer but that discussion is for another post of the series, after having seen the Managed data type first.

We can already improve the code a little by deriving the implementation of log, to do so we can look at deriveLifted:



import { tag } from "@effect-ts/core/Classic/Has"
import * as T from "@effect-ts/core/Effect"
import { pipe } from "@effect-ts/core/Function"
import { runMain } from "@effect-ts/node/Runtime"

// define a module
export interface ConsoleModule {
  log: (message: string) => T.UIO<void>
}

// define a tag for the service
export const ConsoleModule = tag<ConsoleModule>()

// access the module from environment
export const { log } = T.deriveLifted(ConsoleModule)(["log"], [], [])

// T.Effect<Has<ConsoleModule>, never, void>
export const program = pipe(
  log("hello"),
  T.andThen(log("world")),
  T.andThen(log("hello")),
  T.andThen(log("eatrh"))
)

// Run the program providing a concrete implementation
pipe(
  program,
  T.provideService(ConsoleModule)({
    log: (message) =>
      T.effectTotal(() => {
        console.log(message)
      })
  }),
  runMain
)

The 3 array parameters of deriveLifted represent the field names of the functions to derive and they represent in sequence:

non generic, effect-returning functions
constant effects
constant values

Usage:



import { tag } from "@effect-ts/core/Classic/Has"
import * as T from "@effect-ts/core/Effect"
import { pipe } from "@effect-ts/core/Function"
import { runMain } from "@effect-ts/node/Runtime"

// define a module
export interface ConsoleModule {
  prefix: T.UIO<string>
  end: string
  log: (message: string) => T.UIO<void>
}

// define a tag for the service
export const ConsoleModule = tag<ConsoleModule>()

// access the module from environment
export const { end, log, prefix } = T.deriveLifted(ConsoleModule)(
  ["log"],
  ["prefix"],
  ["end"]
)

// T.Effect<Has<ConsoleModule>, never, void>
export const program = pipe(
  prefix,
  T.chain(log),
  T.andThen(log("world")),
  T.andThen(log("hello")),
  T.andThen(pipe(end, T.chain(log)))
)

// Run the program providing a concrete implementation
pipe(
  program,
  T.provideService(ConsoleModule)({
    log: (message) =>
      T.effectTotal(() => {
        console.log(message)
      }),
    prefix: T.succeed("hello"),
    end: "earth"
  }),
  runMain
)

Additional combinators like deriveAccess and deriveAccessM can be used to derive access-like function signatures like:



import { tag } from "@effect-ts/core/Classic/Has"

import * as T from "@effect-ts/core/Effect"

import { pipe } from "@effect-ts/core/Function"

import { runMain } from "@effect-ts/node/Runtime"

// define a module

export interface ConsoleModule {

  prefix: string

  end: string

  log: (message: string) => T.UIO<void>

}

// define a tag for the service

export const ConsoleModule = tag<ConsoleModule>()

// access the module from environment

export const { log } = T.deriveLifted(ConsoleModule)(["log"], [], [])

export const { end: accessEndM, prefix: accessPrefixM } = T.deriveAccessM(

  ConsoleModule

)(["end", "prefix"])

// T.Effect<Has<ConsoleModule>, never, void>

export const program = pipe(

  accessPrefixM(log),

  T.andThen(log("world")),

  T.andThen(log("hello")),

  T.andThen(accessEndM(log))

)

// Run the program providing a concrete implementation

pipe(

  program,

  T.provideService(ConsoleModule)({

    log: (message) =>

      T.effectTotal(() => {

        console.log(message)

      }),

    prefix: "hello",

    end: "earth"

  }),

  runMain

)

Stay tuned

The next week's article of the series will be about the Managed data-type to encode Resources that can be safely acquired and released.

Effect-TS Core: ZIO-Prelude Inspired Typeclasses & Module Structure

Michael Arnaldi — Tue, 06 Oct 2020 10:43:34 +0000

In the first article of the series we described the principles behind the unique encoding of HKTs used in @effect-ts/core, it's now time to take a look at the details.

We will start by exploring the Type-Classes available and we will progressively make some examples of usage.

At the end we will discuss the module structure and what's available a-la-carte.

Project Setup

Let's start with a simple new project (be bothered only if you come from Scala/Haskell, ignore if you know TS):



mkdir effect-ts-series; 
cd effect-ts-series;
npm init -y;
yarn add typescript@next @effect-ts/core @types/node;
mkdir src;

Let's create a file tsconfig.json as follow:



{
  "compilerOptions": {
    "strict": true,
    "target": "ES5",
    "outDir": "lib",
    "lib": ["ESNext"]
  },
  "include": ["src/**/*.ts"]
}

Let's create a file src/index.ts with the following content:



import * as T from "@effect-ts/core/Effect";
import { pipe } from "@effect-ts/core/Function";

pipe(
  T.effectTotal(() => {
    console.log("Hello world");
  }),
  T.runMain
);

and add a build script to your package.json as follows:



{
  "name": "effect-ts-series",
  "version": "1.0.0",
  "description": "Effect-TS Series",
  "main": "lib/index.js",
  "scripts": {
    "build": "tsc",
    "start": "node lib/index.js"
  },
  "keywords": [],
  "author": "Michael Arnaldi",
  "license": "MIT",
  "dependencies": {
    "@effect-ts/core": "^0.2.0",
    "@types/node": "^14.11.2",
    "typescript": "^4.1.0-dev.20201004"
  }
}

We should be able to compile the project with:



yarn build

And run it:



$ yarn start
yarn run v1.22.4
$ node lib/index.js
Hello world
Done in 0.46s.

Introduction to ZIO-Prelude's Type-Classes

First of all let's start with a little bit of theory and reasons why to revise the classical type-classes hierarchy.

Statically Typed Functional Programming as we know it today effectively has roots in haskell and in its design principles; for years we have, as a community, gone through an exercise of borrowing principles one by one and finding its way into different languages.

The process of porting features from one language to another is not an easy one and it requires multiple steps, the first of which is finding similar encodings and secondly improving upon the basics.

Haskell’s type-system is inspired by category theory, but mathematically speaking it's only an “approximation” that focuses on a specific subset of the theory that makes sense in languages of the HM family. We should not be blind to the rest of the theory especially when extending the concepts to different languages because the same assumptions made in haskell might not hold in ours (like for example all of the functions being curried).

ZIO Prelude can be considered the second step of abstraction and adaptation of functional programming concepts to Scala, it is designed for Scala and leverages all the features available in the language.

Lucky for us the features of Scala as a language are very similar to the features of TypeScript at the type-system level and in some cases the TypeScript type-system is even more flexible (i.e. supporting intersection & union types).

Furthermore ZIO Prelude takes a look at a broader range of constructs from mathematics that have previously been perceived as secondary.

Let's take a look at Functor from fp-ts, we will list only one definition to keep things small:



export interface Functor<F> {
  readonly URI: F
  readonly map: <A, B>(fa: HKT<F, A>, f: (a: A) => B) => HKT<F, B>
}

Similarly defined in other fp-languages like purescript & haskell this typeclass shows a bias, in fact in category theory a Functor can be Covariant or Contravariant while here we associate the Functor name with a specific case

Let's now take a look at how a Functor is defined categorically:

A Functor between Categories is a mapping of both objects and morphisms that preserves the categorical structure, there are at least 2 types of Functors, one that preserves the direction of the morphisms and one that inverts the direction.

Those are called Covariant Functor & Contravariant Functor.

From the above definition from fp-ts we realise the haskell bias, everything is pointed towards Covariant Functors.

ZIO Prelude use different naming and leverages an extremely orthogonal design (i.e. minimal type-classes, easily composable), conceptually the same but more close to the actual laws the typeclass respect.

Covariant

Let's take a look at the equivalent of Functor in @effect-ts/core:



export interface Covariant<F extends HKT.URIS, C = HKT.Auto> extends HKT.Base<F, C> {
  readonly map: <A, B>(
    f: (a: A) => B
  ) => <N extends string, K, Q, W, X, I, S, R, E>(
    fa: HKT.Kind<F, C, N, K, Q, W, X, I, S, R, E, A>
  ) => HKT.Kind<F, C, N, K, Q, W, X, I, S, R, E, B>
}

Code at: core/src/Prelude/Covariant/index.ts

The name used is Covariant as in Covariant Functor.

Let's take a look at some instances for known data-types:



export const Covariant = P.instance<P.Covariant<[EitherURI], V>>({
  map: E.map
})

Where E is the Either module, V = Prelude.V<"E", "+"> to indicate the covariance of the parameter E (in Either the error channel E mixes with union type as we will see later).

Monad

Let's take a look at the dear loved Monad:



export type Monad<F extends URIS, C = Auto> = IdentityFlatten<F, C> & Covariant<F, C>

export type IdentityFlatten<F extends URIS, C = Auto> = AssociativeFlatten<F, C> &
  Any<F, C>

export interface Any<F extends HKT.URIS, C = HKT.Auto> extends HKT.Base<F, C> {
  readonly any: <
    N extends string = HKT.Initial<C, "N">,
    K = HKT.Initial<C, "K">,
    Q = HKT.Initial<C, "Q">,
    W = HKT.Initial<C, "W">,
    X = HKT.Initial<C, "X">,
    I = HKT.Initial<C, "I">,
    S = HKT.Initial<C, "S">,
    R = HKT.Initial<C, "R">,
    E = HKT.Initial<C, "E">
  >() => HKT.Kind<F, C, N, K, Q, W, X, I, S, R, E, any>
}

export interface AssociativeFlatten<F extends HKT.URIS, C = HKT.Auto>
  extends HKT.Base<F, C> {
  readonly flatten: <
    N extends string,
    K,
    Q,
    W,
    X,
    I,
    S,
    R,
    E,
    A,
    N2 extends string,
    K2,
    Q2,
    W2,
    X2,
    I2,
    S2,
    R2,
    E2
  >(
    ffa: HKT.Kind<
      F,
      C,
      N2,
      K2,
      Q2,
      W2,
      X2,
      I2,
      S2,
      R2,
      E2,
      HKT.Kind<
        F,
        C,
        HKT.Intro<C, "N", N2, N>,
        HKT.Intro<C, "K", K2, K>,
        HKT.Intro<C, "Q", Q2, Q>,
        HKT.Intro<C, "W", W2, W>,
        HKT.Intro<C, "X", X2, X>,
        HKT.Intro<C, "I", I2, I>,
        HKT.Intro<C, "S", S2, S>,
        HKT.Intro<C, "R", R2, R>,
        HKT.Intro<C, "E", E2, E>,
        A
      >
    >
  ) => HKT.Kind<
    F,
    C,
    HKT.Mix<C, "N", [N2, N]>,
    HKT.Mix<C, "K", [K2, K]>,
    HKT.Mix<C, "Q", [Q2, Q]>,
    HKT.Mix<C, "W", [W2, W]>,
    HKT.Mix<C, "X", [X2, X]>,
    HKT.Mix<C, "I", [I2, I]>,
    HKT.Mix<C, "S", [S2, S]>,
    HKT.Mix<C, "R", [R2, R]>,
    HKT.Mix<C, "E", [E2, E]>,
    A
  >
}

Code at: core/src/Prelude/Monad/index.ts

Apart from being slightly verbose, @effect-ts/core supports up to 10 different type parameters that can mix dynamically based on the variance annotation specified at the instance level.

We can see how well Monad is separated orthogonally across different, more specific, lawful type-classes.

We read Monad is a Covariant functor with an identity and an Associative flatten operation.

Pretty much describes itself the laws a Monad has to respect.

Let's take a look at a few instances of Monad for various data-types and let's have a look at how variance works.

We will first introduce a generic operation to showcase how to write code that works with any kind, we will take a look at the generic chain function that given an instance of Monad performs a series of operations where the second operation depends on the result of the first.



export function chainF<F extends HKT.URIS, C = HKT.Auto>(
  F: Monad<F, C>
): <N2 extends string, K2, Q2, W2, X2, I2, S2, R2, E2, A, B>(
  f: (a: A) => HKT.Kind<F, C, N2, K2, Q2, W2, X2, I2, S2, R2, E2, B>
) => <N extends string, K, Q, W, X, I, S, R, E>(
  fa: HKT.Kind<
    F,
    C,
    HKT.Intro<C, "N", N2, N>,
    HKT.Intro<C, "K", K2, K>,
    HKT.Intro<C, "Q", Q2, Q>,
    HKT.Intro<C, "W", W2, W>,
    HKT.Intro<C, "X", X2, X>,
    HKT.Intro<C, "I", I2, I>,
    HKT.Intro<C, "S", S2, S>,
    HKT.Intro<C, "R", R2, R>,
    HKT.Intro<C, "E", E2, E>,
    A
  >
) => HKT.Kind<
  F,
  C,
  HKT.Mix<C, "N", [N2, N]>,
  HKT.Mix<C, "K", [K2, K]>,
  HKT.Mix<C, "Q", [Q2, Q]>,
  HKT.Mix<C, "W", [W2, W]>,
  HKT.Mix<C, "X", [X2, X]>,
  HKT.Mix<C, "I", [I2, I]>,
  HKT.Mix<C, "S", [S2, S]>,
  HKT.Mix<C, "R", [R2, R]>,
  HKT.Mix<C, "E", [E2, E]>,
  B
>
export function chainF<F>(F: Monad<HKT.UHKT<F>>) {
  return <A, B>(f: (a: A) => HKT.HKT<F, B>) => flow(F.map(f), F.flatten)
}

Code at: core/src/Prelude/DSL/dsl.ts

Let's use this generic chainF function on a few different instances:



import * as IO from "@effect-ts/core/XPure/XIO";
import * as Either from "@effect-ts/core/Classic/Either";
import * as Effect from "@effect-ts/core/Effect";
import { pipe } from "@effect-ts/core/Function";
import { chainF } from "@effect-ts/core/Prelude/DSL";

const chainIO = chainF(IO.Monad);
const chainEither = chainF(Either.Monad);
const chainEffect = chainF(Effect.Monad);

// IO.XIO<number>
const io = pipe(
  IO.succeed(0),
  chainIO((n) => IO.succeed(n + 1))
);

const checkPositive = (n: number): Either.Either<string, number> =>
  n > 0 ? Either.right(n) : Either.left("error");

// Either.Either<string, number>
const either = (n: number) =>
  pipe(
    n,
    checkPositive,
    chainEither((n) => Either.right(n + 1))
  );

// Effect.Effect<{ s: string; } & { n: number; }, string | number, number>
const effect = pipe(
  Effect.accessM((_: { n: number }) =>
    Effect.ifM(Effect.succeed(_.n > 0))(() => Effect.succeed(_.n))(() =>
      Effect.fail("error")
    )
  ),
  chainEffect((n) =>
    Effect.accessM((_: { s: string }) =>
      Effect.ifM(Effect.succeed(_.s.length > 1))(() =>
        Effect.succeed(n + _.s.length)
      )(() => Effect.fail(0))
    )
  )
);

As we can see parameters R, E are mixed differently depending on the variance of the instance specified as:



// for Effect
export type V = P.V<"R", "-"> & P.V<"E", "+">

// for Either
export type V = P.V<"E", "+">

Applicative

Let's take a look at the good old friend Applicative, the first thing to note is that Applicative is completely independent from Monad not really like in Haskell land!



export type Applicative<F extends URIS, C = Auto> = IdentityBoth<F, C> & Covariant<F, C>

export type IdentityBoth<F extends URIS, C = Auto> = AssociativeBoth<F, C> & Any<F, C>

export interface AssociativeBoth<F extends HKT.URIS, C = HKT.Auto>
  extends HKT.Base<F, C> {
  readonly both: <N2 extends string, K2, Q2, W2, X2, I2, S2, R2, E2, B>(
    fb: HKT.Kind<F, C, N2, K2, Q2, W2, X2, I2, S2, R2, E2, B>
  ) => <N extends string, K, Q, W, X, I, S, R, E, A>(
    fa: HKT.Kind<
      F,
      C,
      HKT.Intro<C, "N", N2, N>,
      HKT.Intro<C, "K", K2, K>,
      HKT.Intro<C, "Q", Q2, Q>,
      HKT.Intro<C, "W", W2, W>,
      HKT.Intro<C, "X", X2, X>,
      HKT.Intro<C, "I", I2, I>,
      HKT.Intro<C, "S", S2, S>,
      HKT.Intro<C, "R", R2, R>,
      HKT.Intro<C, "E", E2, E>,
      A
    >
  ) => HKT.Kind<
    F,
    C,
    HKT.Mix<C, "N", [N2, N]>,
    HKT.Mix<C, "K", [K2, K]>,
    HKT.Mix<C, "Q", [Q2, Q]>,
    HKT.Mix<C, "W", [W2, W]>,
    HKT.Mix<C, "X", [X2, X]>,
    HKT.Mix<C, "I", [I2, I]>,
    HKT.Mix<C, "S", [S2, S]>,
    HKT.Mix<C, "R", [R2, R]>,
    HKT.Mix<C, "E", [E2, E]>,
    readonly [A, B]
  >
}

Code at: core/src/Prelude/Applicative/index.ts

Nothing easier, as we read an Applicative is a Covariant functor with an identity and an Associative operation Both.

It is theoretically the same as the classic variant with ap but much more clear from the laws point of view and from the usability standpoint.

Also if we go by the theory, we can read from ncatlab.org:

In computer science, applicative functors (also known as idioms) are the programming equivalent of lax monoidal functors with a tensorial strength in category theory.

If you know the terms involved you will recognise that this definition at the end is much closer to the theory compared to the classic ap.

Let's take a look at some DSL available for Applicative functors:



import * as Either from "@effect-ts/core/Classic/Either";
import * as DSL from "@effect-ts/core/Prelude/DSL";

const struct = DSL.structF(Either.Applicative);
const tupled = DSL.tupledF(Either.Applicative);

// Either.Either<never, { a: number; b: number; c: number; }>
const resultStruct = struct({
  a: Either.right(0),
  b: Either.right(1),
  c: Either.right(2),
});

// Either.Either<never, [number, number, number]>
const resultTupled = tupled(Either.right(0), Either.right(1), Either.right(2));

We leave it as an exercise for the reader to derive the Monad & Applicative declarations of fp-ts from this one and vice versa (hint: you can use functions available in Prelude/DSL).

Traversable

Let's take a look at the dear old friend Traversable:



export interface Foreach<F extends HKT.URIS, C = HKT.Auto> {
  <G extends HKT.URIS, GC = HKT.Auto>(G: IdentityBoth<G, GC> & Covariant<G, GC>): <
    GN extends string,
    GK,
    GQ,
    GW,
    GX,
    GI,
    GS,
    GR,
    GE,
    A,
    B
  >(
    f: (a: A) => HKT.Kind<G, GC, GN, GK, GQ, GW, GX, GI, GS, GR, GE, B>
  ) => <FN extends string, FK, FQ, FW, FX, FI, FS, FR, FE>(
    fa: HKT.Kind<F, C, FN, FK, FQ, FW, FX, FI, FS, FR, FE, A>
  ) => HKT.Kind<
    G,
    GC,
    GN,
    GK,
    GQ,
    GW,
    GX,
    GI,
    GS,
    GR,
    GE,
    HKT.Kind<F, C, FN, FK, FQ, FW, FX, FI, FS, FR, FE, B>
  >
}

export interface Traversable<F extends HKT.URIS, C = HKT.Auto>
  extends HKT.Base<F, C>,
    Covariant<F, C> {
  readonly foreachF: Foreach<F, C>
}

Code at: core/src/Prelude/Traversable/index.ts

Nothing exceptionally different from the classic version apart from the name of the foreachF function (originally called traverse).

Let's take a look at its usage:



import * as Either from "@effect-ts/core/Classic/Either";
import * as Array from "@effect-ts/core/Classic/Array";
import * as Record from "@effect-ts/core/Classic/Record";
import { pipe } from "@effect-ts/core/Function";
import { sequenceF } from "@effect-ts/core/Prelude";

const foreachArray = Array.Traversable.foreachF(Either.Applicative);

// Either.Either<string, Array.Array<number>>
const resultArray = pipe(
  [0, 1, 2, 3],
  foreachArray((n) => (n > 2 ? Either.left("error") : Either.right(n)))
);

const foreachRecord = Record.Traversable.foreachF(Either.Applicative);

// Either.Either<string, Readonly<Record<"a" | "b" | "c" | "d", number>>>
const resultRecord = pipe(
  {
    a: 0,
    b: 0,
    c: 0,
    d: 0,
  },
  foreachRecord((n) => (n > 2 ? Either.left("error") : Either.right(n)))
);

const sequenceArray = sequenceF(Array.Traversable)(Either.Applicative);

// Either.Either<string, Array.Array<number>>
const sequenceArrayResult = sequenceArray([
  Either.left("error"),
  Either.right(0),
  Either.right(1),
  Either.right(2),
]);

Identity

The dear old Monoid:



export interface Identity<A> extends Associative<A> {
  readonly identity: A
}

export interface Associative<A> extends Closure<A> {
  readonly Associative: "Associative"
}

export interface Closure<A> {
  combine(r: A): (l: A) => A
}

Like before without previously knowing the laws we can read that a Monoid has a combine associative operation with an identity element.

Foldable

Nothing special about Foldable:



export type Foldable<F extends URIS, C = Auto> = ReduceRight<F, C> &
  Reduce<F, C> &
  FoldMap<F, C>

export interface Reduce<F extends HKT.URIS, C = HKT.Auto> extends HKT.Base<F, C> {
  readonly reduce: <A, B>(
    b: B,
    f: (b: B, a: A) => B
  ) => <N extends string, K, Q, W, X, I, S, R, E>(
    fa: HKT.Kind<F, C, N, K, Q, W, X, I, S, R, E, A>
  ) => B
}

export interface ReduceRight<F extends HKT.URIS, C = HKT.Auto> extends HKT.Base<F, C> {
  readonly reduceRight: <A, B>(
    b: B,
    f: (a: A, b: B) => B
  ) => <N extends string, K, Q, W, X, I, S, R, E>(
    fa: HKT.Kind<F, C, N, K, Q, W, X, I, S, R, E, A>
  ) => B
}

export interface FoldMap<F extends HKT.URIS, C = HKT.Auto> extends HKT.Base<F, C> {
  readonly foldMap: FoldMapFn<F, C>
}

export interface FoldMapFn<F extends HKT.URIS, C = HKT.Auto> {
  <M>(I: Identity<M>): <A>(
    f: (a: A) => M
  ) => <N extends string, K, Q, W, X, I, S, R, E>(
    fa: HKT.Kind<F, C, N, K, Q, W, X, I, S, R, E, A>
  ) => M
}

Let's take a look at using some Foldable instances.



import  as Array from "@effect-ts/core/Classic/Array";

import  as Record from "@effect-ts/core/Classic/Record";

import * as Identity from "@effect-ts/core/Classic/Identity";

const fromArray = Record.fromFoldable(Identity.string, Array.Foldable);

// Readonly<Record<string, string>>

const record = fromArray([

  ["a", "foo"],

  ["b", "bar"],

]);

Module Structure

The @effect-ts/core package is organized in directories as follow:

@effect-ts/core/Classic : lightweight modules and commonly used type-classes, to be used everywhere (browser, node)
@effect-ts/core/Effect : effect based modules, primarily targeting node development this set of modules is a full suite to structure highly concurrent & well testable services with a variety of data types including: Fiber, FiberRef, Layer, Managed, Promise, Queue, Ref, RefM, Schedule, Scope, Semaphore, Stream, Supervisor . It can be used in frontend development too but there is a cost-benefit to be considered, if the project is large enough it might be beneficial because of project based amortisation in smaller projects and specific use data types from Classic like Async are preferrable.
@effect-ts/core/Function : function based utilities like pipe
@effect-ts/core/Newtype: newtype definition and common newtypes
@effect-ts/core/Utils: small set of utilities for pattern matching and intersection
@effect-ts/core/XPure: data-types based on XPure, an efficient synchronous data-type that support Contravariant State Input, Covariant State Output, Contravariant Reader, Covariant Error, Output. The purpose of XPure is to serve as a basis to construct multiple data-types that can satisfy specific capabilities. It is also very lightweight and can be especially efficient if used across different data-types. XPure is also used to back the Classic/Sync data-type that is natively included as a primitive of Effect.
@effect-ts/core/Modules: internal usage

Encoding HKTs in TS4.1

Michael Arnaldi — Thu, 01 Oct 2020 21:22:53 +0000

There have been many attempts at finding valid HKT encodings in TypeScript. Currently the most used and reliable one is implemented by fp-ts.

In fp-ts all the types are recorded into a number of type-level maps that index URI -> Concrete Type and this map is different for each kind:

export interface URItoKind<A> {}

export interface URItoKind2<E, A> {}

export interface URItoKind3<R, E, A> {}

export interface URItoKind4<S, R, E, A> {}

Those type-level records are filled progressively by using the module augmentation feature. Let's look at how Either & Option are wired in those records:

export const URI = "Either"

export type URI = typeof URI

declare module "./HKT" {
  interface URItoKind2<E, A> {
    readonly [URI]: Either<E, A>
  }
}

export const URI = "Option"

export type URI = typeof URI

declare module "./HKT" {
  interface URItoKind<A> {
    readonly [URI]: Option<A>
  }
}

After augmentation the records will look like:

export interface URItoKind<A> {
    Option: Option<A>
    ...
}

export interface URItoKind2<E, A> {
    Either: Either<E, A>
    ...
}

export interface URItoKind3<R, E, A> {
    ReaderTaskEither: ReaderTaskEither<R, E, A>
    ...
}

export interface URItoKind4<S, R, E, A> {
    StateReaderTaskEither: StateReaderTaskEither<S, R, E, A>
    ...
}

Accessing those types requires getting the proper record key and filling in the params:

type Result = URItoKind<string, number>["Either"]

Which corresponds to:

Either<string, number>

Using this method fp-ts defines:

export type URIS = keyof URItoKind<any>
export type URIS2 = keyof URItoKind2<any, any>
export type URIS3 = keyof URItoKind3<any, any, any>
export type URIS4 = keyof URItoKind4<any, any, any, any>

export type Kind<URI extends URIS, A> = URI extends URIS ? URItoKind<A>[URI] : any
export type Kind2<URI extends URIS2, E, A> = URI extends URIS2
  ? URItoKind2<E, A>[URI]
  : any
export type Kind3<URI extends URIS3, R, E, A> = URI extends URIS3
  ? URItoKind3<R, E, A>[URI]
  : any
export type Kind4<URI extends URIS4, S, R, E, A> = URI extends URIS4
  ? URItoKind4<S, R, E, A>[URI]
  : any

we can now access:

type Result = Kind2<"Either", string, number>

With these constructs it’s possible to write generic interfaces that don't specify the URI.

For example, we can write:

export interface Functor1<F extends URIS> {
  readonly URI: F
  readonly map: <A, B>(fa: Kind<F, A>, f: (a: A) => B) => Kind<F, B>
}

and have:

const functorOption: Functor1<"Option"> = {
    URI: "Option",
    map: ... // map is now correctly typed to work with Option<*>
}

Clearly, this is not enough to generalise over different kinds. In fp-ts you will find multiple definitions of every typeclass (interface with a generic URI, for this matter) for each of the kind.

export interface Functor1<F extends URIS> {
  readonly URI: F
  readonly map: <A, B>(fa: Kind<F, A>, f: (a: A) => B) => Kind<F, B>
}

export interface Functor2<F extends URIS2> {
  readonly URI: F
  readonly map: <E, A, B>(fa: Kind2<F, E, A>, f: (a: A) => B) => Kind2<F, E, B>
}

export interface Functor2C<F extends URIS2, E> {
  readonly URI: F
  readonly _E: E
  readonly map: <A, B>(fa: Kind2<F, E, A>, f: (a: A) => B) => Kind2<F, E, B>
}

export interface Functor3<F extends URIS3> {
  readonly URI: F
  readonly map: <R, E, A, B>(fa: Kind3<F, R, E, A>, f: (a: A) => B) => Kind3<F, R, E, B>
}

export interface Functor3C<F extends URIS3, E> {
  readonly URI: F
  readonly _E: E
  readonly map: <R, A, B>(fa: Kind3<F, R, E, A>, f: (a: A) => B) => Kind3<F, R, E, B>
}

export interface Functor4<F extends URIS4> {
  readonly URI: F
  readonly map: <S, R, E, A, B>(
    fa: Kind4<F, S, R, E, A>,
    f: (a: A) => B
  ) => Kind4<F, S, R, E, B>
}

As we can see, in addition to the 4 kinds, we also have *C interfaces that are used to add a constraint to the E parameter. This is used in Validation where E represents the Error channel and we ask Monoid<E> to eventually combine errors together.

Let's now look at how to use those typeclasses. How can we write a function that consumes a generic Functor?

Starting from the base case, we want a generic function addOne that works by mapping a number output by adding one:

function addOne<URI extends URIS>(F: Functor1<URI>) {
  return (fa: Kind<F, number>): Kind<F, number> => F.map(fa, (n) => n + 1)
}

Calling this function with the appropriate typeclass instance it will yield a specific function for the data-type.

const addOneOption = addOne(functorOption) // (fa: Option<number>) => Option<number>

We can generalise further and support different kinds via overloading:

function addOne<URI extends URIS4, E>(
  F: Functor4C<URI, E>
): <S, R>(fa: Kind4<F, S, R, E, number>) => Kind4<F, S, R, E, number>
function addOne<URI extends URIS4>(
  F: Functor4<URI>
): <S, R, E>(fa: Kind4<F, S, R, E, number>) => Kind4<F, S, R, E, number>
function addOne<URI extends URIS3, E>(
  F: Functor3C<URI, E>
): <R>(fa: Kind3<F, R, E, number>) => Kind3<F, R, E, number>
function addOne<URI extends URIS3>(
  F: Functor3<URI>
): <R, E>(fa: Kind3<F, R, E, number>) => Kind3<F, R, E, number>
function addOne<URI extends URIS2, E>(
  F: Functor2C<URI, E>
): (fa: Kind2<F, E, number>) => Kind2<F, E, number>
function addOne<URI extends URIS2>(
  F: Functor2<URI>
): <E>(fa: Kind2<F, E, number>) => Kind<F, E, number>
function addOne<URI extends URIS>(
  F: Functor1<URI>
): (fa: Kind<F, number>) => Kind<F, number>
function addOne(F: any) {
  return (fa: any): any => F.map(fa, (n) => n + 1)
}

The only trouble is defining the very scary base case (any, any, any).

In fp-ts we can use HKT defined as:

export interface HKT<URI, A> {
  readonly _URI: URI
  readonly _A: A
}

export interface HKT2<URI, E, A> extends HKT<URI, A> {
  readonly _E: E
}

export interface HKT3<URI, R, E, A> extends HKT2<URI, E, A> {
  readonly _R: R
}

export interface HKT4<URI, S, R, E, A> extends HKT3<URI, R, E, A> {
  readonly _S: S
}

Now we can define a specific Functor interface for HKT like:

export interface Functor<F> {
  readonly URI: F
  readonly map: <A, B>(fa: HKT<F, A>, f: (a: A) => B) => HKT<F, B>
}

and use this to type the base case:

function addOne<URI extends URIS4, E>(
  F: Functor4C<URI, E>
): <S, R>(fa: Kind4<F, S, R, E, number>) => Kind4<F, S, R, E, number>
function addOne<URI extends URIS4>(
  F: Functor4<URI>
): <S, R, E>(fa: Kind4<F, S, R, E, number>) => Kind4<F, S, R, E, number>
function addOne<URI extends URIS3, E>(
  F: Functor3C<URI, E>
): <R>(fa: Kind3<F, R, E, number>) => Kind3<F, R, E, number>
function addOne<URI extends URIS3>(
  F: Functor3<URI>
): <R, E>(fa: Kind3<F, R, E, number>) => Kind3<F, R, E, number>
function addOne<URI extends URIS2, E>(
  F: Functor2C<URI, E>
): (fa: Kind2<F, E, number>) => Kind2<F, E, number>
function addOne<URI extends URIS2>(
  F: Functor2<URI>
): <E>(fa: Kind2<F, E, number>) => Kind<F, E, number>
function addOne<URI extends URIS>(
  F: Functor1<URI>
): (fa: Kind<F, number>) => Kind<F, number>
function addOne<URI>(F: Functor<URI>) {
  return (fa: HKT<URI, number>): HKT<URI, number> => F.map(fa, (n) => n + 1)
}

Short and practical, isn't it? Let’s write a composition of functors:

export interface FunctorComposition<F, G> {
  readonly map: <A, B>(fa: HKT<F, HKT<G, A>>, f: (a: A) => B) => HKT<F, HKT<G, B>>
}

export interface FunctorCompositionHKT1<F, G extends URIS> {
  readonly map: <A, B>(fa: HKT<F, Kind<G, A>>, f: (a: A) => B) => HKT<F, Kind<G, B>>
}

export interface FunctorCompositionHKT2<F, G extends URIS2> {
  readonly map: <E, A, B>(fa: HKT<F, Kind2<G, E, A>>, f: (a: A) => B) => HKT<F, Kind2<G, E, B>>
}

export interface FunctorCompositionHKT2C<F, G extends URIS2, E> {
  readonly map: <A, B>(fa: HKT<F, Kind2<G, E, A>>, f: (a: A) => B) => HKT<F, Kind2<G, E, B>>
}

export interface FunctorComposition11<F extends URIS, G extends URIS> {
  readonly map: <A, B>(fa: Kind<F, Kind<G, A>>, f: (a: A) => B) => Kind<F, Kind<G, B>>
}

export interface FunctorComposition12<F extends URIS, G extends URIS2> {
  readonly map: <E, A, B>(fa: Kind<F, Kind2<G, E, A>>, f: (a: A) => B) => Kind<F, Kind2<G, E, B>>
}

export interface FunctorComposition12C<F extends URIS, G extends URIS2, E> {
  readonly map: <A, B>(fa: Kind<F, Kind2<G, E, A>>, f: (a: A) => B) => Kind<F, Kind2<G, E, B>>
}

export interface FunctorComposition21<F extends URIS2, G extends URIS> {
  readonly map: <E, A, B>(fa: Kind2<F, E, Kind<G, A>>, f: (a: A) => B) => Kind2<F, E, Kind<G, B>>
}

export interface FunctorComposition2C1<F extends URIS2, G extends URIS, E> {
  readonly map: <A, B>(fa: Kind2<F, E, Kind<G, A>>, f: (a: A) => B) => Kind2<F, E, Kind<G, B>>
}

export interface FunctorComposition22<F extends URIS2, G extends URIS2> {
  readonly map: <FE, GE, A, B>(fa: Kind2<F, FE, Kind2<G, GE, A>>, f: (a: A) => B) => Kind2<F, FE, Kind2<G, GE, B>>
}

export interface FunctorComposition22C<F extends URIS2, G extends URIS2, E> {
  readonly map: <FE, A, B>(fa: Kind2<F, FE, Kind2<G, E, A>>, f: (a: A) => B) => Kind2<F, FE, Kind2<G, E, B>>
}

export interface FunctorComposition23<F extends URIS2, G extends URIS3> {
  readonly map: <FE, R, E, A, B>(fa: Kind2<F, FE, Kind3<G, R, E, A>>, f: (a: A) => B) => Kind2<F, FE, Kind3<G, R, E, B>>
}

export interface FunctorComposition23C<F extends URIS2, G extends URIS3, E> {
  readonly map: <FE, R, A, B>(fa: Kind2<F, FE, Kind3<G, R, E, A>>, f: (a: A) => B) => Kind2<F, FE, Kind3<G, R, E, B>>
}

And that’s not even complete...

Another limitation is that every single type needs to be independently indexed. This makes typeclass transformers extremely impractical.

Our goal is to have the same features (or a bit more) with significantly less boilerplate.

Let’s now look at a restricted version of the encoding used in @effect-ts/core. While @effect-ts/core allows for as much as 10 type parameters (2 of which are used to encode generic keys, i.e. nominal keys for records, generic keys for maps, integer keys for arrays, etc), let’s limit ourselves to 4 for simplicity (the same number as in fp-ts).

The full code is available at:
https://github.com/Matechs-Garage/matechs-effect/tree/master/packages/core/src/Prelude/HKT

And the typeclasses (inspired by zio-prelude):

https://github.com/Matechs-Garage/matechs-effect/tree/master/packages/core/src/Prelude

The first idea is to shrink the number of type-level records, instead of having multiple records we are only going to have one:

export interface URItoKind<S, R, E, A> {
  XPure: XPure<S, S, R, E, A>
  Effect: Effect<R, E, A>
  Either: Either<E, A>
  Option: Option<A>
}

export type URIS = keyof URItoKind<any, any, any, any>

We can then temporarily define the Kind to be:

export type Kind<F extends URIS, S, R, E, A> = URItoKind<S, R, E, A>[F]

This already removes quite a bit of boilerplate. We can then define typeclasses as follows:

export interface Functor<F extends URIS> {
  URI: F
  map: <A, A2>(
    f: (a: A) => A2
  ) => <S, R, E>(fa: Kind<F, S, R, E, A>) => Kind<F, S, R, E, A2>
}

and instances:

export const functorOption: Functor<"Option"> = {
  URI: "Option",
  map: (f) => (fa) => (fa._tag === "None" ? fa : { _tag: "Some", value: f(fa.value) })
}

and Bifunctor

export interface Bifunctor<F extends URIS> {
  URI: F
  bimap: <A, A2, E, E2>(
    f: (a: A) => A2,
    g: (a: E) => E2
  ) => <S, R>(fa: Kind<F, S, R, E, A>) => Kind<F, S, R, E2, A2>
}

and instances:

export const bifunctorEither: Bifunctor<"Either"> = {
  URI: "Either",
  bimap: (f, g) => (fa) =>
    fa._tag === "Left"
      ? { _tag: "Left", left: g(fa.left) }
      : { _tag: "Right", right: f(fa.right) }
}

Looking better, but how are we going to encode constraints like fixing "E" to a specific value?

The answer is to add a generic C to hold the constraints:

export type Param = "S" | "R" | "E"

export interface Fix<P extends Param, K> {
  Fix: {
    [p in P]: K
  }
}

export type OrFix<P extends Param, C, V> = C extends Fix<P, infer K> ? K : V

export type Kind<F extends URIS, C, S, R, E, A> = URItoKind<
  OrFix<"S", C, S>,
  OrFix<"R", C, R>,
  OrFix<"E", C, E>,
  A
>[F]

and change our typeclasses to become:

export interface Functor<F extends URIS, C = {}> {
  URI: F
  map: <A, A2>(
    f: (a: A) => A2
  ) => <S, R, E>(fa: Kind<F, C, S, R, E, A>) => Kind<F, C, S, R, E, A2>
}

export interface Bifunctor<F extends URIS, C = {}> {
  URI: F
  bimap: <A, A2, E, E2>(
    f: (a: A) => A2,
    g: (a: OrFix<"E", C, E>) => OrFix<"E", C, E2>
  ) => <S, R>(fa: Kind<F, C, S, R, E, A>) => Kind<F, C, S, R, E2, A2>
}

the code of the instances didn't change at all, but we can now create a constrained instance in this way:

export const bifunctorStringValidation: Bifunctor<"Either", Fix<"E", string>> = {
  URI: "Either",
  bimap: (f, g) => (fa) =>
    fa._tag === "Left"
      ? { _tag: "Left", left: g(fa.left) }
      : { _tag: "Right", right: f(fa.right) }
}

// <A, A2, E, E2>(f: (a: A) => A2, g: (a: string) => string) => <S, R>(fa: Either<string, A>) => Either<string, A2>
export const bimapValidation = bifunctorStringValidation.bimap

We have a couple more unused parameters but we get the signature we wanted.

Unfortunately there is no way to remove those phantom types without multiple registries and much more boilerplate, but in the end, who cares?

In a single definition we can now compact multiple kinds and multiple potential constraints, in fact Fix is safe for intersection so we could have written Fix<"E", string> & Fix<"S", number>.

Our addOne function looks much better:

function addOne<URI extends URIS, C>(
  F: Functor<URI, C>
): <S, R, E>(fa: Kind<URI, C, S, R, E, number>) => Kind<URI, C, S, R, E, number> {
  return F.map((n) => n + 1)
}

We could leave it here and already have a decent save but there is a downside; Errors in the generic implementation tend to become unreadable. The reason being, URIS is a very big union and any type error will basically try every possible combination generating unusable error messages.

We can take inspiration from fp-ts in defining one HKT to write the base implementation and leave the rest to overloads, but we don't really want to define separated typeclasses for HKT so we will add HKT into the registry:

export interface F_<A> {
  _tag: "F_"
  A: A
}

export interface URItoKind<S, R, E, A> {
  F_: F_<A>
  XPure: XPure<S, S, R, E, A>
  Effect: Effect<R, E, A>
  Either: Either<E, A>
  Option: Option<A>
}

and we can now define the base case in terms of "F_":

function addOne<URI extends URIS, C>(
  F: Functor<URI, C>
): <S, R, E>(fa: Kind<URI, C, S, R, E, number>) => Kind<URI, C, S, R, E, number>
function addOne(F: Functor<"F_">): (fa: F_<number>) => F_<number> {
  return F.map((n) => n + 1)
}

Any type error in the implementation will now be specific to "F_".
However, there is a problem with this solution of "F_".

If we have a single HKT it's fine but what if we have multiple?

For example in cases like getFunctorComposition:

export function getFunctorComposition<F, G>(F: Functor<F>, G: Functor<G>): FunctorComposition<F, G> {
  return {
    map: (fa, f) => F.map(fa, (ga) => G.map(ga, f))
  }
}

For that we are going to add multiple "fake" types in the registry taking care of discriminating them using a "_tag" field:

export interface F_<A> {
  _tag: "F_"
  A: A
}
export interface G_<A> {
  _tag: "G_"
  A: A
}
export interface H_<A> {
  _tag: "H_"
  A: A
}

export interface URItoKind<S, R, E, A> {
  F_: F_<A>
  G_: G_<A>
  H_: H_<A>
  XPure: XPure<S, S, R, E, A>
  Effect: Effect<R, E, A>
  Either: Either<E, A>
  Option: Option<A>
}

In this way we can safely "name" multiple HKTs making sure that they cannot be mixed, with a bit more work we could also extend the logic of Kind to accept another form of non primitive URIs that allow for embedding of custom parameters and with that discriminate HKTs like functor-composition-in-core.

Good enough? Not even close, we are going to attempt transformers.
What if we want a Functor for Either<E, Option<A>>? How can we do that without reindexing?

The idea is to make the URI of Kind composable, we can use a variadic tuple to represent a list of URIS and have Kind recursively build up the type:

export type KindURI = [URIS, ...URIS[]]

export type Kind<F extends KindURI, C, S, R, E, A> = ((...args: F) => any) extends (
  head: infer H,
  ...rest: infer Rest
) => any
  ? H extends URIS
    ? Rest extends KindURI
      ? URItoKind<
          OrFix<"S", C, S>,
          OrFix<"R", C, R>,
          OrFix<"E", C, E>,
          Kind<Rest, C, S, R, E, A>
        >[H]
      : URItoKind<OrFix<"S", C, S>, OrFix<"R", C, R>, OrFix<"E", C, E>, A>[H]
    : never
  : never

our typeclasses and instances change accordingly:

export interface Functor<F extends KindURI, C = {}> {
  URI: F
  map: <A, A2>(
    f: (a: A) => A2
  ) => <S, R, E>(fa: Kind<F, C, S, R, E, A>) => Kind<F, C, S, R, E, A2>
}

export interface Bifunctor<F extends KindURI, C = {}> {
  URI: F
  bimap: <A, A2, E, E2>(
    f: (a: A) => A2,
    g: (a: OrFix<"E", C, E>) => OrFix<"E", C, E2>
  ) => <S, R>(fa: Kind<F, C, S, R, E, A>) => Kind<F, C, S, R, E2, A2>
}

export const functorOption: Functor<["Option"]> = {
  URI: ["Option"],
  map: (f) => (fa) => (fa._tag === "None" ? fa : { _tag: "Some", value: f(fa.value) })
}

export const bifunctorEither: Bifunctor<["Either"]> = {
  URI: ["Either"],
  bimap: (f, g) => (fa) =>
    fa._tag === "Left"
      ? { _tag: "Left", left: g(fa.left) }
      : { _tag: "Right", right: f(fa.right) }
}

export const bifunctorStringValidation: Bifunctor<["Either"], Fix<"E", string>> = {
  URI: ["Either"],
  bimap: (f, g) => (fa) =>
    fa._tag === "Left"
      ? { _tag: "Left", left: g(fa.left) }
      : { _tag: "Right", right: f(fa.right) }
}

function addOne<URI extends KindURI, C>(
  F: Functor<URI, C>
): <S, R, E>(fa: Kind<URI, C, S, R, E, number>) => Kind<URI, C, S, R, E, number>
function addOne<C>(F: Functor<["F_"], C>): (fa: F_<number>) => F_<number> {
  return F.map((n) => n + 1)
}

But now we can do:

export const functorEitherOption: Functor<["Either", "Option"], {}> = {
  URI: ["Either", "Option"],
  map: (f) => (fa) =>
    fa._tag === "Left"
      ? { _tag: "Left", left: fa.left }
      : fa.right._tag === "None"
      ? { _tag: "Right", right: { _tag: "None" } }
      : { _tag: "Right", right: { _tag: "Some", value: f(fa.right.value) } }
}

Without any reindex we have created an instance of a composed type. To prove it works we can use our addOne:

// <S, R, E>(fa: Either<E, Option<number>>) => Either<E, Option<number>>
const addOneEitherOption = addOne(functorEitherOption)

Apart from the excessive phantom types the signature is what we wanted (like before).

In addition, @effect-ts/core uses the C parameter to encode parameter variance and defines utility types to mix generic parameters which respect the annotated variance.

For that check the code at https://github.com/Matechs-Garage/matechs-effect!

This article code extracted to: https://gist.github.com/mikearnaldi/7388dcf4eda013d806858d945c574fbb

DEV Community: Michael Arnaldi

A bright future for Effect

Acknowledging the Past

Where Are We Now?

The Future

Immutability: what a monster...

Copy On Write

Persistent Data Structures

Conclusion

Corollary

What would be better?

The Case for TS+

Table Of Contents

Why TS+?

The Development of Fluent APIs

The Development of Pipeable APIs

Introducing TS+

Installing TS+

Using TS+ (Fluent Methods)

Using TS+ (Call Extension)

Using TS+ (Binary Operators)

Using TS+ (Lazy Arguments)

Using TS+ (Type Unification)

Using TS+ (Global Imports)

Using TS+ (Runtime Types and Derivation)

Using TS+ (Module Structure and Config)

Using TS+ (Compiled Output)

What's next for TS+

Encoding HKTs in TypeScript (Once Again)

Why do we need HKTs?

Let me tell you about

Single Parameter Encoding

Multi Parameter Encoding

Inference with TypeClasses

The End

Building custom DSLs in TypeScript

Final Encoding

Initial Encoding

Phantom Types & GADTs

Existential Types

Optimized Variant

End

Abusing TypeScript Generators

Early approaches

Enter the world of generators

Generator based do for Effect

Multi-Shot Monads

Generalisation

Bonus:

Stay tuned

The Effect Data Types: Managed & Layer

Bracket

Managed

Composing Managed Resources

Layer

The Has & Tag utilities

Stay Tuned

The Effect Data Types: Effect

The Effect Type

Snippets Location

Variance

Handling Checked Errors

Unchecked Exceptions

Handling full Cause

Environment

Has & Tag

Stay tuned

Effect-TS Core: ZIO-Prelude Inspired Typeclasses & Module Structure

Project Setup

Introduction to ZIO-Prelude's Type-Classes

Covariant

Monad

Applicative

Traversable

Identity

Foldable

Module Structure

Encoding HKTs in TS4.1

Generator based `do` for `Effect`