DEV Community: Jake Runzer

Babel Macros

Jake Runzer — Tue, 10 Dec 2019 00:00:00 +0000

Babel macros are a way to apply code transformations without having to install a new plugin for each transformation. They are implemented in a clever way and can be very useful. And best of all, they work out of the box with Create React App, GatsbyJS, and NextJS. This post will dive into what they are, how to use them, and how to make them using TypeScript.

Why?

Regular Babel plugins are great, but come with a few downsides.

They can be difficult for users to setup and configure, especially when using tools like CRA and Gatsby. CRA straight up prevents you from editing the config.
Plugins can lead to confusion as it can be unclear what in the code is being transpiled and what isn't.
They can conflict in non-intuitive ways. The ordering you define the plugins in is important and when the ordering causes a bug, it can be impossible to debug.

Babel macros to the rescue!

What?

Babel macros are possible because of the babel-plugin-macros library (which is itself a Babel plugin).

This library enables other libraries to create macros by

treating all imports that end in .macro or /macro as a macro.
providing a createMacro function that gives access to Babel.

But TypeScript?

If you use TypeScript you can still benefit from the power of macros. Instead of using the TypeScript compiler (tsc) to compile your code, you can use Babel with the TypeScript preset instead. This is what CRA, Gatsby, and Next are already configured to do.

Usage

If you are not using CRA, Gatsby, or Next and have full control over the Babel pipeline, you can enable using Babel macros in your project by adding the dependency (if you are using these tools, then you don't have to install or configure anything).

npm i --save-dev babel-plugin-macros

and adding it to your Babel config

// .babelrc
{
  "plugins": ["macros"]
}

Use a macro just by importing it like any other JavaScript dependency. For example, the count.macro.

import { lines } from "count.macro";

console.log(`This file has ${lines} lines`);

Development

Let's make a macro! The macro will place the contents of a file into a variable at compile time. A user will be able to use it like this:

import file from "my.macro";

const contents = file("my-file.txt");

This section assumes you have some knowledge of ASTs and how to manipulate them with Babel. The plugin handbook and AST explorer are great resources for learning or familiarising yourself with these concepts.

Caveats

There are a few caveats you have to keep in mind when developing a macro.

Macros are evaluated at compile time. This means that the code you write must be able to run in Node on whatever machine is using the macro. This means no import/export syntax... unless you compile your macro before using it (meta). In this example we will use TypeScript and compile with tsc, but you can just as easily write your macro in ES6 and compile it with Babel. Just remember, after updating your macro, remember to recompile.
They must be synchronous. If you do in file IO, make sure to use the sync version.
To enable type checking when the macro is used in TypeScript, you must lie about the type.

TypeScript

We are going to develop it in TypeScript. If you are not using TypeScript you can still follow along, just remove the type definitions.

Setup

First we need to install our dependencies. We will use yarn for this.

yarn add babel-plugin-macros
yarn add --dev typescript jest ts-jest @types/babel__core @types/node @babel/cli @babel/preset-typescript

If using TypeScript, create a tsconfig file. Something like this. Also, if you are using TypeScript you will want to add some definitions for babel-plugin-macros (the ones on DefinitelyTyped are a bit outdated at the time of writing this. There is a PR open to update them). You can grab the updated ones from here.

Next we can add some npm/yarn scripts.

"scripts": {
  "build": "tsc -p .",
  "test": "jest",
  "compile": "yarn run build && babel --plugins=babel-plugin-macros --presets=@babel/preset-typescript"
}

The compile script will first compile our macro, then compile an example file with Babel. This is really helpful when testing and developing.

Finally, we can create our macro and an example to test with.

// src/index.macro.ts

// TODO

// examples/test.ts
const x = 1;

Run our example with

yarn compile examples/test.ts

You should see the contents of the file output to the console. Awesome! We are now setup.

Macro Definition

A macro is created by calling the createMacro function from babel-plugin-macros. The macro must be the default export.

import { createMacro } from "babel-plugin-macros";

const myMacro = createMacro(/* ... */);
export default myMacro;

The first argument to createMacro is a function that takes an single parameter: an object with a references, state, and Babel field. This is the type definition:

export type References = {
  [key: string]: Babel.NodePath[];
};

export interface MacroParams {
  references: { default: References } & References;
  state: any;
  babel: typeof Babel;
}

export type MacroHandler = (params: MacroParams) => void;

export function createMacro(handler: MacroHandler, options?: Options);

`references`

This is the main thing our macro will use to manipulate the AST. References is an object that contains Babel NodePaths all the places your macro was imported. For example, if I have the following source.

import foo, { bar } from "my.macro";

foo(); // line 3
foo(); // line 4

const x = bar + 1; // line 6

References will be:

{
  default: [/* NodePath line 3 */, /* NodePath line 4 */],
  bar: [/* NodePath line 6 */]
}

This is important to note and tripped me up when I was first learning about macros.

The default export of your macro file is always called with references to all places it was imported and used. Even if the user imports a named export from your macro, the default export will still be called.

Since references contain Babel NodePaths, they allow use to manipulate the ASTs. This is what we will use below.

`state`

The state of the file being traverse. This is the second argument received in a visitor function of a normal Babel plugin. It will look something like:

{
  "file": {
    "declarations": {},
    "path": NodePath,
    "ast": Node,
    "code": string,
    "opts": {
      /* ... */
    },
    "scope": Scope
    // ...
  },
  "filename": string
  // ...
}

`babel`

Same as require("babel-core"). We can use it to get Babel types.


const myMacro = ({ references, state, babel }) => {
  const t = babel.types;
};

Our Macro

With the above knowledge, here is our macro.

import { createMacro, MacroHandler } from "babel-plugin-macros";
import * as fs from "fs";
import * as path from "path";

const myMacro: MacroHandler = ({ references, state, babel }) => {
  const t = babel.types;
  const currentFilename = state.file.opts.filename;

  references.default.map(({ parentPath }) => {
    if (
      t.isCallExpression(parentPath.node) &&
      t.isStringLiteral(parentPath.node.arguments[0])
    ) {
      const filename = path.resolve(
        path.dirname(currentFilename),
        parentPath.node.arguments[0].value
      );
      const contents = fs.readFileSync(filename, "utf8");
      parentPath.replaceWith(t.stringLiteral(contents));
    }
  });
};

export default createMacro(myMacro) as (filename: string) => string;

You can see I had to manually case the type to (filename: string) => string. This way TypeScript users will get type checking and autocomplete when using this macro. If my macro was a named export instead, I could type with:

export const file = null as (filename: string) => string;

Finally, we can change the example to:

// examples/test.ts
import file from "../lib/index.macro";

const contents = file("test.ts");

The tranpsiled code will be,

const contents = "import file from \"../lib/index.macro\";\n\nconst contents = file(\"test.ts\");\n";

Conclusion

Babel macros are a great way to add compile time transformations to your project. They are low effort to setup and already work if you are using CRA, Gatsby, or Next. There are already a lot of macros available (check out awesome-babel-macros), but I believe there is still a lot of untapped potential. They are also fairly easy to create, especially if you already have some experience writing Babel plugins.

TypeScript Friendly State Management

Jake Runzer — Tue, 29 Oct 2019 10:52:29 +0000

TypeScript allows you to write safer and more robust code, while also improving the developer experience with things like better autocomplete, jump to definition, and type inference. However, setting up and using TypeScript with state management libraries has notoriously been difficult. Things are getting better, but there is still room for improvement. In this article I go over the current state of using TypeScript with Redux, MobX, and Overmind, and discuss what an even better solution would look like.

Existing Solutions

There are several existing frameworks with partial support for TypeScript. In a lot of these cases though, TypeScript is an afterthought and getting it setup is cumbersome and painful.

Redux

Redux has detailed documentation on how to setup TypeScript, but like many other areas of Redux, there is a lot of boilerplate involved. Especially if you want to have async actions using libraries like Thunk or Saga.

Creating actions is one area where the amount of code you need to write for TypeScript is almost double the amount for JavaScript. Let’s take a look at the example from the Redux documentation.

// src/store/chat/types.ts
export const SEND_MESSAGE = "SEND_MESSAGE";
export const DELETE_MESSAGE = "DELETE_MESSAGE";

interface SendMessageAction {
  type: typeof SEND_MESSAGE;
  payload: Message;
}

interface DeleteMessageAction {
  type: typeof DELETE_MESSAGE;
  meta: {
    timestamp: number;
  };
}

export type ChatActionTypes = SendMessageAction | DeleteMessageAction;


// src/store/chat/actions.ts
import { Message, SEND_MESSAGE, DELETE_MESSAGE, ChatActionTypes } from './types'

export function sendMessage(newMessage: Message): ChatActionTypes {
  return {
    type: SEND_MESSAGE,
    payload: newMessage
  }
}

export function deleteMessage(timestamp: number): ChatActionTypes {
  return {
    type: DELETE_MESSAGE,
    meta: {
      timestamp
    }
  }
}

The types file is basically duplicated in the actions file. This means that for every new action you create, you will need to create a new constant, create a new action type interface, and create the action creator. All this and you haven’t implemented any actual logic. This is boilerplate. Type checking reducers is a bit better, except you need to manually type the action and return a value instead of it being inferred.

// src/store/chat/reducers.ts
import {
  ChatState,
  ChatActionTypes,
  SEND_MESSAGE,
  DELETE_MESSAGE
} from './types'

const initialState: ChatState = {
  messages: []
}

export function chatReducer(
  state = initialState,
  action: ChatActionTypes
): ChatState {
  switch (action.type) {
    case SEND_MESSAGE:
      return {
        messages: [...state.messages, action.payload]
      }
    case DELETE_MESSAGE:
      return {
        messages: state.messages.filter(
          message => message.timestamp !== action.meta.timestamp
        )
      }
    default:
      return state
  }
}

The above examples show the effort required to make TypeScript play nice with standard Redux. What if we want async actions? When using Redux thunk you will have thunk actions with the type:

ThunkAction<void, StateType, ThunkExtraArguments, ActionType>

Typing this throughout your codebase, even for smaller apps, makes things much more complicated than they need to be. In one project at Prodo we ended up with the following file:

import * as firebase from "firebase/app";
import { AnyAction } from "redux";
import { ThunkAction } from "redux-thunk";
import { State } from "..";
import { Database } from "../../database";

export interface ThunkExtraArguments {
  firebase: firebase.app.App;
  reactReduxFirebase: any;
  database: Database;
}

export type Thunk<R = any> = ThunkAction<
  R,
  State,
  ThunkExtraArguments,
  AnyAction
>;

export const typedThunk = <T extends string, R>(
  type: T,
  args: any[],
  thunk: ThunkAction<R, State, ThunkExtraArguments, AnyAction>,
): Thunk<R> & { type: T; args: any[] } => {
  (thunk as any).type = type;
  (thunk as any).args = args;
  return thunk as Thunk<R> & { type: T; args: any[] };
};

Even as someone involved in the project from the start, I struggle to understand at a glance what the code is doing. Onboarding employees to the project was difficult because they needed to learn all of this TypeScript overhead.

When connecting React components to the store, the most common pattern I’ve seen is using Props and EnhancedProps .Props is the type for props that will be passed by the parent component and EnhancedProps is the type for props that come from the connect function.

import * as React from "react"
import { connect } from "react-redux"
import { State } from "./types"

interface Props { /* ... */ }
interface EnhancedProps { /* ... */ }

const MyComponent: React.FC<Props & EnhancedProps> = props => (
  /* ... */
)

const mapStateToProps = (state: State, ownProps: Props) => ({
  /* ... */
})

export default connect(mapStateToProps)(MyComponent)

MobX

MobX is currently the second most popular state framework for the web. Up until recently, TypeScript support was very limited when using the inject function. However, the support has been much nicer since mobx-react version 6.0 when it started relying on React hooks.

Defining your store and actions is fully typed.

import { observable, action } from "mobx";
import newUUID from "uuid/v4";

export class Store {
  todos = observable<{
    [id: string]: {
      text: string;
      done: boolean;
    };
  }>({});

  newTodo = action((text: string) => {
    this.todos[newUUID()] = { text, done: false };
  });

  toggle = action((key: string) => {
    this.todos[key].done = !this.todos[key].done;
  });
}

export default new Store();

Observing part of the store in a component is accomplished by creating a useStores hook.

import { Store } from "./types"
import { MobXProviderContext } from 'mobx-react'

export const useStores = (): Store => {
  return React.useContext(MobXProviderContext)
}

and using it in a component wrapped with observe.

import * as React from "react";
import { useStore } from "../store";

const MyComponent = observer((props: Props) => {
  const store = useStores();

  return (/* ... */;
});

There are a few gotchas with this method, but they are well documented on the mobx-react website.

TypeScript support in MobX is much nicer than Redux, but there are other aspects of the library that make it not suitable for all projects, such as when you want time travel debugging and uni-directional data flow.

Overmind

Overmind is another library for managing state that offers a very minimal and friendly API. It is less popular than Redux or MobX, but has strong support behind it. It was developed in TypeScript itself so offers good support. The online editor CodeSandbox has even started to adopt Overmind, TypeScript being one of the main reasons.

There are two approaches you can use when setting up TypeScript for overmind in your project. The first is the declare module approach.

// src/overmind/index.ts
import { IConfig } from 'overmind'

const config = {
  state: {
    count: 0
  },
  actions: {
    increaseCount({ state }) {
      state.count++;
    },
    decreaseCount({ state }) {
      state.count--;
    }
  }
};

declare module 'overmind' {
  // tslint:disable:interface-name
  interface Config extends IConfig<typeof config> {}
}

The benefit of this is that all of the imports coming from overmind are typed to your application. The downside is that you can only have a single overmind instance in your app. Overriding types for a library might also make experienced TypeScript users a bit uncomfortable.

The second and more common approach is explicitly typing everything.

// src/overmind/index.ts
import {
  IConfig,
  IOnInitialize,
  IAction,
  IOperator,
  IDerive,
  IState
} from 'overmind'

export const config = {
  state: { /* ... */ },
  actions: { /* ... */ }
}

export interface Config extends IConfig<typeof config> {}
export interface OnInitialize extends IOnInitialize<Config> {}
export interface Action<Input = void, Output = void> extends IAction<Config, Input, Output> {}
export interface AsyncAction<Input = void, Output = void> extends IAction<Config, Input, Promise<Output>> {}
export interface Operator<Input = void, Output = Input> extends IOperator<Config, Input, Output> {}
export interface Derive<Parent extends IState, Output> extends IDerive<Config, Parent, Output> {}

In both of these approaches, you must type actions explicitly. Unfortunately when you are manually typing something, TypeScript inference is longer used and you have to manually specify the return types.

import { Action } from './overmind'

export const noArgAction: Action = (context, value) => {
  value // this becomes "void"
}

export const argAction: Action<string> = (context, value) => {
  value // this becomes "string"
}

export const noArgWithReturnTypeAction: Action<void, string> = (context, value) => {
  value // this becomes "void"

  return 'foo'
}

export const argWithReturnTypeAction: Action<string, string> = (context, value) => {
  value // this becomes "string"

  return value + '!!!'
}

Using your state in a component can be done by first creating a hook:

// src/overrmind/index.ts
export const config = {
  state: { /* ... */ },
  actions: { /* ... */ }
}

export const useOvermind = createHook<typeof config>()

And using it in your components

import * as React from "react";
import { useOvermind } from "./overmind";

const Counter: React.FC = () => {
  const { state, actions } = useApp();
  return (
    <div className="App">
      <h1>{state.count}</h1>
      <button onClick={() => actions.decreaseCount()}>decrease</button>
      <button onClick={() => actions.increaseCount()}>increase</button>
    </div>
  );
}

What we want

The TypeScript authors have done an amazing job making it fit into the existing JavaScript ecosystem. Community efforts like DefinitelyTyped work really well and allow you to type JavaScript libraries that were created before TypeScript even existed. However, libraries that were designed with TypeScript in mind from the start, offer a more seamless developer experience.

With that in mind, the following are some features we would like to see in a state management framework when using TypeScript.

Type Inference
Framework extensions are fully typed
The initial state is fully typed
Jump to definition works seamlessly

Prodo

Here at Prodo we have taken the ideas above and created our own state management framework. We believe it is a step in the right direction and will allow you to develop applications with the speed of JavaScript and with the security and developer benefits of TypeScript. In comparison to the libraries mentioned above, Prodo has an API most similar to Overmind.

Defining your state is as simple as creating an interface.

// src/model.ts
import { createModel } from "@prodo/core";

interface State {
  count: number;
}

export const model = createModel<State>();
export const { state, watch, dispatch } = model.ctx;

Your initial state is fully typed when you create the store.

import { model } from "./model";

const { Provider } = model.createStore({
  initState: {
    count: 0,
  },
});

This provider is a React context provider and can be used to wrap your root level component.

Actions can be defined anywhere and are fully typed. The following examples are possible by using the Babel plugin.

import { state } from "./model";

const increaseCount = () => {
  state.count++;
};

const decreaseCount = () => {
  state.count--;
};

Components are similarly typed

import * as React from "react";
import { state, watch, dispatch } from "./model";
import { increaseCount, decreaseCount } from "./actions";

export const App = () => (
  <div>
    <button onClick={() => dispatch(decreaseCount)()}>-</button>
    <h1>Count: {watch(state.count)}</h1>
    <button onClick={() => dispatch(increaseCount)()}>+</button>
  </div>);

The above code is from the current version of our framework. We are also experimenting with different syntax and ways of doing state management. A post describing this can be found here.

We have open-sourced Prodo on Github at github.com/prodo-dev/prodo. Please consider giving this repo a star if you like the direction we’re taking. You can also join our Slack community if you want to continue the discussion.