Converting TypeScript decorators into static code!

(originally posted on Medium)

TL;DR; 🤯

Decorators are cool, and they seem like a great way to write powerful APIs. Unfortunately, they have a few pretty big drawbacks:

• They’re still unstandardised and experimental 🔬. That means they’re likely to change in the future.

• They can also mess up tree-shaking 🌳, which is a big deal when you’re writing a library.

The Angular framework heavily relies on decorators, but at build-time they compile them into static code to avoid these issues. We can do the same with our own decorators by using TypeScript transforms!

Want to know how? Read on… 🤓🤓🤓

---

This post is the story of my adventures with custom decorators. It touches on some Angular stuff, but is definitely not specific to that framework!

1. First, I’m going to cover what a decorator is and why we’d use them.

2. Then, I’ll tell you about how I got super excited about decorators, and what I thought was a good use case for one.

3. Next, we will look at some of the issues I discovered, and why they may not be a good idea right now.

4. Finally, we’ll go into some different approaches for dealing with the issues, and how you can implement them!

It’s a bit of a long one, with as many new questions as there are answers, so buckle in 🤞, maybe get a nice piece of fruit 🍏, and enjoy! 🎉

What is a decorator?

Decorators are pretty fascinating, and very powerful! They allow us to treat our code as data, and to change the behaviour of our code at runtime. If you’ve used Angular, then you’ve almost definitely used decorators in your code. They might be first-class decorators like @Inject() or @ContentChild(), or they could be from a library like @Effect(). So what do they do?

Let’s look at how we might decorate a class in current JavaScript. You can fake it by passing a class as an argument to a function:

n this example, we’ve got a function that modifies the behaviour of a class by adding a new static property. We’ve extracted common behaviour to this function, and we can apply that behaviour to any class that we like. A real-world use-case could be to add logging for when someone instantiates the class, or for making sure that the class is a singleton.

If we want to decorate a property or a method, we have to jump through a few more hoops. We need to use Object.defineProperty:

This works, but it requires a lot more code, and isn’t the easiest to read. We would like to be able to do what we did with the class earlier, and pass the property or method to a function. We can imagine that it might look like this:

This is invalid with current JavaScript, but you get the idea! Something like this would be a better way to wrap the creation of the property or method and change its behaviour.

That bring us to the decorators proposal. The idea is to change the syntax of the JavaScript language to allow us to do something like the above. TypeScript already has an experimental implementation of decorators, using the following syntax:

The point of this article isn’t to go into the nitty-gritty of decorators. If you’re curious to find out more, check out this article by Addy Osmani

I love this syntax! As a developer, it’s a nice way to attach new meta-behaviour to my code. Other library authors have thought the same thing. Angular, NgRx, and other libraries use decorators for their APIs, even though they are experimental.

As it turns out, I had a great reason to try them out! 😍😍😍

The perfect use case

Around the time when I first found out about decorators, I was working at Trade Me in New Zealand. We were upgrading our component library from AngularJS to Angular, and having trouble with component APIs. Boolean attributes weren’t working how they used to. Check out the following example:

As far as I can tell, this doesn’t match up with how the HTML specification works, or with the Angular documentation. I’m sure there are good reasons, but either way, it messes with my head! I’d prefer it if you didn’t have to use the [] syntax to bind to a property to get the expected behaviour of a boolean input. One way to solve this would be by using get and set on the property, and fiddling with the values:

And now it works!

It’s great that we can get this to behave, but we have had to add quite a bit of code to our inputs. This is actually the approach that Angular Material takes with their components. I thought it would be a much nicer developer experience if we moved this behaviour to a decorator.

Turns out, it was fairly easy to do:

And even easier to use as a consumer:

This is pretty great! We now have a generalised piece of behaviour, with hardly any code. We can use the decorator on all the boolean inputs of our components:

As we upgraded the component library, we actually took this a step further and created the @trademe/ensure library. Instead of having a specific isBool decorator, there is a generic @Value() decorator. It lets you apply any number of guards to an input:

We used this library on most of the component library, and it seemed to work well. You can pass through any guard function that you like, and we used it to handle null checks and type casts. We also use it to check specific requirements like isLessThan5 or isMutuallyExclusive. The developer experience and expressiveness are pretty great! But all was not sunshine ☀️ and rainbows 🌈…

Just a few little issues

The first big issue with decorators is that they have not been standardised yet. They could change drastically by the time they get standardised, or not get standardised at all. I’m sure the TypeScript team (or the Angular team) will help us migrate at a language or framework level. But doing it for all custom decorators could be problematic.

Unfortunately, it gets even worse… 😕😕😕

As our application grew and we used Angular’s lazy-loading features, we noticed we had a big problem. No matter what we did, we always ended up with the whole component library in our main bundles. Whenever we built our application, the tree-shaking and optimisation steps weren’t working. The more I dug around, the more I came to suspect an uncomfortable truth:

Decorators stop tree-shaking from working

⚠️ Disclaimer️ ⚠️:
I’d like to mention that this suspicion is partly based on rumour, vague guesses, and speculation. I don’t have any concrete evidence that this is the case, just the occasional hint from comments on GitHub issues, or conversations on Twitter.

When creating a library, it’s important to consider how tree-shakable your library is. Tree-shaking is an optimisation that excludes unused library code from production JavaScript bundles. Tree-shaking analyses the import and export statements of some code, and removes unused exports. It can only work if the tool is confident that your code doesn’t have any side effects. I don’t fully understand this, but as far as I can tell, it is impossible to statically determine if a decorator is a side-effect or not. You can read more discussion about this here.

During my hunt for more information about this topic, I came across this issue. It is a proposal for removing the @Effect() decorator from NgRx. One comment in particular stood out. It said:

“Decorators have been irresistible for framework authors. Meanwhile, as maintainers of the TypeScript infrastructure at Google, we don’t like decorators.”

and

“Angular has decided to handle them through the AOT compiler”

The first point there is already interesting, but it seems to be based on the same worries about the experimental nature of decorators. But the second point is particularly interesting. What does “handle them” mean…?

Handling them

We might be able to get some clues 🔍 from how the TypeScript compiler deals with decorators. First, here’s a stripped-down Angular component, which also contains a non-Angular decorator:

If we run this code through the TypeScript compiler, we get the following:

Now this, is pretty difficult to read. The important thing to note is that we have two calls to the __decorate function. This is how the generated TypeScript runtime handles decorators. In this case we can see that the Angular decorators, and the custom decorator get treated the same.

We can compare that to what happens if we compile the TypeScript code through the Angular CLI:

Huh. This time we only have one call to __decorate, for our custom @Value() decorator. The other decorators have turned into static properties on the class.

This is how ngc, the Angular compiler, handles Angular decorators. ngc is a wrapper around around tsc (the TypeScript compiler). This wrapper layer handles many things, including converting TypeScript decorators to static code. This is what we see in the code examples above. ngc converts code that could contain side-effects into code that definitely doesn’t. That means better tree-shakabilily. This extra layer is a bit magical, but the Angular community has adopted it without much concern.

Can we do something similar? 🤔🤔🤔

Transforming our decorator

Let’s have a look at how we can do the same thing as the Angular compiler. We want to take our expressive decorator-heavy code, and transform it into something else at build time. Lucky for us, we already have the code we want to end up with! From our first attempt at fixing the boolean inputs:

This combination of getter and setter does the boolean coercion for us. We moved that behaviour to a decorator to give us a nicer developer experience. Can we come up with a generic way to transform our @Value decorator?

TypeScript actually makes it pretty easy for us, with transforms. We can use a transform like this:

The important bit here is the transform function. It allows us to take a representation of a TypeScript file (the SourceFile), and run a transformer over it. Let’s break that down a bit.

• SourceFile: the Abstract Syntax Tree (AST) represents the structure of the code we want to change. The AST is a tree of Node objects.

• Node: A representation of a small syntactic part of the source code. Some examples of nodes are Literal, FunctionDeclaration, Decorator, BinaryExpression.

• Transformer: a function which traverses the AST and can inspect the nodes. If the transformer returns a different node than it was give, the new node replaces the original node.

I like to use ASTExplorer to look at the AST of any code I’m working with. Let’s take a look at what we have:

And here’s what we’d like to end up with:

Now that we know what we’re dealing with, let’s start building up our transformer. We start off with a very basic structure. We have a TransformerFactory which creates new instances of the Transformer. Our transformer visits each node, then visits all its child nodes. It always returns the original node, so no actual transform will take place.

Next, we need to introduce a filter so we can find only the specific nodes we care about. We’re going to use tsquery to do that, which you can read more about here. It works like this:

Next, we need to create the nodes that we’d like to insert back into the AST. The TypeScript APIs for creating nodes tend to involve quite a bit of work, so we’re going to use tstemplate. It is a TypeScript port of estemplate which aims to make creating AST nodes easier. Let’s add it to our transformer:

We first have a template (lines 9–21) for the new code we want to generate. It uses the tstemplate interpolation syntax, <%= nodeName %>. Then we use pass the values we want to the template, which generates a new, valid AST. Because properties and accessors cannot exist without a class, we have to include the class in the template! We then use tsquery again to access the PropertyDeclaration, GetAccessor, and SetAccessor nodes.

At this point we’re actually most of the way there. We’ve removed the old PropertyDeclaration, and replaced it with the new private property, and a new public getter and setter. Running this transform over our component would give us this:

We’re not quite done though, there’s still a few things missing 🚧. We’ve removed our @Input() decorator, and we’ve lost the actual isBool and isNotNull magic. The first part of this is straightforward. We need to select all the decorators that aren’t the @Value() decorator, and move them over to the GetAccessor:

The last bit is more nuanced, and we need to go into the implementation details of @trademe/ensure. The library gives us the ability to set whether a particular check will happen in the get or in the set. The main example of this is isNotNull. We only want to check if the property value is not null during the getter. The setter runs all the other built in checks. Our transform needs to handle both these cases.

There are four important things happening here.

1. We take the decorator node and use tsquery to find all the arguments. These are the identifiers for the guard functions.

2. For each of the found identifiers, we use tstemplate again to generate AST nodes. We are also generating the propertyName string literal that the original decorator expected. It is extra code, but it also gives us better error messages.

3. We split the generated nodes into getters and setters.

4. We insert the generated nodes into the function bodies of the new get and set accessors.

Tada! 🎉🎉🎉

And that’s it! If we run our transform over the Component from before we get the expected result:

The one thing we haven’t done is to tidy up the Value from our list of imports, as it is no longer necessary. I’ll leave it to you to think about how you’d do that.

The completed transform looks like this:

Take a few moments to digest all that. A few queries, a few templates, and a little bit of logic. I reckon it’s not too bad for 83 lines of code, and shows you how powerful AST transforms can be!

What do we do with this?

At this point, we’re left with a bit of a philosophical decision. There are two clear options:

1) Do we run this transform once, stop using the decorator, and not use it again? We would lose the nice developer experience, and end up with quite a bit more source code. The code becomes much more obvious, and easier to understand.

2) Do we take Angular’s approach, and insert this transform in the build process? This way we get to keep the expressiveness of the decorator syntax, and get terser components. The cost is that there’s quite a bit of magic there to understand.

As per usual, the answer to this is “it depends”. All these things are trade-offs.

Generating this code once and moving on seems like a straightforward way to get away from the custom decorators. As I mentioned earlier, they’re still experimental, and thus risky. Is slightly more readable, terse code worth that risk? I’m not sure.

On the other hand, it’s entirely possible to insert the transform into a build process, keep the nice developer experience, and mitigate the tree-shaking issues.

It all comes down to who is going to be maintaining that build process. It is quite a bit more challenging to write the transform than to write the original decorator. If that is something that a library author wanted to take on then it seems fine. It could be an idea space for something like the Angular CLI to provide the ability to insert transforms like this one? I understand that there has been chatter about moving the coercion helpers into Angular itself. That seems like a good solution to me.

I left Trade Me before a decision was reached, so I don’t even know what happened in their situation! It might even still be up in the air.

What do you think about the trade-offs? Please reach out and let me know!

Phew! 😅

Okay, that was a big one. While this is a pretty specific story about a pretty specific situation, I hope it will encourage you to think about how you can use techniques like this to automate refactoring, or to make development easier.

To recap what we covered:

1. TypeScript decorators are cool, but still experimental and not standardised.

2. They give us a new ability to write nice APIs to modify how our code works at runtime. One example is to change how getting and setting an Angular Input works.

3. Decorators come at a cost, and sometimes break tree-shaking when they’re used in a library.

4. The Angular team gets around that problem by transforming their custom decorators.

5. We can write our own TypeScript transforms using tools like tsquery and tstemplate!

6. Theses transforms are pretty magical, but also very very powerful! It’s up to you to figure out when they’re appropriate.

That’s all I’ve got, I hope that was interesting and somewhat useful. If you got this far, thank you ❤️. Please get in touch and give me some feedback!

Huge shout out to Kevin Cartwright at Trade Me, who paired with me on this in the weeks before I left 👋👋👋

Comments

[Joe Pea]

Now what happens if you want to export your decorators and someone with a non-TypeScript JavaScript project was hoping to import and use your decorators?

[Craig ☠️💀👻]

I guess you do what Angular does, and ship the build tools with your library? 😄