In 2017 Tom Dale, wrote Compilers are the New Frameworks. And he was right. In 2017 things were already heading that way and have only continued on that trend since.
If you look at the whole range of build tools we use every framework is enhanced by some build ahead process. And if you want to take it to its natural extent you might land on, as @swyx did in his article Language Servers are the new Frameworks, down to a language itself.
But there are more steps still to go on this path. This trend of UI Framework in JavaScript being a language goes back much further. Elm(2012), Marko(2014), and Imba(2015) are just handful. But fast-forward to 2021 and we have many more libraries in this space.
And that's why it's more important to familiarize yourself with compilation in JavaScript frameworks. To understand what they are doing and more importantly what they can and cannot do.
What is a Compiled JavaScript Framework?
Ones where end user code is run through a compiler to produce the final output. To be fair this might be a bit too loose but I want to show that the approach is a spectrum rather than a single target. The term most often gets associated with frameworks like Svelte or Marko where everything ends up getting processed. But almost all popular frameworks use some form of ahead of time(AOT) compilation on their templates.
The reason is simple. Declarative interfaces are easier to reason about when you have systems where the inputs can come from many points and propagate through many related or non-related outputs. Most of these compiled frameworks are an extension of their templating languages. So that is the most reasonable place to start.
While there have been a few approaches over the years in the compiled camp now there are two main ones that stick out currently. HTML-first templating languages like Svelte, Vue, and Marko, and JavaScript-first templating languages like JSX.
<section>
<h1>My favorite color</h1>
<div>${input.color.toUpperCase()}</div>
</section>
<shared-footer/>
HTML-first templating languages treat the source file like it is an enhancement of HTML and often will work as a perfectly valid HTML partial if used with pure HTML. Some of the earliest form used HTML string attributes for expressions, but most now use JavaScript expressions in their binding syntax.
export default FavoriteColor(props) {
return <>
<section>
<h1>My favorite color</h1>
<div>{props.color.toUpperCase()}</div>
</section>
<SharedFooter />
</>;
}
JSX provides HTML like syntax that can be inlined expressions in your JavaScript. You can view it as almost a different syntax for a function call, and in many cases that is all it is. But JSX is not part of the JavaScript standard so several frameworks actually leverage its well-defined syntax the same way HTML based templates do.
Optimizing Templates
A lot of the motivation for compiled frameworks has come from the desire to optimize these templates further. But there is a lot that can be done with the base templating language. They can be compiled differently for server and browser. They can serve as a means for feature detection to aggressively tree shake. And many frameworks use templating languages as way of doing ahead of time static analysis to optimize the the code that is generated for performance.
Most template-generated code is creation logic, whether it is a bunch of VDOM nodes or real DOM nodes. When looking at a template you can almost immediately identify which parts will never change like literal values in attributes, or fixed groupings of elements. This is low hanging fruit for any templating approach.
A VDOM library like Inferno uses this information to compile its JSX directly into pre-optimized node structures. Marko hoist their static VDOM nodes outside of their components so that they don't incur the overhead of recreating them on every render. Vue ups the ante collecting dynamic nodes reducing subsequent updates to just those nodes.
Svelte separates its code between create and update lifecycles. Solid takes that one step further hoisting the DOM creation into clone-able Template elements that create whole portions of the DOM in a single call, incidentally a runtime technique used by Tagged Template Literal libraries like @webreflection's uhtml and Lit.
// Solid's compiled output
const _tmpl$ = template(
`<section><h1>My favorite color</h1><div></div></section>`
);
function FavoriteColor(props) {
const _el$ = _tmpl$.cloneNode(true),
_el$2 = _el$.firstChild,
_el$3 = _el$2.nextSibling;
insert(_el$3, () => props.color.toUpperCase());
return [_el$, createComponent(SharedFooter, {})];
}
export default FavoriteColor;
With non-VDOM libraries, like Svelte or Solid, we can further optimize for updates as well since the framework is not built on a diff engine. We can use the statically known information like attributes and directly associate template expressions with them, without necessarily understanding much about those expressions. This is basically loop unwinding. Instead of iterating over a list of unknown properties we compile in the inline update expressions. You can think of it like:
if (isDirty(title)) el.setAttribute("title", title);
We can even make some further assumptions from the input data in some cases. For example, Solid's compiler knows that simple variable bindings are not reactive as the tracking system relies on getters. So it can choose not to put that code under the update path.
There are still limits to what can be analyzed ahead of time. Spreads have to fallback to runtime approaches as do dynamic components like Svelte's <svelte:component> or Vue's <component>.
The other dynamic parts like loops and conditionals are always done at runtime in every framework. We cannot diff at build time. We can just narrow down the possibilities for the runtime. But for things like managing lists there are no shortcuts. Their reconciliation methods make a up a good chunk of the pulled in runtime for any framework. Yes, even compiled frameworks have runtimes.
Beyond Templates
Now it is arguable when you have Single File Components if you shouldn't view the whole file as the template and a library like Svelte or Marko basically treats it as such. There are certain assumptions that can be made when you know that your file represents a single component.
In the case of Svelte this determines the reactive tracking boundary. All reactive atoms declared within a file on change tell the component to update. In so Svelte can basically compile away their reactive system, removing the need to manage any subscriptions, by simply augmenting every assignment with a call to update the component ($$invalidate).
// excerpt from Svelte's compiled output
function instance($$self, $$props, $$invalidate) {
let { color } = $$props;
$$self.$$set = $$props => {
if ("color" in $$props)
$$invalidate(0, color = $$props.color);
};
return [color];
}
This is relatively easy for static analysis since the decision can be made by looking at where variables are defined in the scope and update all places they are used. But this is much harder to do automatically when these reactive atoms need to come outside the template. Svelte uses a $ naming convention to signify the stores so the compiler can know how to setup subscriptions.
A similar local optimization is how Marko looks for classes in their components to know if they are stateful. Depending on what life-cycles are present on them and the types of bindings being used in the template you can determine if these component need to be sent to the browser or only include them on the server. This simple heuristic with some bundler magic makes for a simple approach to Partial Hydration.
Both of these approaches use specific syntax to denote understanding the nature of their state. Their data has become part of their language. While not enforced, have you ever wondered about the potential value of the use prefix on React hooks?
Beyond Modules?
The biggest limitation to compilation is the scope of what it can reasonably analyze. While we can do tricks to inform the compiler, like Svelte's $, we tend to not see beyond import statements. This means we have to assume the worst when looking at what inputs come into our components (is it dynamic?). We don't know if children components use our stateful data in dynamic manner.
This hinders our ability for efficient composition. We need to fallback to usually different runtime mechanisms to fill this gap instead of leveraging the compiler's strengths. What if you could tell how a piece of data could affect the whole app at compile time?
So, for the most part we focus on local optimization. However, bundlers and minifiers get to work with final output code. While there is a lot we can do ahead of time to generate output that plays nice with their ability to optimize, at a certain point compilers will want to get in there too.
What we are doing through specific language is better understanding the developer's intent. Especially with heavy use of declarative constructs. This information is useful at all stages. This is something that is harder to do with general purpose programming languages.
Conclusion
We are just scratching the surface of compiled JavaScript frameworks, but the techniques that we associate with pure compiled frameworks are working their way into others. For example, Vue has been exploring new data-level language in their Single File Components. And it is easy since the groundwork is already there.
The approach(HTML-first vs JS-first) each Framework takes to templating is mostly a superficial differentiator. There is very little meaningful difference here. But the devil is in the details when it comes feature support. Every framework has places where they have no choice but to lean heavier on their runtimes and these boundaries are commonly crossed in any significant application. So even code size isn't a clear benefit.
Where compilation excels is abstracting the complexity. From simpler syntax to interact with data and updates, to specialized output for server versus browser. This is a DX tool much like Hot Module Replacement on your bundler's Dev Server. It feeds into better IDE support since the program better understands your intent. And it also can bring performance gains.
Today, the biggest limitation to compiled approaches is that they are module scoped. If compiled approaches want to scale like runtime approaches this is a hurdle we will have to overcome. For now hybrid approaches might be the best solution. But even today, compilers are capable of so much it's hard to picture a future without them being a significant part.
Top comments (19)
One thing that worries me about this development is that compiled frameworks seem to phase more or less general-purpose compiled languages like ReScript or PureScript or ClojureScript. Sure, both Svelte and SolidJS support TypeScript, but that requires effort on the framework maintanters' part, so how likely is it that, say, Solid will ever support ReScript? Sure, you can somehow marry the compiled framework and an AltJs language (I think in ReScript's case you could even add genType to generate TS, but it's still not a perfect setup: Svelte components just consume ReScript functions, so there's no ReScript features in UI: no concise sum types with exhaustive pattern matching, no runtime currying or pipes.
So the whole situation only reinforces the TS monopoly. And it's not that TS is an awful language, but any monopoly leads to stagnation. And also I think there are still better languages than TypeScript, so the whole situation leaves you with the choice between a modern compiled framework with TypeScript and a nice business-logic friendly compiled language that wraps good old React (ReScript, PureScirpt, ClosureScript), or Preact (Mint), or its own VDom-based implementation (Elm). Or, if you're desperate enough, you could try to convince the language maintainers to alter their JSX compilation to support Solid 🤔
The latter is what we've been looking at with Solid. They support React. Honestly if we could just
preserveJSX equivalent. Or even HyperScript can be converted back to JSX is something looked at. All that being said still requires some amount of language support. This is the active discussion: github.com/solidjs/solid/discussio....It's a tricky one though. TS was hard enough to support for language features. It still restricts what we can do with JSX to this day. I think at least in Solid's case the solution is making the JSX types more adaptable. If ReScript had the same sort of flexibility around JSX like TS does there would be no pause to support it.
I'm not sure it's possible for ReScript compiler to leave JSX as is, for several reasons. For instance, at least from the typechecking perspective, every rescript-react component has a
makefunction (that takes a props object) and amakePropsfunction with named arguments. Alsobarin<Foo bar />in ReScript is a shortcut forbar=bar, not forbar=true.So I guess it all requires a bit of effort on the teams part. Here's hoping they're going to find time for it at some point. There is some interest, and I hope SolidJS gains some traction and the interest will only increase in the coming months and years.
Svelte is extensible enough that if you wanted to, you could add support for ResCript via a preprocessor: svelte.dev/docs#svelte_preprocess.
Yeah mind you does this work in the rest of the template? I read this as a way to process blocks. I know like Riot and Knockout used to also support setting the language in the template. Like in the expressions but on quick read I didn't see that.
I imagine it is possible for JavaScript based framework to import Rescript files and if they aren't working on custom Single File Component formats then they can get the full power of it without any preprocessor. But getting the syntax inside the template I think is the more interesting challenge. JSX feels like it could be that bridge with the right support.
Or an alternative approach is to import external res files into your svelte components: github.com/sabinbajracharya/Svelte...
That's always a possibity. You can even generate TypeScript types from ReScript. I was also thinking of using ReScript for domain logic and TypeScript for (React) JSX, because TypeScript has less ceremony for conditional rendering. But that scenario is not perfect. For one thing, if ReScript is your language of choice, not being able to use it in your views is not much better than just using templates with a DSL for loops and conditions. No big deal, but I'm already spoiled by the flexitibily of JSX. Also, having to compile (or at least typecheck) both ReScript and TypeScript means maintaining more dependencies. Also, TypeScirpt error messages are kinda meh. Also, context switching. So, all things being equal, I think I'd prefer to use less languages.
I always learn from your posts, thank you for continuing to contrast and compare approaches and libraries. For those of us who don't write libraries, but consume them, it helps us to think like the library creator and understand why decisions were made and avoid silly mistakes.
What do you think about eg Rust+WASM?
One day there probably will be something really powerful there. By that time will JavaScript be even more ubiquitous, and the reasons justifying it need to be greater? Maybe.
I think one of the interesting things is how much goes into what I describe in the article on the JavaScript side of things. I do a lot of benchmarking and we all know that hand optimized vanilla code is going to be the fastest. Rust+WASM has been closing the gap to the point now it is in dead heat in a JS targeted benchmark like the JS Frameworks Benchmark with a library like Solid.
However, that is low level vanilla code. We all acknowledge there there are more unlocks for WASM so I expect the gap to disappear with Vanilla JS and maybe even be quicker one day. However, the fastest WASM declarative libraries are much slower. Even ones using fine-grained reactivity. Now a portion of that is the same overhead I was referring to before that will be going away, but how much of it is because in JavaScript the template code we output doesn't even necessarily resemble what we input. We've added a layer to compile away the declarative code into more performant optimized code. By comparison I wonder how much longer before things built in Rust will follow suit. Will they ever be so inclined?
Size is also a real consideration in the browser. Every WASM solution I've seen is significantly larger. Maybe comparable to the larger JS frameworks like Angular or Ember but compared to say Svelte or HyperApp they seem monstrous. Something has to change in the way we consume/architect websites for this to be a reality.
So I think right now WASM delegated to special purpose things where high performance is necessary. Places where you can defer loading. Can we break free of that? Not sure. I watch WASM with great interest, but for different reasons. It seems as likely or more likely we see more performant server code written in JavaScript(TypeScript) than we get from Node today. Deno hints at it but I'm not sure that is the end of the story. Serverless WASM binaries seem pretty attractive as a deployment artifact for ecosystem flooded with JavaScript users.
EDIT: Speak of the devil this was posted today: bytecodealliance.org/articles/maki...
Thanks for the very comprehensive response, and link. I didn't fully appreciate the issues with size, either.
What do you think about framework that mostly render in webgl/webgpu?
(e.g Flutter, PCUI, ...)
I mean as long as the devices support them (which I imagine they do) there is a clear performance benefit here. I think there is concern that any work in this way to do general UI would involve basically re-inventing the core interaction points of the browser that we are used to. Things like accessibility etc.. I imagine any approach to do this in a general way basically would be building a DOM on top these technologies. You want declarative UIs etc... I think the biggest gap for them might simply being that they aren't the native implementation so there is always more work or code needed to run these. It is possible albeit unlikely at this point that a different standard could immerge.
I wanted to do some benchmarking with Flutter but the abstraction makes it hard to do apples to apples comparisons except with different versions of Flutter itself.
Hey @ryansolid just wanted to thank you for all your write up on frameworks, I read all of them and learn a lot every time, I never comment but wanted to say thanks !
thanks for the shoutout Ryan! yeah i agree that the next frontier is whole-app compilation. this is ironically something that only React is even starting to think about with Server Components.
Yeah that article came out shortly after I joined the Marko team, and having not being subject to it I was like, well there is always JSX. But seriously once I started working on Marko so much of it comes down to the language, IDE support, formatting, templating hints and errors. It's so much about building that experience.
Whole app compilation is something the Marko team at eBay was looking into the past couple years and why I was so excited to join up. It might be a lesser known framework/player but it has been great working through these sort of problems and coming up with solutions. We've been creating a system for cross module analysis that we've been using to inform compilation and are ironing out using it to do compile away reactivity similar to what Svelte does on a per file basis.
What is really interesting to me is different projects seem to be working on different parts of it to see what works. There is React Server Components. There is Astro, probably the first to get progressive hydration working on top of islands. Things like Prism have optimized localized compilation and hydration better than I've seen anywhere else. Even projects like Builder.io's Qwik which are less on compilation but more on the mechanical aspects of ultra granular progressive hydration, being built by both the creators of Angular and Stencil who left those projects to develop this.
It's crazy to think this far in things are still developing at this rate. But this is why I love web dev.
I agree compilers are the future. Nice read very informative.
Same here. I think Virtual DOM is a nice episode which gave foundation (it popularised) for declarative UI development. Since there are already build-time diffing tools, we have a proof it can be done. And since we transpile our JS/TS anyway, it feels obvious to expect the tools to spit out the most performant code that doesn't require any runtime housekeeping like DOM diffing.
Obviously, it's more complex and there are times when runtime diffing can be useful e.g. rendering dynamically generated code in web based code editors. And I don't know how it's in Svelte, but Solid comes up with HyperScript version which gives that "edge-case" alternative.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.