With my focus recently going back to look at optimization for the Solid 1.0 release, I thought I'd revisit my The Real Cost of UI Components article. When I wrote the original article I wasn't really sure what I'd find and I was a bit cautious not wanting to offend anyone. I let every framework have their showcase at level 0 and then just built on that.
The shortcoming of not equalizing the implementations is I didn't actually show the tradeoffs of the Virtual DOM and I completely glazed over the overhead of Web Components. So I wanted to look at this again with that in mind.
Why now? I recently have been benchmarking Stencil and the new Lit. And it was sort of bugging me since neither of these support Native Built-ins. This is a problem since with a benchmarks using
HTMLTableElements meant they can't just insert random Custom Elements. So these implementations were all done in a single large Component. I wanted to see if I could better approximate the way these scale.
Mandatory Disclaimer: I wrote Solid, but I did not create this benchmark. Take it for what it is. I hope that your takeaway is more than Solid is fast. Different technologies scale differently and that should be where the focus is.
The test is once again a modification of JS Frameworks Benchmark. This is our TodoMVC app on steroids. It will blast our implementations with some absurd data but we will quickly be able to see any bottlenecks.
The important thing to note is given the limitation of around Native built-ins we will be using hand optimized Web Component solutions. This means better performance than you'd typically find for Lit. So things are slightly skewed in its favor but it's the best I can do.
When I first started I did the tests on the new M1 Macbook Air but given the issues with applying CPU throttling(known issue) I also ran these on a Intel i7 Macbook Pro. This muddies the narrative a little but it can help view the difference between running on the latest greatest and on a slower device(via CPU throttling).
- Level 1: The whole benchmark is implemented in a single Component.
- Level 2: A Component is made per row and per button.
- Level 3: Each row is further subdivided into Cell Components for each of the four table columns and the remove Icon is also made into a Component.
2. Lit: Google-backed Tagged Template render library. Given the lack of support for Native Built-ins I'm using optimized hand written Custom Element wrappers. I also kept explicit event delegation in which is an advantage compared to every non-vanilla implementation. Source [1, 2, 3]
Instead of focusing a framework at a time I think it will be easier to just look at this in terms of levels. Relative positioning speaks a lot more to the trends. Since our baseline is moving with us by using Vanilla JS with Web Components, even though libraries are getting slower as we add more components by how much differs.
We are going to make heavy use of looking at the averaged geometric mean(the bottom row) to holistically look at how these libraries compare. It is important to look at the individual results for more information but this gives us an easy way to determine relative positioning.
One component/app is all you get. While for most libraries this is the most optimal version this is not true of the VDOM where components are really important for managing update performance.
This is probably the worst you've ever seen Inferno perform and it's not its fault. This is what would happen if everyone wrote VDOM code the way it is described in Rich Harris' The Virtual DOM is pure overhead. Hopefully most people don't do that. It actually isn't bad for most things but really takes a hit on the selection benchmark and where the updates are more partial.
This is what I'd consider the pretty typical scenario for a lot of frameworks in terms of the component breakdown. The VDOM now has enough components to operate.
Thanks to adding Web Components to Vanilla the gap between it and Solid has disappeared. Inferno is significantly faster now that it has enough components. The gap between Lit, Svelte, and Vanilla are keeping pace. So it looks like their components have comparable cost.
At this level every table cell is a Component. This breakdown might seem a bit extreme to some. In Virtual DOM land we are used to this sort of wrapping. Things like Styled Components and Icon libraries push us to these patterns without flinching. Just how expensive is this?
Adding Web Components to our optimal Vanilla JS has actually made it more expensive than the equivalent Solid example. Inferno has now closed the gap considerably with Vanilla JS. And Svelte and Lit have continue to drop a few more points. On the slower system, Svelte is really getting hurt at this point by it's memory usage on benchmarks like clear rows:
If anything this test was setup in Web Components favor. There is no Shadow DOM or extra elements inserted. Those things you would find in real world would make them even heavier solution. I didn't want any contention so I kept in things like explicit event delegation which only benefits Lit in this test. This is really the most optimistic look at Web Components.
But it is pretty clear that frameworks that scale with well with more components, such as Virtual DOM libraries like React or Inferno or "component-less" libraries like Solid, don't experience as much overhead.
This doesn't come as much as revelation to me this time around. But maybe by looking at a few numbers we can better extrapolate where we should cautious. This is just a brutal microbenchmark that only really shows us the framework level bottlenecks and the real ones happen usually in our user code. But for those looking to evaluate on pure technological approach maybe there is some value here.
Results in a single table Intel w/ Slowdowns