DEV Community

Cover image for Resumable JavaScript with Qwik
Ryan Carniato for This is Learning

Posted on • Updated on

Resumable JavaScript with Qwik

When Misko Hevery (creator of AngularJS) approaches you to look at his new framework, well, you stop and listen. I was already aware of Qwik and saw the potential, but this was as good as any to stop and take a closer look.

Qwik is a unique JavaScript framework in that it is the only one that exists today that can hydrate out of order at a component level. It goes beyond that though Qwik introduces a new concept to the vernacular of JavaScript: The Resumable Framework.


Resumable Framework?

Image description

These days it is common for our JavaScript Frameworks to be isomorphic. That is to be able to render on the server and in the browser. But for most frameworks this ability was tacked on afterwards. A natural extension to their client side paradigms. But what if the framework was built on SSR in the first place?

Well before Qwik we've seen surprisingly few. Meteor, Marko, maybe a couple others. However, the modern isomorphic landscape is built on the back of libraries like React, Vue, and Svelte that weren't initially created for server rendering.

So unsurprisingly their core mechanics weren't designed in a way to leverage this information. If you knew that your app was always going to be rendered on the server first what sort of concessions could you make?

The most powerful one might be not redo any of the work in the browser that was already done on the server. A JavaScript framework designed to do less work in the browser. Not the first framework to do so. But perhaps the first to realize the idealized hydration execution.

Hydration is the process of adding interactivity to server rendered HTML. It involves attaching event handlers, and initializing application state. For most JavaScript libraries this is a top-down process very similar to re-rendering the whole app again in the browser, even if it doesn't actually create any new DOM nodes. For a more complete guide on Hydration see Why Efficient Hydration in JavaScript Frameworks is so Challenging.


The Journey to Resumability

Image description

Creating a hydration approach that does not redo work in the browser is not an easy path. You don't simply pick up your existing Single Page App framework of choice and get here. We've been working on the same problem in Marko the last couple years and although the approach different it does really come down to a few key things.

  1. Ability to break apart the code needed for hydration (events, effects), from code needed to render the view and manage stateful updates.

  2. Understanding what data is stateful(can update) and what depends on it. To resume work it must be done at a more granular level than the components as re-running components during hydration would be unnecessary work.

  3. Serializing sufficient data so that unrelated changes don't require re-calculation and so that that parts of the app can be hydrated independently and out of order.

Some frameworks do maybe one of these but almost no frameworks do all three. Qwik accomplishes this through following rules on Component authoring, reactive primitives(similar to React Hooks), and the use of a compiler leverages markers in the JSX to indicate how code should be broken apart.

Image description

Notice the $ indicators to separate component code from view and from event handler in this header section of TodoMVC.


What about Lazy-loading?

Beyond being resumable one the most standout features of Qwik is its progressive hydration. It progressively loads the JavaScript as needed. It can start with 0kb of bundled JavaScript component code and scale up to what needs to be on the page no matter how interactive it is.

Qwik does this very differently. Others that seek to solve this problem have used knowledge of what is server versus client to make choices. Those solutions rely on islands, special file extensions, or even advanced compiler analysis. From where I'm sitting this is the 80% of the problem to solve. Most pages are mostly static once you remove async data loading and routing considerations. But what if the page is incredibly interactive? What if most of the page could and would be loaded in the browser?

Well, in that case progressive hydration may be the only way to get a responsive page during initial load. And it isn't as simple just delaying the inevitable. That just pushes the full cost to when the user first interacts with the page. No. What makes Qwik interesting is those same qualities that allow it to be resumable also allow any part of the page to hydrate independently.

Yes. That button half way down the page can load the necessary code to add an item to your cart before any JavaScript higher up in the hierarchy is loaded. This isn't how typical frameworks work. If you have components that contain other components and pass props throughout things need to run top down.


So Problem Solved?

Image description

Well, maybe. But probably not in the way you are picturing. Understanding what I explained above I thought it would be fun to come up with a demo really showcase these unique features of Qwik. I mean picture it:

Writing your typical single page application(SPA) with JSX and reactive data like you've grown accustom to, except when the page loads barely any JS loads. And when you scroll down a bit you find something that interests you and the JavaScript just for that section loads and works. Content with that you click a link, suddenly the client side router loads, and client navigation takes over. Seamless SPA experience with perfect on demand hydration.

Until you realize that when you navigate to that new page you are loading routing information about the whole app and you are suddenly loading dozens of new mini JS files to render the whole page in the browser. At first you go, maybe this isn't great. But then you think well we can do something smarter here with bundling. And Qwik is looking into some smart approaches with bundling. But this goes beyond that.

For a framework that optimizes everything to reduce JavaScript in the browser why would you even want to render the whole next page in the browser?

Well, you wouldn't. And that's when everything starts to really make sense. Evaluating Qwik on the merits of existing frameworks is pointless. It seems like Qwik is the panacea for React's bundle size, but really it is entirely something else.


It's a Brave New World

Image description

So what is Qwik? It's everything I've mentioned in this article. It's a framework that is optimized to do the least initial work in the browser regardless of app composition. More importantly it suggests the potential of a new paradigm in how we build apps in the frontend. Not one that just transitions from SPA but is built entirely with getting the most from the server in mind.

It's still relatively new. A lot of features are undocumented. And there are problems that still need solving.

It clearly benefits from classic server multi-page app routing to persist its experience even as we move to new pages. When going to new locations server rendering allows Qwik to continue to send no JavaScript by default. I anticipate that we will see more developments in this space in general to bring server rendered pages and partials without full page reloads.

Progressive Hydration is still a tricky problem to solve because it does have a cost. Critical interactions shouldn't be lazy-loaded and things should be loaded together in logical ways to prevent code split waterfalls. Qwik has an optimizer that gives you control over how things are bundled. In the future you will be able to feed your site analytics, how your users interact with your page, to inform bundling. Wild, I know. But part of the consideration of such an approach. You can play with the optimizer today in their online playground.

Data loading and serialization is still a consideration. Some other partial hydration solutions use the fact they know what is server only to only serialize the required data. Leveraging the fact they'd need to be passed in as props to top-level browser components can significantly reduce the double data problem (representing it both as JSON and the rendered HTML). Qwik doesn't innately have this knowledge but its approach to hydration isn't a limitation here. So it will be interesting to see what approach they take.


Conclusion

Having had the pleasure of making a couple demos now (Hackernews, JS Framework Benchmark) in Qwik I see the beginnings of a very promising framework. But it is also one a bit difficult to evaluate in the current environment because I feel we are yet to see the full picture. Not just because Qwik is still under development, but because the wider ecosystem and tooling hasn't really caught up to this shift. But that is only a matter of time.

In the meanwhile Qwik provides one of the most unique takes on how to solve the problem of too much JavaScript. Handling 100% Lighthouse scores without breaking a sweat. If you are in the market for minimizing your Time to Interactive and wanting to try something new, you could hardly make a better a choice.


Interest more in how Qwik works? Misko Hevery has written a great article series on the topic:

Discussion (17)

Collapse
fractal profile image
Fractal

Great article! A few points I’d like to challenge:

But what if the framework was built on SSR in the first place?

Sveltekit is a framework designed from the ground up with SSR at its core! I think that’s important to note considering the statements following that quote.

But perhaps the first to realize the idealized hydration execution.

I thought this was called Partial Hydration - and not really a new thing. Astro has been pioneering this with their Islands Architecture. The Svelte community has had this for years with Elder JS.

Or perhaps I’m misunderstanding the difference?

Collapse
ryansolid profile image
Ryan Carniato Author • Edited on
  1. Sveltekit sure.. but not Svelte. I mean at the actual mechanical framework level. Like how they handle rendering and hydration. Not the extra dressing that the metaframework adds. Sveltekit is great but there are mechanical differences in Qwik based on this assumption of server first. Certain decisions not present in Svelte.

  2. Sort of. Partial Hydration has existed for a long time. The earliest trace I found is Marko from eBay back in 2013/2014 period about 6 years before anyone else. But even with common partial hydration today for the parts that are hydrated they still run top down. Basically the islands are like their own apps and follow the traditional flow. And you can't nest them (although you can pass "server components" in "client components"/islands through child projections).

What Qwik is doing actually changes how hydration occurs. It actually does less work at hydration time (only attaches event handlers), and it can break things apart beyond the island level. It can hydrate child before parent.

To be clear other frameworks (like Marko) are working on the same space but this goes well beyond what Astro or Elder are doing. If I sound critical I'm not. I think Astro and Elder are great, and I push Astro a lot with my UI library SolidJS. But it is incredibly difficult to take an existing framework like React, Vue, or Svelte and do what Qwik is doing. You'd need the core framework to change its behavior.

Collapse
mindplay profile image
Rasmus Schultz

Do you know about Opa?

opalang.org/

From 2011. 😄

This went way beyond just partial hydration - you basically just wrote programs, and the compiler would "slice" client/server concerns for you.

It was quite revolutionary at the time, I think - but for some reason was largely DOA with practically no one paying a lick of attention.

Years later there was Meteor, which has some of that, and suddenly everyone was like ooh yay isomorphic JavaScript wooh.

I think it was way ahead of it's time - and I actually loved the idea then.

In my opinion, these new tools don't go far enough. And I know that may seem like the opposite of what I was saying the other day, but the thing is, if there was a language that gave you all of this as part of the language, not just another framework with all the limitations of an existing language - I think that's cleaner and more interesting than mangling the semantics of an existing language to accomplish something that's not quite as good and will likely be superseded in another 6 months. 😄

Collapse
mindplay profile image
Rasmus Schultz

Interesting.

But I've yet to encounter a project that actually needs this, or SSR in general.

Most projects have maybe 2 or 3 page types with mixed static and interactive content - and typically, those pages are mostly static with a few interactive elements, none of which tend to be important for SEO.

Take your typical e-commerce site - your product listing or product detail page is mostly static, with a few interactive elements that generally aren't relevant for SEO. Then you have some informational pages with nothing interactive. And then your checkout pages and maybe a contact form with nothing SEO relevant on them at all.

Maybe this is just my personal experience, but the projects I've seen in 23 years of experience, for the most part, fairly naturally partition into client or server rendered, or mostly server rendered with some interactive elements with no SEO relevance, so...

What's the problem?

Just render those few pages with mixed static and interactive content without the interactive bits, and sprinkle those on the client side - the way it was always done before SSR.

Like, who is this for? It seems like these solutions are mostly engineered for developer feel-good and some idealistic principle about insisting on a single source.

But why? Why is that important?

Why is it even considered a good thing? Maybe it's actually better to have some logical, physical and mental separation between static content and interactive islands within that content. We could call this "separation of concerns", haha. 😉

No, but seriously. I get it. I understand what you're doing - I understand why. I'm just not sure I agree that the problem is large or important enough to even bother solving.

Can you point to a site where static and interactive content is so intertwined that any of this substantially pays off?

Other than maybe facebook.com (the biggest and messiest example I can think of) who really needs this? 🤔

Collapse
peerreynders profile image
peerreynders

Just render those few pages with mixed static and interactive content without the interactive bits, and sprinkle those on the client side - the way it was always done before SSR.

There is nothing wrong with that style of development but if one steps back for a moment and surveys what has been going on over the last 15 or so years then (with the benefit of hindsight of course) it's almost inevitable that something as ambitious as Qwik would be attempted given the emerging realities of the mobile web.

The whole CSR-based SPA movement was a drive from the document web towards the application web (whether the problem being solved required it or not). However many pure-CSR solutions suffered from delayed first contentful paint (trying the visitor's patience) leading to the introduction of SSR.

Unlike dynamic server rendering SSR dictates JavaScript on the server side because it has to run the JS-based client side framework in order to render the HTML. Of course being more "application like" something else gets rendered: client side application state - and combining both the application view (HTML) and the state client side lead to the concept of hydration (which at times lead to false expectations).

Now generating client side state on the server side wasn't anything new; it's a well established practice that was a known tactic even in the jQuery era but in most cases it was simpler to "just" make some initial calls with the existing AJAX functionality rather than having to manually add and accept the additional complexity of "hydrating" client state directly from the HTML.

With SSR technologies "hydration" was automatic requiring no intervention by the developer. But "hydration" comes at a cost that can vary wildly depending on the underlying client side framework leading to the "Uncanny Valley" - the time where content is visible but not interactive.

Once again, the solution to the problem only changes the problem.

The obvious answer was to ship less JavaScript in order to minimize the uncanny valley and that's exactly what Svelte and Solid did. Among other measures both used compilation tools - unlike the established frameworks which tended to often rely on runtime solutions both Svelte and Solid reduced their runtime code output by doing work at compile time whenever possible. But in the end even those applications are monolithic affairs, so short of any manual bundle splitting to arrive at some critical core (akin to critical CSS), the whole application has to be downloaded and parsed before it can become interactive.

This is where Qwik comes in as it attacks the monolithic nature of client side JavaScript. By design the HTML and UI state is server rendered and shipped to the browser but the included (tiny) loader will only download compact JS bundles on interaction to support that particular interaction (progressive hydration) - it is gradually deploying the UI, only as necessary. There are also hints that Qwik will act much less as the center of the universe so it may be more suited to off-main-thread architecture.

Does all this sound incredibly complex?

Sure but in this case the complexity is in the service of addressing the uncertainties of the modern web when it comes to connection quality and client device computational capability. Back in 2010 Responsive Design emerged in response to the reality of unpredictable screen ratios and sizes due to the rise of the mobile web. Qwik is a phenomenon of the rest of front end development catching up to that reality as it impacts the remaining aspects of the web.

So Qwik isn't challenging the approach of "sprinkled interactivity" where that is sufficient but the assumption that "application-like" behaviour and feel will always require an SPA.

It's easy to quip about how we have now gone back to rendering HTML on the server. But these days the boundaries are very different. With PHP (1995) and Rails (2005) the boundary pretty much stopped at the server; it rendered pages and accepted submissions. jQuery made it fairly easy to use XMLHttpRequest in 2006 though AJAX was in use before that; servers would now also render data for the ad hoc consumption of page clients. Technically there is some strong coupling between the server and the client but for the most part there is a boundary between them.

I think it could be argued that in 2014 Marko (open sourced 2017) tried to to push the boundary from the server to envelop the browser - but at the time the rest of the world was too busy to notice while adopting SPAs.

So Qwik seems to be the latest attempt to view server and browser as part of the same system (application) while fully respecting the constraints of their separation/distribution. Now some might claim that Phoenix LiveView (2018), Laravel LiveWire (2019) and Rails Hotwire (2020) do the same (while being primarily developed in the server language) - but those assume availability of high bandwidth, low latency connections. Conceptually at least Qwik seems to be designed to work even when conditions are less than ideal.


Off topic: I think the above overview should give an impression of how different web development actually is from anything else. So whenever somebody predicts or hopes that technology X (current favourite: your favourite language compilation to WASM) will act as the great homogenizer to bring web development in line with native and backend development [1], I can only express my disbelief.

In my mind Qwik is an example of the outside of the box thinking that is sometimes required on the web to ensure that a certain class of solutions will work under most circumstances rather than just under the best of circumstances.

I'm just not sure I agree that the problem is large or important enough to even bother solving.

I think Qwik tries to get beyond the limitations of the current generation of SSR-SPAs perhaps in the form of an MPA. Whether or not any particular solution needs to be "app like" is an altogether different discussion; so far the SPA fashion trend seems to be continuing.

Collapse
mindplay profile image
Rasmus Schultz

Heh, this should have been an article on it's own. 😄

Not that I needed it, thanks - I've been doing both front-end and back-end development since long before anyone even bothered making that distinction.

The whole idea of the "document web" versus the "application web" is completely silly to me. There is one web. It has documents and applications - but for the most part, it's documents with little applications here and there. If you insist on building those sites in application frameworks, of course you're going to need increasingly complex frameworks.

I dunno. It doesn't appeal to me. I like simple things.

If I'm building an application, I'll pick a front-end framework - if I'm building mostly-documents site, I'll use something for templating on the server-side.

Easy. I don't know why the trend is to find complex solutions to simple problems.

I'm sure there are a couple of projects that might need this, but in my opinion, there's no reason this or anything like it should be mainstreamed. Right tool for the job and all.

Thread Thread
peerreynders profile image
peerreynders • Edited on

The whole idea of the "document web" versus the "application web" is completely silly to me.

Fair enough but as a page takes on more and more interactivity it becomes more and more "app like" and even though in 2014 Scott Jensen described applications as "technology tillers" with reference to the web a lot of the industry is continuing to chase the "rich user interface" experience (also [1]).

In my judgement it was the often reported jQuery spaghetti code that resulted from sprinkling too much interactivity on a page that pushed many developers to adopt pure client side rendering with the hope that it should be much easier to maintain given that the DOM is now entirely managed in one place - the client - rather than always having to go back to the server code to make the necessary adjustments to the various templates. So it could be argued the move to pure-CSR marked the beginning of the "application web".

no reason this or anything like it should be mainstreamed.

I don't think anything is being mainstreamed here. It's a development of interest against the background of Paul Lewis's observation six years ago and a change in direction from the status quo. If you look into the tweet thread:

"Paul, any examples of frameworks that actually play nicely with PR+B?" [2]

"No, and that’s my issue. Most (if not all) seem to expect all the JS up front before giving the user anything" [3]

In an earlier blog (2015) Paul Lewis aimed squarely at React to state: developer ergonomics should be less important than our users’ needs.

That message fell largely on deaf ears as evidenced by the continually growing JS payloads delivered to clients. "The network will save us" and "client devices are becoming more performant every year" are the typical excuses despite evidence that networks are at risk of becoming over subscribed and large numbers of less performant devices are entering the market while even flag ship devices are architected with a larger number of small, low power cores when the current generation of web technology requires peak single thread performance.

Back in 2016 there were some suggestions that FaceBook BigPipe and Drupal BigPipe were examples of "Progressive Rendering and Bootstrapping". (Perhaps Marko's introduction of async fragments in 2014 should at least count as "progressive rendering".)

So while Qwik emphasizes "Resumability" and "Progressive Hydration" it seems to embrace the "Progressive Rendering and Bootstrapping" approach which would likely be impossible to implement with the current generation of mainstream frameworks (Miško Hevery introduced the concepts in 2019). Interestingly during Qwik's development developer experience was only problem/priority #9 - echoing "developer ergonomics should be less important than our users’ needs".

I don't know why the trend is to find complex solutions to simple problems.

Now this is complete speculation on my part but Qwik is being developed at Builder.io - and they feature low-code services. Reportedly services like Wix, Squarespace and Webflow tend to result in fairly hefty JS payloads (likely requiring developer/engineering intervention to correct, defeating the "low-code" concept) which harms user experience. Qwik (or some derivative) could be the foundation for a low-code web site builder that has a performance advantage over those established services.

Collapse
ryansolid profile image
Ryan Carniato Author

I see you aren't saying don't server render.. but rather why bother hydrating? Like Server render what you need on the server and then sprinkle on some client side JavaScript where needed interactivity.

Yeah this is mostly a developer experience consideration, but I don't think it is too far of a stretch. It is a bit like asking do you need to use frameworks, and the answer, of course, is no. Yet we still do. All of this is to allow authoring a single app experience and have it just work across the board. One that leverages declarative patterns for organizing code. There are organizational/scaling concerns too but not every project needs to worry about that.

It seems you are familiar with most the reasoning and in the same way I won't convince people who want to use Vanilla JS to do everything instead of using frameworks, I'm not going to convince you otherwise. All I got for that is, been there, tried that before, and wasn't completely satisfied. So let's see what else we can do.

Collapse
matthewharwood profile image
matthew harwood

🙏 Awesome write up!

Collapse
cjsmocjsmo profile image
Charlie J Smotherman

Nice write up.

How does Qwick compare to htmx they sound like they are similar?

Happy coding

Collapse
ryansolid profile image
Ryan Carniato Author

HTMX is a mechanism to piecewise server render. Where Qwik is a way to piecewise hydrate. The difference is that once in the browser Qwik doesn't go back to the server for things that don't need to go back to the server. So something like HTMX scales perfectly in that it just needs a small runtime and then everything runs on the server as needed. Click a button, replace part of the view. Click it again and go to the server and replace it again. You never need any more JavaScript.

Qwik on the other hand request some JavaScript when you click that button. And that JavaScript will render the new content in the browser instead of going in the server. And when you click the button again, well it already has the code so it updates immediately.

Basically the main difference in premise is that client rendering tends to be faster than server rendering when you already have JavaScript in the browser. Qwik focuses on how to take an already server rendered page and load the JavaScript as needed. HTMX is always going back to the server even when it otherwise wouldn't need to.

Now that being said in an ideal world Qwik would use HTMX approach for big changes like routing. When you need to swap out 80% of the page then lets go render on the server, as more than likely we needed to load new data etc... But for the micro-interactions keep them fast and performant in the browser.

The more subtle difference is when you consider the way you author. Now I'm sure someone could extend HTMX page and add some client side interaction and JS on top and also get the sort of best of both worlds scenario I'm talking about. However with Qwik you just build the whole app like it s a declarative stateful app like the way you would with React and this just happens. In our hybrid HTMX example we'd be using a combination of hand crafted JavaScript and mechanical directives that HTMX provides. This means for a very different sort of developer experience.

Collapse
drsensor profile image
drsensor

I guess every event handler will be executed in the microtask since it came from `await import()'. I wonder what would happen if I run physics engine computation on mouse click 🤔
(normally it should be run inside a webworker but running in the main thread give a better startup time)

Collapse
peerreynders profile image
peerreynders • Edited on

How we cut 99% of our JavaScript with Qwik + Partytown

Partytown and Qwik are both projects at BuilderIO. Going by a recent interview they have every intention for Qwik applications to leverage web workers to immediately download and run the heavier parts - restricting the main thread to only what is essential to support the UI.


Earlier in that same interview:

"... at some point you know the client might go back to the server and say like did any of these things change or i'm deleting one or whatever right and that's up to application to kind of figure out how that that works.

How do you dictate that in Qwik?

You don't, like that's outside of the responsibility Qwik you just have data, you just have objects".

The implication is that Qwik isn't running the entire client side application but that Qwik is focused entirely on the UI aspect and its supporting state - so "the rest of the client side application" could partly or as a whole be running on a web worker.

Collapse
drsensor profile image
drsensor

Here is a simple scenario in gamedev (with parallelize ECS) where you don't want to lazy load the event handler nor put them in the queuMicrotask

<button on:click=(() => {
  for (const [typedArray, offset, [entity, health]] of queryRaw(enemyHealth)) {
    Atomics.sub(typedArray, offset, 20)
    physicsKnockback(entity, 50)
  }
})>
Enter fullscreen mode Exit fullscreen mode
Collapse
natxoski profile image
Iggy

"If you have components that contain other components and pass props throughout things need to run top down."

If you run into this situation you need to rethink your approach.

Also all this seems very familiar with Svelte/Sveltekit. IMHO it's just the same thing, but with a different name...

Collapse
ryansolid profile image
Ryan Carniato Author

It is how every SPA framework works. React/Svelte/Vue/Angular/Solid/Lit you name it. And how every metaframework built on them works Next/Remix/Redwood/Nuxt/Sapper/SvelteKit/Universal/SolidStart. Things like Astro/Iles/Slinkity/Elder/Marko optimize over that model with partial hydration, but Qwik is the only shipping resumable today of known libraries.

Collapse
ninjin profile image
Jin

There is comparison of two approaches: resumable hydration vs offline first.

youtu.be/N-QkXQh-TOk