Yeah the reason I didn't focus too much on resumable hydration here is I don't think true resumable hydration exists yet. Qwik does succeed at doing out of order hydration in that you can hydrate the child before the parent. And with that partitioning it is possible to assume that data initializes with initial state from the server at a component level. You have the inputs and you have the outputs, so you can confidently just not do the re-render of that component in the browser at hydration time and only attach the event handlers, or any defined side effects.
However, the reason I don't consider this true resumable hydration is that when any data would update for this component it needs to re-run and redo unrelated work that may have already been done on the server. It's like the motivation for the React forget compiler example if you saw that.. if you have some state that has nothing to do with a list you are rendering and you update the state the list re-renders. This is unsurprising. Picture if that unrelated work triggered an async data request etc..
VDOM based solutions have some challenges with resumable hydration. Qwik's approach is more like skipping hydration and then doing the work when you interact with it. Resumable Hydration should be able to work not only skipping unnecessary hydration but be able to not redo computations and derivations in the browser after the fact when unrelated data changes. This is possible if you can slice up your component logic the other way more similar to the fine-grained graphs that Solid produces. And you are probably unsurprised to hear this is exactly what we are working on for Marko 6.
I am playing with Qwik right now and seeing just how effective their resumable approach is, because I do want to confirm my understanding is correct.
UPDATE: Talking to Misko and the team Qwik is already working on a solution for precisely the concern that I have. They have a runtime based reactive system similar to Solid and are serializing the dependencies at runtime on the server with some new primitives that resemble useEffect and useMemo. When this lands they should be able to do the type of fine-grained resumability I'm talking about.
I don't think true resumable hydration exists yet.
Given your article's introduction the characterization of "replayable hydration" (for the current state of SSR) versus the hypothetical ideal of "resumable hydration" seemed useful.
I had a most excellent chat with Ryan offline and wanted to share my thoughts here, which boil down to "Qwik is truely resumable." Maybe not right this moment, as we have a few more things to implement, but we have designs for all of these.
To be truly resumable, you need to have a few things:
As Ryan pointed out, you need to attach to existing components out of order.
You need to serialize the whole state of the application.
You need to understand which component knows about what state. (Subscriptions)
If you don't have #3, then once the data changes, you don't have a choice but to re-render the whole app because you don't know which components are invalidated and need to be re-rendered. This forces you to download the whole app, and that is expensive.
But Qwik does know the answer to #3. Though Proxys Qwik knows which component cares about which data, Qwik serializes all of this information to the DOM. Then the data gets mutated Qwik can run querySelectorAll on HTML, which will tell it which components are subscribed to the data and hence have to be re-rendered. This is significant because it allows most of the components to stay in their unhydrated state for the duration of the app's life.
I think of this problem as layers in onion:
Most obvious is how to serialize listeners. (If you can't do that, than you have to rehydrate)
How do I render components out of order (If you can't do that that you will be forced to render parent/children)
Know which component is invalidated on state change (If you can't do that, you will be forced to re-render the whole app)
Have ways to watch data without running the code (useEffect equivalent) (If you don't have that you will have to eagerly download code)
Have ways to deal with data that is not Serializable. (if you can't do that, than the kinds of apps you can build will be limited)
On each of the levels, there are interesting problems to solve. Qwik's aim is to solve all of them, and so far, we have been able to do that and have plans for the reminder, which is consistent with Qwik's mental model.
This is why I think that Qwik is (will be) truly resumable.
I appreciate both yours & Ryan's work so this thread is gold. Thank you for listing out the hydration scenarios. Here is my concern as an app & library developer. I really like isomorphic JS & am building around that concept. Up till now isojs hydration has been a black box. Making the isojs hydration process more transparent & even programmable will be appreciated.
I would like to learn more about your plans re: step 4 & 5. Also, do you forsee Qwik or its concepts as being compatible with Solid & other isojs component libraries?
I love Misko's framing here as 4/5 are big part of what we are working on with Marko as well. 4 is a fairly natural extension of 3 in the same way Solid's granular reactivity is over say MobX + VDOM. The way to do this involves serializing the dependencies, then you don't need to run it once. In Marko's case it is a compiler that detects these similar to Svelte, except Marko's works cross file and traces dependencies even hoisted. For Qwik it's runtime similar to Solid, just that if it runs on the server you know the dependencies from the last run and can continue only when one updates. The way they serialize them is simply to take the key off their shared proxy store that all components use (that manages the Dependency injection).
I do want to talk to Misko more about this because there are different types of scenarios. Derivations definitely want to run on the server and be treated this way but effects in most frameworks are also a way to do client only code that you don't run on the server, like include a jQuery chart. In those cases you'd want the effect to run at hydration time so it isn't really a problem since you don't need it to run on the server to collect dependencies. In Marko we make that deliberation that it is effects and event handlers that run at hydration time. The interesting piece for Qwik is do they code split on this too putting the effects in with the event handlers and separate from the other component code.. once you follow this thinking Qwik may actually approach closer to how Marko 6 is breaking up code.
For Marko we actually split it from input/props through Reactive graph so a component consists of many different fine-grained pieces. Which is interesting since instead of things being component level the parent can know based on whether it passes static or reactive data to even import the dynamic child part to run it. And since we know exactly what can be stateful we can eliminate whole subtrees through the component structure.
I'm excited to talk more about this because the implications get pretty interesting. Because even when things are stateful instead of serializing everything we can optimize it further to serialize things based on the leaf nodes of the reactive graph. It's only things read from an DOM, event handler or effect directly that need to be serialized. With one except reactive convergences need each source branch serialized as well so one side can update without needing to run again. In a sense I've shown how Marko today can only serialize data that is used in components based on what is passed in to top level components input/props. With this approach we'd get field level serialization of only what is actually used/usable. Because of convergence we know that and in other cases we know that it would be impossible change what data we are seeing without refetching (ie triggering an async source) so we can be confident we don't need to have it in the browser initially.
Of course this all comes down to what is serializable. I'm interested to learn more about Qwiks approach. With Marko since the compiler sets the boundaries we can always step up one level with serialization if it isn't serializable as long as all root sources are. But we are working on techniques to serialize typically unserializable things like promises or functions and closures.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Yeah the reason I didn't focus too much on resumable hydration here is I don't think true resumable hydration exists yet. Qwik does succeed at doing out of order hydration in that you can hydrate the child before the parent. And with that partitioning it is possible to assume that data initializes with initial state from the server at a component level. You have the inputs and you have the outputs, so you can confidently just not do the re-render of that component in the browser at hydration time and only attach the event handlers, or any defined side effects.
However, the reason I don't consider this true resumable hydration is that when any data would update for this component it needs to re-run and redo unrelated work that may have already been done on the server. It's like the motivation for the React forget compiler example if you saw that.. if you have some state that has nothing to do with a list you are rendering and you update the state the list re-renders. This is unsurprising. Picture if that unrelated work triggered an async data request etc..
VDOM based solutions have some challenges with resumable hydration. Qwik's approach is more like skipping hydration and then doing the work when you interact with it. Resumable Hydration should be able to work not only skipping unnecessary hydration but be able to not redo computations and derivations in the browser after the fact when unrelated data changes. This is possible if you can slice up your component logic the other way more similar to the fine-grained graphs that Solid produces. And you are probably unsurprised to hear this is exactly what we are working on for Marko 6.
I am playing with Qwik right now and seeing just how effective their resumable approach is, because I do want to confirm my understanding is correct.
UPDATE: Talking to Misko and the team Qwik is already working on a solution for precisely the concern that I have. They have a runtime based reactive system similar to Solid and are serializing the dependencies at runtime on the server with some new primitives that resemble
useEffectanduseMemo. When this lands they should be able to do the type of fine-grained resumability I'm talking about.Given your article's introduction the characterization of "replayable hydration" (for the current state of SSR) versus the hypothetical ideal of "resumable hydration" seemed useful.
I think it just did, with Qwik 1.0 just having been released recently.
I had a most excellent chat with Ryan offline and wanted to share my thoughts here, which boil down to "Qwik is truely resumable." Maybe not right this moment, as we have a few more things to implement, but we have designs for all of these.
To be truly resumable, you need to have a few things:
If you don't have #3, then once the data changes, you don't have a choice but to re-render the whole app because you don't know which components are invalidated and need to be re-rendered. This forces you to download the whole app, and that is expensive.
But Qwik does know the answer to #3. Though Proxys Qwik knows which component cares about which data, Qwik serializes all of this information to the DOM. Then the data gets mutated Qwik can run
querySelectorAllon HTML, which will tell it which components are subscribed to the data and hence have to be re-rendered. This is significant because it allows most of the components to stay in their unhydrated state for the duration of the app's life.I think of this problem as layers in onion:
useEffectequivalent) (If you don't have that you will have to eagerly download code)On each of the levels, there are interesting problems to solve. Qwik's aim is to solve all of them, and so far, we have been able to do that and have plans for the reminder, which is consistent with Qwik's mental model.
This is why I think that Qwik is (will be) truly resumable.
I appreciate both yours & Ryan's work so this thread is gold. Thank you for listing out the hydration scenarios. Here is my concern as an app & library developer. I really like isomorphic JS & am building around that concept. Up till now isojs hydration has been a black box. Making the isojs hydration process more transparent & even programmable will be appreciated.
I would like to learn more about your plans re: step 4 & 5. Also, do you forsee Qwik or its concepts as being compatible with Solid & other isojs component libraries?
I love Misko's framing here as 4/5 are big part of what we are working on with Marko as well. 4 is a fairly natural extension of 3 in the same way Solid's granular reactivity is over say MobX + VDOM. The way to do this involves serializing the dependencies, then you don't need to run it once. In Marko's case it is a compiler that detects these similar to Svelte, except Marko's works cross file and traces dependencies even hoisted. For Qwik it's runtime similar to Solid, just that if it runs on the server you know the dependencies from the last run and can continue only when one updates. The way they serialize them is simply to take the key off their shared proxy store that all components use (that manages the Dependency injection).
I do want to talk to Misko more about this because there are different types of scenarios. Derivations definitely want to run on the server and be treated this way but effects in most frameworks are also a way to do client only code that you don't run on the server, like include a jQuery chart. In those cases you'd want the effect to run at hydration time so it isn't really a problem since you don't need it to run on the server to collect dependencies. In Marko we make that deliberation that it is effects and event handlers that run at hydration time. The interesting piece for Qwik is do they code split on this too putting the effects in with the event handlers and separate from the other component code.. once you follow this thinking Qwik may actually approach closer to how Marko 6 is breaking up code.
For Marko we actually split it from input/props through Reactive graph so a component consists of many different fine-grained pieces. Which is interesting since instead of things being component level the parent can know based on whether it passes static or reactive data to even import the dynamic child part to run it. And since we know exactly what can be stateful we can eliminate whole subtrees through the component structure.
I'm excited to talk more about this because the implications get pretty interesting. Because even when things are stateful instead of serializing everything we can optimize it further to serialize things based on the leaf nodes of the reactive graph. It's only things read from an DOM, event handler or effect directly that need to be serialized. With one except reactive convergences need each source branch serialized as well so one side can update without needing to run again. In a sense I've shown how Marko today can only serialize data that is used in components based on what is passed in to top level components input/props. With this approach we'd get field level serialization of only what is actually used/usable. Because of convergence we know that and in other cases we know that it would be impossible change what data we are seeing without refetching (ie triggering an async source) so we can be confident we don't need to have it in the browser initially.
Of course this all comes down to what is serializable. I'm interested to learn more about Qwiks approach. With Marko since the compiler sets the boundaries we can always step up one level with serialization if it isn't serializable as long as all root sources are. But we are working on techniques to serialize typically unserializable things like promises or functions and closures.