[progressive-hydration] conditions for first level hydration timing about community-protocols

[daKmoR]

This probably should be combined with defer-hydration.

Basically, everything is defer-hydration and based on the loading="..." we can define which one to hydrate.

For "parents" that will never hydrate on their own but only "as needed" based on their children it could be something like hydrate:asNeeded.

Additionally, there is now a need to define that the hydration should wait on "what"?

A suggestion would be to rename all "modifiers" to with...

• withDelay (former onIdle)
• withWaitOnIdle (former OnIdle)
• withParentFirst

which results in

<my-form loading="hydrate:asNeeded" defer-hydration>
  <my-input loading="hydrate:onClick && withParentFirst" defer-hydration></my-input>
</my-form>

Now if you click inside my-input the following happens

1. loader sees my-input should hydrate as all conditions are now true (modifiers as withParentFirst are always true)
2. withParentFirst triggers hydration of parent first (await parent.updateComplete;?)
3. Hydrate my-input (now it gets registered within my-form)
4. Replay click event (focus now within the my-input)

[daKmoR]

Good question 🤔

In my current implementation, I test it by using a component that "manually" exposes it.

in lit this.updated only gets called on the client after hydration... so we can use that as a "hook" to say I am hydrated.

import { LitElement, html, css } from 'lit';

export class MyEl extends LitElement {
  static properties = {
    msg: { type: String },
    hydrated: { type: Boolean, reflect: true },
  };

  constructor() {
    super();
    this.hydrated = false;
  }

  updated(props) {
    super.updated(props);
    this.hydrated = true;
  }

  render() {
    return html`<p>Hello World</p>`;
  }

  static styles = css`
    :host([hydrated]) {
      background: green;
      display: block;
    }
  `;
}

but right - the "HydrationLoader" (at least that's how it's called in my current implementation) could add the attribute after the component is loaded, registered and upgraded... and it could fire an hydrated event...

I'm a little wary of the scope - so I would call these "optional" goals for now - as they look more like nice to have (especially as we could add them later) - don't get me wrong if there is fast agreement on all the other stuff we could include it - or it could be an iteration 🤗

[daKmoR]

I really try to follow all these hydration discussions - but still, none of the explanations made it clear to me what defines a hard cut between full, partial, progressive hydration or resumeability.

imho it's quite a blurry line 😅

but yeah this seems more like "progressive hydration"... as it's not only about hydration but also about loading code (why that is not important for all of the hydration strategies I don't really understand 🙈)

[thescientist13]

Yeah, it has taken me a little while to try and wrap my arms around them all as well, especially all their pros / cons and subtle implementation details. though I definitely do not claim to be an expert at the time of writing this, here's my best take on what it all means. 😅

---

Graphic courtesy of Ryan Carniato

So using the above definitions, and given this bit of code that has some state and some event handling, I'll try and apply the expected output for each of those techniques as best I can to reflect their distinctions.

class Counter extends HTMLElement {
  constructor() {
    super();

    this.count = 0;

    if(this.shadowRoot) {
      this.shadowRoot.querySelector('button#dec').addEventListener('click', this.dec.bind(this));
      this.shadowRoot.querySelector('button#inc').addEventListener('click', this.inc.bind(this));
    } else {
      this.attachShadow({ mode: 'open' });
      this.shadowRoot.innerHTML = this.render();
    }
  }

  inc() {
    this.count = this.count + 1;
    this.update();
  }

  dec() {
    this.count = this.count - 1;
    this.update();
  }

  update() {
    this.shadowRoot.querySelector('span#count').textContent = this.count;
  }

  render() {
    return `
      <div>
        <button id="inc">Increment</button>
        <span>Current Count: <span id="count">${this.count}</span></span>
        <button id="dec">Decrement</button>
      </div>
    `;
  }
}

customElements.define('wcc-counter', Counter);

Progressive Hydration

Load JS / hydrate on event / interaction.

Basically what we're talking about here, where the loading of the JS is not eagerly done on load, but rather via something like an IntersectionObserver or MutationObserver. (Think Astro and its islands)

The two big issue with top down hydration are:

• Double the data - You need to sync the data from the server side to restore it correctly in the client and hydrate into the correct state for the UI.
• Double the JavaScript - Even though we might only be attaching event handlers when hydrating, we still had to effectively ship all the JavaScript to the client that was already run on the server.

There is a lot of nuance in this section when it comes to top down (like React or Lit) and their hydrate functions and being coarse grained, vs something like Solid which is much more fine grained in its reactivity. One draw back with islands is that while you get vertical isolation for each island, it may be harder to achieve any horizontal state sharing, since you've now introduced a parent and are required to render top down.

Partial Hydration

Use knowledge of the server vs client to only ship code / serialize data needed in the browser.

Now this is where things start to get interesting, and where something like React Server Components come into the picture. As mentioned with hydration, we're often finding ourselves shipping the work and the data from the server to the client again, to keep things in sync. So what if the server code that ran and wasn't needed again on the client (like say just rendering the template) didn't get shipped?

So in a partial hydration scenario, a compiler or build tool could examine our Counter components and see that the template never changes, and the only interactivity comes from the event handlers. So why not just ship only the event handlers?

class Counter extends HTMLElement {
  constructor() {
    super();

    this.count = 0;
    
    if(this.shadowRoot) {
      this.shadowRoot.querySelector('button#dec').addEventListener('click', this.dec.bind(this));
      this.shadowRoot.querySelector('button#inc').addEventListener('click', this.inc.bind(this));
    } else {
      this.attachShadow({ mode: 'open' });
    }
  }

  inc() {
    this.count = this.count + 1;
    this.update();
  }

  dec() {
    this.count = this.count - 1;
    this.update();
  }

  update() {
    this.shadowRoot.querySelector('span#count').textContent = this.count;
  }
}

customElements.define('wcc-counter', Counter);

This effectively aims to solve the double data / JS issue with top down hydration, but may or may not incur a little initial runtime overhead to glue some of these pieces together. (I think Solid does this)

Resumable

Do not repeat any work in the browser already done on the server.

In this case, we're now going in a completely different direction from hydration, so much so that per Misko, we shouldn't be thinking of resumable as hydration at all.

So, given our starting Counter component, in a resumable scenario, we would effectively just get HTML at runtime instead of JS, with the state and closures all neatly handled by the framework and tucked away into little pockets of JS that are lazily loaded as needed, without the need for a second render on the client side, since it has already been done once on the server side.

<div>
  <button id="inc" onclick="() => this.count + 1">Increment</button>
  <span>Current Count: <span id="count">${this.count}</span></span>
  <button id="dec" onclick="() => this.count - 1">Decrement</button>
</div>

This is why Qwik is Progressive and Resumable, but not Partial. The onclick is being managed by Qwik and actually calls little lazy loaded chunks that correctly map all the state and closures from our component, but it could just as easily be inlined into the HTML as well, which I did here for demonstration purposes.

And so this is what is meant by resumable; in that you could copy / paste the active HTML of this output at any time and paste it into the document of another app as innerHTML, and you would get all the functionality and state exactly as you left off in the previous tab.

---

Anyway, that's where I am at so far with all of these, and I'm sure I missed a bunch of nuance and not nearly all the pros / cons, but I can't do it any justice the way Ryan Carniato of Solid / Marko does it, so I'll just link to his blog posts and live streams, which is where all my knowledge has come from effectively.

I think some of these have interesting promise as community protocols, since given how much the latter two solutions require a pretty complex system / framework to operate in, having bespoke implementations all over the place could be tough for developers moving between projects. So if we can align on terms, and maybe even some interfaces, I think at least portability of concepts, and hopefully code, can be achievable without getting in the way. ✌️

[matthewp]

Some of my own perspective:

1. As the bundler/framework you'll need to know which hydration strategies are used at build time, that's why I think putting that in the attribute value is the wrong way to go: loading="hydrate:onMedia('(max-width: 320px)')". Especially for media, people will want to put their queries into a common .js file to share. So they might put the entire attribute value string into a common file, and that will prevent the framework from knowing that onMedia is used. This is why in Astro we use a colon syntax client:media. However that's not very webby so I like the idea of having separate attributes for "loading strategy" and "loading args". Maybe something like: <my-el loading-strategy="onMedia" loading-args="(max-width: 320px)">. Can bikeshed on the names but I think you get the idea. This way you can easily enforce that loading-strategy be a static string, but the args can be dynamically determined at runtime.

2. Custom elements are different from non-CE based framework components in that once you load a component it will be hydrated by the CE callback functions. In other words if you do:

• <my-el loading="onVisible">
• <my-el loading="onIdle">
  Do you expect the first element to hydrate on visible? So they hydrate separately? If so you need more coordination with the rendering library. Personally I think a "soonest win" rule makes sense. This way no coordination is required; all instances will render whenever the first load occurs.

[daKmoR]

strategy could also be called loading as an alternative 🤔

might make it feel more "in line" with the web?

<img loading="lazy" />
<my-list loading="client"></my-list>
<my-list loading="hydrate"></my-list>
<my-heavy-chart loading="hydrate:onVisible || onMedia('(min-width: 768px)')"></my-heavy-chart>

[georges-gomes]

+1 with loading!

[michaelwarren1106]

does/should this proposal also come with a way to tell when a component has been hydrated? stencil has the hydrated class/attr they add when components are updated, so i think this proposal could add something similar so that app features that depend on components that haven't been hydrated yet have something to await so that we're not limited to setTimeouts()or setInterval().

Could there be an eventing strategy so that a hydrated component could notify other components when it gets hydrated in like a pub/sub kind of way?

[renoirb]

Looks neat!

Thing is that we may not want to confuse what DOM string templates from the "what looks like HTML" that becomes ECMAScript code (e.g. JSX, Vue template, Lit's html tagged template function)

So if we want two types of condition, separate the "loading" per input type. And keep strings as strings.

Using the string as a "channel" is looking good though.

Can't we find a way to make the "loading" be one for a name, the other for MQ. But there's <style media /> already. Having a way to leverage this on other elements. There's CSS Element Queries (see other examples) but isn't about adding on the element like we are seeing here.

How about we do something like

<e
  loading="whatever"
  media="(max-width: 320px)"
/>

So we tell the host that this component has a "loading" so we broadcast it subscribe to it.

Then have the window subscribe for the conditional in JavaScript ECMAScript, stay in ECMAScript.

window.addEventListener('loading:whatever', (e) => { /* ... */ })
//                  Yeah, this ^ bugs me about addEventListener. 
//  Don't want to mixup namespacing and passing an argument to.
// Maybe have another way. Like instantiating a "service" and hook it up as a consumer when "loading" occurs.

[thescientist13]

Would it make sense to rename the title of this issue to progressive hydration? Following along with chats on Twitter about this and assuming we would all abide by Ryan / Misko / etc acknowledgement of these definitions, I then think partial hydration would be an entirely different proposal / protocol than progressive hydration. (IMO)

[daKmoR]

@thescientist13

thank you for writing this summary - especially the code sample that made it way more understandable to me.

As far as I understand most of these things will only work for a "full framework" that is in full control of the whole rendering server & client side (e.g. it needs to know all possible way that can trigger a change or an interaction).

Seems not such a good fit for web components? especially with shadow dom and strong encapsulation in mind?
or what do you think?

@matthewp

thank you for taking a look 🤗

1. I'm not sure what you mean? I think you mean

   import { mobileLoading } from '...';
   
   const foo = html`<my-el loading=${foo}></my-el>`;

   but that is still fine? at least in my case as I create the full html output and then parse it to check what needs to be rendered and when... e.g. the output HTML will be <my-el loading="hydrate:onMedia('(max-width: 320px)')"> and then I know it's a hydration which I write back to the "source"...

2. Yes it's definitely first one wins e.g. as soon as my-el gets hydrated all my-el gets hydrated
   Sidenote: It's also "highest impact mode" wins... e.g. if you have loading="client" and loading="hydrate:*" it will only use client. Some docs

Available in alpha version of Rocket

The above implementation has been released in @rocket/engine.

Try it for yourself 💪

👨‍💻 npx @rocket/create@latest

and select the "Hydration Starter"

Twitter Announcement: https://twitter.com/daKmoR/status/1519263600371814400?s=20&t=ZqeIxf-_s0lQ0tT-Y6OD0g

Docs: https://twitter.com/daKmoR/status/1519263600371814400?s=20&t=ZqeIxf-_s0lQ0tT-Y6OD0g

[thescientist13]

As far as I understand most of these things will only work for a "full framework" that is in full control of the whole rendering server & client side (e.g. it needs to know all possible way that can trigger a change or an interaction). Seems not such a good fit for web components? especially with shadow dom and strong encapsulation in mind? or what do you think?

Yes, to my knowledge all these strategies, hydration included, imply some sort of orchestration on the client side because they are all in some way building on top of what has already been done on the server for rendering. The issue then with current implementation of hydration include doubling up of the work on the client side, including:

1. the template and tracking the dynamic parts
2. the data

(Perhaps how hydration markers are implemented could be a community protocol ??)

I think what's worth taking away from these other approaches is seeing if we can do better than top-down hydration because it is effectively a lot of duplicate work, even if in the client side it is "just attaching event handlers". Now, not saying it is not a viable strategy, or one worth supporting because I'm not sure any of these strategies (maybe Resumable since it is not hydration per se) are a silver bullet, but I think it's interesting to explore just how much we can take advantage of what the server has already done to avoid doubling up any of that work on client. I think it could end being a combination of a couple of these strategies in the end though since you will definitely need at least a single pass on the server, streaming or not.

I think if anything, Resumable is great in theory, but it further pushes your code into framework land, and is very manual by the author. At least a community protocol here could standardize on that implementation detail so WC authors don't have to think about the mechanims and their relative portability. I think it is definitely a fair point to call out that typically the more fine-grained you want to go, the more DSL-y your code becomes, and is very intertwined with the framework, etc. So cool output, but I definitely worry about the vendor lock-in nature to any of these attribute based implementations, which is what inspired me to explore #33 .

In my mind at least, I think this is something WCs could work really well for, because Shadow DOM, and in particular Declarative Shadow DOM provides a great encapsulation mechanism already built into the browser, and can easily be baked into HTML, like in a <script type="application/json"> for data in the shadow root. This may technically duplicate the data but perhaps with something like Context API, just the slice of the data that is needed for that component instance would be SSR'd into the HTML, avoiding the bloat of the entire top-level data structure being repeated multiple time per rendered instance. This could be the gateway to enabling fine-grained hydration strategies.

Admittedly I'm still exploring the space and I know Ryan Carniato ran into certain issues using WCs to implement his earlier version of SolidJS, but I haven't looked at his work in that space deeply enough to know if what he encountered was an issue at the spec level, or how he wanted to implement fine-grained reactivity, or something else entirely Maybe I'll reach out to him.

So yeah, I guess technically at this moment I don't know what specifically would prevent WCs from participating in all these various strategies and captured as community protocol, but I'm here to see what is possible. 🤓