HTTP/2 server push support about fetch

[annevk]

@DanielBaulig I think we'd want to give full control, yes. The main tricky point is how to layer H/2 push.

[rektide]

Please help. This deserves to be worked upon. This issue speaks to one of the best, most interesting, most enabling new capabilities of http 2, and leaving it chronically un-usable is so sad.

[annevk]

You'll have to be a bit more concrete with what kind of API you're expecting for this.

[martinthomson]

Sure. I was thinking that an event handler on Request, maybe called onpush might work. You get pushes before and during response delivery. Each push turns up in the form of a Request, and you need to actually fetch that to get the actual response.

[annevk]

So what are the use cases for this? I wonder if this should be part of the fetch registry proposal https://www.w3.org/Bugs/Public/show_bug.cgi?id=23878#c16 and some kind of associated network observer API.

[martinthomson]

Not understanding the full scope of what your registry proposal is, it seems plausible, but I'd have to trust your judgment in this case.

The use cases I can think of, without getting into too much detail:

• HTTP APIs that use server push
• replacing long-polling (webpush uses server push)

[brendanlong]

This bug report has a lot of information about use-cases. Copying it here for convenience: https://www.w3.org/Bugs/Public/show_bug.cgi?id=28083

http://lists.w3.org/Archives/Public/public-whatwg-archive/2015Feb/0030.html

The use-case I have in mind is MPEG DASH or HLS live streaming:

A server updates the manifest (MPD or m3u8) and pushes it. The client browser will cache it, but JavaScript applications don’t know that it exists. For newly generated segments, we need the MPD to use them anyway, but for adaptive streaming we need to know when the request started and when it finished so we can estimate bandwidth. If a server is pushing MPD updates, we want a way to tell it to stop (RST_STREAM on a pushed stream to end the transfer, or RST_STREAM on the original client request to prevent future pushes for that request).

The obvious question to ask is “why not just poll the server”? The answer its that live streaming latency depends (among other things) on how quickly you poll. Unless you can perfectly predict when the server will have an update available, you need to either poll slightly late (introducing latency) or poll significantly more often than the server creates updates. Using server push is equivalent to to polling infinitely fast, while simultaneously reducing load on the server by making fewer requests (win/win).

I imagine that this would be useful for JavaScript implementations of HTTP/2 REST API’s, especially API’s using something like ATOM or RSS. Firefox has an API for addons to detect pushed streams, so presumably they think there is some use for this (but they didn’t propose this as a public API, so maybe not?).

What's needed:

• Event when PUSH_PROMISE is detected, containing a new Response.
• Ability to signal to a UA that a stream should left in the "half closed" state (to allow server pushes).

I'm not sure fetch is the best API for this, since there's currently no way to cancel fetches or use a partial response before it finishes, but it seems to be the only API anyone wants to make changes to.

[igrigorik]

@annevk +1 on registry overlap. I think this is another instance of "observer use case": #65. Specifically, in this instance the receipt of the PUSH_PROMISE would register the request in the registry... which would trigger appropriate observer callbacks, etc.

[rektide]

I have an example of this usage pattern in a server-to-server case- I stream IRC messages from one microservice to another.
https://github.com/rektide/tele-pump/blob/master/tele-muc-pump.js#L17

There's also an earlier implemented of this capability that I created by recording all push_promises i'd sent, then attaching an X-Associated-Content header to any replies made to the client with this list. Then I polyfilled XMLHttpRequest to read these headers, and postMessage to to the window to inform any listeners. By periodically issuing requests to the server, the client gets notice of what has been pushed.
https://github.com/rektide/pushchannel

This latter technique was something inspired by Jetty, who also has X-associated-content: in that implementation, the header informs the server to push that resource to the client; I took it one step further. This should be a readily available capability for the browser, without polyfilling XHR. I would love for the first project, tele-muc-pump, to be usable from the client (without having to devise an out of band signalling like the pushchannel XHR monkeypatching, or using a Server-Sent-Events or websockets).

[jakearchibald]

I don't think this API fits on request objects, as far as I can tell PUSH_PROMISE frames aren't linked to another request.

I don't think this API should be on pages because the lifecycle doesn't match up. As far as I know one H2 connection can cover many pages. Are H2 connections even per origin?

[martinthomson]

@jakearchibald, actually PUSH_PROMISE is tied to a request and that binding is sometimes (often maybe) important.

That an h2 connection can cover multiple origins makes this less easy, I agree. In some cases you will be receiving a cross-grained request-response without a pre-flight. We have to ask whether we even want to surface the existence of something like that.

[rektide]

I appreciate @martinthomson pointing out that PUSH_PROMISE is affiliated with a specific request (and that's super important as in the case with protocol specs like his WebPush Protocol). That makes fetch a sensible place to expose push events.

As a developer, it would also be convenient to have one high level place where I could perform observations on pushed data. If Service Workers also offered a way to see PUSH_PROMISES for their domain, for example, that would be useful and handy. It would both prevent developers from having to explicitly set up each fetch to listen, and secondarily, it would be answer to how to handle PUSH_PROMISES sent to non-AJAX/non-fetch-based PUSH_PROMISES.

[martinthomson]

Trying to record the result of in-person discussion...

Two options seem to be most attractive:

1. A fetch-group-local service that provides events when requests are made. See #65.
2. Attaching events to Response that surface when a server push is made.

In both these cases, a Request would be generated for each server push. These could then be passed to fetch() to attach to the existing request and acquire the answer. This would likely need to build on Response.clone() to deal with the fact that multiple clients could consume the answer.

The main shortcoming of the second option is that you would delay the receipt of pushes until the response header fields are populated. h2 permits PUSH_PROMISE to be sent prior to the generation of the response header block, but fetch() depends on the header block to construct the Response.

[igrigorik]

In favor of (1): some RUM vendors wrap all the JS APIs + attach onload / onerror's to all new elements that initiate fetches via MutationObserver to capture "start of request". Exposing an event would eliminate that overhead.. which would be nice.

[annevk]

There's also some discussion on the semantics of server push and @mnot might write a draft on that. Fetch-wise the semantics would roughly be:

1. All push promises end up in the HTTP cache (effectively). This way they can be used for navigation. (There's many ways to put something in the HTTP cache already so that wasn't really a valid concern.)
2. Noncacheable push promises disappear together with the fetch group.

[DanielBaulig]

Is there any consideration of being able to cancel/abort an ongoing push from a future API through a reset stream? I don't think the proposed solutions in this thread allow for this.

[rektide]

@slightlyoff providing a more tangible discussion point here:
https://gist.github.com/slightlyoff/18dc42ae00768c23fbc4c5097400adfb#gistcomment-2227534

[ed: jake archibald has also reiterated some ideas begat in fetch observer in second comment, and it's worth reviewing!! see: comment#2]

[jakearchibald]

A lot of PUSH stuff was removed from Chrome https://groups.google.com/a/chromium.org/g/blink-dev/c/K3rYLvmQUBY/m/vOWBKZGoAQAJ

[mitar]

Nothing has been removed from Chrome (yet).

[jakearchibald]

I'm pretty sure we never enabled PUSH over HTTP/3

[mitar]

So? Your comment was that that PUSH was removed from Chrome, referencing a mailing list post where nothing to that effect was posted, but only that there is an intend and that they are conducting an experiment. So, what exactly has been already removed?

[rektide]

This should still be followed up on, even if it's winter-cg that can use it & the browsers remain unmoved about letting dev's ever explore what the world could do with push.