FetchObserver (for a single fetch)

[annevk]

In #447 (comment) @jakearchibald sketched some APIs based on @stuartpb's work which @bakulf then implemented:

• https://developer.mozilla.org/en-US/docs/Web/API/FetchObserver
• https://dxr.mozilla.org/mozilla-central/source/dom/webidl/FetchObserver.webidl
• https://dxr.mozilla.org/mozilla-central/source/dom/webidl/Request.webidl#59

That issue got closed after we landed a subset of that API in Fetch (you can finally abort a fetch). Let's discuss the remainder here (though not priorities, that's #436).

cc @wanderview

[rektide]

Both #51 and #65 are related.

I would love to see this feature added. It would enable some very interesting new reactive models that mesh well with existing HTTP-centric web-systems. Using websocket & SSE is un-ideal in that messaging done there isn't inherently resource-oriented, unlike Push support.

[annevk]

@RikkiGibson I noticed you're working for Microsoft (not that this is a requirement by all means, but having an employer that supports standards development definitely helps). I don't know how much you're into standards and work with the Edge team, but if you have some time you could help make this happen. We basically need to formalize the proposal and work on tests. There's ample implementer interest already.

[jakearchibald]

@rolivo The proposal should probably be formalised first, but if you're interested in contributing tests, https://github.com/web-platform-tests/wpt has the info. https://github.com/web-platform-tests/wpt/tree/master/fetch/api/abort - these are the tests for aborting fetch, in case they're a useful reference.

[asutherland]

As a heads up, we're probably going to remove Firefox's (never shipped, always disabled via preference) WIP (as implemented in parts 4 and 5 of this bug) related to FetchObserver in an upcoming refactor.

[piranna]

I'm not so much interested on the uploaded bytes, that I can count them myself before sending the data, that by the acknowledgedBytes, the actual bytes that the server has received, and that can be get from the TCP ACK packages but there's no high level APIs for that except if you do it yourself at application level.

[jimmywarting]

I guess with the new addition of duplex stream (sending & receiving data at the same time) also has to be taken into consideration now for how to design it

[annevk]

#607 (comment) is still applicable, but implementer interest might be reduced? As far as I know nobody is actively working on this, but if someone is interested I'd be happy to help them. Best to reach out on https://whatwg.org/chat and not comment here as it ends up pinging a lot of people.

[wanderview]

FWIW, one reason I have seen to implement FetchObserver is that using streams to observe progress of wasm bundle downloads ends up breaking bytecode caching optimizations.

[jakearchibald]

It's also unreliable for uploads right? Since you're tracking the time the data is taken from the stream, which isn't the same as the data being transferred over the network.

[piranna]

It's also unreliable for uploads right? Since you're tracking the time the data is taken from the stream, which isn't the same as the data being transferred over the network.

One thing is when data is sent over the wire, and another want is the acknowledge of the received bytes at the other end. They are different counters, and the useful and needed one is the second one, former one can be already computed at application level just by counting how much bytes we are writting on the stream, but we currently don't have a way to know how much bytes has been acknowledged on the receiver side.

[jimmywarting]

Hmm, just did some backend server stuff where i receive a request event (in particular true for deno, service worker and cloudflare workers) where things are not initiated by a fetch call
And i would like to get the progress out of them too
So a fetchObserver init option where you pass it down via 2nd argument to fetch() wouldn't work...

but this could:

globalThis.addEventListener('fetch', evt => {
  const request = evt.request
  const response = new Response(...)

  response.onprogress = 
  request.onprogress = console.log

  evt.respondWith(response)
})

[trusktr]

I think this is the last piece for fetch to have a good enough DX. Wanting to measure progress has always been tricky, so having any sort of simple API would be great.

Why not an EventTarget-based solution somewhere in the fetch API? Is there a good reason to have a new event pattern when we already have one?