Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4.6 Integrate the processing model with Fetch #252

Closed
16 tasks done
yoavweiss opened this issue Jan 18, 2021 · 10 comments
Closed
16 tasks done

4.6 Integrate the processing model with Fetch #252

yoavweiss opened this issue Jan 18, 2021 · 10 comments
Assignees

Comments

@yoavweiss
Copy link
Contributor

yoavweiss commented Jan 18, 2021

  • The algorithm should be called (probably from Fetch) with clear inputs.
  • Step 1 - need to replace "fetched by a cross-origin stylesheet fetch with no-cors policy" with defined concepts related to the Request object.
  • Step 2 - replace "triggered by..." with well-defined concepts related to the Request object
  • Step 4 - startTime needs to be recorded as part of Fetch and communicated to the processing mode
  • Step 5 - The processing model of initiatorType needs to be defined in terms of Fetch. See initiatorType processing model needs to be defined in terms of Fetch #246
  • Step 6 - define in terms of Fetch. See Definition of name needs to be more precise #244
  • Step 7 - fetchStart needs to be recorded as part of Fetch and communicated to the processing mode
  • Step 8 - Seems spurious if a fetch is not initiated or terminated. Needs to fall out from the integration between Fetch and this processing model. May require definition of the Memory/Preload Cache. See Define the "Preload Cache" whatwg/fetch#590
  • Step 9 - workerStart definition should be set by Fetch/ServiceWorkers and communicated to this processing model
  • Step 12 - replace "resource is fetched from" with well-defined concepts
  • Steps 13-15 - domain lookup and connection times are not currently defined in Fetch. Need to investigate to see if/how they can be better defined.
  • Step 16 - "secure transport is used" needs to be replaced with Fetch concepts. Better definition of the connection times needs to be investigated.
  • Step 17-19 - Set requestStart, responseStart and responseEnd of part of Fetch processing.
  • Step 19.2 - define transferSize, encodedBodySize and decodedBodySize in terms of Fetch. Note that on transferSize, we may want to omit the header size: transferSize might reveal HttpOnly cookies #238
  • Step 21 - Define redirect processing as part of the Fetch processing model
  • Define if entry is queued before or after the load event Clarify when entry is added to performance entry buffer #82
@noamr
Copy link
Contributor

noamr commented Mar 1, 2021

I'm seeing to ways to possible ways to go about this:

  1. keep the data as a struct inside fetch, and call resource timing only at the end of the response with all the info. This would basically mean keeping a lot of timing information inside the fetch spec.
  2. Call resource timing from multiple locations with different parameters.

I'm currently leaning towards (2), but would love to run it by you first, @annevk and @yoavweiss. Note that the implementations look more like (1), where the network stack saves all the info internally in its own data structure until it's time to "mark".

@annevk
Copy link
Member

annevk commented Mar 1, 2021

What do you mean with "at the end of the response"? When it's returned to the caller or when it reaches end-of-body? If the data structures are actually tightly coupled with a response I think it would be better to specify it that way. That also enables new types of APIs and seems like a more solid foundation than calling out to something. (It's also not immediately clear to me how that calling out would work across conceptual threads and such.)

@noamr
Copy link
Contributor

noamr commented Mar 1, 2021

What do you mean with "at the end of the response"? When it's returned to the caller or when it reaches end-of-body? If the data structures are actually tightly coupled with a response I think it would be better to specify it that way. That also enables new types of APIs and seems like a more solid foundation than calling out to something. (It's also not immediately clear to me how that calling out would work across conceptual threads and such.)

Yes, they're pretty tightly coupled. It's maybe better to save everything as a structure in fetch and pass it as a parameter to resource-timing when it's time to mark (as the implementations do anyway).

@annevk
Copy link
Member

annevk commented Mar 1, 2021

Yeah, we have structs to be clear, see https://fetch.spec.whatwg.org/#fetch-params for an example. (Although I suspect here we'd store it on the response.)

@noamr
Copy link
Contributor

noamr commented Mar 2, 2021

What do you mean with "at the end of the response"? When it's returned to the caller or when it reaches end-of-body? If the data structures are actually tightly coupled with a response I think it would be better to specify it that way. That also enables new types of APIs and seems like a more solid foundation than calling out to something. (It's also not immediately clear to me how that calling out would work across conceptual threads and such.)

Looking closer, data structures are more tightly coupled with the request than with the response. For example, some metrics such as workerStart and fetchStart need to be measured in some cases before the response object is constructed.
We can either create one struct and associate it with the request, or create separate request and response timing structs, which seems perhaps unnecessary.

@annevk
Copy link
Member

annevk commented Mar 2, 2021

That makes sense and some of that duplication exists today as well. I don't have a good handle on when these things are exposed, but if they are not exposed before a response is exposed, I think we should end up exposing them on the response and use "fetch params" or maybe "request" as a place to do bookkeeping until there is a response. If they are exposed at some other point fetching might need new callbacks, but I hope that's not needed.

@noamr
Copy link
Contributor

noamr commented Mar 2, 2021

OK, I'll put it as a field of fetch params and later expose it in the response.

noamr added a commit to noamr/fetch that referenced this issue Mar 2, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 7, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 7, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 7, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 8, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 8, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 8, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 9, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 9, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 9, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 10, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 10, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 11, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 11, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 11, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 11, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 11, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 11, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 15, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 15, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 15, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 16, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/xhr that referenced this issue Mar 17, 2021
When response is a network error or body is fully read,
finalize and report the response.

Depends on whatwg/fetch#1185

See:
w3c/resource-timing#261
and w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 17, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 17, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
noamr added a commit to noamr/xhr that referenced this issue Mar 17, 2021
When response is a network error or body is fully read,
finalize and report the response.

Depends on whatwg/fetch#1185

See:
w3c/resource-timing#261
and w3c/resource-timing#252
noamr added a commit to noamr/fetch that referenced this issue Mar 24, 2021
Created a "fetch timing info" struct to hold the bookkeeping
necessary for resource timing.

Populated it with some of the required values, leaving some of
them for later patches as this is a big undertaking.

See w3c/resource-timing#252
annevk added a commit to whatwg/fetch that referenced this issue Mar 25, 2021
This defines the foundation for the PerformanceResourceTiming API and its properties. Note that the names used here are not a 1:1 match, e.g., post-redirect start time is fetchStart in the API.

Note that Fetch doesn't decide when to create and enqueue PerformanceResourceTiming objects, but rather creates a data structure that holds the different time values, and it's up to the caller to invoke the newly-added finalize and report timing. Fetch also has a new callback, processResponseDone, for when the response errors or EOFs, which lets callers like the fetch() API invoke finalize and report timing at the right time. This change also makes fetch() adopt that.

It's the first phase for addressing w3c/resource-timing#252, which contains a link to the other required steps to address Resource Timing/Navigation Timing/HTML/Fetch integration.

Corresponding Resource Timing PR: w3c/resource-timing#261. (Once that lands there'll be a subsequent change to Fetch to resolve the remaining TODO.)

Co-authored-by: Anne van Kesteren <annevk@annevk.nl>
@noamr
Copy link
Contributor

noamr commented Mar 29, 2021

  • The algorithm should be called (probably from Fetch) with clear inputs.

Will be closed by #261

  • Step 1 - need to replace "fetched by a cross-origin stylesheet fetch with no-cors policy" with defined concepts related to the Request object.

Closed by whatwg/fetch#1185

  • Step 2 - replace "triggered by..." with well-defined concepts related to the Request object

Closed by whatwg/fetch#1185

  • Step 4 - startTime needs to be recorded as part of Fetch and communicated to the processing mode

Closed by whatwg/fetch#1185

Will be closed by #261

Closed by whatwg/fetch#1185

  • Step 7 - fetchStart needs to be recorded as part of Fetch and communicated to the processing mode

Closed by whatwg/fetch#1185

  • Step 9 - workerStart definition should be set by Fetch/ServiceWorkers and communicated to this processing model

Closed by Closed by whatwg/fetch#1185
Note that for navigation timing workerStart means something different, see w3c/ServiceWorker#1575

  • Step 12 - replace "resource is fetched from" with well-defined concepts

Closed by whatwg/fetch#1185

  • Steps 13-15 - domain lookup and connection times are not currently defined in Fetch. Need to investigate to see if/how they can be better defined.

Closed by whatwg/fetch#1185

  • Step 16 - "secure transport is used" needs to be replaced with Fetch concepts. Better definition of the connection times needs to be investigated.

Closed by whatwg/fetch#1185

  • Step 17-19 - Set requestStart, responseStart and responseEnd of part of Fetch processing.

Closed by whatwg/fetch#1185

#238 still holds. The rest are closed by whatwg/fetch#1185

  • Step 21 - Define redirect processing as part of the Fetch processing model

Closed by whatwg/fetch#1185

@noamr
Copy link
Contributor

noamr commented Nov 8, 2021

I believe this can be closed, and we can continue tracking whatwg/html#6542.
WDYT @yoavweiss ?

@yoavweiss
Copy link
Contributor Author

LGTM

noamr added a commit to noamr/xhr that referenced this issue Dec 14, 2021
When response is a network error or body is fully read,
finalize and report the response.

Depends on whatwg/fetch#1185

See:
w3c/resource-timing#261
and w3c/resource-timing#252
annevk pushed a commit to whatwg/xhr that referenced this issue Feb 11, 2022
When response is a network error or body is fully read, finalize and report the response.

Depends on whatwg/fetch#1185.

Also see w3c/resource-timing#261 and w3c/resource-timing#252.
Bishwarupjee pushed a commit to Bishwarupjee/xhr that referenced this issue Jan 31, 2024
When response is a network error or body is fully read, finalize and report the response.

Depends on whatwg/fetch#1185.

Also see w3c/resource-timing#261 and w3c/resource-timing#252.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants