Prefetching resources degrades TBT score and over performance score

[shappir]

FAQ

• Yes, my issue is not about variability or throttling.
• Yes, my issue is not about a specific accessibility audit (file with axe-core instead).

URL

https://www.nextinsurance.com/

What happened?

Prefetching resources increases TBT and lower performance score as measured by Lighthouse. Checked using PageSpeed Insights and local Lighthouse installation, and experienced consistent behavior in both cases. We repeatedly enabled and disabled the prefetch and saw the same behavior.

1. Prefetching is implemented by dynamically adding <link rel="prefetch" ...> into the HTML
2. The number of prefetches is large: ~150 resources
3. Adding prefetch links happens late in the session: after JS is downloaded and then after requestIdleCallback
4. Adding the links is split every 25ms and waits on requestIdleCallback again to continue

Graph shows TBT improvement when prefetch was disabled

What did you expect?

No impact of late executing prefetch on TBT or overall score
Especially given that code never executes for more than 25ms and the continues after requestIdleCallback

What have you tried?

1. Disabling prefetch reduces TBT and increases score
2. Enabling prefetch increases TBT and reduces score
3. No impacts was seen either way on field CWV scores
4. Testing using Dev Tools profile tab showed no long tasks due to prefetch, regardless of network or CPU throttling

How were you running Lighthouse?

node, PageSpeed Insights, Chrome DevTools

Lighthouse Version

10.3.0

Chrome Version

No response

Node Version

No response

OS

No response

Relevant log output

No response

[adamraine]

Is it possible to see two version of the page, one with prefetch enabled and one with prefetch disabled? Or perhaps some other minimum repro case would work.

This is possibly related to our requestIdleCallbackShim used for simulated throttling.

Can you try enabling DevTools throttling and report the difference in page performance?

[shappir]

The vast majority of our prefetches are to resources in the app.nextinsurance.com subdomain. Could it be sufficient for your tests to block that subdomain?

If not we could add a URL param that disables prefetching. Would that be good enough?

My guesses:

1. Maybe prefetch requests extend TTI since they count as network activity. If that's the case then since TBT is measured until TTI, this will impact its value. I think prefetch, and also beacon, should be ignored for TTI. (Maybe ignore all network requests that have "lowest" priority.)
2. While we have limited the tasks that generate the prefetch DOM nodes to 25ms, maybe the simulated throttling stretches them to appear to be longer than that.

[adamraine]

If not we could add a URL param that disables prefetching. Would that be good enough?

Depending on how the resources are used this may have side-effects that dilute the results. I'll give it a try though.

Maybe prefetch requests extend TTI since they count as network activity. If that's the case then since TBT is measured until TTI, this will impact its value. I think prefetch, and also beacon, should be ignored for TTI. (Maybe ignore all network requests that have "lowest" priority.)

This is an interesting point, however I would expect prefetch to make TTI shorter since the resources would be fetched near the beginning of the navigation rather than the end.

[adamraine]

Just blocking prefetch requests does reduce TTI and reduces TBT in my testing, however this is expected since we are removing the requests completely rather than removing their prefetch. @shappir in your original results, were you testing with the requests completely blocked or with prefetch disabled?

[shappir]

however I would expect prefetch to make TTI shorter since the resources would be fetched near the beginning of the navigation rather than the end.

This is prefetch, not preload. It is used to fill the cache with resources for the next navigation, not the current one. This is why it's done so late in load process.

were you testing with the requests completely blocked or with prefetch disabled?

Wholly disabled - we turned off the code that generated the prefetch links. I will create a URL param that completely disables prefetch and update you as soon as it's deployed. Hopefully later today.

[shappir]

Sorry for the delay - I've been ill.

We added a URL param that disables the prefetch mechanism: no_prefetch (regardless of value):

• With prefetch: https://www.nextinsurance.com/
• Without prefetch: https://www.nextinsurance.com/?no_prefetch

Due to A/B tests occasionally the page may be redirected. This can impact the results so watch out for that.

I've run PSI with and without the URL parameter and see a significant impact on TBT (mobile):

• Without parameter TBT is 1,530 ms and overall score is 67
• With parameter TBT is 880 ms and overall score is 74
  These results are pretty consistent for me - images attached.

Without parameter:

With parameter:

[adamraine]

The TTI is somewhat worse when the prefetch links are included, but I don't think this can explain the large difference in TBT alone. FWIW I tested both urls using DevTools throttling and the results are similar:

With Prefetch
Without Prefetch

Perhaps something can be adjusted in our lantern simulations?

[shappir]

This website is implemented using NextJS / React, which uses SSR. The prefetch is implemented on client-side only (not part of the SSR). The prefetch elements are added to the DOM by client-side script only. This script is triggered by requestIdleCallback, which itself is only invoked after hydration finishes.

Given this, for real-world sessions the prefetch mechanism should not have any impact on FCP. Yet in these LH recordings it does. (Note that in my PSI tests it doesn't.)

The prefetch mechanism is intentionally implemented in this way and executed late in the page's load process precisely in order to precent adverse impact on load performance.