DBW: use navigator.sendBeacon in beforeunload/pagehide handlers

[ebidel]

How to check:

• look for beforeunload/pagehide handlers (can reuse https://github.com/GoogleChrome/lighthouse/blob/master/lighthouse-core/gather/gatherers/dobetterweb/document-event-listeners.js)
• check for sync xhrs or fetch calls in the handler

because user agents typically ignore asynchronous XMLHttpRequests made in an unload handler.

To solve this problem, analytics and diagnostics code have historically made a synchronous XMLHttpRequest call in an unload or beforeunload event handler to submit the data. The synchronous XMLHttpRequest blocks the process of unloading the document, which in turn causes the next navigation appear to be slower. There is nothing the next page can do to avoid this perception of poor page load performance, and the result is that the user perceives that the new web page is slow to load, even though the true issue is with the previous page.

https://developer.mozilla.org/en-US/docs/Web/API/Navigator/sendBeacon

[olingern]

@ebidel I looked at this for a while today, and wanted to ping you about a couple of things.

filename for the audit?

• uses-sendbeacon
• no-xhr-beforeUnload-pagehide

Watching network requests
Since I'm still new to the codebase, I wanted to ask a couple of questions about implementing the checks for this.

1. I see that, if I define an beforeUnload event, it's available to me through PageLevelEventListeners on the artifacts object thats passed into audit. It seems like the best way to implement this would be, check for beforeUnload or pagehide events, if present - monitor for sync xhrs or fetch like you mentioned before.
2. I'm still searching around the codebase for example on hooking into events. It seems that if either of the two closing events are present, then I should dispatch a close event and listen for sync xhrs / fetch. Do you have anywhere in the codebase where you could link me to an example of invoking events? As for monitoring, it seems that NetworkRecorder inside of lighthouse-core/lib will be of most help for this.

Cheers

[ebidel]

"uses-sendbeacon" sgtm.

For the network stuff, it looks like something clever like this could work. I didn't realize you could dispatching beforeunload yourself works!

// Inject this into page in beforePass
const observer = new PerformanceObserver(list => {
  const xhrsRequests = list.getEntries().filter(entry => {
    // Note: fetch() does not have initiatorType - https://crbug.com/649571
    return entry.initiatorType === 'xmlhttprequest';
  });
  if (xhrsRequests.length) {
    console.log(xhrsRequests);
  }
}).observe({entryTypes: ['resource']});

// Example of page code that you should add to lighthouse-cli/test/fixtures/dobetterweb/dbw_tester.html
window.addEventListener('beforeunload', e => {
  const xhr = new XMLHttpRequest();
  xhr.open('GET', 'https://www.chromestatus.com/features', false);
  xhr.send();
});

// Inject into page in `afterPass`
window.dispatchEvent(new CustomEvent('beforeunload'));
window.dispatchEvent(new CustomEvent('pagehide'));

You could wrap the observer and dispatching part in a function that gets injected into the page. Example. You might need to play around with delaying dispatching the events.

This would't tell you that the XHR was sync (unless you also grep the console output for the warning chrome throws when sync xhrs are used), but I don't think it would hurt to always suggest navigator.sendBeacon for all requests made in beforeunload and pagehide handlers.

[igrigorik]

Does this need to be instrumented in the page itself? Instead, could you look at the initiator field in the trace event log and detect any !beacon requests initiated after unload?

PerfObserver does not guarantee synchronous delivery of these events, and since page is being unloaded, there is high likelihood that you may miss requests with the above pattern -- e.g. inject an image beacon and allow page to continue unloading, etc.

[ebidel]

Just looking at the --save-artifacts output from running against https://googlechrome.github.io/samples/beacon/....`navigator.sendBeacon` requests don't seem to show up in our network trace. Perhaps this is because they're not captured during the pass (e.g. the request is made after the fact in a pagehide handler).

The other thing we miss out on with using the network trace is line/col/source info:

[ebidel]

@igrigorik I also found that navigator.sendBeacon requests don't show up in PerformanceObserver. Is that known? Should they?

[igrigorik]

Yes, they should. Sounds like a bug. If you have a test page.. can you please file a bug?

[olingern]

@ebidel @igrigorik is this something I can tackle without navigator.sendBeacon exposed, or should I hold off until the bug is fixed?

[ebidel]

@olingern the bug with PerformanceObserver doesn't apply here. We don't care about pages using navigator.sendBeacon. They're doing the right thing. We only care about pages that are using XHR on page unload :)

I'd experiment with the technique I highlighted in #871 (comment) to see if we catch PerformanceObserver callbacks in unbeforeunload. They're not guaranteed to be delivered on page unload, but I believe they were when I tried this out quickly.

Definitely add some tests to https://github.com/GoogleChrome/lighthouse/blob/master/lighthouse-cli/test/fixtures/dobetterweb/dbw_tester.html and let us know.

[paulirish]

Need to get a complete new audit proposal on this one: https://github.com/GoogleChrome/lighthouse/blob/master/docs/new-audits.md

It's definitely worthwhile, but will take some work.

[paulirish]

in m75, sync xhr in "page dismissal" is attempted to be removed.
https://bugs.chromium.org/p/chromium/issues/detail?id=827324

I'm pretty sure there is a console.log/violation we could use.

Seems good, though we'd have to dismiss the page during gathering. We could try to fire their handlers, though..

But if this removal sticks, then it already did our job for us.

[connorjclark]

window.addEventListener('beforeunload', function () {
   var request = new XMLHttpRequest();
request.open('GET', '/bar/foo.txt', false);  // `false` makes the request synchronous
request.send(null);

if (request.status === 200) {
  console.log(request.responseText);
}
});

VM26:1 Uncaught DOMException: Failed to execute 'send' on 'XMLHttpRequest': Failed to load 'https://developer.mozilla.org/bar/foo.txt': Synchronous XHR in page dismissal.