content-visibility API

[victorporof]

Request for Mozilla Position on an Emerging Web Specification

• Specification Title: Display Locking rendersubtree subtree-visibility content-visibility
• Specification or proposal URL: https://github.com/WICG/display-locking https://drafts.csswg.org/css-contain-2/#content-visibility
• Mozillians who can provide input: @victorporof @dbaron

Other information

We’ve identified two main problem categories with this proposal, detailed blow.

Relationship between “wip” and “live” subtrees

Display artifacts (issue 39)

We’re not convinced that painting retained “snapshots” at the old size is desirable: when layout is altered on “locking roots”, some subtrees could have unpainted areas while others would clip. This is especially evident in suggested use-cases such as “resizing multi-pane UI with complex layout within each pane” as described here.

Starving work (issue 20)

Exhausting budgets and starving cooperative updates excessively is possible, given an unbounded number of new nodes or other potential DOM operations, over an unbounded number of frames.

It’s not clear how the implementation would avoid situations where committing a lock results in long-lasting jank, by accumulating outstanding work.

Possible suggestions involve increasing priority for budgeted update phases, or switching to synchronous work.

Starving memory (issue 14)

It’s not clear how the implementation would avoid situations where excessive amounts of memory are retained in order to cache the rendering output, by locking elements much larger than the viewport.

Possible suggestions involve checkerboarding.

Event handling (spec)

It’s not clear how the implementation would handle user input targeted at subtrees which are locked for display.

Possible suggestions involve queuing up input and replaying, or ignoring input altogether. We think that both variants lead to undesirable behavior:

• Replaying input can cause unexpected interactions, since the user may be targeting an unexpected element (script could have mutated the DOM before unlocking);
• Ignoring input can lead to potential user frustration (page would appear frozen);

Forced layout (issue 21)

It’s not clear how the implementation would handle layout queries under subtrees which are locked for display. It’s difficult to determine what layout information should be returned ideally, when querying properties which otherwise normally cause reflow.

Possible suggestions involve:

• Synchronously releasing the lock, forcing reflow;
• Returning a fully computed alternative layout, which isn’t used otherwise;
• Returning a partially computed alternative layout, determined by current budgeting;
• Returning the initially snapshotted layout, stored when the lock was acquired;
• Returning bogus values or throwing an exception;

There’s a preference for aborting and synchronously committing the lock, forcing reflow.

Altering containment (issue 28)

The API requires containment for paint, style and layout for lockable subtrees, to better reason about expected behavior of the entire page. This is done with the contain: content CSS property.

It’s not clear how the implementation would handle situations where "contain" is altered while an element is locked.

Possible suggestions involve aborting by synchronously committing the lock.

Assumptions

The proposal has a risk of prematurely baking in assumptions about current browser engine designs into future web standards.

Parallelisation assumptions

Several assumptions are made which can be at odds with parallelizable browser designs: that layout and painting can synchronously access element locking information, that there is no interruptible layout, and that layout and script are serialised.

There’s a risk to alternate browser engine designs when standardising these assumptions, especially with regards to parallelizable layout. There’s sufficient prior art for how old standards are limiting modern browser design.

Reinforcing a disconnect between DOM and VDOM

Knowing what to acquire a lock on requires certain heuristics, which might not be trivial to define, or compute. This is especially true if the burden for figuring this out is given to frameworks, which are intended to be the primary users.

Possible implementations can be fairly close in nature to existing component instance tree traversals (early stages of “diffing”), or comparing previous and next presentational intent (late stages of “diffing”, or reconciliation). This can mean multiple things:

• The proposal fits in well with existing virtual DOM architectures;
• The proposal doesn’t solve impedance mismatches between the DOM and declarative modern web framework design.

If frameworks still need to do “some kind of diffing” (loosely defined) in order to talk to the browser (acquire a lock on certain subtrees), due to their declarative architecture, this could have a risk of reinforcing the current disconnect.

[dbaron]

Thanks for writing up this list of concerns, and sorry to take a while to get to this...

I think there's a substantial list of real concerns here. Some of them can be addressed by adding complexity to the proposal -- either complexity to the browsers (particularly the browser engines that aren't the first one to implement, given how much it will likely be tied to now-unspecified browser internals), or complexity to the mental model that users of the API would need to maintain. Some of them seem fundamental to the proposal.

I think my conclusion is that (1) there's some real harm that's likely to result if this happened, although some of it could be remedied but that (2) I'd rather see the effort that was going into this work be put into other efforts to improve performance and responsiveness for the use cases that this was trying to address (or maybe into other performance improvements for other use cases; I don't want to take a position of the importance of those particular performance use cases relative to others) -- and hopefully efforts that fit in a bit better into the Web's existing API surface.

I think my inclination is to mark this as harmful, although I'd concede that harmful might be a strong word for this sort of case.

[rniwa]

FYI, the WebKit team at Apple discussed this feature internally and we've concluded that we do not support this proposal due to the issues listed here and beyond. We think the underlying use case of the need to avoid the junk whilst making a large scale DOM mutation is real and valid but like @dbaron stated above, we prefer solving this underlying issue some other way.

[chrishtr]

Some comments inline about the concerns listed regarding Display Locking. But first: I agree that locking stale pixels for content on-screen is harder and has its own set of challenges. The current MVP version of Display Locking under development does not have stale pixels. This avoids the memory starvation and hit testing issues mentioned below, so I won't respond to those comments at this time, and instead focus on the remaining ones that may still apply.

Starving work (issue 20)

Exhausting budgets and starving cooperative updates excessively is possible, given an unbounded number of new nodes or other potential DOM operations, over an unbounded number of frames.

It’s not clear how the implementation would avoid situations where committing a lock results in long-lasting jank, by accumulating outstanding work.

Possible suggestions involve increasing priority for budgeted update phases, or switching to synchronous work.

I don't think starvation or jank are unique to Display Locking. It is something that happens in many scenarios already. Increasing priority is one well-known technique for avoiding starvation. Falling
back to "synchronous" work is no worse than how DOM works today.

Forced layout (issue 21)

It’s not clear how the implementation would handle layout queries under subtrees which are locked for display. It’s difficult to determine what layout information should be returned ideally, when querying properties which otherwise normally cause reflow.

The current plan is to not force-unlock, but instead perform forced style or layout as necessary on the locked subtree, and return results which are relative to the locked root. See issue 45.

Altering containment (issue 28)

The API requires containment for paint, style and layout for lockable subtrees, to better reason about expected behavior of the entire page. This is done with the contain: content CSS property.

It’s not clear how the implementation would handle situations where "contain" is altered while an element is locked.

Possible suggestions involve aborting by synchronously committing the lock.

The current plan is to do just this - synchronously commit the lock and reject its promise. See issue 28.

Assumptions

The proposal has a risk of prematurely baking in assumptions about current browser engine designs into future web standards.

Parallelisation assumptions

Several assumptions are made which can be at odds with parallelizable browser designs: that layout and painting can synchronously access element locking information, that there is no interruptible layout, and that layout and script are serialised.

Display Locking does not assume that layout and script are serialized. Also, I don't see why layout and paint would not have access to the element locking information of the commit state from which they are layouting/painting.

There’s a risk to alternate browser engine designs when standardising these assumptions, especially with regards to parallelizable layout. There’s sufficient prior art for how old standards are limiting modern browser design.

Do you have a comparable example in mind?

Reinforcing a disconnect between DOM and VDOM

Knowing what to acquire a lock on requires certain heuristics, which might not be trivial to define, or compute. This is especially true if the burden for figuring this out is given to frameworks, which are intended to be the primary users.

Possible implementations can be fairly close in nature to existing component instance tree traversals (early stages of “diffing”), or comparing previous and next presentational intent (late stages of “diffing”, or reconciliation). This can mean multiple things:

• The proposal fits in well with existing virtual DOM architectures;
• The proposal doesn’t solve impedance mismatches between the DOM and declarative modern web framework design.

If frameworks still need to do “some kind of diffing” (loosely defined) in order to talk to the browser (acquire a lock on certain subtrees), due to their declarative architecture, this could have a risk of reinforcing the current disconnect.

I don't quite follow the concern here. Is it that Display Locking may encourage frameworks to become more declarative/separate from DOM in the future?

[victorporof]

rendersubtree

The following is an update to reflect the recent changes to Display Locking, now known as “rendersubtree”. I've also updated the title.

Summary

We have two overarching concerns with the current proposal, detailed below. In its current form, this proposal isn’t something we'd implement, and can be harmful given how it makes accessibility technology detectable. If the lack of specifics and accessibility issues are addressed, we'll reassess and consider prototyping.

Lack of specifics

Determining whether the general approach of element activation and the ability to control pre-rendering is one that we would support (and satisfies the virtual list use cases) isn’t currently possible without more detail. Being vague doesn’t just block implementing, but blocks understanding the complexity of the proposal and thus judging whether it’s worth doing.

It isn’t clear whether or not it’s expected for the final proposal to be exhaustive in defining which data about an element may be consumed and how.

To address the proposal further, we���ll need an enumeration of the expected algorithms involved in the locking and unlocking process: understanding which things do or don’t cause something to be “activated” or “deactivated” (i.e. cause the browser to change the respective displaylocking attributes).

Accessibility issues

Most (if not all) performance benefits disappear when accessibility technology is present. As a further consequence, the proposal makes accessibility technology detectable.

There are a number of issues that arise from this. One of them is fingerprinting, where clear differences in how this API behaves based on whether or not accessibility is used makes user identification easier. The other is having direct access to such data that the user might not want to share.

It isn’t clear whether the solution is to avoid having locked subtrees directly available for accessibility, such that users won’t be able to see all the content, or something else entirely. It would be good to find an implementation that also preserves the overall intended accessibility benefits, so that the proposal actually provides value over other virtual scroller implementations.

[vmpstr]

Since the last update, we've moved to a CSS version of the feature called content-visibility, with a spec draft available here: https://drafts.csswg.org/css-contain-2/#content-visibility

My hope is the spec addresses the ambiguity and other concerns. Our intent is to ship this feature in Chromium in the near future. Could you please revisit this issue, since I think it has matured substantially since the initial proposal.

[victorporof]

Thank you @vmpstr, I'll have another close look at the updated proposal and reach out if there's additional concerns.

[victorporof]

We've discussed this and have all agreed to proceed with worth-prototyping. The use cases for the feature are worth solving and the state of the proposal now is an improvement for the platform. Thank you @chrishtr and @vmpstr for engaging with us and addressing our feedback.

I believe this proposal can incentivise using the web platform directly instead of third party frameworks, specifically in case of single-page apps with “content-visibility: hidden/visible”, as well as in the case of virtual scrollers with “content-visibility: auto”. In situations where the behaviour might be rough around the edges, it’s no worse than what third party libraries provide, and the user agent is now in control of the UX.

[victorporof]

Updated the title to reflect the new name.

[bholley]

@victorporof Can you make a PR to close this out? Thanks!