VIDEO tag

[nicjansma]

I think we should provide some guidance on how browsers should handle the <video> (and probably <audio>) tags.

See this repro page: http://dev.nicj.net/video-tag.html

Findings:

Chrome

• Shows poster="" images as initiatorType="video"
• For the actual videos, Chrome sometimes (pre)downloads all or part of a video (e.g. via range requests)
  • Some, but not all of these range requests end up in ResourceTiming: In one example, one video had 3 range requests and 2 RT entries, another video had 3 range requests and 1 RT entry.
  • All of these have initiatorType="" (blank).

Edge

• Shows poster="" images as initiatorType="img"
• Doesn't show any videos in RT, even after hitting Play

Firefox

• Shows poster="" images as initiatorType="img"
• For the actual videos, Firefox sometimes (pre)downloads all or part of a video (e.g. via range requests)
  • All of the range requests show up as individual RT entries
  • All of these have initiatorType="other"

Safari

• Shows poster="" images as initiatorType="video"
• For the actual videos, Safari sometimes (pre)downloads all or part of a video (e.g. via range requests)
  • All of the range requests show up as individual RT entries
  • All of these have initiatorType="video"
  • No size is available

Unfortunately, it looks like none of the UAs agree on how ResourceTiming should reflect media elements.

I think there's a couple questions:

1. What should the image from a <video> poster="" attribute show up as?
2. What should the video from a <video> with a src="" attribute show up as?
3. What should the video from a <video><source src=""></video> element show up as?
4. How should range requests be reflected? (more talk in RT should handle multi-request fetches #21 Clarify what constitutes a "downloadable resource" #100)

My initial take: Safari's behavior seems most logical to me (though its Range request entries are hampered by lack of ResourceTiming2 transferSize / etc attributes).

Thoughts:

1. The spec suggests images should show up as the localName, i.e. video. Though this makes it harder to differentiate both the image and video downloaded from the same tag. Maybe it could be video-poster instead?
2. It seems logical that the video from a <video> with a src="" attribute shows up as video
3. A <video><source src=""></video> element could go either way: video or source, but I think video is more helpful.
4. I like consistently seeing all range requests in ResourceTiming.

[plehegar]

Indeed, Safari behavior is the most logical when reading the spec.

1. you can always use 'name' to distinguish the poster from the video (they'll have different URLs).
2. agreed
3. this would break the current spec assumption (source). Did we consider the case of custom elements? Might be worth opening a separate issue on this front.
4. agreed.

[toddreifsteck]

Agree with @plehegar on all comments

[nicjansma]

For <video src=""> and <video poster=""> differentiation, my main concern is that the name (URL) is not always sufficient to guarantee you know what type of content it is. While you could look at file extensions, they aren't always set. Since we don't get the MIME type, knowing which file is the poster vs src isn't guaranteed.

I've filed #132 to help look at the bigger picture, and determine if there are patches we want to do to Level 2 and/or if we should design something different for Level 3.

[nicjansma]

Let's build tests for all of the VIDEO cases in #138 to lock in Level 2 behavior.

We're separately considering what to do in Level 3 in #132

[yoavweiss]

@plehegar and @toddreifsteck -
While fixing up initiatorType values in Blink I realized that the current spec for the <video><source></video> case implied that it should be the local name of the element which processing triggered the download. That element is the HTMLVideoElement (HTMLSourceElement does nothing on its own...).
Compound that to the fact that the implementation would be significantly trickier if we want it to say "source", as well as the fact that it's not useful from a developer perspective, nor currently implemented by any browser... makes me want to change the test to say "video" and add a spec note indicating that this is the case.

Since both of you indicated supporting "source" over "video", I'm wondering if there was a particular reason for that. If not, I'll happily land the CL and change the test :)

[toddreifsteck]

The idea was to be as specific as possible. Given that source only functions with another "view" element, I can imagine that audio/video/picture could be the correct string and would be ok with that change.

[nicjansma]

Having <video><source /></video> report initiatorType as video makes a lot more sense to me, and as Yoav states, aligns with "... is a result of processing the element" as the <source> element does nothing on its own.