Incorrect detection of MediaCapabilities on Chromecast, resulting in MEDIAKEYS_UNSUPPORTED error

[alexgerv]

Have you read the FAQ and checked for duplicate open issues?
yes

What version of Shaka Player are you using?
4.3.4 (default version included in CAF)

Can you reproduce the issue with our latest release version?
Did not attempt yet, as issue is not reproducible at will, only sporadically.

Can you reproduce the issue with the latest code from main?
Did not attempt yet, as issue is not reproducible at will, only sporadically.

Are you using the demo app or your own custom app?
custom receiver app using CAF

If custom app, can you reproduce the issue using our demo app?
No. Issue is not reproducible at will, only sporadically.

What browser and OS are you using?
Chromecast (all models)

What are the manifest and license server URIs?
Issue happens before starting playback. We can provide URLs if necessary.

What configuration are you using?
Default Chromecast configuration.

What did you do?
We have a spike of MEDIAKEYS_UNSUPPORTED errors after we released a new version of our Chromecast Receiver App with the latest CAF (3.0.0111) which also updated Shaka to version 4.3.4.

From the data we have, a user would try to start a new casting session and immediatly get this error, but after disconnecting and trying to cast again, it would work.

Reverting the Shaka version to 3.2.2 (the one bundled with the previous CAF) solves this issue, but we are trying to understand the cause of this error so we can use a more recent version of Shaka.

We never managed to reproduce the issue ourselves, but looking at the data we have from production, it seems like there is sometimes a problem during initialization which causes the device/Chrome/Shaka to report as not supporting any video codecs. We see that when the error happens, our calls to canDisplayType on the CAF context also always report false for that session even for parameters we know should work on that device. For example, a Chromecast v3 would report as not being able to display a 1080p 30fps avc1.640028 stream which it easily should be able to do.

The fact this error is only happening with newer versions of Shaka is really puzzling us, since from what we can see Shaka simply calls navigator.requestMediaKeySystemAccess and throws REQUESTED_KEY_SYSTEM_CONFIG_UNAVAILABLE or NO_RECOGNIZED_KEY_SYSTEMS when it fails and those errors are then mapped to MEDIAKEYS_UNSUPPORTED by the CAF.

The only thing we can think of is that there were some changes in the timings of the player initialization or in the polyfills used to manage that.

[alexgerv]

Hi @joeyparrish :) knowing the Cast focus of the Shaka team, do you think you could have a look at this issue ?
Very though one because we don't have repro steps, but also very annoying for our customers because they get random failures to initiate streams. Thanks in advance !

[avelad]

@theodab @JulianDomingo can you review it? Thanks

[simonlary]

From looking more into it, the issue is caused by the changes in #4727. The navigator.mediaCapabilities.decodingInfo() polyfill now checks cast.__platform__.canDisplayType() on Chromecast devices, but this sometimes returns false when the device is actually able to play the media.

This bad reporting seems to happen in setups where the Chromecast device is being casted to, but the display is not actually opened. Like when a user starts to cast to have the audio on their AV receiver, but don't open their TV.

Overriding the new polyfill with the old one fixes the issue, but we would obviously like a better long-term solution. 🙂

[alexgerv]

@JulianDomingo would you have any opinion on the reliability of cast.__platform__.canDisplayType() ? It seems this check sometimes fails, which causes misdetection of capability in some cases when display is not fully ready upon loading the cast receiver application.

[theodab]

I spent a while examining the code behind cast.__platform__.canDisplayType().

My current theory is that in the situation @simonlary described above (display has not yet opened), it might not yet know what the maximum resolution is. Before the maximum resolution is known, it looks like the code defaults to it being 0 by 0. If that is the case, it would reject any content with a known width or height... and we attach the width and height of the video, in that polyfill.

I don't have much more time to look into this issue, unfortunately. It sounds like you have a setup to test it, though, so you could try commenting out these lines in the polyfill:

shaka-player/lib/polyfill/media_capabilities.js

Lines 271 to 272 in 3a14047

	displayType +=
	`; width=${videoConfig.width}; height=${videoConfig.height}`;

If we stop providing the width and height and suddenly the issue stops reproducing, that would suggest it is due to the canDisplayType method rejecting content with width and height values before the maximums are known.

If that is the case, we might be able to handle this on the Shaka Player level by periodically running a query along the lines of:

cast.__platform__.canDisplayType('video/mp4; codecs="avc1.42E01E"; width=1; height=1')

And delaying calls to decodingInfo until that query passes. Perhaps with a timeout attached, in case the display never powers on. It'd be crude, but a more immediate fix than something that would involve changes to the Cast end.

[joeyparrish]

I tested this on both models I have at home, and I found that the Android-based devices (latest) do not have this problem. Only the older Linux-based models have this issue.

Sadly, this means we can't rely on a firmware update to fix it, since some of these are devices that no longer receive updates. We will have to work around it either in the Cast SDK or in the player. I have not yet determined if a fix in the Cast SDK is possible.

[joeyparrish]

In my judgement, filtering by width/height is not critical. Streams should already be limited by resolution on Cast devices in a very simplistic way through the max HW resolution. There, we either cap streams as 1080p or 4k. If a 4k-capable device is unplugged when a stream starts, it would act as if it couldn't play 4k.

Given this, we could either:

1. Always ignore resolution in canDisplayType
   or
2. Run a synchronous 1x1 check on load, and skip resolution checks if that fails

Either way, a basic resolution cap will still be implemented. The decoders on these devices should generally still be able to decode 1080p, which will be the fallback cap. So I don't see the harm in it.

I am open to thoughts from everyone else before we proceed.

[alexgerv]

Hi Joey,
I confirm your observations from above, only Linux-based devices were affected.

From @simonlary and my perspective, solution number 1 (always ignoring the resolution in canDisplayType, like before) would be perfectly fine.

[joeyparrish]

Internally, we are discussing several related steps, though only the first addresses this issue:

1. removing width/height in canDisplayType
2. memoize/cache canDisplayType results to reduce startup latency (these are synchronous calls involving IPC)
3. using MediaCapabilities on certain Cast devices instead of always using canDisplayType (needs research into which devices have trustworthy implementations of MCap)

[avelad]

@joeyparrish can we enable MCap for Android Cast-devices? (See: 65903aa, I think we should remove native TS support everywhere for this to work properly)

[joeyparrish]

@joeyparrish can we enable MCap for Android Cast-devices? (See: 65903aa, I think we should remove native TS support everywhere for this to work properly)

It depends on whether or not MCap is correctly implemented there. We're investigating implementations with the various platform teams. There are broadly three different types of OSes and runtime implementations on Cast devices: Linux, Fuchsia, and Android.

[avelad]

@joeyparrish can we enable MCap for Android Cast-devices? (See: 65903aa, I think we should remove native TS support everywhere for this to work properly)

It depends on whether or not MCap is correctly implemented there. We're investigating implementations with the various platform teams. There are broadly three different types of OSes and runtime implementations on Cast devices: Linux, Fuchsia, and Android.

Do you have the result of the investigation? Thanks!

[simonlary]

@avelad You closed this issue referring to #6656, but that pull request disables the problematic polyfill only on Android-based and Fuschia-based based Chromecast devices while the incorrect detection only happens on Linux-based Chromecast devices. Was something else changed to fix the problem on Linux-based Chromecast devices?

[avelad]

Sorry, was a mistake