Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] The media could not be loaded #4027

Closed
Yumega opened this issue Aug 4, 2023 · 35 comments · Fixed by #4037
Closed

[Bug] The media could not be loaded #4027

Yumega opened this issue Aug 4, 2023 · 35 comments · Fixed by #4037
Labels
bug Something isn't working

Comments

@Yumega
Copy link

Yumega commented Aug 4, 2023

Describe the bug

most videos can play, but only a few videos can't play

Steps to Reproduce

xxx.com/watch?v=Srjk6_j8J04
Logs

The media could not be loaded, either because the server or network failed or because the format is not supported.
Screenshots

image

Additional context

@Yumega Yumega added the bug Something isn't working label Aug 4, 2023
@unixfox
Copy link
Member

unixfox commented Aug 4, 2023

Is this a public instance or a private one?

@Yumega
Copy link
Author

Yumega commented Aug 4, 2023

Is this a public instance or a private one?

my selfhost invidous

@AncientMicrowave
Copy link

I've been having the same problem with my private instance lately as well. It seems to happen more often with newer videos, but I haven't done any testing to confirm.

@unixfox
Copy link
Member

unixfox commented Aug 4, 2023

Can you look at the Network tab from the devtools and check the requests, see if there are any errors. If so please report them

I did test with the video ID given and that was working fine for me: https://invidious.projectsegfau.lt/watch?v=Srjk6_j8J04

@rockerBOO
Copy link

rockerBOO commented Aug 4, 2023

Specifically for me it returns

2023-08-04 14:34:00 UTC [info] 403 GET /videoplayback?expire=1691181044&...

And does so in rapid fashion (3 per second at least) on the frontend. Maybe it's making too many requests?

Current version: 2023.08.03-bfca043c

@unixfox
Copy link
Member

unixfox commented Aug 4, 2023

Specifically for me it returns

2023-08-04 14:34:00 UTC [info] 403 GET /videoplayback?expire=1691181044&...

And does so in rapid fashion (3 per second at least) on the frontend. Maybe it's making too many requests?

Current version: 2023.08.03-bfca043c

Which instance (public or private)? Which video ID?

@rockerBOO
Copy link

Specifically for me it returns

2023-08-04 14:34:00 UTC [info] 403 GET /videoplayback?expire=1691181044&...

And does so in rapid fashion (3 per second at least) on the frontend. Maybe it's making too many requests?
Current version: 2023.08.03-bfca043c

Which instance (public or private)? Which video ID?

Private, videoId: nxKNibbjrbQ

Tried HD720 and DASH quality.

HD720 generally works OK.

But DASH does the rapid fire and 403. Now getting 206s in rapid fire.

2023-08-04 15:54:13 UTC [info] 206 GET /videoplayback?expire=1691185995&...

Noting that this happens on newer videos only as many older videos work fine. But occasionally older videos might stop working for me but work fine after waiting some time. I think it might be related to trying to watch newer videos and then some rate limiting is happening (due to the rapid fire of the requests) or something.

@xortuna
Copy link

xortuna commented Aug 4, 2023

Having similar issues on multiple instances, but only on very new videos:
https://invidious.projectsegfau.lt/watch?v=uMxrDOvzD1o

I know there was an old issue with youtube being slow with their back end processing 720p that caused a similar issue last year, but this is happening on dash and 720p this time.

Seem to be getting 403 errors on videoplayback:
Screenshot 2023-08-04 190137

EDIT:
and just like that - the video referenced is now working. So probably some weird YouTube backend stuff (this was after watching the video in embedded mode on YT)

@a7maadf
Copy link

a7maadf commented Aug 4, 2023

I'm encountering the same problem on my private instance. I have attempted nearly every solution, including rebuilding the instance from scratch. For now, my temporary workaround is viewing the video on a different public instance.

@thebozzcl
Copy link

thebozzcl commented Aug 5, 2023

EDIT: NVM, I just tried disabling Pi-hole and the problem went away. Time to look at my DNS logs.

@anoo2niem
Copy link

anoo2niem commented Aug 5, 2023

Running private instance here, lately running into this issue more frequently aswell.

Edit; sometimes running into this issue in a video a already half watched before.
Mostly systemctl restart invidious.service resolves the issue at my end. But the frequency of this happening is increasing indeed.

@mycosavant
Copy link

mycosavant commented Aug 5, 2023

I'm having this same issue on multiple public instances, including the links provided in this thread.

image

@absidue
Copy link
Contributor

absidue commented Aug 5, 2023

Question for everyone in the thread, have you enabled video proxying?

I'm wondering if YouTube might not like the IP of the server requesting the watch page being different to the computer that is actually requesting the video and audio data.

@anoo2niem
Copy link

Question for everyone in the thread, have you enabled video proxying?

I'm wondering if YouTube might not like the IP of the server requesting the watch page being different to the computer that is actually requesting the video and audio data.

For me i can confirm, proxying enabled

@xortuna
Copy link

xortuna commented Aug 5, 2023

Tried enabled and disabled didn’t seem to make a difference at the time, yet to have the issue today

@a7maadf
Copy link

a7maadf commented Aug 5, 2023

Allso, D

Question for everyone in the thread, have you enabled video proxying?

I'm wondering if YouTube might not like the IP of the server requesting the watch page being different to the computer that is actually requesting the video and audio data.

When I use proxying, I notice the issue on all videos. But when I turn it off, only a few videos have that problem. I've been scratching my head trying to figure out what links those specific videos. The issue seems to pop up with the 'googlevideo.com' link. I even tried turning off my VPN, disabling my custom DNS, and switching both devices and networks. But no luck – none of these tweaks made any difference.

image

@FavoritoHJS
Copy link

FavoritoHJS commented Aug 5, 2023

you beat me to making this issue...guess i'll copy-paste what I would have submitted in case it is helpful.


Describe the bug

Every now and then, a video fails to load with a "The media could not be loaded[...]" error. After some time the error disappears.

Steps to Reproduce

  1. Watch a lot of videos. This issue is extremely rare [EDIT 05-08:or it was back in the 27th of July when I started writing this report...], don't be surprised if it takes a long time to trigger.
  2. One of the videos will say "The media could not be loaded[...]".
    Notice the video continually refreshes, like it's trying to load... and failing every time.
  3. After a few minutes, the video magically loads like normal.

Additional context

Something interesting is that, when this bug happens, it attempts to play from the instance, like using a proxy... but the option to use a proxy is disabled.
I wonder if this could be poor handling of rate limiting or general network instability, as forcing a proxy is normal behavior for if a video fails to load.

Comparing the request of a video with this problem, and videos without this problem, I found the following:

  • The "spc" argument is considerably longer with a faulty video than the same video (or most other videos!) when actually working(42 instead of 31, a change of 11 characters)
  • Additionally, it had a longer than usual "sig"... though I'm doubtful about the importance of this, since a known-good video also had these properties.
@Joshndroid
Copy link

Private, self hosted instance.
I have been seeing this quite recently as well.
I can see that on a newer video I seem to get it more often.
I also seem to get it more on DASH quality than on HD720 (so i have mine just set to HD720)
Proxy on or off didn't make a difference

@andre4ik3
Copy link

andre4ik3 commented Aug 6, 2023

Related: NewPipe had the same issue, but they fixed it by implementing a workaround in v0.25.2.

(As far as I understand, correct me if I'm wrong) This is a new integrity check YouTube is currently rolling out through A/B testing. The 403 error means Invidious is failing the integrity check and thus YouTube is denying it access to the video.

The reason that it only happens on some videos, and that the rate seems to be increasing, is that YouTube is rolling it out as an A/B test, meaning they start with a small % of videos, then increase as time goes on until in the end all videos have this protection and Invidious won't be able to play anything at all.

The workaround that works the best for me is just waiting. Go watch a different video, come back in 5-10 mins, it should work. If it doesn't, repeat until it does. (You are basically rolling the dice whether you will get this protection or not every time you load the video)

@unixfox
Copy link
Member

unixfox commented Aug 6, 2023

Thanks @andre4ik3. We were made aware of this special parameter a month ago but did not yet implement: https://github.com/TeamNewPipe/NewPipeExtractor/pull/1084/files#diff-e07bd9a3dfb657e0ac68c2847533825c0c25d9cc8486c9a5a0dab1d413d54d42R986

We will implement it.

@a7maadf
Copy link

a7maadf commented Aug 6, 2023

Thanks @andre4ik3. We were made aware of this special parameter a month ago but did not yet implement: https://github.com/TeamNewPipe/NewPipeExtractor/pull/1084/files#diff-e07bd9a3dfb657e0ac68c2847533825c0c25d9cc8486c9a5a0dab1d413d54d42R986

We will implement it.

Thank you for your efforts

@anoo2niem
Copy link

Thanks @andre4ik3. We were made aware of this special parameter a month ago but did not yet implement: https://github.com/TeamNewPipe/NewPipeExtractor/pull/1084/files#diff-e07bd9a3dfb657e0ac68c2847533825c0c25d9cc8486c9a5a0dab1d413d54d42R986
We will implement it.

Thank you for your efforts

Agreed, many thanks for this project

@syeopite
Copy link
Member

syeopite commented Aug 6, 2023

Here's a quick patch that implements NewPipe's workaround. Can anyone see if this fixes the problem? Please clear or wait until the video cache expires when doing so.

Patch
From 2f6b2688bb8042c29942e46767dc78836f21fb57 Mon Sep 17 00:00:00 2001
From: syeopite <syeopite@syeopite.dev>
Date: Sun, 6 Aug 2023 12:20:05 -0700
Subject: [PATCH] Use workaround for fetching streaming URLs

YouTube appears to be A/B testing some new integrity checks. Adding the
parameter "CgIQBg" to InnerTube player requests appears to workaround
the problem

See https://github.com/TeamNewPipe/NewPipeExtractor/pull/1084
---
 src/invidious/videos/parser.cr | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/src/invidious/videos/parser.cr b/src/invidious/videos/parser.cr
index 9cc0ffdc..2a09d187 100644
--- a/src/invidious/videos/parser.cr
+++ b/src/invidious/videos/parser.cr
@@ -55,8 +55,9 @@ def extract_video_info(video_id : String, proxy_region : String? = nil)
   client_config = YoutubeAPI::ClientConfig.new(proxy_region: proxy_region)
 
   # Fetch data from the player endpoint
-  # 8AEB param is used to fetch YouTube stories
-  player_response = YoutubeAPI.player(video_id: video_id, params: "8AEB", client_config: client_config)
+  # CgIQBg is a workaround for streaming URLs that returns a 403.
+  # See https://github.com/iv-org/invidious/issues/4027#issuecomment-1666944520
+  player_response = YoutubeAPI.player(video_id: video_id, params: "CgIQBg", client_config: client_config)
 
   playability_status = player_response.dig?("playabilityStatus", "status").try &.as_s
 
@@ -135,8 +136,9 @@ end
 
 def try_fetch_streaming_data(id : String, client_config : YoutubeAPI::ClientConfig) : Hash(String, JSON::Any)?
   LOGGER.debug("try_fetch_streaming_data: [#{id}] Using #{client_config.client_type} client.")
-  # 8AEB param is used to fetch YouTube stories
-  response = YoutubeAPI.player(video_id: id, params: "8AEB", client_config: client_config)
+  # CgIQBg is a workaround for streaming URLs that returns a 403.
+  # See https://github.com/iv-org/invidious/issues/4027#issuecomment-1666944520
+  response = YoutubeAPI.player(video_id: id, params: "CgIQBg", client_config: client_config)
 
   playability_status = response["playabilityStatus"]["status"]
   LOGGER.debug("try_fetch_streaming_data: [#{id}] Got playabilityStatus == #{playability_status}.")
-- 
2.41.0

@andre4ik3
Copy link

Looks like the patch is working. After setting up a Linux VM and figuring out how to install all the dependencies, I did a before and after of the patch. Before the patch, clicking on random videos in trending, I'd say there is like a 30% chance of getting the error. After applying the patch and clicking more videos (lower down in the trending page, not the same ones because clicking the same ones will still result in error until video cache clears), I haven't had a single media error. Nice job!

@Despernal
Copy link

I have patched the source and built the docker container locally and switched my docker stack over and have not had the problem anymore. I was having it happen non stop all day, have clicked and played about 60 videos now with no problem.
Thank you for the patch and your hard work on this project, very thankful.

@m4teh
Copy link

m4teh commented Aug 6, 2023

Fantastic work all. Definitely been hit hard in the past week with most videos 403'ing. I was worried the "crackdown" was going to be the start of the end of the Invidious glory days, but replies above rest for now.

@a7maadf
Copy link

a7maadf commented Aug 6, 2023

Here's a quick patch that implements NewPipe's workaround. Can anyone see if this fixes the problem? Please clear or wait until the video cache expires when doing so.
Patch

Deployed it to my instance; no issues whatsoever.
Would you consider opening a pull request?

@anoo2niem
Copy link

anoo2niem commented Aug 6, 2023

Hi

I changed parser.cr according to the provided patch, then i ran make from the invidious directory.
i've restarted the invidious service, but on the video /watch?v=IHs-QqUik4E i still get the issue?

Did i do something wrong?

Edit: The video now seems to be loading.
After applying the patch, i've clicked 10 - 15 videos in a row to check if the issue was gone.
It was gone on all the videos, but remained persistent on the video mentioned above.
After writing this and retrying it worked without changing anything on the server.

@a7maadf
Copy link

a7maadf commented Aug 6, 2023 via email

@anoo2niem
Copy link

Sorry if i sound stupid but didn't i just rebuild it with make?

@a7maadf
Copy link

a7maadf commented Aug 7, 2023

Sorry if i sound stupid but didn't i just rebuild it with make?

Rewriting cause apparently GitHub missed up the formatting

to be honest I'm really not sure since I always use the invidious updater, but yeah it seems like the make​ command is what rebuilds it

Update a manual install

su - invidious
cd invidious
git pull
make
exit
systemctl restart invidious.service

I would try clearing the browser cache

@bbsixzz

This comment was marked as off-topic.

@iBicha
Copy link
Contributor

iBicha commented Aug 7, 2023

Adding to the conversation, I've seen the same issue using the API (iBicha/playlet#125)

What's interesting here is that the video plays for a few seconds before hitting the 403
Even though the DASH manifest has many streams, eventually it errors out "Error seen on all bitrates lasterror:403" indicating that all streams were tried but none of them worked

Another detail that does indicate it is an integrity check: inspecting the stream url that hit a 403 in devtools, when opening these links in their own browser tab, they work.

@syeopite
Copy link
Member

syeopite commented Aug 7, 2023

Here's a quick patch that implements NewPipe's workaround. Can anyone see if this fixes the problem? Please clear or wait until the video cache expires when doing so.

Thanks for testing everyone! I've went ahead and opened up #4037 which implements the patch

@alex27riva

This comment was marked as off-topic.

@iv-org iv-org locked as resolved and limited conversation to collaborators Aug 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working