YouTube video(s) not downloadable (or viewable?) from certain IP address(es): "Sign in to confirm you’re not a bot. This helps protect our community"

[holta]

Many yt-dlp users appear to be affected by YouTube blocking their IP addresses in recent weeks. Including our own test machine (LRN2.ORG) which has been affected for ~9 days now.

Error message:

Sign in to confirm you’re not a bot. This helps protect our community

FYI this appears to be quite different from past YouTube blocks of IP addresses — blocks which (typically, in the past) resolved themselves after 12+ hours?

@deldesir has identified others looking into and working on this:

• [youtube] Sign in to confirm you’re not a bot. This helps protect our community yt-dlp/yt-dlp#10128

Possible fix: (appears very promising)

• PR [ie/youtube] Add po_token, visitor_data and data_sync_id extractor-args yt-dlp/yt-dlp#10648

Related:

• PR Adjust error message for stuck/slow videos [if elapsed_time >= timeout, e.g. due to unavailable fragments &/or xklb media_check] #223 (comment)
• https://www.reddit.com/r/youtubedl/comments/1e6bzu4/ytdlp_error_sign_in_to_confirm_youre_not_a_bot/

[holta]

It seems likely (or very possible!) that cloud VPS IP addresses might be more affected [more vulnerable to banning / blocking by YouTube] than residential/home/mobile IP addresses? As mentioned on...

• An attempt to download [a YouTube channel] resulted in an accumulation (long list) of "unavailable videos" in Tasks, from all current and previously attempted failed downloads [why is the very same video individually downloadable?] #238 (comment)

[holta]

@deldesir @chapmanjacobd

What's the best way to surface very important WARNINGS like Sign in to confirm you’re not a bot (example below) — making this visible to all users?

Is xklb's list of about 107 yt_unrecoverable_errors relevant and the right way to solve this?

# tail /var/log/xklb.log
2024-08-28 19:05:10 - [Info] [youtube] ecsfN_0wK8I: Downloading web creator player API JSON
2024-08-28 19:05:10 - [Debug] WARNING: [youtube] Sign in to confirm you’re not a bot. This helps protect our community. Learn more
2024-08-28 19:05:10 - [Debug] WARNING: No video formats found!
2024-08-28 19:05:10 - [Debug] no such table: playlists
2024-08-28 19:05:10 - [Debug] Importing playlist-less media https://www.youtube.com/watch?v=ecsfN_0wK8I
2024-08-28 19:05:10 - [Debug] media.playlist_media_add 1.0493
2024-08-28 19:05:10 - [Debug] WARNING: Requested format is not available
2024-08-28 19:05:10 - [Debug] ydl.extract_info done 0.1031
2024-08-28 19:05:10 - [Info]
2024-08-28 19:05:10 - [Info] lb-wrapper's xklb command (tubeadd) completed successfully.

[deldesir]

I tried to add ".*Sign in to confirm you’re not a bot" in /root/.local/share/pipx/venvs/xklb/lib64/python3.12/site-packages/xklb/data/yt_dlp_errors.py under yt_unrecoverable_errors without success. (I expected the error to be added in column media.error in xklb-metadata.db)

[holta]

June 2024 news that (might) be related?! (Links from @deldesir)

• "[YouTube breaks third party clients once again"
  https://discuss.techlore.tech/t/youtube-breaks-third-party-clients-once-again/8944

• "YouTube is breaking ad blockers again, and trials a viewing limitation (Update: Google statement)
  The change could make it much tougher to block ads in the future."
  https://www.androidauthority.com/youtube-server-side-ads-3451248/

[moorchegue]

Being affected by this, I've found that access to the actual video files is not restricted, only to some metadata pages which are required to obtain correct URLs.

Would it be feasible to set up a 3rd party metadata repository that would be updated by users with unrestricted access?