All downloads from YouTube appear to fail: "'NoneType' object is not subscriptable" [vague error messages need to become relevant + clear in "Tasks" view]

[holta]

1. Also confirmed with a fresh install of the very latest IIAB using the very latest Raspberry Pi OS, on a Raspberry Pi 400 (all attempts to download individual videos appear to fail, in this same way).

   So here's just one example ("'NoneType' object is not subscriptable") when trying to download https://youtu.be/3-bLf_nRx58 using LRN2 button "Download to IIAB"

2. Secondarily, all IIAB Calibre-Web errors will need to contain clear English in future — explaining roughly what appears to be failing and approximately why. (If technical diagnostics are also included in Calibre-Web's Tasks view even better, to help contextualize + accelerate meaningful debugging!)

   A short English explanation is all that's needed initially — outlining what high-level category of error is occurring + contextualizing broken preconditions (i.e. broken assumptions) insofar as possible — so community debugging becomes increasingly Much More Efficient. 💯

Two iiab-diagnostics https://paste.centos.org/view/a6fc55a5 excerpts are pasted in below to help our understanding...

/var/log/calibre-web.log excerpt:

[2024-08-11 23:45:50,953]  INFO {cps.server:319} webserver stop (restart=False)
[2024-08-11 23:45:50,970]  INFO {apscheduler.scheduler:212} Scheduler has been shut down
[2024-08-11 23:45:50,979]  INFO {cps.server:298} Performing shutdown of Calibre-Web
[2024-08-11 23:45:58,333]  INFO {cps:168} *** "html2text" version does not meet the requirements. Should: <2024.2.26, Found: 2024.2.26, please consider installing required version ***
[2024-08-11 23:45:58,336]  INFO {cps:168} *** "py7zr" version does not meet the requirements. Should: >=0.15.0, Found: 0.11.3+dfsg, please consider installing required version ***
[2024-08-11 23:45:58,337]  INFO {cps:177} Starting Calibre Web...
[2024-08-11 23:45:58,348] DEBUG {cps.services:26} Cannot import goodreads, showing authors-metadata will not work: No module named 'goodreads'
[2024-08-11 23:45:58,349] DEBUG {cps.services:34} Cannot import simpleldap, logging in with ldap will not work: No module named 'flask_simpleldap'
[2024-08-11 23:45:58,350] DEBUG {cps.services:42} Cannot import SyncToken, syncing books with Kobo Devices will not work: No module named 'jsonschema'
[2024-08-11 23:45:58,352] DEBUG {cps.services:49} Cannot import gmail, sending books via Gmail Oauth2 Verification will not work: No module named 'google_auth_oauthlib'
[2024-08-11 23:45:58,394]  WARN {py.warnings:110} /usr/local/calibre-web-py3/lib/python3.12/site-packages/flask_limiter/extension.py:337: UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
  warnings.warn(
 
[2024-08-11 23:45:58,681] DEBUG {cps.gdriveutils:83} Cannot import pydrive, httplib2, using gdrive will not work: No module named 'pydrive'
[2024-08-11 23:45:59,401]  INFO {apscheduler.scheduler:181} Scheduler started
[2024-08-11 23:45:59,402]  INFO {apscheduler.scheduler:895} Added job "delete temp" to job store "default"
[2024-08-11 23:45:59,403]  INFO {apscheduler.scheduler:895} Added job "end scheduled task" to job store "default"
[2024-08-11 23:45:59,403]  INFO {apscheduler.scheduler:895} Added job "immediately delete temp" to job store "default"
[2024-08-11 23:45:59,404]  INFO {apscheduler.executors.default:123} Running job "immediately delete temp (trigger: date[2024-08-11 23:45:59 CDT], next run at: 2024-08-11 23:45:59 CDT)" (scheduled at 2024-08-11 23:45:59.403286-05:00)
[2024-08-11 23:45:59,406]  INFO {apscheduler.scheduler:641} Removed job bbbff94516094761a9b0eddd58c65dc8
[2024-08-11 23:45:59,406] DEBUG {cps.services.worker:91} Add Task for user: System - Clean up
[2024-08-11 23:45:59,407]  INFO {apscheduler.executors.default:144} Job "immediately delete temp (trigger: date[2024-08-11 23:45:59 CDT], next run at: 2024-08-11 23:45:59 CDT)" executed successfully
[2024-08-11 23:45:59,407] DEBUG {cps.tasks.clean:43} Deleted expired session_keys
[2024-08-11 23:45:59,725] DEBUG {cps.comic:46} Cannot import comicapi, extracting comic metadata will not work: No module named 'comicapi'
[2024-08-11 23:45:59,732] DEBUG {cps.comic:53} Cannot import rarfile, extracting cover files from rar files will not work: No module named 'rarfile'
[2024-08-11 23:46:00,026] DEBUG {cps.admin:82} Cannot import Flask-Dance, login with Oauth will not work: No module named 'flask_dance'
[2024-08-11 23:46:00,036] DEBUG {cps.gdrive:44} Cannot import googleapiclient, using GDrive will not work: No module named 'googleapiclient'
[2024-08-11 23:46:00,238] DEBUG {cps.search_metadata:62} Import error for metadata source: lubimyczytac - No module named 'markdown2'
[2024-08-11 23:46:00,271] DEBUG {cps.search_metadata:62} Import error for metadata source: scholar - No module named 'scholarly'
[2024-08-11 23:46:00,583]  INFO {cps.server:268} Starting Tornado server on :8083
[2024-08-11 23:46:18,144] DEBUG {cps.web:1434} You are now logged in as: 'Admin'
[2024-08-11 23:46:18,155] DEBUG {cps.ub:84} Login and store session : 9efdb20e7fa9da67d5c219fcb547ff8133f2cbaabac3f23469f904258fca3e9d757cf56eb45fa262565d097c48fddd9b604aa2c48ea24ed4a1c0ba84bea1c14f
[2024-08-11 23:47:19,695] DEBUG {cps.services.worker:91} Add Task for user: Admin - Metadata fetch task for https://youtu.be/3-bLf_nRx58
[2024-08-11 23:47:19,696]  INFO {cps.tasks.metadata_extract:155} Starting to fetch metadata for URL: https://youtu.be/3-bLf_nRx58
[2024-08-11 23:47:26,531] DEBUG {cps.services.worker:91} Add Task for user: Admin - Download task for https://www.youtube.com/watch?v=3-bLf_nRx58
[2024-08-11 23:47:26,532]  INFO {cps.tasks.download:43} Subprocess args: ['lb-wrapper', 'dl', 'https://www.youtube.com/watch?v=3-bLf_nRx58']
[2024-08-11 23:47:41,065] ERROR {cps.tasks.download:132} An error occurred during the subprocess execution: 'NoneType' object is not subscriptable
[2024-08-11 23:47:41,066]  INFO {cps.tasks.download:139} Download task for https://www.youtube.com/watch?v=3-bLf_nRx58 completed successfully
[2024-08-11 23:47:41,066] ERROR {cps.services.worker:202} 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/usr/local/calibre-web-py3/cps/tasks/download.py", line 93, in run
    requested_file = conn.execute("SELECT path FROM media WHERE webpath = ? AND path NOT LIKE 'http%'", (self.media_url,)).fetchone()[0]
                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable
 
During handling of the above exception, another exception occurred:
 
Traceback (most recent call last):
  File "/usr/local/calibre-web-py3/cps/services/worker.py", line 199, in start
    self.run(*args)
  File "/usr/local/calibre-web-py3/cps/tasks/download.py", line 133, in run
    self.message = f"{self.media_url_link} failed to download: {self.read_error_from_database()}"
                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/calibre-web-py3/cps/tasks/download.py", line 149, in read_error_from_database
    error = conn.execute("SELECT error FROM media WHERE webpath = ?", (self.media_url,)).fetchone()[0]
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable
[2024-08-11 23:49:26,689] DEBUG {cps.web:1609} Start video watching for 522
[2024-08-11 23:49:26,699]  INFO {cps.web:1231} Serving book: PPP 040601 - The State Monad in practice - IOG Academy

/var/log/xklb.log excerpt:

2024-08-11 23:47:20 - [Info] Using yt-dlp 2024.08.06
2024-08-11 23:47:20 - [Info] Running xklb command: lb tubeadd '/library/calibre-web/xklb-metadata.db' 'https://youtu.be/3-bLf_nRx58' --force -vv
2024-08-11 23:47:21 - [Debug] library v2.9.021
2024-08-11 23:47:21 - [Debug] ['/usr/local/bin/lb', 'tubeadd', '/library/calibre-web/xklb-metadata.db', 'https://youtu.be/3-bLf_nRx58', '--force', '-vv']
2024-08-11 23:47:21 - [Debug] {'force': True, 'paths': ['https://youtu.be/3-bLf_nRx58']}
2024-08-11 23:47:21 - [Debug] {'noprogress': True, 'skip_download': True, 'lazy_playlist': True, 'extract_flat': True, 'no_check_certificate': True, 'ignore_no_formats_error': True, 'skip_playlist_after_errors': 21, 'playlistend': 20000, 'force': True}
2024-08-11 23:47:22 - [Debug] yt-dlp initialized 0.2287
2024-08-11 23:47:22 - [Info] [youtube] Extracting URL: https://youtu.be/3-bLf_nRx58
2024-08-11 23:47:22 - [Info] [youtube] 3-bLf_nRx58: Downloading webpage
2024-08-11 23:47:23 - [Info] [youtube] 3-bLf_nRx58: Downloading ios player API JSON
2024-08-11 23:47:23 - [Info] [youtube] 3-bLf_nRx58: Downloading web creator player API JSON
2024-08-11 23:47:23 - [Info] [youtube] 3-bLf_nRx58: Downloading player 28fd7348
2024-08-11 23:47:25 - [Debug] Importing playlist-less media https://youtu.be/3-bLf_nRx58
2024-08-11 23:47:25 - [Debug] media.playlist_media_add 3.1249
2024-08-11 23:47:26 - [Debug] ydl.extract_info done 0.8878
2024-08-11 23:47:26 - [Info]
2024-08-11 23:47:26 - [Info] lb-wrapper's xklb command (tubeadd) completed successfully.
2024-08-11 23:47:27 - [Info] Using yt-dlp 2024.08.06
2024-08-11 23:47:27 - [Info] Running xklb command: lb dl '/library/calibre-web/xklb-metadata.db' --video --search 'https://www.youtube.com/watch?v=3-bLf_nRx58' --format best --format-sort 'tbr~1000' --write-thumbnail --subs --live --live-from-start -o '/library/downloads/calibre-web/%(extractor_key,extractor)s/%(uploader,uploader_id)s/%(title).170B_%(view_count)3.2D_[%(id).64B].%(ext)s' -vv
2024-08-11 23:47:27 - [Debug] library v2.9.021
2024-08-11 23:47:27 - [Debug] ['/usr/local/bin/lb', 'dl', '/library/calibre-web/xklb-metadata.db', '--video', '--search', 'https://www.youtube.com/watch?v=3-bLf_nRx58', '--format', 'best', '--format-sort', 'tbr~1000', '--write-thumbnail', '--subs', '--live', '--live-from-start', '-o', '/library/downloads/calibre-web/%(extractor_key,extractor)s/%(uploader,uploader_id)s/%(title).170B_%(view_count)3.2D_[%(id).64B].%(ext)s', '-vv']
2024-08-11 23:47:28 - [Debug] {'include': ['https://www.youtube.com/watch?v=3-bLf_nRx58'], 'subs': True, 'live': True, 'paths': []}
2024-08-11 23:47:28 - [Debug] None
2024-08-11 23:47:28 - [Debug] {'id': 11, 'playlists_id': None, 'path': 'https://www.youtube.com/watch?v=3-bLf_nRx58', 'playlist_path': None, 'title': 'Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman', 'duration': 188, 'time_created': 1723438045, 'size': 0, 'time_modified': 0, 'time_downloaded': 0, 'time_deleted': 0, 'error': None, 'extractor_config': None, 'extractor_key': None}
2024-08-11 23:47:28 - [Debug] {'noprogress': True, 'lazy_playlist': True, 'noplaylist': True, 'no_check_certificate': True, 'ignore_no_formats_error': True, 'skip_playlist_after_errors': 21, 'playlistend': 20000, 'extractor_args': {'youtube': {'skip': ['authcheck']}}, 'check_formats': 'selected', 'logger': <xklb.createdb.tube_backend.download.<locals>.DictLogger object at 0x770561e0aba0>, 'postprocessors': [{'key': 'FFmpegMetadata'}, {'key': 'FFmpegEmbedSubtitle'}], 'playlist_items': '1', 'extractor_retries': 3, 'retries': 12, 'retry_sleep_functions': {'extractor': <function download.<locals>.<lambda> at 0x770561e22340>, 'http': <function download.<locals>.<lambda> at 0x770561e223e0>, 'fragment': <function download.<locals>.<lambda> at 0x770561e22480>}, 'outtmpl': {'default': '/library/downloads/calibre-web/%(extractor_key,extractor)s/%(uploader,uploader_id)s/%(title).170B_%(view_count)3.2D_[%(id).64B].%(ext)s'}, 'progress_hooks': [<function download.<locals>.<lambda> at 0x770561e22520>], 'subtitlesformat': 'srt/best', 'writesubtitles': True, 'writeautomaticsub': True, 'format': 'best', 'format_sort': ['tbr~1000'], 'writethumbnail': True, 'live_from_start': True, '_warnings': ['"-f best" selects the best pre-merged format which is often not the best option.\n         To let yt-dlp download and merge the best available formats, simply do not pass any format selection.\n         If you know what you are doing and want only the best pre-merged format, use "-f b" instead to suppress this warning'], 'include': ['https://www.youtube.com/watch?v=3-bLf_nRx58'], 'subs': True, 'live': True}
2024-08-11 23:47:28 - [Debug] [yt-dlp]: Downloading https://www.youtube.com/watch?v=3-bLf_nRx58
2024-08-11 23:47:35 - [Debug] [https://www.youtube.com/watch?v=3-bLf_nRx58]: Downloaded to /library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4
2024-08-11 23:47:36 - [Debug] ('systemd-run', '--user', '-p', 'MemoryMax=4G', '-p', 'MemorySwapMax=1G', '--pty', '--pipe', '--same-dir', '--wait', '--collect', '--service-type=exec', '--quiet', '--', 'ffmpeg', '-nostdin', '-hide_banner', '-nostats', '-xerror', '-v', '16', '-err_detect', 'explode', '-ss', '40.00', '-i', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4', '-t', '1', '-f', 'null', '/dev/null')
2024-08-11 23:47:36 - [Debug] Failed to connect to bus: No medium found
2024-08-11 23:47:36 - [Debug] [systemd-run --user -p MemoryMax=4G -p MemorySwapMax=1G --pty --pipe --same-dir --wait --collect --service-type=exec --quiet -- ffmpeg -nostdin -hide_banner -nostats -xerror -v 16 -err_detect explode -ss 40.00 -i '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4' -t 1 -f null /dev/null] exited 1
2024-08-11 23:47:36 - [Debug] ('systemd-run', '--user', '-p', 'MemoryMax=4G', '-p', 'MemorySwapMax=1G', '--pty', '--pipe', '--same-dir', '--wait', '--collect', '--service-type=exec', '--quiet', '--', 'ffmpeg', '-nostdin', '-hide_banner', '-nostats', '-xerror', '-v', '16', '-err_detect', 'explode', '-ss', '0.00', '-i', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4', '-t', '1', '-f', 'null', '/dev/null')
2024-08-11 23:47:36 - [Debug] Failed to connect to bus: No medium found
2024-08-11 23:47:36 - [Debug] [systemd-run --user -p MemoryMax=4G -p MemorySwapMax=1G --pty --pipe --same-dir --wait --collect --service-type=exec --quiet -- ffmpeg -nostdin -hide_banner -nostats -xerror -v 16 -err_detect explode -ss 0.00 -i '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4' -t 1 -f null /dev/null] exited 1
2024-08-11 23:47:36 - [Debug] ('systemd-run', '--user', '-p', 'MemoryMax=4G', '-p', 'MemorySwapMax=1G', '--pty', '--pipe', '--same-dir', '--wait', '--collect', '--service-type=exec', '--quiet', '--', 'ffmpeg', '-nostdin', '-hide_banner', '-nostats', '-xerror', '-v', '16', '-err_detect', 'explode', '-ss', '60.00', '-i', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4', '-t', '1', '-f', 'null', '/dev/null')
2024-08-11 23:47:36 - [Debug] Failed to connect to bus: No medium found
2024-08-11 23:47:36 - [Debug] [systemd-run --user -p MemoryMax=4G -p MemorySwapMax=1G --pty --pipe --same-dir --wait --collect --service-type=exec --quiet -- ffmpeg -nostdin -hide_banner -nostats -xerror -v 16 -err_detect explode -ss 60.00 -i '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4' -t 1 -f null /dev/null] exited 1
2024-08-11 23:47:36 - [Debug] ('systemd-run', '--user', '-p', 'MemoryMax=4G', '-p', 'MemorySwapMax=1G', '--pty', '--pipe', '--same-dir', '--wait', '--collect', '--service-type=exec', '--quiet', '--', 'ffmpeg', '-nostdin', '-hide_banner', '-nostats', '-xerror', '-v', '16', '-err_detect', 'explode', '-ss', '20.00', '-i', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4', '-t', '1', '-f', 'null', '/dev/null')
2024-08-11 23:47:36 - [Debug] Failed to connect to bus: No medium found
2024-08-11 23:47:36 - [Debug] [systemd-run --user -p MemoryMax=4G -p MemorySwapMax=1G --pty --pipe --same-dir --wait --collect --service-type=exec --quiet -- ffmpeg -nostdin -hide_banner -nostats -xerror -v 16 -err_detect explode -ss 20.00 -i '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4' -t 1 -f null /dev/null] exited 1
2024-08-11 23:47:36 - [Debug] ('systemd-run', '--user', '-p', 'MemoryMax=4G', '-p', 'MemorySwapMax=1G', '--pty', '--pipe', '--same-dir', '--wait', '--collect', '--service-type=exec', '--quiet', '--', 'ffmpeg', '-nostdin', '-hide_banner', '-nostats', '-xerror', '-v', '16', '-err_detect', 'explode', '-ss', '80.00', '-i', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4', '-t', '1', '-f', 'null', '/dev/null')
2024-08-11 23:47:36 - [Debug] Failed to connect to bus: No medium found
2024-08-11 23:47:36 - [Debug] [systemd-run --user -p MemoryMax=4G -p MemorySwapMax=1G --pty --pipe --same-dir --wait --collect --service-type=exec --quiet -- ffmpeg -nostdin -hide_banner -nostats -xerror -v 16 -err_detect explode -ss 80.00 -i '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4' -t 1 -f null /dev/null] exited 1
2024-08-11 23:47:36 - [Debug] ('systemd-run', '--user', '-p', 'MemoryMax=4G', '-p', 'MemorySwapMax=1G', '--pty', '--pipe', '--same-dir', '--wait', '--collect', '--service-type=exec', '--quiet', '--', 'ffmpeg', '-nostdin', '-hide_banner', '-nostats', '-xerror', '-v', '16', '-err_detect', 'explode', '-ss', '160.00', '-i', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4', '-t', '1', '-f', 'null', '/dev/null')
2024-08-11 23:47:36 - [Debug] Failed to connect to bus: No medium found
2024-08-11 23:47:36 - [Debug] [systemd-run --user -p MemoryMax=4G -p MemorySwapMax=1G --pty --pipe --same-dir --wait --collect --service-type=exec --quiet -- ffmpeg -nostdin -hide_banner -nostats -xerror -v 16 -err_detect explode -ss 160.00 -i '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4' -t 1 -f null /dev/null] exited 1
2024-08-11 23:47:36 - [Debug] ('systemd-run', '--user', '-p', 'MemoryMax=4G', '-p', 'MemorySwapMax=1G', '--pty', '--pipe', '--same-dir', '--wait', '--collect', '--service-type=exec', '--quiet', '--', 'ffmpeg', '-nostdin', '-hide_banner', '-nostats', '-xerror', '-v', '16', '-err_detect', 'explode', '-ss', '100.00', '-i', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4', '-t', '1', '-f', 'null', '/dev/null')
2024-08-11 23:47:36 - [Debug] Failed to connect to bus: No medium found
2024-08-11 23:47:36 - [Debug] [systemd-run --user -p MemoryMax=4G -p MemorySwapMax=1G --pty --pipe --same-dir --wait --collect --service-type=exec --quiet -- ffmpeg -nostdin -hide_banner -nostats -xerror -v 16 -err_detect explode -ss 100.00 -i '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4' -t 1 -f null /dev/null] exited 1
2024-08-11 23:47:36 - [Debug] ('systemd-run', '--user', '-p', 'MemoryMax=4G', '-p', 'MemorySwapMax=1G', '--pty', '--pipe', '--same-dir', '--wait', '--collect', '--service-type=exec', '--quiet', '--', 'ffmpeg', '-nostdin', '-hide_banner', '-nostats', '-xerror', '-v', '16', '-err_detect', 'explode', '-ss', '120.00', '-i', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4', '-t', '1', '-f', 'null', '/dev/null')
2024-08-11 23:47:36 - [Debug] Failed to connect to bus: No medium found
2024-08-11 23:47:36 - [Debug] [systemd-run --user -p MemoryMax=4G -p MemorySwapMax=1G --pty --pipe --same-dir --wait --collect --service-type=exec --quiet -- ffmpeg -nostdin -hide_banner -nostats -xerror -v 16 -err_detect explode -ss 120.00 -i '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4' -t 1 -f null /dev/null] exited 1
2024-08-11 23:47:36 - [Debug] ('systemd-run', '--user', '-p', 'MemoryMax=4G', '-p', 'MemorySwapMax=1G', '--pty', '--pipe', '--same-dir', '--wait', '--collect', '--service-type=exec', '--quiet', '--', 'ffmpeg', '-nostdin', '-hide_banner', '-nostats', '-xerror', '-v', '16', '-err_detect', 'explode', '-ss', '140.00', '-i', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4', '-t', '1', '-f', 'null', '/dev/null')
2024-08-11 23:47:36 - [Debug] Failed to connect to bus: No medium found
2024-08-11 23:47:36 - [Debug] [systemd-run --user -p MemoryMax=4G -p MemorySwapMax=1G --pty --pipe --same-dir --wait --collect --service-type=exec --quiet -- ffmpeg -nostdin -hide_banner -nostats -xerror -v 16 -err_detect explode -ss 140.00 -i '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4' -t 1 -f null /dev/null] exited 1
2024-08-11 23:47:36 - [Debug] ('systemd-run', '--user', '-p', 'MemoryMax=4G', '-p', 'MemorySwapMax=1G', '--pty', '--pipe', '--same-dir', '--wait', '--collect', '--service-type=exec', '--quiet', '--', 'ffmpeg', '-nostdin', '-hide_banner', '-nostats', '-xerror', '-v', '16', '-err_detect', 'explode', '-ss', '186.00', '-i', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4', '-t', '1', '-f', 'null', '/dev/null')
2024-08-11 23:47:36 - [Debug] Failed to connect to bus: No medium found
2024-08-11 23:47:36 - [Debug] ('systemd-run', '--user', '-p', 'MemoryMax=4G', '-p', 'MemorySwapMax=1G', '--pty', '--pipe', '--same-dir', '--wait', '--collect', '--service-type=exec', '--quiet', '--', 'ffmpeg', '-nostdin', '-hide_banner', '-nostats', '-xerror', '-v', '16', '-err_detect', 'explode', '-ss', '180.00', '-i', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4', '-t', '1', '-f', 'null', '/dev/null')
2024-08-11 23:47:36 - [Debug] [systemd-run --user -p MemoryMax=4G -p MemorySwapMax=1G --pty --pipe --same-dir --wait --collect --service-type=exec --quiet -- ffmpeg -nostdin -hide_banner -nostats -xerror -v 16 -err_detect explode -ss 186.00 -i '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4' -t 1 -f null /dev/null] exited 1
2024-08-11 23:47:36 - [Debug] Failed to connect to bus: No medium found
2024-08-11 23:47:36 - [Debug] [systemd-run --user -p MemoryMax=4G -p MemorySwapMax=1G --pty --pipe --same-dir --wait --collect --service-type=exec --quiet -- ffmpeg -nostdin -hide_banner -nostats -xerror -v 16 -err_detect explode -ss 180.00 -i '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4' -t 1 -f null /dev/null] exited 1
2024-08-11 23:47:40 - [Debug] ('ffprobe', '-show_entries', 'stream=r_frame_rate,nb_read_frames,duration', '-select_streams', 'v', '-count_frames', '-of', 'json', '-threads', '1', '-v', '0', '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].mp4')
2024-08-11 23:47:40 - [Debug] {
2024-08-11 23:47:40 - [Debug] "programs": [
2024-08-11 23:47:40 - [Debug]
2024-08-11 23:47:40 - [Debug] ],
2024-08-11 23:47:40 - [Debug] "streams": [
2024-08-11 23:47:40 - [Debug] {
2024-08-11 23:47:40 - [Debug] "r_frame_rate": "24000/1001",
2024-08-11 23:47:40 - [Debug] "duration": "187.520667",
2024-08-11 23:47:40 - [Debug] "nb_read_frames": "4496"
2024-08-11 23:47:40 - [Debug] }
2024-08-11 23:47:40 - [Debug] ]
2024-08-11 23:47:40 - [Debug] }
2024-08-11 23:47:40 - [Debug] Extra media data {'time_modified': 0, 'time_downloaded': 0, 'time_deleted': 0, 'requested_subtitles_en_ext': 'vtt', 'requested_subtitles_en_url': 'https://www.youtube.com/api/timedtext?v=3-bLf_nRx58&ei=4ZO5Zq2NGZGY2_gPpMGlqAY&caps=asr&opi=112496729&exp=xbt&xoaf=5&hl=en&ip=0.0.0.0&ipbits=0&expire=1723463249&sparams=ip%2Cipbits%2Cexpire%2Cv%2Cei%2Ccaps%2Copi%2Cexp%2Cxoaf&signature=2B65AFE3B82DCF5391CC0ED443906AB1E1AB08F5.6FDEEE80361ED404CD76B46A872081B15C2429B6&key=yt8&kind=asr&lang=en&fmt=vtt', 'requested_subtitles_en_name': 'English', 'requested_subtitles_en_filepath': '/library/downloads/calibre-web/Youtube/Afrobarometer/Afrobarometer is one of the most important breakthroughs in data - Nic Cheeseman_25.00_[3-bLf_nRx58].en.vtt', 'corruption': 0}
2024-08-11 23:47:40 - [Info] lb-wrapper's xklb command (dl) completed successfully.

Issues also mentioning to "'NoneType' object is not subscriptable":

• When no format is available (downloadable) for a video [e.g. member-only videos, ETC] notify user & explain why ['NoneType' object is not subscriptable] #114
• 2 videos "failed to download: 'NoneType' object is not subscriptable" — yt-dlp --format best fails to download complete video (same 1503232-byte .mp4.part fragment every time!) #135
• No suitable format found for some (many?) Vimeo (and YouTube) downloads — "failed to download: 'NoneType' object is not subscriptable" #164
• 3 kinds of errors on a fresh test: (1) 'NoneType' object cannot be interpreted as an integer (2) 'NoneType' object is not subscriptable (3) Download appears to be stuck #167
• "Test at least 10 different Vimeo video downloads" #171
• (1) download.py: "sqlite3.OperationalError: no such column: error" after "TypeError: 'NoneType' object is not subscriptable" (2) editbooks.py: SAWarning: Object of type <Books> not in session, add operation along 'Authors.books' won't proceed #186
• Formerly live videos fail to download: "failed to download [download] ... does not pass filter (live_status=?not_live); skipping" #188
• Formerly live https://youtu.be/4BL65HElOPg fails to download [same "no such column: error" issue as #186 ?] [unavailable fragments?] #215
• All videos failed to download: "'NoneType' object is not subscriptable" — "Some formats may be missing" #222

[deldesir]

Please upgrade IIAB Calibre-Web to use the latest xklb (underlying media library). A fix was shipped quickly by its author.

[avni]

Confirmed! Woohoo!

[holta]

"Tasks" view needs clear English error messages in coming months!

• Articulating which precondition/assumption was violated (in English!)

• Supplemented with diagnostic detail or relevant/insightful error message (whenever realistic!)

• IN SHORT: Context, not vague error messages (which kind of error at minimum! hint at probable cause[s] when possible!)

Regardless of the particulars of the xklb 2.9.021 regression (below) that caused this ticket's problem: (now fixed as part of xklb 2.9.022 or 2.9.023!)

1. lb dl creates a duplicate record that uses only 3 columns (time_modified, path and the new download_attempts).

2. When --search was used, only the duplicate row was kept.

Related:

• Adjust error message for stuck/slow videos [if elapsed_time >= timeout, e.g. due to unavailable fragments &/or xklb media_check] #223 (comment)