-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Why ~100 minute ETA to complete "Metadata Fetch" stage (seems very slow) for a playlist of 35 short videos? [Metadata Fetch: ... failed: unsupported operand type(s) for /: 'NoneType' and 'int'] [hundreds of videos stuck in xklb-metadata.db, painstakingly processed every time, but never downloading?] #189
Comments
Just FYI the "Metadata Fetch" stage has now been running for more than an hour, and its Progress column currently shows 70%. |
The "Metadata Fetch" stage failed after ~86 minutes — is this related to an earlier video stuck in some queue possibly? Possibly related: |
Just FYI regardless of the root cause for the (failed) ~86 minute "Metadata Fetch", videos that keep reappearing in "Tasks" view (e.g. https://www.youtube.com/watch?v=n_iRE9alO44) should be handled, to explain what's happening. FWIW in this one particular case, browsing to https://www.youtube.com/watch?v=n_iRE9alO44 shows "This live stream recording is not available.", "No views" and "Streamed 1 year ago" — error messages that in an ideal world might become more visible via "Tasks" view?
|
Can I have access to your xklb-metadata? https://www.youtube.com/watch?v=n_iRE9alO44 looks out-of-norm and might not have its error recorded in UPDATE: Please ignore the request. I'll try to download it myself. |
Just FYI installing the very latest IIAB Calibre-Web onto LRN2 did not help: Trying to "Download to IIAB" https://www.youtube.com/playlist?list=PLybg94GvOJ9ELZEe9s2NXTKr41Yedbw7M is currently working through a ~100 minute "Metadata Fetch".
|
@deldesir: at first I suspected Calculus playlist https://www.youtube.com/playlist?list=PLybg94GvOJ9ELZEe9s2NXTKr41Yedbw7M might be pulling in more than its official 35 videos? As if you watch the running of
But that theory proved false: as https://www.youtube.com/watch?v=45lckolNRm8 is "PARLONS EN Les études à l’étranger. Expérience des anciens étudiants congolais en Chine" from YouTube channel TOP CONGO FM — it seems these hundreds of videos must somehow be stuck within IIAB Calibre-Web — it keeps trying to reprocess or redownload them somehow... |
A quick way to confirm this is to use a clean xklb-metadata.db. |
Good idea. Feel free to run...
...and then try again to "Download to IIAB" https://www.youtube.com/playlist?list=PLybg94GvOJ9ELZEe9s2NXTKr41Yedbw7M on LRN2? |
"Minimal Reproducer" URL? Any Minimal Reproducer URL that "pollutes" xklb-metadata.db, would be incredibly useful for testing — and to help us all understand this better — if we can identify one or several such URLs! 🙏 |
I confirm "lb tubeadd" is taking longer to process the videos. I don't see any pollution with irrelevant videos. Update: pollution happening for me with 3-videos playlist https://www.youtube.com/playlist?list=PLJicmE8fK0Eg8cBExYSr9r3tzg9ZQo3gD. Using a clean db is a temporary measure to get around this. |
Trying to download playlist https://www.youtube.com/playlist?list=PLybg94GvOJ9ELZEe9s2NXTKr41Yedbw7M on LRN2 is going extremely slowly. The "Progress" for "Metadata Fetch" has increased by 1 percentage point every minute or so, as if this initial stage alone will take about 100 minutes:
The playlist's 35 videos are each about 5-12min long.
Any idea why the "Metadata Fetch" stage is going so slowly? Hopefully the
/var/log/xklb.log
logs below provide hints to help explain?The text was updated successfully, but these errors were encountered: