Set outtmpl externally [to avoid missing thumbnails; NOTE: archival-quality book & video titles are preserved in 2 SQLite db's]

[deldesir]

Set "out directory" template externally so video file + thumbnail can be retrieved. This should fix the missing thumbnail issue explained in #171 (comment).

Tested on Ubuntu 24.04

[holta]

Is the idea to further strengthen thumbnail resiliency, building on xklb 2.8.059 ?

Background — xklb 2.8.059 (if I'm understanding correctly!?) also seeks to fix some of these punctuation problems — that have been leading to missing thumbnails like:

• "Test at least 10 different Vimeo video downloads" #171 (comment)

[deldesir]

Is the idea to further strengthen thumbnail resiliency, building on xklb 2.8.059 ?

Yes. Now the issue is resolved. This PR is tested with xklb 2.8.059.

[deldesir]

To test this PR, follow the instructions outlined in https://github.com/iiab/calibre-web/wiki#how-do-i-test-a-pull-request-pr-, then do:

• cp /usr/local/calibre-web-py3/scripts/lb-wrapper /usr/local/bin/

[holta]

@deldesir is there a specific example this PR fixes, that xklb 2.8.059 does not fix?

[deldesir]

@deldesir is there a specific example this PR fixes, that xklb 2.8.059 does not fix?

xklb 2.8.059 added support to get both video file and thumbnail in the same folder when outtmpl is set externally. Video files and thumbnails are no longer named after the video_id. The directories are no longer placed into a folder named after the extractor name (YouTube, Vimeo...) and outtmpl downloads everything where the script is run, not inside TMP_DOWNLOADS_DIR anymore. Calibre-Web has to adapt with all these changes, thus #177.

[holta]

xklb 2.8.059 added support to get both video file and thumbnail in the same folder when outtmpl is set externally.

1. Are your saying outtmpl is absolutely necessary, even with 2.8.059?

Video files and thumbnails are no longer named after the video_id

directories are no longer placed into a folder named after the extractor name (YouTube, Vimeo...)

2. Please include a specific/full example of an actual PATH/FILENAME to make the code below much more clear for all, Thank You:

OUTTMPL="'${TMP_DOWNLOADS_DIR}/%(extractor_key,extractor)s/%(uploader,uploader_id)s/%(title).170B_%(view_count)3.2D_[%(id).64B].%(ext)s'"

[holta]

Central Design Question that might deserve its own ticket:

• Are original video titles being preserved in full in xklb-metadata.db? (To protect all characters, no matter how strange, from all languages — and no matter how long the video title?)

• Likewise for playlist name, channel name and similar?

• (Even if literary-not-video E-Book titles & metadata might not yet similarly be preserved with archival quality?!)

[holta]

🙏 This PR is opening up existentially critical questions — that families and educators care about deeply in all countries — so learning materials their lives depend on don't disappear or get excessively / accidentally mangled! 🙏

Even if we're not quite there yet obviously! ✊

[deldesir]

xklb 2.8.059 adde

1. Are your saying outtmpl is absolutely necessary, even with 2.8.059?

For now yes

2. Please include a specific/full example of an actual PATH/FILENAME to make the code below much more clear for all, Thank You:

Will do

[deldesir]

Central Design Question that might deserve its own ticket:

• Are original video titles being preserved in full in xklb-metadata.db? (To protect all characters, no matter how strange, from all languages — and no matter how long the video title?)
• Likewise for playlist name, channel name and similar?
• (Even if literary-not-video E-Book titles & metadata might not yet similarly be preserved with archival quality?!)

We retrieve them directly from xklb-metadata.db already per

https://github.com/iiab/calibre-web/blob/f28a0b7701e0ef0bcb012ac25453f2d0c97ca092/cps/uploader.py#L259C13-L263C37

[holta]

We retrieve them directly from xklb-metadata.db already per

https://github.com/iiab/calibre-web/blob/f28a0b7701e0ef0bcb012ac25453f2d0c97ca092/cps/uploader.py#L259C13-L263C37

Great News!

ASIDE: Does Calibre-Web similarly preserve literary-not-video E-Book titles & metadata, in a completely intact / archival way — or not quite yet?

[deldesir]

We retrieve them directly from xklb-metadata.db already per
https://github.com/iiab/calibre-web/blob/f28a0b7701e0ef0bcb012ac25453f2d0c97ca092/cps/uploader.py#L259C13-L263C37

Great News!

ASIDE: Does Calibre-Web similarly preserve literary-not-video E-Book titles & metadata, in a completely intact / archival way — or not quite yet?

Yes, in /library.calibre-web/metadata.db > books.title

[holta]

Extremely valuable explanations, thanks to @deldesir !

AI² = Articulate Assumptions + Intuition + Intention 🏞️