New upstream instructions for Calibre-Web manual installation

[deldesir]

Hey @EMG70 and @jllhub! Can you test these instructions on Ubuntu 24.04 VM?

[holta]

Here are NEW Instructions (to install Calibre-Web all alone, without IIAB) that @deldesir helped write:

https://github.com/janeczku/calibre-web/wiki/Manual-Installation

[holta]

Hi @EMG70 and @jllhub !

1. The instructions at https://github.com/janeczku/calibre-web/wiki/Manual-Installation were dramatically improved over the past few hours, so please reload that page before you begin. 👍

2. Please run hostnamectl and paste the output into this ticket (regardless how far you get, hopefully everything works!) so that we know what OS you tested, Thanks! (e.g. In my case I tested Ubuntu 24.10 VMs, as you can see below...)

ubuntu@u2410-cw3:/opt/calibre-web$ hostnamectl
 Static hostname: u2410-cw3
       Icon name: computer-vm
         Chassis: vm 🖴
      Machine ID: c2d5357da3a0495f91815784a48d9093
         Boot ID: c3632cbe980f41f3baad84eb3076572a
    AF_VSOCK CID: 4140098444
  Virtualization: kvm
Operating System: Ubuntu 24.10
          Kernel: Linux 6.11.0-8-generic
    Architecture: x86-64
 Hardware Vendor: QEMU
  Hardware Model: Standard PC _Q35 + ICH9, 2009_
Firmware Version: unknown
   Firmware Date: Wed 2022-02-02
    Firmware Age: 2y 8month 6d

[holta]

@deldesir

Can you confirm that Step 6. of the instructions is definitely NOT REQUIRED on the current 64-bit Raspberry Pi OS Lite?

./venv/bin/python3 -m pip install cryptography

[deldesir]

Ok, will test to check if so.

[holta]

Ok, will test to check if so.

Please also clarify if IIAB's install of Calibre-Web is now broken — lacking pip install cryptography — and why or why not?!

[deldesir]

./venv/bin/python3 -m pip install cryptography is REQUIRED on the current 64-bit Raspberry Pi OS Lite as per my testing. Python3-cryptography did not help following the Manual Installation instructions.

UPDATE: Installing Calibre-Web using IIAB unittest succeeded. Python3-cryptography was installed with this method.

[holta]

./venv/bin/python3 -m pip install cryptography is REQUIRED on the current 64-bit Raspberry Pi OS Lite as per my testing. [apt install python3-cryptography did not help.]

Interesting that this appears to be a change compared to 1-2 weeks ago!! 🙃

@OzzieIsaacs is it possible that pip install cryptography is a new Calibre-Web requirement — that should perhaps be made more official in requirements.txt — instead of in Step 6. of the current https://github.com/janeczku/calibre-web/wiki/Manual-Installation ?

[holta]

@OzzieIsaacs @deldesir here's what seems to be happening, at a deeper lever:

1. When IIAB installs Calibre-Web, the missing pip dependency (cryptography) is pulled in from the OS's apt package python3-cryptography to the venv — as a result of PR Experimenting with various apt package installs [e.g. python3-cryptography, so Calibre-Web unit test works on Raspberry Pi OS] iiab#3815 (important on distros like Raspberry Pi OS!) and --system-site-packages here:

   python3 -m venv --system-site-packages /usr/local/calibre-web-py3
   

   As you can see on Line 128 here:

   https://github.com/iiab/iiab/blob/3651ac666e8dedfc96f567408081954dc53e2d1b/roles/calibre-web/tasks/install.yml#L123-L129

2. Whereas Calibre-Web "Manual Installation" Step 4. is not using the --system-site-packages flag at the moment. In other words, a cheap hack/workaround (if Calibre-Web really wants to impose the python3-cryptography apt package as a formal requirement?!) would be to change Step 4. of Calibre-Web's "Manual Installation" to:

   python3 -m venv venv --system-site-packages
   

   And while that would work quite reliably... it sounds like an even better solution... might be for Calibre-Web's requirements.txt to fully declare its dependency on cryptography ❓

FWIW an even crazier option (might!) also be considered to make things even more resilient:

Is it possible that imposing both approaches (i.e. apt install python3-cryptography and also pip install cryptography) hypothetically might help provide even better coverage — as OS's (Linux distributions, etc) AND the cryptography package constantly evolve month-by-month and year-by-year — e.g. when Python's latest cryptography package and wheel are inevitably / regularly broken-or-not-ready on Raspberry Pi etc❓ (Would need to be battle-tested of course!!)

[holta]

CLARIF:

python3 -m venv venv --system-site-packages and then ./venv/bin/python3 -m pip install -r requirements.txt do work when following "cheap hack/workaround" above (item 2.) — but just FYI this warning appears (as usual!?) when running Calibre-Web in the final step:

root@u2410-cw2:/opt/calibre-web# ./venv/bin/python3 cps.py
[2024-10-10 11:33:34,645]  WARN {py.warnings:112} /opt/calibre-web/cps/string_helper.py:22: SyntaxWarning: invalid escape sequence '\s'
  return re.sub("(^[\s\u200B-\u200D\ufeff]+)|([\s\u200B-\u200D\ufeff]+$)","", text)

[holta]

1. https://github.com/janeczku/calibre-web/wiki/Manual-Installation is dramatically improved!

2. Thanks to @deldesir and @avni!

3. Now the evidence needs to presented upstream ~ with a PR accompanying Should Calibre-Web's requirements.txt install cryptography ? janeczku/calibre-web#3183 ~ so we're all eating the same food ~ as a tree that falls in the forest does not make a sound, if nobody's around! 😉

[holta]

1. https://github.com/janeczku/calibre-web/wiki/Manual-Installation is dramatically improved!

And much further improved now, thanks to upstream fix (janeczku@c2ffa94) off of:

• Should Calibre-Web's requirements.txt install cryptography ? janeczku/calibre-web#3183

@deldesir, @avni please review doc cleanup below if you can, Thank You!

"Tighten up Manual Installation, thx to janeczku#3183 & c2ffa94"

https://github.com/janeczku/calibre-web/wiki/Manual-Installation/_compare/84902a0f9002d6ee98ac8a5d879e10d6010de32a...0a20d71ec2af591cf4507a3f62e18ec597bd4acc

(Can this ticket be closed now?)