Add config option to restrict asset downloads to logged-in users

[Martchus]

This PR is now only covering the web app and probably not useful on its own. You can still see how the approach using basic auth/htpasswd would look like on https://github.com/Martchus/openQA/tree/asset-auth-nginx-basic-auth.

---

See https://progress.opensuse.org/issues/170380

[okurz]

Seems like you try to incorporate changes applicable to pure mojo+nginx+apache at the same time. Wouldn't it be easier if we focus for a start on mojo especially to avoid the tricky authentication synchronisation between htpasswd and the database?

[Martchus]

Maybe we can use https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-subrequest-authentication instead of maintaining an htpasswd file.

[Martchus]

It makes not much sense to focus on the web app as a start because this change would be useless if one can just access assets via NGINX anyway. However, I now split the commits because the changes to the web app are of course independent and I probably want to try out https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-subrequest-authentication for NGINX.

So this PR is now only covering the web app. You can still see how the approach using basic auth/htpasswd would look like on https://github.com/Martchus/openQA/tree/asset-auth-nginx-basic-auth.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.98%. Comparing base (f0e8a3c) to head (55410ef).
Report is 10 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #6076   +/-   ##
=======================================
  Coverage   98.98%   98.98%           
=======================================
  Files         396      396           
  Lines       39527    39532    +5     
=======================================
+ Hits        39127    39132    +5     
  Misses        400      400           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Martchus]

Using a subquery for authentication would actually be quite easy: https://github.com/Martchus/openQA/tree/asset-auth-nginx-subrequest

With this we still need to adapt other code, e.g. so assets are still accessible via a logged-in web UI session. With Chromium this just works (except when using the incognito mode) but with Firefox it doesn't and I cannot get it to work using tokenauth (e.g. via https://github.com/Martchus/openQA/tree/asset-auth-nginx-subrequest-with-redirect).

Although tokenauth works in general with this approach, e.g.

curl 'http://Demo:…:…@localhost/assets/other/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20230109-Media.iso.sha256'

works as well as:

curl -u Demo:…:… http://localhost/assets/other/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20230109-Media.iso.sha256

[Martchus]

I tried to use the cache service:

notes:
  has_download_error: false
  lock: openSUSE-Tumbleweed-DVD-x86_64-Snapshot20241203-Media.iso.http://localhost
  output: |
    [info] [#1120] Cache size of "/hdd/openqa-devel/openqa/worker/cache" is 46 GiB, with limit 50 GiB
    [info] [#1120] Downloading "openSUSE-Tumbleweed-DVD-x86_64-Snapshot20241203-Media.iso" from "http://localhost/tests/4401/asset/iso/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20241203-Media.iso"
    [info] [#1120] Size of "/hdd/openqa-devel/openqa/worker/cache/localhost/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20241203-Media.iso" is 33 KiB, with ETag ""
    [info] [#1120] Download of "/hdd/openqa-devel/openqa/worker/cache/localhost/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20241203-Media.iso" successful (169 KiB/s), new cache size is 46 GiB

This appears to work despite C&P-ing http://localhost/tests/4401/asset/iso/openSUSE-Tumbleweed-DVD-x86_64-Snapshot20241203-Media.iso into Firefox gives a 403 response. However, the only thing being downloaded here is unfortunately the openQA index page. It contains "Logged in as Demo" so the auth worked but the redirection is not handled correctly.

Unfortunately Firefox also gives a 403 response with a logged-in session under http://localhost which also affects asset downloads from the web UI as mentioned before. Now it also doesn't work with Chromium. Probably it still had basic auth credentials cached in my test on Thursday (and restarting Chromium didn't help to clean the cache).

[Martchus]

I summarized my findings so far in https://progress.opensuse.org/issues/174154.