Replace URL signing and verification in LMS and Via 3 with Referer

[seanh]

Via 3 no longer signs the Via HTML URLs that it redirects to: instead Via HTML's authentication just allows https://lms.hypothes.is/ as a Referer header prefix, which is much simpler. (When LMS injects Via 3 and Via 3 redirects to Via HTML, the browser sets Referer to LMS not Via 3).

We should do the same in Via 3: when LMS injects a Via 3 /route or /pdf URL, and also when Via 3's /route redirects to its /pdf, the browser will send a Referer header that begins with https://lms.hypothes.is/. We should change Via 3's /route and /pdf to authenticate requests if they have a Referer that begins with https://lms.hypothes.is/.

Via 3's NGINX endpoint can continue using NGINX secure links generated by the /pdf endpoint as it currently does: while it may be possible to change NGINX to authenticate based on Referer instead, it's probably not worth the effort at this stage. NGINX changes are much more risky and difficult to do. The Python change is much easier and will allow us to simplify a lot of code.

Pros

• Referer is much simpler than URL signing
• More reliable / less likely to contain bugs: we've already had a couple of issues with URL signing and might have more
• Temporary, signed URLs are a hazard: we need to make sure that they never get into the browser's location bar because then users might copy and share them, not realising that the URL will stop working after a time. This isn't a problem in the LMS context but may be when Via 3 replaces legacy Via in the public context

Cons

• It's less secure. Anyone can write a curl script that can proxy anything through Via 3 by spoofing the Referer. Example with HTTPie: http GET https://via3.hypothe.is/proxy/?url=https://example.com Referer:https://lms.hypothes.is. This can be used by malware to phone home through Via, for example.

  This is already possible to do with Via HTML though, so I'm not sure that making it possible with Via 3 decreases our overall security any: if we block these kinds of scripts in Via 3 the attackers can just use Via HTML instead.

  This also may not be a big security concern for us: it doesn't enable phishing campaigns or malware distribution etc to be done through Via, which is the main thing that we're worried about. I'm not sure that we've seen any evidence that the phoning-home attack has actually been used.

• We need to make sure to shard our caching (e.g. in Cloudflare) by Referer and Sec-Fetch-Site (using the Vary header).

  This adds complexity as we now have a security need to ensure that caching is set up in a particular way for all Via 3 instances.

  This may hurt the effectiveness of our caching a little, although URL signing already hurts it (by adding a query param that changes over time). We can help the caching a little by adding a referrer-policy that ensures that the domain only (not the path) is sent to us in the Referer header.