Link check improvement

[Kovah]

This topic continues the discussion of a post from the old community forum: "Link check improvement" by Alexander_Stepanov
A backup of the full conversation can be viewed on the Internet Archive.

---

There are a couple of parts to this request.

1. Some working links are marked as broken. It will be helpful if each link marked as broken will have information on why and when. Time and code returned by a site. In this way, it will be easier to understand what happened and why the working link is marked as broken. For example https://owennelson.co.uk/accessing-a-modem-through-a-ubiquiti-usg/
2. It will be also helpful to divide broken links into groups. At least put in the separate group links that return “moved” code.
   I would propose to separate broken links into three categories
   Returning code 404 “deleted”
   Returning code 301,302 “moved”
   All other “broken”
3. I think before the link is marked as broken it should be checked several times. I would propose to recheck a link let’s say 3 times (once a day). And it appears broken all 3 times only then mark it as broken.

[joepitt91]

I'd suggest an .env option to try a GET request if HEAD fails, per #547

[martadinata666]

Just some thoughts after my email notification spammed by link check error, add ability to disable or change number of check daily, weekly. Thanks.

[naquad]

Hello there! I've recently switched from a DIY thing to LinkAce, kudos for lists & the advanced search, very cool features :)

There are a couple of things to consider though regarding the link check:

1. A configuration option to control the link check notification. Email is too much for this, I would prefer to disable that. Maybe a site notification during the next visit or something. For now, I've added an explicit return in the App\Console\Commands\CheckLinksCommand::sendNotification, but that's not a real solution.
2. More control over the link checker. Extra headers, options, and behavior to emulate the real browser. I've updated the linked checker to add headers Accept and Accept-Encoding (values from Chromium) and as mentioned above use GET rather than HEAD.

After the second change, I went from 94 broken links to 11 (actually broken). Mostly they're from Reddit, Amazon, and HackerNews.

P. S. Just in case here's the result: https://gist.github.com/d3f0e1765548b7c1beaebd02d3373320