Duplicate entries

[thibaultmol]

I noticed that certain urls in the smallweb list are duplicates, because there is no consistency when it comes to trailing slashes

for ex:

smallweb/smallweb.txt

Lines 10443 to 10444 in b065639

	https://shkspr.mobi/blog/feed
	https://shkspr.mobi/blog/feed/

From what I can tell these are the duplicates:

https://benbrubaker.com/feed
https://blog.lukaszolejnik.com/rss
https://blog.thestateofme.com/feed
https://chrismaury.com/rss
https://computer.rip/rss.xml
https://defungames.com/feed
https://drbenjaminhabib.com/feed
https://fearoflanding.com/feed
https://feeds.feedburner.com/saneef
https://furbo.org/feed
https://georgemauer.net/rss.xml
https://ghuntley.com/rss
https://gregoryszorc.com/blog/feed
https://heatherburns.tech/feed
https://interactionmagic.com/feed
https://kbr.sh/rssfeed.xml
https://marcin.juszkiewicz.com.pl/feed
https://shkspr.mobi/blog/feed
https://smallcultfollowing.com/babysteps/atom.xml
https://solarboi.com/feed
https://thefoggiest.dev/feed
https://www.joshwcomeau.com/rss.xml

I can fix it but I don't know what we're standardizing on
(/ should we have a github actions that then checks if it's consistent when doing PR's)?

[vprelovac]

you can sanitize to what the site resolves with in your browser