-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add downloader for AVCX #202
base: main
Are you sure you want to change the base?
Conversation
Adds a downloader for AVCX (American Values Club Crosswords). These are popular crosswords from a variety of creators, see https://avxwords.com/about-us/. This is a subscription-only crossword series, and requires authentication. This is handled in exactly the same way as NYT. This downloader may not seem to serve an obvious purpose, given that AVCX emails subscribers an AcrossLite compatible .puz file for every new release. However, I'm thinking it will be useful for the following features: * Automatic downloading of new crosswords, e.g. using a crontab. * Would allow downstream software like Gnome Crosswords to fetch AVCX automatically. * I do a few fixups on the .puz files and include and difficulty metadata. * Some crosswords are not available in .puz from the AVCX website, e.g. barred crosswords like https://avxwords.com/puzzles/1621. As these are available in JPZ I think it would be nice to do a very minimal translation into a close .puz approximation.
Just noticed there's sort of a JPZ parser already in |
|
||
def find_solver(self, url): | ||
if "puzzles" in url: | ||
url = url.removesuffix("/") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is 3.9+ and 3.8 is not quite EOL yet but I'm okay with that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, normally I'd be all for backward compatibility, but 3.8 will probably be EOL before the next release of xword-dl, and at this point even Debian oldstable has 3.9. Still, willing to to change it if you'd prefer.
At a glance, this is great! I want to poke around at it some and test it out, but as an AVCX subscriber I would totally use this. |
@thisisparker Question about using |
It's likely my cleanup function being a little overzealous. These are |
Yep, my AVCX code slaps several bits of metadata into the notes with |
I had a look myself, this is an issue with using Also, what's the intended purpose of using this library? It converts HTML to a Markdown equivalent, but does the AcrossLite PUZ specification support Markdown text representation? Are there specific programs that display it correctly? I tried putting the HTML markup directly in |
The intention behind html2text is to convert from something that looks "marked up" to something that doesn't, because some clients don't render html and e.g. That's all probably a matter of opinion! Which is why I added the |
Yes, that fixes the issue with removing new lines. |
Alright! Then one option is to pass it at runtime each time, or another would be to put a |
Hmm, well me not liking the look of it is one thing, but it removing any new lines in the notes string is another. That seems like it should be avoided. Should downloaders that have plain text notes replace Seems like this ought to affect the Puzzle Society downloader too, although that one is currently disabled. |
I think that using Semantically it's probably even a touch better to just wrap paragraphs in |
self.descriptions.append(f"Edited by {parts[2]}.") | ||
|
||
if self.descriptions: | ||
puzzle.notes = "\n\n".join(self.descriptions) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Per my last comment, I think you could do this like
puzzle.notes = "\n\n".join([f"<p>{d}</p>" for d in self.descriptions])
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, I've mostly done that. There's a little more tinkering to get a nice plain text rendering; I'm replacing any links with just the text when preserve-html
is off, hopefully you think that's a reasonable compromise if no one is actually rendering the Markdown at present.
Adds a downloader for AVCX (American Values Club Crosswords). These are popular crosswords from a variety of creators, see https://avxwords.com/about-us/.
This is a subscription-only crossword series, and requires authentication. This is handled in exactly the same way as NYT.
This downloader may not seem to serve an obvious purpose, given that AVCX emails subscribers an AcrossLite compatible .puz file for every new release. However, I'm thinking it will be useful for the following features: