Opening up puzzle moderation

[fakelvis]

Triggered by the conversation on reddit, I thought I'd open a discussion here first, rather than an issue.

Some initial thoughts…

I would presume that there would need to be some guidance written in terms of what you, as the guiding voice of the project, want moderation to achieve. For instance:

• Identifying potential copyright infringement -- how do you currently do this? On puzzle titles/copyright metadata of .puz uploads, or also a quick Google search for marque entries in manually-created puzzles?
• Do you have a list of entries or clues that you wouldn't want to get posted to the site?
• Observing your process from the other side, I see a best practice would be to also engage immediately with approved puzzles via comments and slight editorial-style comments on puzzles (which is great, as it builds engagement and also helps identify puzzle errors before wider circulation).
• Other?

How would you envisage opening up puzzle moderation to more people?

• Opening the moderation queue to trusted users?
• Double moderation during an initial period?
• Technical restrictions on which puzzles get moderated, or work solely on trust (i.e. can someone moderate their own puzzles)?

As mentioned, these are just some initial thoughts to start a discussion. Interested in hearing your take on some of this.

[mdirolf]

Thanks for starting this discussion! To date, there has actually been very little need for extensive moderation from a copyright/content perspective 🙌. It's mostly been needed just to decide what gets featured on the homepage / pick daily minis / give an initial comment to the puzzles. I'll start by describing how the process works currently:

• First, when somebody posts a new puzzle there is a very simplistic check for some "watermarks" from major publishers to try to automatically warn folks not to post copyrighted puzzles.
• New puzzles are live immediately but also get sent to a moderation queue where I can go through them when I have a chance.
• If a puzzle is private I usually just sanity check that the title isn't crazy offensive or obviously infringement and then mark it as moderated - since it's not eligible for the public areas of the site I'm OK w/ keeping the check very simple
• If a puzzle is public (or 'private until xxx') I try my best to give it a solve. As you point out, even if it doesn't end up getting featured I like to try to make a comment / give some feedback on almost every public puzzle just to encourage the community.
• In terms of copyright checking, I haven't started doing extensive google searching, etc - so far the one or two times ppl have posted copyrighted stuff it's been pretty immediately obvious from the title, etc.
• After solving, if a puzzle is a mini and the cluing seems good and doesn't have any real controversial entries then I'll often moderate it by marking it as a daily mini for a particular date - I try to keep the next month or so of these scheduled, so once I've got a month out filled I might opt to put a new mini as a featured puzzle instead of scheduling as a daily mini.
• If a puzzle wasn't selected as a daily mini but it has good cluing that generally follows the "rules" of crosswords I'll usually mark it as featured so it'll go on the homepage.
• Other puzzles just get marked as moderated so they leave the queue.

[mdirolf]

As for how I envision opening this up - it's something I haven't devoted much time to thinking about so my thoughts are pretty nascent.

There are a variety of technical limitations that will make multiple moderators tricky - things that were easy to implement when I knew only one person would be moderating - for example let's say 2 moderators open a puzzle from the queue simultaneously and one decides to "feature" it while the other decides to schedule it as a daily mini. Or they decide to schedule as daily mini for different dates, etc. etc.

Work could be done to eliminate those race conditions but here's another idea:

• A new page gets added that shows unmoderated public or "private until" puzzles in the queue. This page is available only to folks who are tagged as trusted moderators.

• After finishing a puzzle + commenting on it a trusted moderator can give it a thumbs up or down (or flag it for deletion?) - essentially saying whether they think it is worthy of being featured based on good cluing, following the rules of crosswords, not using offensive terms, etc. Flagging would be available for content that is copyright infringement or very offensive. Maybe even a comment or explanation of why you voted how you did.

• At least at first (until those race conditions get sorted) I'll continue to be the only one who can mark a puzzle as featured or select it as a daily mini or delete it entirely. But the advantage is I'll now be able to see upvote/downvote/flags/(comments?) from anybody who already moderated it. If I'm working through a huge backlog I could potentially skip solving and just say "5 mods liked this one, I'll feature it" or "nobody thought this was good enough to feature, I can skip it".

Thoughts on that idea? Suggestions?