Simplify data collection

[ysthakur]

This PR aims to simplify our data collection process by storing everything in an SQL database and using a less complicated process for randomly generating comment IDs to request.

Regarding choice of database, I used an SQLite database for now for simplicity, but that's temporary. We can switch to a Postgres database later (I just didn't want to put in the work before we figure out how to get 4CAT to read from our own DB). It will require updating some code, though (like using %s rather than ? for formatting SQL queries). It's not actually necessary to store stuff in an SQL database, and I could modify this PR to go back to using a CSV, but using SQLite does simplify things.

The out/ directory is no longer used because we don't have to worry about separating data from different runs. All data is now stored in a data.db file right in the data-collection directory. The logs for each run go into the logs folder, in a file named in the format %Y-%m-%d-%H-%M-%S.log.

For requesting comments now, I gave up on making it deterministic and switched to the following easy method:

• Make a list of all time bins that still need comments
• Randomly pick a bin
• Randomly generate 100 IDs from that bin

I initially checked if any of the generate IDs were duplicates by looking in the database, but this was slow. Instead, I made the id column the primary key for both the comments and misses tables. Now, each time we want to insert a comment/missed ID, we can use INSERT OR IGNORE and it won't add the comment/miss if it's a duplicate. The problem with INSERT OR IGNORE is that it ignores all conflicts, not just primary key duplicates, but I think that's fine. This approach ended up being faster.

I threw in a script (csv_to_db.py) that can take comments.csv and missed-ids.txt files from the old data collector and put that data into a data.db file. We can just import all the data we've collected so far and continue collecting based off of that.

I also made it so the data collector no longer needs to talk to the web dashboard. The web dashboard goes through the database on its own, so it's totally separate. That said, as we collect more comments, updating the web dashboard frequently might not be feasible, so we might have to go back to having it get updates from the data collector. This changes sacrifices performance for simplicity.