Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initial scrapy, actions setup #1

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

pjsier
Copy link
Collaborator

@pjsier pjsier commented Dec 14, 2021

I adapted a lot of this from City-Bureau/city-scrapers, but this should give us a decent starting point for writing scrapers and running them on GitHub Actions. We'll want to disable GitHub Actions on the repo for now while this is still a dummy spider

I realized I wasn't sure what data we want for each item, so this is mostly initial guesses

  • Sets up poetry for dependency management on Python 3.9
  • Sets up a basic scrapy project that doesn't currently write anywhere
  • Initial scrapy item with some validation built in
  • Adds a basic scraper and test file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant