Skip to content

mediacloud/sitemap-tools

Repository files navigation

Simple tools for consuming sitemap files

Goals:

  • Simple
  • Composable
  • Parser process a single page without recursion
  • Crawler class has "visit_one" method to allow background crawling
  • For internal use (not adding to PyPI, at this time, anyway)

TO CREATE DEVELOPMENT ENVIRONMENT

make installs development environment, pre-commit hook, runs pre-commit