Automate Validation

[acannistra]

TODO:

• Create a Docker image with process_results.yml conda environment as base that will allow for the production of validation statistics
  • Modify Travis to test-build this new image also
• Produce a Makefile which automates the following workflow:
  • Reversibly set AWS credentials provided to make as the default profile to allow for parameterization of credentials
  • Parameterize local or S3-based configuration file locations + download remote (s3) config files + store locally
  • Run prediction on selected validation image bucket (parameterized via env variable) given model output checkpoint (again via env or make variable)
  • Gather prediction/mask/image tiles into central location
  • Produce mean/std validation metrics (precision/recall/f1) for the mask/pred pairs

cc: @ajijohn