Metric analysis

[aimalz]

• read in parametrized catalogs via Make the approximate PDFs #56
• loop over N_metrics, N_floats, N_parametrizations
• plot the results

[drphilmarshall]

Let's make a new notebook, data_analysis and use the Ensemble class to analyze all of Melissa's PDFs.

[aimalz]

I'm going to use this issue to group together issues for specific metrics, namely #47, #49, and #42. As qp will be the backend for the LSST-DESC PZ DC1 paper, it will need to have the following metrics implemented in the qp.Ensemble class:

• RMSE (Quantify quantile parametrization approximation with KL divergence #9)
• KLD (Quantify quantile parametrization approximation with KL divergence #9)
• KS/AD/CvM (KS test #47)
• QQ (Q-Q Plot #42)
  (Is the plan still to do all of these @sschmidt23?)
  Some of these may require qp to have some notion of the true redshift, which is also a requirement of LSST SRD photo-z metrics #53 (which in turn suggests Use moments as a metric #49).

[aimalz]

Update: I'm pretty happy with the status of issue/57/analysis and will run something like the analysis notebook on NERSC as soon as it's back on line.