End-to-end submission time for each project and graphs of kpis

[parthshahva]

As a DCP member, I would like to see the end-to-end time from first bundle exported to last analysis bundle deposited and a graph of mean end-to-end over time graph in order to determine if the DCP is improving the time it takes for data to be made available

[parthshahva]

New event dynamo table that stores time for major submission events... 1) new submission in submitted/processing state (ingest), 2) timestamp for first primary bundle in DSS, 3) timestamp for first/last workflow submitted, workflow started, workflow completed 4) timestamp for ingest submission complete 5) timestamp for azul primary complete 6) timestamp for analysis complete 7) timstamp for matrix complete 8) timestamp for azul analysis complete

[parthshahva]

Graphs

1. Total number of cells, line chart (last 90 days)
   1. Tracker puts a cloud watch metric every time it runs
2. Total number of projects, line chart (last 90 days)
   1. Tracker puts a cloud watch metric every time it runs
3. % of projects analyzed, line chart (last 90 days)
   1. Tracker puts a cloud watch metric every time it runs
4. % of projects available in matrix service, line chart (last 90 days)
   1. Tracker puts a cloud watch metric every time it runs
5. DCP Test reliability (running % over last 90 days), single number
   1. Tracker puts a cloud watch metric every time it runs
6. Distribution of projects by species, pie chart
   1. Tracker has an api endpoint
7. Distribution of projects by method, pie chart
   1. Tracker has an api endpoint
8. % of projects that completed without error (both with and without analysis)
   1. Tracker has an api endpoint
   2. How to tell if error occurred?
      1. Primary submission has error
      2. Workflow failed
      3. Analysis submission envelope failed

Submission Time of last 5 projects
- api endpoint
Primary submission time - first project update date - first project submission date
Analysis time - last envelope update date - update date of primary submission envelope

[parthshahva]

First cut can be found at https://tracker.data.humancellatlas.org/metrics.html. Additional modifications and improvements will be made over time