Pipeline stabilization

[kltm]

Project link

https://github.com/orgs/geneontology/projects/149

Project description

The purpose of this project is to increase the number of releases (snapshot and release) made and decrease the amount of time per release attempt. While a complete pipeline refactor may be in the cards in the future, this project is aimed at, more or less, "working with what we have".

To this end, the focus will be:

• buy new machine, to:
  • try and ameliorate In some cases, the pipeline fails due to container not stopping fast enough pipeline#316
  • increase overall throughput
• look at blessing snapshots to release, to:
  • help with things like Explore getting the pipeline completing again by adjusting settings and runtime parameters pipeline#349
  • prevent conflicting run timings when snpashot or release not going well
• drop Solr indices from the Zenodo archive bundle, to:
  • make the final stages a little more "stable"
  • help deal with recent Zenodo changes Can no longer upload to Zenodo; due to recent Zenodo update? pipeline#345

PI

Chris

Product owner (PO)

Pascale

Technical lead (TL)

Seth

Other personnel (OP)

TBD

Technical specs

No new libraries or technologies. The only "interesting" additions would likely be:

• an extension of the current Zenodo scripts to assemble an upload package from a given snapshot and push it
• snapshots to get an associated date and location
• figure out what to do about testing

Other comments

N/A

[kltm]

@pgaudet I have a note that says that this can be considered under way (lit. "move to working"); does that sound right to you?

[kltm]

@pgaudet confirms above.

[kltm]

@kltm believes that the remainder can be folded into the GOC / GOA pipeline project, as the work here is not completely informed or dependent decisions in that project.

[kltm]

closed; remaining tickets moved to
#93
https://github.com/orgs/geneontology/projects/159