[WIP] test script and venv requirements

[cat-bro]

A script to clone training-material and find workflows with tests then run the tests + non-GTN-workflow tests in this repo.

The virtual environment requires planemo and parsec. There is a conflict installing these requirements and it ends up using bioblend version 0.14.0 which is too new for parsec, but this seems to be fine for the parsec calls used in the script.

@bgruening what code is on Jenkins for this currently? I'd like to make the history.html links and the planemo reports available after the test run. The planemo report is useful if the test has failed because of tool versions, history links are useful for just about everything else.

TODO: Makefile, automatic updating of the readme

[hexylena]

There is a conflict installing these requirements and it ends up using bioblend version 0.14.0 which is too new for parsec, but this seems to be fine for the parsec calls used in the script.

yeah, should be fine, the diff between 0.14.0 and what parsec was pinned to, should be very minimal.

what code is on Jenkins for this currently

pip install planemo==0.62.1
pip install galaxy-parsec
export PLANEMO_OPTIONS="--verbose"
export GALAXY_URL="https://usegalaxy.eu"

bash run_galaxy_workflow_tests.sh $WORKFLOW

this is everything that's in jenkins currently.

And the following glob chooses which files to archive:

**/tool_test_output.html, **/history.html

[hexylena]

On jenkins this is done through a plugin that creates a venv (shining panda), so this step isn't necessary. But the pip install is useful.

[cat-bro]

Probably best to pip install -r requirements.txt within code on jenkins

[hexylena]

Suggested change

	workflow_list=workflows_to_test.txt
	workflow_list=$(mktemp)

[hexylena]

I've never seen this construction before, cool! 👍

[hexylena]

Merged test reports will be SO much nicer!!

[cat-bro]

@hexylena thanks for reviewing.

I've taken out a few lines. There will be more that needs to change because I hadn't considered that Jenkins was running a build for each test. So the workflow_list and the test_output folder will both need to be outside of the repo.

[hexylena]

Or we can switch it to running a single build! I used the multi-build strategy only because I wanted the nice pass/fail badges for each WF, but, this looks so much more conceptually simple/correct than the old design (where updating the workflow list was super difficult) that it's definitely worth sacrificing the badges for, we can find another, better solution for those.