Add integration tests for galaxy packages against tpv

[nuwang]

In the past, we've run into issues with TPV testing due to PyPI packaging, normally because a file was overlooked while bundling galaxy PyPI packages.

#16308
#14018
#14020

Most recently, this: #17748

TPV uses a fairly comprehensive gamut of Galaxy packages, so it seems like a good integration test target to make sure PyPI packaging is not overlooking required files.

How to test the changes?

(Select all options that apply)

• I've included appropriate automated tests.
• This is a refactoring of components with existing test coverage.
• Instructions for manual testing are as follows:
  1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[nuwang]

I'm moving this back to draft because it didn't catch this issue: #17748
Installing from an editable location won't cut it I suppose, since the symlinks will resolve all missing files. I'm now thinking of creating a wheel first, and testing through a wheel based installation.

[nuwang]

This is temporarily pointing to my upstream to simulate some additional missing files.

[mvdbeek]

Can we turn this around and have TPV runs tests against multiple Galaxy versions ? That's the bioblend model and I think it works well ? Running this against each PR doesn't seem necessary, instead we can include the addition of new releases in the Galaxy release process ?

[nuwang]

Are you saying that the github event trigger should change from PR to release? As long as we can detect releases with broken packages prior to release, that's fine I think.

[mvdbeek]

No, I'm suggesting to run the TPV tests in a cronjob from the TPV repo. We'll make sure you'll run against all relevant branches.

[nsoranzo]

No, I'm suggesting to run the TPV tests in a cronjob from the TPV repo.

For BioBlend, we use this: https://github.com/galaxyproject/bioblend/blob/main/.github/workflows/test.yaml#L2-L7

[nuwang]

There tests are meant to catch Galaxy packaging errors, not TPV errors, so I think it defeats the purpose if Galaxy integration tests don't fail when packaging fails. In fact, TPV tests have broken numerous times (see linked issues above), when the breakage could have been detected and fixed at the point the breaking change was introduced into Galaxy. If the worry is that some TPV changes can break these tests, we can pin to a specific commit.

[mvdbeek]

I don't think it makes sense to run TPV tests to detect packaging issues, at least not on the Galaxy side as that's very indirect. If you want to contribute to more strict packaging tests that would of course be welcome.