Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wave with Docker and Spack #4940

Draft
wants to merge 73 commits into
base: wave
Choose a base branch
from
Draft

Wave with Docker and Spack #4940

wants to merge 73 commits into from
+204 −18

Commits on Feb 16, 2024

  1. build: Add wave

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    9db4c91 View commit details
    Browse the repository at this point in the history

  2. build: Set strategy to dockerfile, conda then container

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    fe7665f View commit details
    Browse the repository at this point in the history

  3. refactor: Remove container

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    59d3019 View commit details
    Browse the repository at this point in the history

  4. build: Add a repo to push to

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    67c62bd View commit details
    Browse the repository at this point in the history

  5. ci(wave): Add wave build 

    https://github.com/nodejs/docker-node/blob/3c4fa6daf06a4786d202f2f610351837806a0380/.github/workflows/build-test.yml#L29
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    f1d8649 View commit details
    Browse the repository at this point in the history

  6. ci(wave): Switch to all_changed_files

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    5415a42 View commit details
    Browse the repository at this point in the history

  7. ci(wave): Only look for envronment.ymls

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    36414d1 View commit details
    Browse the repository at this point in the history

  8. dummy: Change env

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    25957b2 View commit details
    Browse the repository at this point in the history

  9. ci(wave): Remove raw format

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    5d3f4f1 View commit details
    Browse the repository at this point in the history

  10. ci(wave): Try a bunch of different things at once

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    c79eb65 View commit details
    Browse the repository at this point in the history

  11. ci(wave): Remove redundant fromJson and wrap in an array

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    fac96ce View commit details
    Browse the repository at this point in the history

  12. ci(wave): I have no idea what I'm doing

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    fc0121d View commit details
    Browse the repository at this point in the history

  13. ci(wave): Wrap it

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    5d875d7 View commit details
    Browse the repository at this point in the history

  14. ci(wave): Found an example 

    https://github.com/tj-actions/changed-files/blob/main/.github/workflows/matrix-test.yml
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    e0ce27d View commit details
    Browse the repository at this point in the history

  15. ci(wave): Maybe quotes?

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    5d37376 View commit details
    Browse the repository at this point in the history

  16. ci(wave): That'll do it

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    0073c28 View commit details
    Browse the repository at this point in the history

  17. ci(wave): Fix wave install

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    5ead848 View commit details
    Browse the repository at this point in the history

  18. ci(wave): Hard code an image

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    1e89bff View commit details
    Browse the repository at this point in the history

  19. ci(wave): Add secrets

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    ea9ab86 View commit details
    Browse the repository at this point in the history

  20. feat: Try a different files structure

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    8571bc9 View commit details
    Browse the repository at this point in the history

  21. ci(wave): First stab at building singularity images

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    d6ef54a View commit details
    Browse the repository at this point in the history

  22. fixup! feat: Try a different files structure

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    6a4af4d View commit details
    Browse the repository at this point in the history

  23. ci(wave): Add profile to matrix

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    5410b46 View commit details
    Browse the repository at this point in the history

  24. ci(wave): Give up on fancy substitution

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    1090212 View commit details
    Browse the repository at this point in the history

  25. ci(wave): Add await 

    Co-authored-by: ewels <[email protected]>
    @edmundmiller @ewels
    edmundmiller and ewels committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    a3bf363 View commit details
    Browse the repository at this point in the history

  26. ci(wave): Switch to quay

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    a3fa9d0 View commit details
    Browse the repository at this point in the history

  27. test(wave): Add freeze and update build repo

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    adafa32 View commit details
    Browse the repository at this point in the history

  28. refactor(wave): What happens if I add a container?

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    59f6b4f View commit details
    Browse the repository at this point in the history

  29. refactor(wave): Have both bowtie modules use the same env 

    For the sake of demonstration
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    d7cbe39 View commit details
    Browse the repository at this point in the history

  30. test: Cut out using wave on tests

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    7edc6d3 View commit details
    Browse the repository at this point in the history

  31. refactor: What happens if we use the singularity one?

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    b737da1 View commit details
    Browse the repository at this point in the history

  32. refactor: Keep container directives for offline download 

    seqeralabs/wave#323
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    5e838e7 View commit details
    Browse the repository at this point in the history

  33. feat: Try new singularity OCI setting 

    nextflow-io/nextflow@f5362a7
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    e89a5b5 View commit details
    Browse the repository at this point in the history

  34. build: Update container name 

    Guess #4327 broke that
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    7312a7a View commit details
    Browse the repository at this point in the history

  35. chore: Bump wave-cli version

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    6ea8d38 View commit details
    Browse the repository at this point in the history

  36. ci: Install runc

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    0b50913 View commit details
    Browse the repository at this point in the history

  37. ci: Switch to singularityhub action 

    nextflow-io/nextflow#4543
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    a4a11c5 View commit details
    Browse the repository at this point in the history

  38. ci: Install new singularity manually 

    Why that action trys to build from source, idk.
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    39bbce3 View commit details
    Browse the repository at this point in the history

  39. ci: Install dependancies for singularity

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    41602b0 View commit details
    Browse the repository at this point in the history

  40. ci: runc => crun

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    bc00761 View commit details
    Browse the repository at this point in the history

  41. ci: Fix cgroup error 

    https://blog.misharov.pro/2021-05-16/systemd-github-actions
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    df744e2 View commit details
    Browse the repository at this point in the history

  42. ci: That'll do it

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    da53a8d View commit details
    Browse the repository at this point in the history

  43. ci: Remove Dockerfile 

    We'll have a seperate action for this I think
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    94aef22 View commit details
    Browse the repository at this point in the history

  44. ci: Update name

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    6d3e32e View commit details
    Browse the repository at this point in the history

  45. ci: Push to the correct repos

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    1759f64 View commit details
    Browse the repository at this point in the history

  46. ci: Remove OCI stuff

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    1c65a71 View commit details
    Browse the repository at this point in the history

  47. ci: Need a full URL

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    2f06ec6 View commit details
    Browse the repository at this point in the history

  48. ci: Fix // in container name

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    255b05c View commit details
    Browse the repository at this point in the history

  49. ci: Remove push 

    Build once, renovate should bump the images automagically
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    f2e8c7a View commit details
    Browse the repository at this point in the history

  50. build: Add containers back

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    509c300 View commit details
    Browse the repository at this point in the history

  51. ci: Add cache repos 

    Idk what this does exactly
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    463af3b View commit details
    Browse the repository at this point in the history

  52. ci: Change registry name to use _ 

    Because "build" is a api end point on quay.io.

So `bowtie/build` doesn't work.

Other plus is this matches the conda env name.
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    6dfdcfc View commit details
    Browse the repository at this point in the history

  53. build: / => _ in container name

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    e0d58c3 View commit details
    Browse the repository at this point in the history

  54. Try ociAutoPull

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    8505eec View commit details
    Browse the repository at this point in the history

  55. chore: Add renovate comments to samtools 

    Just to trigger wave build
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    6961e19 View commit details
    Browse the repository at this point in the history

  56. test: Add ociAutoPull to nf-test

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    e93ad79 View commit details
    Browse the repository at this point in the history

  57. ci: Use latest Nextflow

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    7b1070c View commit details
    Browse the repository at this point in the history

  58. ci: Bump wave version

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    881de0a View commit details
    Browse the repository at this point in the history

  59. chore: Bump containers with new wave version 

    Not sure why that's happening...
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    25f2b00 View commit details
    Browse the repository at this point in the history

  60. ci: Remove Singularity building

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    7a6eb3f View commit details
    Browse the repository at this point in the history

  61. ci: Clean up conda building

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    f61a1ae View commit details
    Browse the repository at this point in the history

  62. ci: Add Docker wave step

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    65b9816 View commit details
    Browse the repository at this point in the history

  63. ci: Add Spack build Step

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    bb5368a View commit details
    Browse the repository at this point in the history

  64. chore: Update container registry

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    802588c View commit details
    Browse the repository at this point in the history

  65. build(bowtie): Try bowtie with spack

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    c2fe72e View commit details
    Browse the repository at this point in the history

  66. chore: Bump action versions

    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    5f0e2da View commit details
    Browse the repository at this point in the history

  67. ci: Follow the example and add "files" 

    https://github.com/tj-actions/changed-files/blob/f0ce41b71c3bb642dfc7cb1ab61d14d3b56af681/.github/workflows/matrix-example.yml
    @edmundmiller
    edmundmiller committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    1e738d1 View commit details
    Browse the repository at this point in the history

Commits on Feb 17, 2024

  1. ci: Try removing quotepath

    @edmundmiller
    edmundmiller committed Feb 17, 2024
    Configuration menu
    Copy the full SHA
    8d2faf8 View commit details
    Browse the repository at this point in the history

  2. ci: That'll do it

    @edmundmiller
    edmundmiller committed Feb 17, 2024
    Configuration menu
    Copy the full SHA
    d3a1fa6 View commit details
    Browse the repository at this point in the history

  3. fix: Spack syntax

    @edmundmiller
    edmundmiller committed Feb 17, 2024
    Configuration menu
    Copy the full SHA
    f5579cd View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2024

  1. ci: Remove spack for now 

    seqeralabs/wave#351
    @edmundmiller
    edmundmiller committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    fc4a9e8 View commit details
    Browse the repository at this point in the history

  2. build: Test a theory for gatk4 packaging bug

    @edmundmiller
    edmundmiller committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    116adce View commit details
    Browse the repository at this point in the history

  3. build: Try passing a url 

    https://simonwillison.net/2022/Apr/24/pip-install-github/
    @edmundmiller
    edmundmiller committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    6243136 View commit details
    Browse the repository at this point in the history