Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed version of nextflow, hopefully this works #118

Merged
merged 8 commits into from
Dec 17, 2024
Merged

Changed version of nextflow, hopefully this works #118

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
59851ff
From the way nextflow is setup, it doesn't work if we use '{workDir}'…
harmonbhasin Nov 27, 2024
829e2d4
Merge pull request #116 from naobservatory/harmon_fix_copying_file_bug
willbradshaw Nov 27, 2024
c71289d
Changed version of nextflow, hopefully this works
harmonbhasin Dec 3, 2024
8d40459
Updated label to be correct
harmonbhasin Dec 3, 2024
222946d
Merge branch 'master' of https://github.com/naobservatory/mgs-workflo…
harmonbhasin Dec 3, 2024
ef3a1bb
Splitting up tests
harmonbhasin Dec 4, 2024
67742e8
Didn't know that I had an out of date verison of this branch.
harmonbhasin Dec 4, 2024
8ae0a93
Forgot to remove validation from run test, hopefully this will test t…
harmonbhasin Dec 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 50 additions & 12 deletions .github/workflows/end-to-end.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ name: End-to-end MGS workflow test
on: [pull_request]

jobs:
test:
test-index:
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- name: Checkout
Expand All @@ -16,10 +17,10 @@ jobs:
java-version: '11'
distribution: 'adopt'

- name: Setup Nextflow latest-edge
- name: Setup Nextflow latest (stable)
uses: nf-core/setup-nextflow@v1
with:
version: "latest-edge"
version: "latest"

- name: Install nf-test
run: |
Expand All @@ -28,19 +29,56 @@ jobs:

- name: Run index workflow
run: nf-test test --tag index --verbose
test-run:
runs-on: ubuntu-latest
timeout-minutes: 10

- name: Clean docker for more space
run: |
docker kill $(docker ps -q) 2>/dev/null || true
docker rm $(docker ps -a -q) 2>/dev/null || true
docker rmi $(docker images -q) -f 2>/dev/null || true
docker system prune -af --volumes
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Clean up nf-test dir
run: sudo rm -rf .nf-test
- name: Set up JDK 11
uses: actions/setup-java@v4
with:
java-version: '11'
distribution: 'adopt'

- name: Setup Nextflow latest (stable)
uses: nf-core/setup-nextflow@v1
with:
version: "latest"

- name: Install nf-test
run: |
wget -qO- https://get.nf-test.com | bash
sudo mv nf-test /usr/local/bin/

- name: Run run workflow
run: nf-test test --tag run --verbose

test-validation:
runs-on: ubuntu-latest
timeout-minutes: 5

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up JDK 11
uses: actions/setup-java@v4
with:
java-version: '11'
distribution: 'adopt'

- name: Setup Nextflow latest (stable)
uses: nf-core/setup-nextflow@v1
with:
version: "latest"

- name: Install nf-test
run: |
wget -qO- https://get.nf-test.com | bash
sudo mv nf-test /usr/local/bin/

- name: Run run_validation workflow
run: nf-test test --tag validation --verbose
run: nf-test test --tag validation --verbose
4 changes: 2 additions & 2 deletions workflows/run.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,9 @@ workflow RUN {
time_ch = Channel.of(start_time_str + "\n").collectFile(name: "time.txt")
version_ch = Channel.fromPath("${projectDir}/pipeline-version.txt")
index_params_ch = Channel.fromPath("${params.ref_dir}/input/index-params.json")
.map { file -> file.copyTo("${workDir}/params-index.json") }
.map { file -> file.copyTo("${params.base_dir}/work/params-index.json") }
index_pipeline_version_ch = Channel.fromPath("${params.ref_dir}/logging/pipeline-version.txt")
.map { file -> file.copyTo("${workDir}/pipeline-version-index.txt") }
.map { file -> file.copyTo("${params.base_dir}/work/pipeline-version-index.txt") }
publish:
// Saved inputs
index_params_ch >> "input"
Expand Down
4 changes: 2 additions & 2 deletions workflows/run_validation.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,9 @@ workflow RUN_VALIDATION {
time_ch = Channel.of(start_time_str + "\n").collectFile(name: "time.txt")
version_ch = Channel.fromPath("${projectDir}/pipeline-version.txt")
index_params_ch = Channel.fromPath("${params.ref_dir}/input/index-params.json")
.map { file -> file.copyTo("${workDir}/params-index.json") }
.map { file -> file.copyTo("${params.base_dir}/work/params-index.json") }
index_pipeline_version_ch = Channel.fromPath("${params.ref_dir}/logging/pipeline-version.txt")
.map { file -> file.copyTo("${workDir}/pipeline-version-index.txt") }
.map { file -> file.copyTo("${params.base_dir}/work/pipeline-version-index.txt") }
publish:
// Saved inputs
index_params_ch >> "input"
Expand Down
Loading