Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates #217

Merged
merged 26 commits into from
Feb 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
1847de9
Update version
chris-cheshire Nov 7, 2023
ae11418
Run multiqc when running some only keywords
chris-cheshire Nov 7, 2023
054e040
Improve igg ctrl matching with target samples
chris-cheshire Nov 8, 2023
ad99e9f
Update version
chris-cheshire Nov 8, 2023
bb59ea5
Add option to change sample grouping in igv output
chris-cheshire Nov 11, 2023
1cea405
Add ability to dump scale factor values out of the pipeline
chris-cheshire Nov 11, 2023
55dac76
Update scale factor dump
chris-cheshire Nov 11, 2023
1575fd0
Update logging
chris-cheshire Nov 11, 2023
81baaf1
Auto scale igv groups together
chris-cheshire Nov 11, 2023
448b192
Group samples on group in igv
chris-cheshire Nov 11, 2023
f0a800a
Update metadata
chris-cheshire Nov 28, 2023
d4fa90d
Update igg control mapping
chris-cheshire Jan 31, 2024
6970c85
black updating
chris-cheshire Jan 31, 2024
a6bdc2c
Update for tech reps
chris-cheshire Jan 31, 2024
19ebcc5
Merge remote-tracking branch 'nf-core/nf-core-template-merge-2.12' in…
chris-cheshire Feb 1, 2024
cbf9cf8
Update schema
chris-cheshire Feb 1, 2024
7e779ef
Prettier formating
chris-cheshire Feb 1, 2024
0d50fff
Update template
chris-cheshire Feb 1, 2024
f518d1e
Update schema
chris-cheshire Feb 1, 2024
80a3e69
Update local versions
chris-cheshire Feb 1, 2024
4238a79
Update schema
chris-cheshire Feb 1, 2024
3bdd809
Update versions
chris-cheshire Feb 1, 2024
da78f72
Update changelog
chris-cheshire Feb 1, 2024
957b7ab
Merge branch 'template-merge' into dev
chris-cheshire Feb 1, 2024
037a62b
Whitespace
chris-cheshire Feb 1, 2024
4407282
whitespace
chris-cheshire Feb 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,11 @@ indent_size = unset

[/assets/email*]
indent_size = unset

# ignore Readme
[README.md]
indent_style = unset

# ignore python
[*.{py}]
indent_style = unset
3 changes: 3 additions & 0 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ If you're not used to this workflow with git, you can start with some [docs from

## Tests

You can optionally test your changes by running the pipeline locally. Then it is recommended to use the `debug` profile to
receive warnings about process selectors and other debug info. Example: `nextflow run . -profile debug,test,docker --outdir <OUTDIR>`.

When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.

Expand Down
1 change: 1 addition & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/cuta
- [ ] If necessary, also make a PR on the nf-core/cutandrun _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core lint`).
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
- [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`).
- [ ] Usage Documentation in `docs/usage.md` is updated.
- [ ] Output Documentation in `docs/output.md` is updated.
- [ ] `CHANGELOG.md` is updated.
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
}
profiles: test_full

- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@v4
with:
name: Tower debug log file
path: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/awstest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
}
profiles: test

- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@v4
with:
name: Tower debug log file
path: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/branch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
# NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets
- name: Post PR comment
if: failure()
uses: mshick/add-pr-comment@v1
uses: mshick/add-pr-comment@v2
with:
message: |
## This PR is against the `master` branch :x:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ jobs:
CAPSULE_LOG: none
steps:
- name: Check out pipeline code
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Install Nextflow
uses: nf-core/setup-nextflow@v1
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/clean-up.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v7
- uses: actions/stale@v9
with:
stale-issue-message: "This issue has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment otherwise this issue will be closed in 20 days."
stale-pr-message: "This PR has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment if it is still useful."
Expand Down
67 changes: 67 additions & 0 deletions .github/workflows/download_pipeline.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
name: Test successful pipeline download with 'nf-core download'

# Run the workflow when:
# - dispatched manually
# - when a PR is opened or reopened to master branch
# - the head branch of the pull request is updated, i.e. if fixes for a release are pushed last minute to dev.
on:
workflow_dispatch:
pull_request:
types:
- opened
branches:
- master
pull_request_target:
branches:
- master

env:
NXF_ANSI_LOG: false

jobs:
download:
runs-on: ubuntu-latest
steps:
- name: Install Nextflow
uses: nf-core/setup-nextflow@v1

- uses: actions/setup-python@v5
with:
python-version: "3.11"
architecture: "x64"
- uses: eWaterCycle/setup-singularity@v7
with:
singularity-version: 3.8.3

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install git+https://github.com/nf-core/tools.git@dev

- name: Get the repository name and current branch set as environment variable
run: |
echo "REPO_LOWERCASE=${GITHUB_REPOSITORY,,}" >> ${GITHUB_ENV}
echo "REPOTITLE_LOWERCASE=$(basename ${GITHUB_REPOSITORY,,})" >> ${GITHUB_ENV}
echo "REPO_BRANCH=${GITHUB_REF#refs/heads/}" >> ${GITHUB_ENV}

- name: Download the pipeline
env:
NXF_SINGULARITY_CACHEDIR: ./
run: |
nf-core download ${{ env.REPO_LOWERCASE }} \
--revision ${{ env.REPO_BRANCH }} \
--outdir ./${{ env.REPOTITLE_LOWERCASE }} \
--compress "none" \
--container-system 'singularity' \
--container-library "quay.io" -l "docker.io" -l "ghcr.io" \
--container-cache-utilisation 'amend' \
--download-configuration

- name: Inspect download
run: tree ./${{ env.REPOTITLE_LOWERCASE }}

- name: Run the downloaded pipeline
env:
NXF_SINGULARITY_CACHEDIR: ./
NXF_SINGULARITY_HOME_MOUNT: true
run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -stub -profile test,singularity --outdir ./results
72 changes: 53 additions & 19 deletions .github/workflows/fix-linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
types: [created]

jobs:
deploy:
fix-linting:
# Only run if comment is on a PR with the main repo, and if it contains the magic keywords
if: >
contains(github.event.comment.html_url, '/pull/') &&
Expand All @@ -13,43 +13,77 @@ jobs:
runs-on: ubuntu-latest
steps:
# Use the @nf-core-bot token to check out so we can push later
- uses: actions/checkout@v3
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
with:
token: ${{ secrets.nf_core_bot_auth_token }}

# indication that the linting is being fixed
- name: React on comment
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
with:
comment-id: ${{ github.event.comment.id }}
reactions: eyes

# Action runs on the issue comment, so we don't get the PR by default
# Use the gh cli to check out the PR
- name: Checkout Pull Request
run: gh pr checkout ${{ github.event.issue.number }}
env:
GITHUB_TOKEN: ${{ secrets.nf_core_bot_auth_token }}

- uses: actions/setup-node@v3
# Install and run pre-commit
- uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5
with:
python-version: 3.11

- name: Install Prettier
run: npm install -g prettier @prettier/plugin-php
- name: Install pre-commit
run: pip install pre-commit

# Check that we actually need to fix something
- name: Run 'prettier --check'
id: prettier_status
run: |
if prettier --check ${GITHUB_WORKSPACE}; then
echo "result=pass" >> $GITHUB_OUTPUT
else
echo "result=fail" >> $GITHUB_OUTPUT
fi
- name: Run pre-commit
id: pre-commit
run: pre-commit run --all-files
continue-on-error: true

- name: Run 'prettier --write'
if: steps.prettier_status.outputs.result == 'fail'
run: prettier --write ${GITHUB_WORKSPACE}
# indication that the linting has finished
- name: react if linting finished succesfully
if: steps.pre-commit.outcome == 'success'
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
with:
comment-id: ${{ github.event.comment.id }}
reactions: "+1"

- name: Commit & push changes
if: steps.prettier_status.outputs.result == 'fail'
id: commit-and-push
if: steps.pre-commit.outcome == 'failure'
run: |
git config user.email "[email protected]"
git config user.name "nf-core-bot"
git config push.default upstream
git add .
git status
git commit -m "[automated] Fix linting with Prettier"
git commit -m "[automated] Fix code linting"
git push

- name: react if linting errors were fixed
id: react-if-fixed
if: steps.commit-and-push.outcome == 'success'
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
with:
comment-id: ${{ github.event.comment.id }}
reactions: hooray

- name: react if linting errors were not fixed
if: steps.commit-and-push.outcome == 'failure'
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
with:
comment-id: ${{ github.event.comment.id }}
reactions: confused

- name: react if linting errors were not fixed
if: steps.commit-and-push.outcome == 'failure'
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
with:
issue-number: ${{ github.event.issue.number }}
body: |
@${{ github.actor }} I tried to fix the linting errors, but it didn't work. Please fix them manually.
See [CI log](https://github.com/nf-core/cutandrun/actions/runs/${{ github.run_id }}) for more details.
65 changes: 13 additions & 52 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,72 +11,33 @@ on:
types: [published]

jobs:
EditorConfig:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- uses: actions/setup-node@v3

- name: Install editorconfig-checker
run: npm install -g editorconfig-checker

- name: Run ECLint check
run: editorconfig-checker -exclude README.md $(find .* -type f | grep -v '.git\|.py\|.md\|json\|yml\|yaml\|html\|css\|work\|.nextflow\|build\|nf_core.egg-info\|log.txt\|Makefile')

Prettier:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- uses: actions/setup-node@v3

- name: Install Prettier
run: npm install -g prettier

- name: Run Prettier --check
run: prettier --check ${GITHUB_WORKSPACE}

PythonBlack:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Check code lints with Black
uses: psf/black@stable

# If the above check failed, post a comment on the PR explaining the failure
- name: Post PR comment
if: failure()
uses: mshick/add-pr-comment@v1
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
message: |
## Python linting (`black`) is failing

To keep the code consistent with lots of contributors, we run automated code consistency checks.
To fix this CI test, please run:

* Install [`black`](https://black.readthedocs.io/en/stable/): `pip install black`
* Fix formatting errors in your pipeline: `black .`

Once you push these changes the test should pass, and you can hide this comment :+1:
python-version: 3.11
cache: "pip"

We highly recommend setting up Black in your code editor so that this formatting is done automatically on save. Ask about it on Slack for help!
- name: Install pre-commit
run: pip install pre-commit

Thanks again for your contribution!
repo-token: ${{ secrets.GITHUB_TOKEN }}
allow-repeats: false
- name: Run pre-commit
run: pre-commit run --all-files

nf-core:
runs-on: ubuntu-latest
steps:
- name: Check out pipeline code
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Install Nextflow
uses: nf-core/setup-nextflow@v1

- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
architecture: "x64"
Expand All @@ -99,7 +60,7 @@ jobs:

- name: Upload linting log file artifact
if: ${{ always() }}
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: linting-logs
path: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/linting_comment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Download lint results
uses: dawidd6/action-download-artifact@v2
uses: dawidd6/action-download-artifact@v3
with:
workflow: linting.yml
workflow_conclusion: completed
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Install dependencies
Expand Down Expand Up @@ -56,7 +56,7 @@ jobs:
bsky-post:
runs-on: ubuntu-latest
steps:
- uses: zentered/bluesky-post-action@v0.0.2
- uses: zentered/bluesky-post-action@v0.1.0
with:
post: |
Pipeline release! ${{ github.repository }} v${{ github.event.release.tag_name }} - ${{ github.event.release.name }}!
Expand Down
3 changes: 3 additions & 0 deletions .gitpod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ tasks:
command: |
pre-commit install --install-hooks
nextflow self-update
- name: unset JAVA_TOOL_OPTIONS
command: |
unset JAVA_TOOL_OPTIONS

vscode:
extensions: # based on nf-core.nf-core-extensionpack
Expand Down
7 changes: 6 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
repos:
- repo: https://github.com/pre-commit/mirrors-prettier
rev: "v2.7.1"
rev: "v3.1.0"
hooks:
- id: prettier
- repo: https://github.com/editorconfig-checker/editorconfig-checker.python
rev: "2.7.3"
hooks:
- id: editorconfig-checker
alias: ec
Loading
Loading