Release 2.4.0 #181

Daniel-VM · 2024-11-05T14:00:52Z

PR checklist

PR description

Release 2.4.0

Major changes are:

Refactored long-reads polishing – Improved the long-reads polishing step for better performance.
Removed params.save_merged – Merged reads are no longer used in downstream analysis.
Updated Kmerfinder module – Enhanced the module with increased memory resources to handle larger datasets.
Replaced local Unicycler module – Switched to the nf-core Unicycler module for consistency and version updates.
Updated to nf-core-tools v3.0.2 – Integrated the latest template updates from nf-core tools v3.0.2.
Added FastQC after-trimming – Included after-trimming FastQC section in MultiQC report for improved quality checks.
Supported automatic concatenation of FastQ files – Simplified input handling by adding automatic concatenation for FastQ files of the same sample.
Multiple fixes in input and metadata handling – Resolved issues in input channels for long reads, params.save_merged, Canu, KmerFinder, and Prokka/Bakta.

This summary covers the main improvements, new features, and fixes in version 2.4.0 of the nf-core/bacass pipeline.

Replace local unicycler module with nf-core module + bump version

Fix missing 'dev' tag in bumped version

Fixed `files_unchanged` section in `.nf-core.yml` to allow a proper linting test.

Fix kmerfinder scripts

Fixed current Zenodo URL of Kmerfinderdb

Update module Kmerfinder

Fix memory issues, handle KmerFinder results, and manage empty fasta files

Fix bakta running only for one sample

Added module to concatenate/merge fastq files

fixed matrix.test_name in linting and missing features from template 3.0.2

Bump version 2.4.0

github-actions · 2024-11-05T14:02:51Z

`nf-core pipelines lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 72cfd85

+| ✅ 211 tests passed       |+
#| ❔   2 tests were ignored |#
!| ❗   1 tests had warnings |!

❗ Test warnings:

pipeline_todos - TODO string in nextflow.config: Optionally, you can add a pipeline-specific nf-core config at https://github.com/nf-core/configs

❔ Tests ignored:

files_exist - File is ignored: conf/igenomes.config
nextflow_config - Config default ignored: params.dfast_config

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-bacass_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-bacass_logo_light.png
files_exist - File found: docs/images/nf-core-bacass_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes_ignored.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-bacass_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowBacass.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Found nf-schema plugin
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: validation.help.enabled
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable found: validation.help.beforeText
nextflow_config - Config variable found: validation.help.afterText
nextflow_config - Config variable found: validation.help.command
nextflow_config - Config variable found: validation.summary.beforeText
nextflow_config - Config variable found: validation.summary.afterText
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config variable (correctly) not found: params.max_cpus
nextflow_config - Config variable (correctly) not found: params.max_memory
nextflow_config - Config variable (correctly) not found: params.max_time
nextflow_config - Config variable (correctly) not found: params.validationFailUnrecognisedParams
nextflow_config - Config variable (correctly) not found: params.validationLenientMode
nextflow_config - Config variable (correctly) not found: params.validationSchemaIgnoreParams
nextflow_config - Config variable (correctly) not found: params.validationShowHiddenParams
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version does not contain dev for release: 2.4.0
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.assembler= unicycler
nextflow_config - Config default value correct: params.assembly_type= short
nextflow_config - Config default value correct: params.polish_method= medaka
nextflow_config - Config default value correct: params.annotation_tool= prokka
nextflow_config - Config default value correct: params.baktadb_download_args= --type light
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
nextflow_config - Config default value correct: params.schema_ignore_params= modules,igenomes_base
nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-bacass_logo_light.png matches the template
files_unchanged - docs/images/nf-core-bacass_logo_light.png matches the template
files_unchanged - docs/images/nf-core-bacass_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 24.04.2, Config: 24.04.2
readme - README Zenodo placeholder was replaced with DOI.
plugin_includes - No wrong validation plugin imports have been found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (0 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: template_version_comment.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
modules_config - conf/modules.config found and not ignored.
modules_config - NANOPLOT found in conf/modules.config and Nextflow scripts.
modules_config - PYCOQC found in conf/modules.config and Nextflow scripts.
modules_config - PORECHOP_PORECHOP found in conf/modules.config and Nextflow scripts.
modules_config - UNICYCLER found in conf/modules.config and Nextflow scripts.
modules_config - CANU found in conf/modules.config and Nextflow scripts.
modules_config - MINIMAP2_ALIGN found in conf/modules.config and Nextflow scripts.
modules_config - MINIMAP2_CONSENSUS found in conf/modules.config and Nextflow scripts.
modules_config - MINIMAP2_POLISH found in conf/modules.config and Nextflow scripts.
modules_config - MINIASM found in conf/modules.config and Nextflow scripts.
modules_config - DRAGONFLYE found in conf/modules.config and Nextflow scripts.
modules_config - RACON found in conf/modules.config and Nextflow scripts.
modules_config - MEDAKA found in conf/modules.config and Nextflow scripts.
modules_config - NANOPOLISH found in conf/modules.config and Nextflow scripts.
modules_config - KRAKEN2 found in conf/modules.config and Nextflow scripts.
modules_config - KRAKEN2_LONG found in conf/modules.config and Nextflow scripts.
modules_config - QUAST found in conf/modules.config and Nextflow scripts.
modules_config - PROKKA found in conf/modules.config and Nextflow scripts.
modules_config - DFAST found in conf/modules.config and Nextflow scripts.
modules_config - MULTIQC_CUSTOM found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 3.0.2
version_consistency - Version tags are numeric and consistent between container, release tag and config.
included_configs - Pipeline config includes custom configs.

Run details

nf-core/tools version 3.0.2
Run at 2024-11-18 13:16:52

uncomment required line for linting in `--release` mode

Fix dfast conda issue by updating version

d4straub

Looks good, but I do have comments ;)

.github/workflows/awsfulltest.yml

.nf-core.yml

conf/modules.config

modules/local/dfast/main.nf

modules/nf-core/unicycler/environment.yml

workflows/bacass.nf

Daniel-VM · 2024-11-18T12:08:19Z

Thanks for the review. @d4straub, suggestions have been added here: #184

Code review for release 2.4.0

d4straub

LGTM!

FloWuenne and others added 30 commits June 26, 2024 16:34

Replaced local unicycler module with nf-core module.

cfe8253

Removed deprecated unicycler parameters from test_dfast.

24f0cbc

Updated CHANGELOG.

2baeb3b

Removed no longer needed unicycler local files.

e922812

Merge pull request #150 from FloWuenne/unicycler_nfcore

03f0dc8

Replace local unicycler module with nf-core module + bump version

fix missing dev tag in bumped version

a49930e

Merge pull request #152 from Daniel-VM/dev

24ec64c

Fix missing 'dev' tag in bumped version

remove files_unchanged items form linting

04091a3

fix files_unchanged to make a proper linting check

1bb3278

added changelog #153

b73c536

Merge pull request #153 from Daniel-VM/dev

cc1882b

Fixed `files_unchanged` section in `.nf-core.yml` to allow a proper linting test.

fix kmerfinder script to download reference

e8442e2

add more resources to kmerfinder module to avoid memory related issues

3bfab15

added changelog #154

3e9cf3f

Merge pull request #154 from Daniel-VM/fix_kmerfinder_scripts

20830af

Fix kmerfinder scripts

fixed current zenodo url to kmerfinderdb

084fec4

update changelog in #157

82adb62

Merge pull request #157 from Daniel-VM/fix_kmerfinderdb

5f64486

Fixed current Zenodo URL of Kmerfinderdb

enhance KmerFinder support parsing multiple file format versions

04def18

update kmerfinderdb description

9280f89

update kmerfinder module

b5b0407

update kmerfinder module dir structure

fd7dd01

update module paths and handle memory-errors in kmerfinder

c2a94d7

Merge pull request #159 from Daniel-VM/update_kmerfinder

9e85db0

Update module Kmerfinder

fix memory issues due to config in kmerfinder

6250ad3

Handle KmerFinder results when no species hit is detected

09a1eb6

handle prokka/bakkta channel when fasta file is empty

ae525c0

update changelog in #160

fcf0ab9

Merge pull request #160 from Daniel-VM/dev

c66630f

Fix memory issues, handle KmerFinder results, and manage empty fasta files

fix saving merged fastq files in fastp module

41fd8c9

Daniel-VM and others added 11 commits October 25, 2024 11:13

Merge pull request #178 from Daniel-VM/dev

064996b

Fix bakta running only for one sample

fixed matrix.test_name in linting

2bccce1

remove parameter section in git ci

fd2b54e

Merge pull request #158 from Daniel-VM/concatenate_fasqs

efe6309

Added module to concatenate/merge fastq files

implement missing features in memory set up from nf-core tools 3.0.2

85bdf85

update changelog #179

aa42f95

Merge pull request #179 from Daniel-VM/fix_linting_test_name

02bf3dd

fixed matrix.test_name in linting and missing features from template 3.0.2

bump version 2.4.0

8434c60

update changelog #180

f1809ce

fix release date in #180

9c95fb7

Merge pull request #180 from Daniel-VM/bump-version-2.4.0

067fb0f

Bump version 2.4.0

uncomment required line for linting in --release mode

262f741

Daniel-VM mentioned this pull request Nov 5, 2024

uncomment required line for linting in --release mode #182

Merged

10 tasks

Daniel-VM and others added 3 commits November 5, 2024 15:29

update changelog in #182

e241ff8

Merge pull request #182 from Daniel-VM/dev

c476f50

uncomment required line for linting in `--release` mode

fix dfast conda issue by updating version

ad68b1f

Daniel-VM mentioned this pull request Nov 6, 2024

Fix dfast conda issue by updating version #183

Merged

10 tasks

Daniel-VM and others added 2 commits November 6, 2024 13:50

update changelog in #183

0f7646a

Merge pull request #183 from Daniel-VM/dev

fc52c3e

Fix dfast conda issue by updating version

Daniel-VM marked this pull request as ready for review November 7, 2024 08:33

saramonzon approved these changes Nov 14, 2024

View reviewed changes

d4straub reviewed Nov 14, 2024

View reviewed changes

added code review from #181

bf367e4

Daniel-VM mentioned this pull request Nov 18, 2024

Code review for release 2.4.0 #184

Merged

10 tasks

fix linting in #184

cd1bf3e

Merge pull request #184 from Daniel-VM/dev

72cfd85

Code review for release 2.4.0

d4straub approved these changes Nov 18, 2024

View reviewed changes

Daniel-VM merged commit be9c064 into master Nov 18, 2024
49 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release 2.4.0 #181

Release 2.4.0 #181

Daniel-VM commented Nov 5, 2024

github-actions bot commented Nov 5, 2024 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

d4straub left a comment

Daniel-VM commented Nov 18, 2024

d4straub left a comment

Release 2.4.0 #181

Release 2.4.0 #181

Conversation

Daniel-VM commented Nov 5, 2024

PR checklist

PR description

github-actions bot commented Nov 5, 2024 • edited Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

d4straub left a comment

Choose a reason for hiding this comment

Daniel-VM commented Nov 18, 2024

d4straub left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 5, 2024 •

edited

Loading

`nf-core pipelines lint` overall result: Passed ✅ ⚠️