From 85f274cdc110b3c57a210ebd347b47d93c67fcfd Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Fri, 6 Dec 2024 16:48:45 +0000 Subject: [PATCH 01/15] Add script recommendations --- .../src/content/docs/guidelines/components/modules.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index 6c130c050e..c81c97a962 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -324,6 +324,14 @@ See the [Bash manual on file operators](https://tldp.org/LDP/abs/html/fto.html) Alternate suggestions include using `grep -c` to search for a valid string match, or other tool which will appropriately error when the expected output is not successfully created. +### Script inclusion + +Where the content of the `script:` block is not a simple call to a command, but rather a script (e.g. in R, Python), you have a set of options available to you: + + * Inline process code: code can still be included in the script block using a shebang like `#!/usr/bin/env python`, and code used inline as with regular shell content. This can, however, be obstructive to understanding of module code when content gets extensive. + * [Module templates](https://www.nextflow.io/docs/latest/module.html#module-templates): templates work exactly the same as inline code (so beware of the need to escape dollar signs etc), however can be stored external to the process definition in the module, in a `templates` subfolder. This is done for the [DESeq2 module](https://github.com/nf-core/modules/tree/master/modules/nf-core/deseq2/differential), for example. + * Externalise the scripts entirely, and access them via a Conda definition. This is probably the cleanest solution, but requires a little more effort and management. + ### Stubs #### Stub block must exist From fb0eb4b4693234f98d7bfddb315c098d326aeffd Mon Sep 17 00:00:00 2001 From: nf-core-bot Date: Fri, 6 Dec 2024 16:53:29 +0000 Subject: [PATCH 02/15] [automated] Fix code linting --- .../docs/src/content/docs/guidelines/components/modules.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index c81c97a962..c149224a1c 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -328,9 +328,9 @@ Alternate suggestions include using `grep -c` to search for a valid string match Where the content of the `script:` block is not a simple call to a command, but rather a script (e.g. in R, Python), you have a set of options available to you: - * Inline process code: code can still be included in the script block using a shebang like `#!/usr/bin/env python`, and code used inline as with regular shell content. This can, however, be obstructive to understanding of module code when content gets extensive. - * [Module templates](https://www.nextflow.io/docs/latest/module.html#module-templates): templates work exactly the same as inline code (so beware of the need to escape dollar signs etc), however can be stored external to the process definition in the module, in a `templates` subfolder. This is done for the [DESeq2 module](https://github.com/nf-core/modules/tree/master/modules/nf-core/deseq2/differential), for example. - * Externalise the scripts entirely, and access them via a Conda definition. This is probably the cleanest solution, but requires a little more effort and management. +- Inline process code: code can still be included in the script block using a shebang like `#!/usr/bin/env python`, and code used inline as with regular shell content. This can, however, be obstructive to understanding of module code when content gets extensive. +- [Module templates](https://www.nextflow.io/docs/latest/module.html#module-templates): templates work exactly the same as inline code (so beware of the need to escape dollar signs etc), however can be stored external to the process definition in the module, in a `templates` subfolder. This is done for the [DESeq2 module](https://github.com/nf-core/modules/tree/master/modules/nf-core/deseq2/differential), for example. +- Externalise the scripts entirely, and access them via a Conda definition. This is probably the cleanest solution, but requires a little more effort and management. ### Stubs From be7200e3716f294de9b8b6e507e7f0507f6de0c4 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Mon, 9 Dec 2024 10:22:21 +0000 Subject: [PATCH 03/15] Update sites/docs/src/content/docs/guidelines/components/modules.md Co-authored-by: James A. Fellows Yates --- .../docs/guidelines/components/modules.md | 58 +++++++++++++++++-- 1 file changed, 54 insertions(+), 4 deletions(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index c149224a1c..7254d65225 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -326,11 +326,61 @@ Alternate suggestions include using `grep -c` to search for a valid string match ### Script inclusion -Where the content of the `script:` block is not a simple call to a command, but rather a script (e.g. in R, Python), you have a set of options available to you: +:::note +You SHOULD NOT use [Nextflow module binaries](https://www.nextflow.io/docs/latest/module.html#module-binaries) where possible, as these are not fully portable across all execution contexts. +::: + +If a module's `script:` block contains not command invocations, but script content, whatever the language (e.g. Bash, R, Python), and if that content is non-trivial in length (>> 10 lines), that MUST be supplied via a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). This makes it clearer when changes have been made to the scientific logic in the script, or to the workflow-relevant logic in the module. + +#### Template location + + The template MUST go in a directory called `templates/` that sits alongside the `main.nf` of the module. + +The template file MUST be named after module itself with a language appropriate file suffix. e.g. the `deseq2/differential` nf-core module will use the `deseq2_differential.R` + +The template file can be referred to within the module itself using the following notation: + + ```nextflow + script: + template 'deseq2_differential/R' + ``` + +An example can be seen with `deseq2/differential` [here](https://github.com/nf-core/modules/blob/master/modules/nf-core/deseq2/differential/main.nf#L47). + +The resulting structure would look like this. + + ```tree + deseq2 + └── differential + ├── environment.yml + ├── main.nf + ├── meta.yml + ├── templates + │   └── deqseq2_differential.R + └── tests + ├── main.nf.test + ├── main.nf.test.snap + └── tags.yml + ``` + +#### Template contents + +:::warning +Be aware that in any script template that Nextflow needs to be escaped in the same way you would in a standard bash `script:` block! +::: + +The script template file MUST generate a `versions.yml` file in the language-appropriate way that contains versions of the base language and all relevant libraries and packages. + +The generated `versions.yml` MUST have the same structure as a standard nf-core module `versions.yml`. + +For example, for R you can refer to the `deseq2/differential` module [here](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/templates/deseq_de.R#L509-L534). + +#### Script template stub + +A script template module MUST also include a stub block following a standard nf-core module structure (i.e., using e.g. `touch` to generate empty files, and versions. You can see the `deseq2/differential` module example [here](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L34-L49). + +An inline command to call the version for libraries for the `versions.yml` MAY be used in this case. For an R example see [deseq2/differential](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L47). -- Inline process code: code can still be included in the script block using a shebang like `#!/usr/bin/env python`, and code used inline as with regular shell content. This can, however, be obstructive to understanding of module code when content gets extensive. -- [Module templates](https://www.nextflow.io/docs/latest/module.html#module-templates): templates work exactly the same as inline code (so beware of the need to escape dollar signs etc), however can be stored external to the process definition in the module, in a `templates` subfolder. This is done for the [DESeq2 module](https://github.com/nf-core/modules/tree/master/modules/nf-core/deseq2/differential), for example. -- Externalise the scripts entirely, and access them via a Conda definition. This is probably the cleanest solution, but requires a little more effort and management. ### Stubs From 7ca3c2602caa00e61baaf46b0c572a752190b134 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Mon, 9 Dec 2024 10:42:24 +0000 Subject: [PATCH 04/15] Apply suggestions from code review Co-authored-by: James A. Fellows Yates --- .../content/docs/guidelines/components/modules.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index 7254d65225..8b25b9c685 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -327,14 +327,14 @@ Alternate suggestions include using `grep -c` to search for a valid string match ### Script inclusion :::note -You SHOULD NOT use [Nextflow module binaries](https://www.nextflow.io/docs/latest/module.html#module-binaries) where possible, as these are not fully portable across all execution contexts. +You SHOULD NOT use [Nextflow module binaries](https://www.nextflow.io/docs/latest/module.html#module-binaries), as these are not fully portable across all execution contexts. ::: If a module's `script:` block contains not command invocations, but script content, whatever the language (e.g. Bash, R, Python), and if that content is non-trivial in length (>> 10 lines), that MUST be supplied via a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). This makes it clearer when changes have been made to the scientific logic in the script, or to the workflow-relevant logic in the module. -#### Template location +#### Module template location - The template MUST go in a directory called `templates/` that sits alongside the `main.nf` of the module. +The template MUST go in a directory called `templates/` that sits alongside the `main.nf` of the module. The template file MUST be named after module itself with a language appropriate file suffix. e.g. the `deseq2/differential` nf-core module will use the `deseq2_differential.R` @@ -363,7 +363,7 @@ The resulting structure would look like this. └── tags.yml ``` -#### Template contents +#### Template or inline script-code contents :::warning Be aware that in any script template that Nextflow needs to be escaped in the same way you would in a standard bash `script:` block! @@ -375,9 +375,9 @@ The generated `versions.yml` MUST have the same structure as a standard nf-core For example, for R you can refer to the `deseq2/differential` module [here](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/templates/deseq_de.R#L509-L534). -#### Script template stub +#### Stubs in templated modules -A script template module MUST also include a stub block following a standard nf-core module structure (i.e., using e.g. `touch` to generate empty files, and versions. You can see the `deseq2/differential` module example [here](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L34-L49). +A templated module MUST have a stub block in the same way as any other module (i.e., using e.g. `touch` to generate empty files, and versions). You can see the `deseq2/differential` module example [here](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L34-L49). An inline command to call the version for libraries for the `versions.yml` MAY be used in this case. For an R example see [deseq2/differential](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L47). From 22ecd5644f30bf9916e43e004aae02beeba31656 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Mon, 9 Dec 2024 10:42:37 +0000 Subject: [PATCH 05/15] Update sites/docs/src/content/docs/guidelines/components/modules.md Co-authored-by: James A. Fellows Yates --- sites/docs/src/content/docs/guidelines/components/modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index 8b25b9c685..f1a6539f43 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -369,7 +369,7 @@ The resulting structure would look like this. Be aware that in any script template that Nextflow needs to be escaped in the same way you would in a standard bash `script:` block! ::: -The script template file MUST generate a `versions.yml` file in the language-appropriate way that contains versions of the base language and all relevant libraries and packages. +The script template file or inline script code (<20 lines) MUST generate a `versions.yml` file in the language-appropriate way that contains versions of the base language and all relevant libraries and packages. The generated `versions.yml` MUST have the same structure as a standard nf-core module `versions.yml`. From 650cf001979bb28574142918b44db8f3b0f8b13a Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Mon, 9 Dec 2024 10:43:55 +0000 Subject: [PATCH 06/15] Update sites/docs/src/content/docs/guidelines/components/modules.md Co-authored-by: James A. Fellows Yates --- .../docs/src/content/docs/guidelines/components/modules.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index f1a6539f43..dce4b0b6f1 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -330,7 +330,12 @@ Alternate suggestions include using `grep -c` to search for a valid string match You SHOULD NOT use [Nextflow module binaries](https://www.nextflow.io/docs/latest/module.html#module-binaries), as these are not fully portable across all execution contexts. ::: -If a module's `script:` block contains not command invocations, but script content, whatever the language (e.g. Bash, R, Python), and if that content is non-trivial in length (>> 10 lines), that MUST be supplied via a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). This makes it clearer when changes have been made to the scientific logic in the script, or to the workflow-relevant logic in the module. +If a module's `script:` block contains not command invocations, but script content, whatever the language (e.g. Bash, R, Python), and if that content is non-trivial in length (> 20 lines), that MUST be supplied via a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). +This makes it clearer when changes have been made to the scientific logic in the script, or to the workflow-relevant logic in the module. + +#### Inline script code + +If the script content is less than 20 lines, the code MAY be embedded directly in the module without a dedicated template file, however should still follow the guidance content as with a template. #### Module template location From 08ac34aa2e0a36c966e997dfca0739025f556ff9 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Mon, 9 Dec 2024 10:47:22 +0000 Subject: [PATCH 07/15] Update sites/docs/src/content/docs/guidelines/components/modules.md --- sites/docs/src/content/docs/guidelines/components/modules.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index dce4b0b6f1..f9800320a0 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -330,6 +330,10 @@ Alternate suggestions include using `grep -c` to search for a valid string match You SHOULD NOT use [Nextflow module binaries](https://www.nextflow.io/docs/latest/module.html#module-binaries), as these are not fully portable across all execution contexts. ::: +:::note +Where script content in a module becomes particularly extensive, you should consider hosting the code externally and provisioning via Conda/ Docker. +::: + If a module's `script:` block contains not command invocations, but script content, whatever the language (e.g. Bash, R, Python), and if that content is non-trivial in length (> 20 lines), that MUST be supplied via a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). This makes it clearer when changes have been made to the scientific logic in the script, or to the workflow-relevant logic in the module. From 99aafc3b8dafccdd69371c24e6692163adb84092 Mon Sep 17 00:00:00 2001 From: nf-core-bot Date: Mon, 9 Dec 2024 10:52:27 +0000 Subject: [PATCH 08/15] [automated] Fix code linting --- .../src/content/docs/guidelines/components/modules.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index f9800320a0..5e46b4a192 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -343,7 +343,7 @@ If the script content is less than 20 lines, the code MAY be embedded directly i #### Module template location -The template MUST go in a directory called `templates/` that sits alongside the `main.nf` of the module. +The template MUST go in a directory called `templates/` that sits alongside the `main.nf` of the module. The template file MUST be named after module itself with a language appropriate file suffix. e.g. the `deseq2/differential` nf-core module will use the `deseq2_differential.R` @@ -378,19 +378,18 @@ The resulting structure would look like this. Be aware that in any script template that Nextflow needs to be escaped in the same way you would in a standard bash `script:` block! ::: -The script template file or inline script code (<20 lines) MUST generate a `versions.yml` file in the language-appropriate way that contains versions of the base language and all relevant libraries and packages. +The script template file or inline script code (<20 lines) MUST generate a `versions.yml` file in the language-appropriate way that contains versions of the base language and all relevant libraries and packages. -The generated `versions.yml` MUST have the same structure as a standard nf-core module `versions.yml`. +The generated `versions.yml` MUST have the same structure as a standard nf-core module `versions.yml`. For example, for R you can refer to the `deseq2/differential` module [here](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/templates/deseq_de.R#L509-L534). #### Stubs in templated modules -A templated module MUST have a stub block in the same way as any other module (i.e., using e.g. `touch` to generate empty files, and versions). You can see the `deseq2/differential` module example [here](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L34-L49). +A templated module MUST have a stub block in the same way as any other module (i.e., using e.g. `touch` to generate empty files, and versions). You can see the `deseq2/differential` module example [here](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L34-L49). An inline command to call the version for libraries for the `versions.yml` MAY be used in this case. For an R example see [deseq2/differential](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L47). - ### Stubs #### Stub block must exist From 0107df0b6e75a27a61a0a39489ee44185bfafb00 Mon Sep 17 00:00:00 2001 From: Jonathan Manning Date: Mon, 9 Dec 2024 10:54:52 +0000 Subject: [PATCH 09/15] Apply suggestions from code review Co-authored-by: James A. Fellows Yates --- .../docs/src/content/docs/guidelines/components/modules.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index 5e46b4a192..6b6770faa3 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -326,12 +326,12 @@ Alternate suggestions include using `grep -c` to search for a valid string match ### Script inclusion -:::note +:::warning You SHOULD NOT use [Nextflow module binaries](https://www.nextflow.io/docs/latest/module.html#module-binaries), as these are not fully portable across all execution contexts. ::: :::note -Where script content in a module becomes particularly extensive, you should consider hosting the code externally and provisioning via Conda/ Docker. +Where script content in a module becomes particularly extensive, we strongly encourage packaging and hosting the code externally and provisioning via Conda/ Docker as a standalone tool(kit). ::: If a module's `script:` block contains not command invocations, but script content, whatever the language (e.g. Bash, R, Python), and if that content is non-trivial in length (> 20 lines), that MUST be supplied via a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). @@ -388,7 +388,8 @@ For example, for R you can refer to the `deseq2/differential` module [here](http A templated module MUST have a stub block in the same way as any other module (i.e., using e.g. `touch` to generate empty files, and versions). You can see the `deseq2/differential` module example [here](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L34-L49). -An inline command to call the version for libraries for the `versions.yml` MAY be used in this case. For an R example see [deseq2/differential](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L47). +An inline command to call the version for libraries for the `versions.yml` MAY be used in this case. +For an R example see [deseq2/differential](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L47). ### Stubs From 7c0cf585443f9a688d32ffd7a8062502b4a6fc6d Mon Sep 17 00:00:00 2001 From: "James A. Fellows Yates" Date: Tue, 10 Dec 2024 10:57:59 +0100 Subject: [PATCH 10/15] Apply suggestions from code review Co-authored-by: Christopher Hakkaart --- .../docs/guidelines/components/modules.md | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index 6b6770faa3..9a2861f7f7 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -331,30 +331,30 @@ You SHOULD NOT use [Nextflow module binaries](https://www.nextflow.io/docs/lates ::: :::note -Where script content in a module becomes particularly extensive, we strongly encourage packaging and hosting the code externally and provisioning via Conda/ Docker as a standalone tool(kit). +Where script content in a module becomes particularly extensive, we strongly encourage packaging and hosting the code externally and provisioning via Conda/Docker as a standalone tool(kit). ::: -If a module's `script:` block contains not command invocations, but script content, whatever the language (e.g. Bash, R, Python), and if that content is non-trivial in length (> 20 lines), that MUST be supplied via a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). -This makes it clearer when changes have been made to the scientific logic in the script, or to the workflow-relevant logic in the module. +If a module's `script:` block contains a script rather than command invocations, regardless of the language (e.g., Bash, R, Python), and the content is more than 20 lines, it MUST be provided through a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). +Using module templates helps distinguish between changes made to the scientific logic within the script and those affecting the workflow-specific logic in the module. This separation improves the code's clarity and maintainability. #### Inline script code -If the script content is less than 20 lines, the code MAY be embedded directly in the module without a dedicated template file, however should still follow the guidance content as with a template. +If the script content is less than 20 lines, the code MAY be embedded directly in the module without a dedicated template file. However, they should still follow the guidance content as with a template. #### Module template location -The template MUST go in a directory called `templates/` that sits alongside the `main.nf` of the module. +The template MUST go into a directory called `templates/` in the same directory as the module `main.nf`. -The template file MUST be named after module itself with a language appropriate file suffix. e.g. the `deseq2/differential` nf-core module will use the `deseq2_differential.R` +The template file MUST be named after the module itself with a language-appropriate file suffix. For example, the `deseq2/differential` nf-core module will use the `deseq2_differential.R`. -The template file can be referred to within the module itself using the following notation: +The template file can then be referred to within the module using the template function: ```nextflow script: - template 'deseq2_differential/R' + template 'deseq2_differential.R' ``` -An example can be seen with `deseq2/differential` [here](https://github.com/nf-core/modules/blob/master/modules/nf-core/deseq2/differential/main.nf#L47). +See [`deseq2/differential`](https://github.com/nf-core/modules/blob/master/modules/nf-core/deseq2/differential/main.nf#L47) for an example of a template in an nf-core pipeline. The resulting structure would look like this. @@ -382,11 +382,11 @@ The script template file or inline script code (<20 lines) MUST generate a `vers The generated `versions.yml` MUST have the same structure as a standard nf-core module `versions.yml`. -For example, for R you can refer to the `deseq2/differential` module [here](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/templates/deseq_de.R#L509-L534). +See the [`deseq2/differential` module](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/templates/deseq_de.R#L509-L534) for an example using R. #### Stubs in templated modules -A templated module MUST have a stub block in the same way as any other module (i.e., using e.g. `touch` to generate empty files, and versions). You can see the `deseq2/differential` module example [here](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L34-L49). +A templated module MUST have a stub block in the same way as any other module. For example, using `touch` to generate empty files and versions. See [`deseq2/differential` module](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L34-L49) for an example in an nf-core module. An inline command to call the version for libraries for the `versions.yml` MAY be used in this case. For an R example see [deseq2/differential](https://github.com/nf-core/modules/blob/4c2d06a5e79abf08ba7f04c58e39c7dad75f094d/modules/nf-core/deseq2/differential/main.nf#L47). From 9233dd5da55332a7f26631fa8875142bbcab6b74 Mon Sep 17 00:00:00 2001 From: "James A. Fellows Yates" Date: Tue, 10 Dec 2024 11:02:16 +0100 Subject: [PATCH 11/15] Apply suggestions from code review --- sites/docs/src/content/docs/guidelines/components/modules.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index 9a2861f7f7..53181dc722 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -339,7 +339,7 @@ Using module templates helps distinguish between changes made to the scientific #### Inline script code -If the script content is less than 20 lines, the code MAY be embedded directly in the module without a dedicated template file. However, they should still follow the guidance content as with a template. +If the script content is less than approximately 20 lines, the code MAY be embedded directly in the module without a dedicated template file. However, they should still follow the guidance content as with a template. #### Module template location @@ -378,7 +378,7 @@ The resulting structure would look like this. Be aware that in any script template that Nextflow needs to be escaped in the same way you would in a standard bash `script:` block! ::: -The script template file or inline script code (<20 lines) MUST generate a `versions.yml` file in the language-appropriate way that contains versions of the base language and all relevant libraries and packages. +The script template file or inline script code (less than approximately 20 lines) MUST generate a `versions.yml` file in the language-appropriate way that contains versions of the base language and all relevant libraries and packages. The generated `versions.yml` MUST have the same structure as a standard nf-core module `versions.yml`. From dd0ba6b04385ab22bd1f3fddaf17cef6bdbc833a Mon Sep 17 00:00:00 2001 From: Christopher Hakkaart Date: Tue, 10 Dec 2024 11:04:14 +0100 Subject: [PATCH 12/15] Update sites/docs/src/content/docs/guidelines/components/modules.md Co-authored-by: James A. Fellows Yates --- .../src/content/docs/guidelines/components/modules.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index 53181dc722..46e8c9ba7c 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -326,17 +326,17 @@ Alternate suggestions include using `grep -c` to search for a valid string match ### Script inclusion -:::warning -You SHOULD NOT use [Nextflow module binaries](https://www.nextflow.io/docs/latest/module.html#module-binaries), as these are not fully portable across all execution contexts. +If a module's `script:` block contains a script rather than command invocations, regardless of the language (e.g., Bash, R, Python), and the content is more than approximately 20 lines, it MUST be provided through a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). +Using module templates helps distinguish between changes made to the scientific logic within the script and those affecting the workflow-specific logic in the module. This separation improves the code's clarity and maintainability. + +:::note +We recommend use of Nextflow templates as they are the most portable method of the separate custom script execution across all execution contexts ::: :::note Where script content in a module becomes particularly extensive, we strongly encourage packaging and hosting the code externally and provisioning via Conda/Docker as a standalone tool(kit). ::: -If a module's `script:` block contains a script rather than command invocations, regardless of the language (e.g., Bash, R, Python), and the content is more than 20 lines, it MUST be provided through a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). -Using module templates helps distinguish between changes made to the scientific logic within the script and those affecting the workflow-specific logic in the module. This separation improves the code's clarity and maintainability. - #### Inline script code If the script content is less than approximately 20 lines, the code MAY be embedded directly in the module without a dedicated template file. However, they should still follow the guidance content as with a template. From 6490837c44b21ae0bd756503cf900bb2ef6e76b4 Mon Sep 17 00:00:00 2001 From: "James A. Fellows Yates" Date: Tue, 10 Dec 2024 12:31:37 +0100 Subject: [PATCH 13/15] Update sites/docs/src/content/docs/guidelines/components/modules.md Co-authored-by: Christopher Hakkaart --- sites/docs/src/content/docs/guidelines/components/modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index 46e8c9ba7c..fbf2c84141 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -326,8 +326,8 @@ Alternate suggestions include using `grep -c` to search for a valid string match ### Script inclusion -If a module's `script:` block contains a script rather than command invocations, regardless of the language (e.g., Bash, R, Python), and the content is more than approximately 20 lines, it MUST be provided through a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). Using module templates helps distinguish between changes made to the scientific logic within the script and those affecting the workflow-specific logic in the module. This separation improves the code's clarity and maintainability. +If a module's `script:` block contains a script rather than command invocations, regardless of the language (e.g., Bash, R, Python), and the content is more than approximately 20 lines, it MUST be provided through a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). :::note We recommend use of Nextflow templates as they are the most portable method of the separate custom script execution across all execution contexts From e742ed72485b74107525eb5ea5960669d82ed2f1 Mon Sep 17 00:00:00 2001 From: "James A. Fellows Yates" Date: Tue, 10 Dec 2024 12:33:42 +0100 Subject: [PATCH 14/15] Apply suggestions from code review --- .../docs/src/content/docs/guidelines/components/modules.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index fbf2c84141..39c02b524d 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -327,7 +327,7 @@ Alternate suggestions include using `grep -c` to search for a valid string match ### Script inclusion Using module templates helps distinguish between changes made to the scientific logic within the script and those affecting the workflow-specific logic in the module. This separation improves the code's clarity and maintainability. -If a module's `script:` block contains a script rather than command invocations, regardless of the language (e.g., Bash, R, Python), and the content is more than approximately 20 lines, it MUST be provided through a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). +If a module's `script:` block contains a script rather than command invocations, regardless of the language (e.g., Bash, R, Python), and the content is more than a readable length (as a rule of thumb, approximately 20 lines), it MUST be provided through a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). :::note We recommend use of Nextflow templates as they are the most portable method of the separate custom script execution across all execution contexts @@ -339,7 +339,7 @@ Where script content in a module becomes particularly extensive, we strongly enc #### Inline script code -If the script content is less than approximately 20 lines, the code MAY be embedded directly in the module without a dedicated template file. However, they should still follow the guidance content as with a template. +If the script content remains at a readable length, the code MAY be embedded directly in the module without a dedicated template file. However, they should still follow the guidance content as with a template. #### Module template location @@ -378,7 +378,7 @@ The resulting structure would look like this. Be aware that in any script template that Nextflow needs to be escaped in the same way you would in a standard bash `script:` block! ::: -The script template file or inline script code (less than approximately 20 lines) MUST generate a `versions.yml` file in the language-appropriate way that contains versions of the base language and all relevant libraries and packages. +The script template file or inline script code (used when at a readable length) MUST generate a `versions.yml` file in the language-appropriate way that contains versions of the base language and all relevant libraries and packages. The generated `versions.yml` MUST have the same structure as a standard nf-core module `versions.yml`. From c4381152c5443bf16cf1a8c90d06818a56eb6311 Mon Sep 17 00:00:00 2001 From: Chris Hakkaart Date: Wed, 11 Dec 2024 19:35:56 +0100 Subject: [PATCH 15/15] Update sites/docs/src/content/docs/guidelines/components/modules.md nice Co-authored-by: Jonathan Manning --- sites/docs/src/content/docs/guidelines/components/modules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sites/docs/src/content/docs/guidelines/components/modules.md b/sites/docs/src/content/docs/guidelines/components/modules.md index 39c02b524d..ace7910ab8 100644 --- a/sites/docs/src/content/docs/guidelines/components/modules.md +++ b/sites/docs/src/content/docs/guidelines/components/modules.md @@ -330,7 +330,7 @@ Using module templates helps distinguish between changes made to the scientific If a module's `script:` block contains a script rather than command invocations, regardless of the language (e.g., Bash, R, Python), and the content is more than a readable length (as a rule of thumb, approximately 20 lines), it MUST be provided through a [Nextflow module template](https://www.nextflow.io/docs/latest/module.html#module-templates). :::note -We recommend use of Nextflow templates as they are the most portable method of the separate custom script execution across all execution contexts +We recommend use of Nextflow templates as they are the most portable method of separating custom script content and execution across all execution contexts ::: :::note