Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Activate multiqc ai summaries #1495

Closed
wants to merge 17 commits into from
Closed

Activate multiqc ai summaries #1495

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
efd25a2
feat: add MultiQC AI summaries support
pinin4fjords Jan 24, 2025
41b2187
feat: Add MultiQC AI summary options with type selection
pinin4fjords Jan 24, 2025
fade208
feat: Add Seqera access token configuration for MultiQC AI
pinin4fjords Jan 24, 2025
d2903c9
Merge branch 'dev' into feat-multiqc-ai-summaries
pinin4fjords Jan 24, 2025
b66571f
bump multiqc
pinin4fjords Jan 27, 2025
68ee86c
fix schema
pinin4fjords Jan 27, 2025
2b054b9
Simplify env var porting
pinin4fjords Jan 27, 2025
17ccfa4
update changelog
pinin4fjords Jan 27, 2025
1747b1b
Account for other providers, move keys to secrets
pinin4fjords Jan 27, 2025
67cb1dc
Use env after all
pinin4fjords Jan 27, 2025
8de6177
Update docs
pinin4fjords Jan 27, 2025
f000479
Add new params to changelog
pinin4fjords Jan 27, 2025
9260b70
Fix schema
pinin4fjords Jan 27, 2025
81f6bb4
what happens if I disable the tower var passthrough?
pinin4fjords Jan 27, 2025
d945758
Revert "what happens if I disable the tower var passthrough?"
pinin4fjords Jan 27, 2025
42756fb
Don't env empty things
pinin4fjords Jan 27, 2025
ae15852
update snaps
pinin4fjords Jan 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- [PR #1482](https://github.com/nf-core/rnaseq/pull/1482) - Update trimgalore module for save_unpaired fix
- [PR #1486](https://github.com/nf-core/rnaseq/pull/1486) - Bump STAR build for multiprocessing fix
- [PR #1490](https://github.com/nf-core/rnaseq/pull/1490) - Make genomic FASTA input optional
- [PR #1495](https://github.com/nf-core/rnaseq/pull/1495) - Activate multiqc AI summaries

## Parameters

| Old parameter | New parameter |
| ------------- | ----------------------- |
| | `--enable_multiqc_ai` |
| | `--multiqc_ai_type` |
| | `--multiqc_ai_provider` |

# 3.18.0 - 2024-12-19

Expand Down
14 changes: 14 additions & 0 deletions docs/usage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,20 @@ It is important to note that the accuracy of Kraken2 is [highly dependent on the

While Kraken2 is capable of detecting low-abundance contaminants in a sample, false positives can occur. Therefore, if only a very small number of reads from a contaminating species are detected, these results should be interpreted with caution.

## AI summaries with MultiQC

As of version 1.27, MultiQC has [AI-driven summarisation functionality](https://docs.seqera.io/multiqc/ai). You can activate this functionality in this workflow by passing `--enable_multiqc_ai`.

You use any of Seqera AI, OpenAI, or Anthropic as your provider of AI services, setting `--multiqc_ai_provider` correctly and defining the appropriate environment variable to provide the API key:

| `--multiqc_ai_provider` | Environment variable |
| ----------------------- | -------------------- |
| seqera | SEQERA_ACCESS_TOKEN |
| openai | OPENAI_API_KEY |
| anthropic | ANTHROPIC_API_KEY |

Default behaviour is a concise summary, you can opt for a fuller summary by setting `--multiqc_ai_type` to 'full'.

## Running the pipeline

The typical command for running the pipeline is as follows:
Expand Down
2 changes: 1 addition & 1 deletion modules.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@
},
"multiqc": {
"branch": "master",
"git_sha": "cf17ca47590cc578dfb47db1c2a44ef86f89976d",
"git_sha": "f0719ae309075ae4a291533883847c3f7c441dad",
"installed_by": ["modules"]
},
"picard/markduplicates": {
Expand Down
2 changes: 1 addition & 1 deletion modules/nf-core/multiqc/environment.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions modules/nf-core/multiqc/main.nf
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion modules/nf-core/multiqc/nextflow.config
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions modules/nf-core/multiqc/tests/main.nf.test
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 12 additions & 12 deletions modules/nf-core/multiqc/tests/main.nf.test.snap
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 15 additions & 7 deletions nextflow.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,9 @@ params {
multiqc_logo = null
max_multiqc_email_size = '25.MB'
multiqc_methods_description = null
enable_multiqc_ai = false
multiqc_ai_type = 'concise'
multiqc_ai_provider = 'seqera'

// Boilerplate options
outdir = null
Expand Down Expand Up @@ -285,15 +288,20 @@ charliecloud.registry = 'quay.io'
// Load igenomes.config if required
includeConfig !params.igenomes_ignore ? 'conf/igenomes.config' : 'conf/igenomes_ignored.config'

// Export these variables to prevent local Python/R libraries from conflicting with those in the container
// The JULIA depot path has been adjusted to a fixed path `/usr/local/share/julia` that needs to be used for packages in the container.
// See https://apeltzer.github.io/post/03-julia-lang-nextflow/ for details on that. Once we have a common agreement on where to keep Julia packages, this is adjustable.

env {
// Export these variables to prevent local Python/R libraries from conflicting with those in the container
// The JULIA depot path has been adjusted to a fixed path `/usr/local/share/julia` that needs to be used for packages in the container.
// See https://apeltzer.github.io/post/03-julia-lang-nextflow/ for details on that. Once we have a common agreement on where to keep Julia packages, this is adjustable.
PYTHONNOUSERSITE = 1
R_PROFILE_USER = "/.Rprofile"
R_ENVIRON_USER = "/.Renviron"
JULIA_DEPOT_PATH = "/usr/local/share/julia"
R_PROFILE_USER = "/.Rprofile"
R_ENVIRON_USER = "/.Renviron"
JULIA_DEPOT_PATH = "/usr/local/share/julia"

// Propagate API keys to MultiQC
System.getenv('SEQERA_ACCESS_TOKEN')?.with { SEQERA_ACCESS_TOKEN = it }
System.getenv('TOWER_ACCESS_TOKEN')?.with { TOWER_ACCESS_TOKEN = it }
System.getenv('OPENAI_API_KEY')?.with { OPENAI_API_KEY = it }
System.getenv('ANTHROPIC_API_KEY')?.with { ANTHROPIC_API_KEY = it }
Comment on lines +300 to +304
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know nf-core wants to be 'batteries included', but this does feel like a lot of stuff when it's user configurable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I quite like the approach, but I'm not sure that it'll be allowed with the new config parser :/

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I don't want to be the user that configures it ;-)

}

// Set bash options
Expand Down
20 changes: 20 additions & 0 deletions nextflow_schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,26 @@
"type": "string",
"description": "MultiQC report title. Printed as page header, used for filename if not otherwise specified.",
"fa_icon": "fas fa-file-signature"
},
"enable_multiqc_ai": {
"type": "boolean",
"description": "Enable AI-powered summaries in MultiQC report",
"fa_icon": "fas fa-robot",
"help_text": "When enabled, MultiQC will use AI to generate summaries of the quality control results."
},
"multiqc_ai_type": {
"type": "string",
"default": "concise",
"description": "MultiQC AI summary type.",
"fa_icon": "fas fa-font",
"enum": ["concise", "full"]
},
"multiqc_ai_provider": {
"type": "string",
"default": "seqera",
"description": "MultiQC AI summary provider.",
"fa_icon": "fas fa-font",
"enum": ["seqera", "openai", "anthropic"]
}
}
},
Expand Down
28 changes: 14 additions & 14 deletions tests/default.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,10 +98,10 @@
]
],
"meta": {
"nf-test": "0.9.0",
"nextflow": "24.10.2"
"nf-test": "0.9.2",
"nextflow": "24.10.4"
},
"timestamp": "2024-12-03T12:26:32.825018567"
"timestamp": "2025-01-28T13:26:07.890871152"
},
"Params: default": {
"content": [
Expand Down Expand Up @@ -367,10 +367,10 @@
"multiqc/star_salmon/multiqc_report_data/multiqc_software_versions.txt",
"multiqc/star_salmon/multiqc_report_data/multiqc_sources.txt",
"multiqc/star_salmon/multiqc_report_data/multiqc_star.txt",
"multiqc/star_salmon/multiqc_report_data/picard_MarkIlluminaAdapters_histogram.txt",
"multiqc/star_salmon/multiqc_report_data/picard_MeanQualityByCycle_histogram.txt",
"multiqc/star_salmon/multiqc_report_data/picard_QualityScoreDistribution_histogram.txt",
"multiqc/star_salmon/multiqc_report_data/picard_deduplication.txt",
"multiqc/star_salmon/multiqc_report_data/picard_histogram.txt",
"multiqc/star_salmon/multiqc_report_data/picard_histogram_1.txt",
"multiqc/star_salmon/multiqc_report_data/picard_histogram_2.txt",
"multiqc/star_salmon/multiqc_report_data/qualimap_gene_coverage_profile_Counts.txt",
"multiqc/star_salmon/multiqc_report_data/qualimap_gene_coverage_profile_Normalised.txt",
"multiqc/star_salmon/multiqc_report_data/qualimap_genomic_origin.txt",
Expand Down Expand Up @@ -1346,11 +1346,11 @@
"multiqc_cutadapt.txt:md5,583b7b9ba76b26162bb9610ed746454b",
"multiqc_fastqc_fastqc_raw.txt:md5,81c3c1a2575a1891a7f2a9637a0f2cc0",
"multiqc_fastqc_fastqc_trimmed.txt:md5,54743154d0e8858980acffeb5b6f6a97",
"multiqc_featurecounts_biotype_plot.txt:md5,5d83930a8c0aebf5fff0a5d12857b42d",
"multiqc_featurecounts_biotype_plot.txt:md5,fb9db2fca737fbdcd1a921e683805d77",
"multiqc_samtools_idxstats.txt:md5,8b8f6cb092004918c18319a8cd64eb4c",
"picard_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_histogram_1.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_histogram_2.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_MarkIlluminaAdapters_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_MeanQualityByCycle_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_QualityScoreDistribution_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"qualimap_gene_coverage_profile_Counts.txt:md5,d86b5743ba4486f5796327cbd2854882",
"qualimap_gene_coverage_profile_Normalised.txt:md5,05618239e3183e8f312552676c4256b5",
"qualimap_rnaseq_cov_hist.txt:md5,ea4b7cf07e5c7590ded5af29161db405",
Expand Down Expand Up @@ -1490,9 +1490,9 @@
]
],
"meta": {
"nf-test": "0.9.0",
"nextflow": "24.10.2"
"nf-test": "0.9.2",
"nextflow": "24.10.4"
},
"timestamp": "2024-12-03T12:39:04.406597796"
"timestamp": "2025-01-28T13:24:46.099038947"
}
}
}
26 changes: 13 additions & 13 deletions tests/featurecounts_group_type.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,10 +98,10 @@
]
],
"meta": {
"nf-test": "0.9.0",
"nextflow": "24.10.2"
"nf-test": "0.9.2",
"nextflow": "24.10.4"
},
"timestamp": "2024-12-03T17:26:42.742119846"
"timestamp": "2025-01-28T13:38:47.824968389"
},
"Params: --featurecounts_group_type false": {
"content": [
Expand Down Expand Up @@ -360,10 +360,10 @@
"multiqc/star_salmon/multiqc_report_data/multiqc_software_versions.txt",
"multiqc/star_salmon/multiqc_report_data/multiqc_sources.txt",
"multiqc/star_salmon/multiqc_report_data/multiqc_star.txt",
"multiqc/star_salmon/multiqc_report_data/picard_MarkIlluminaAdapters_histogram.txt",
"multiqc/star_salmon/multiqc_report_data/picard_MeanQualityByCycle_histogram.txt",
"multiqc/star_salmon/multiqc_report_data/picard_QualityScoreDistribution_histogram.txt",
"multiqc/star_salmon/multiqc_report_data/picard_deduplication.txt",
"multiqc/star_salmon/multiqc_report_data/picard_histogram.txt",
"multiqc/star_salmon/multiqc_report_data/picard_histogram_1.txt",
"multiqc/star_salmon/multiqc_report_data/picard_histogram_2.txt",
"multiqc/star_salmon/multiqc_report_data/qualimap_gene_coverage_profile_Counts.txt",
"multiqc/star_salmon/multiqc_report_data/qualimap_gene_coverage_profile_Normalised.txt",
"multiqc/star_salmon/multiqc_report_data/qualimap_genomic_origin.txt",
Expand Down Expand Up @@ -1313,9 +1313,9 @@
"multiqc_fastqc_fastqc_raw.txt:md5,81c3c1a2575a1891a7f2a9637a0f2cc0",
"multiqc_fastqc_fastqc_trimmed.txt:md5,54743154d0e8858980acffeb5b6f6a97",
"multiqc_samtools_idxstats.txt:md5,8b8f6cb092004918c18319a8cd64eb4c",
"picard_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_histogram_1.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_histogram_2.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_MarkIlluminaAdapters_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_MeanQualityByCycle_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_QualityScoreDistribution_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"qualimap_gene_coverage_profile_Counts.txt:md5,d86b5743ba4486f5796327cbd2854882",
"qualimap_gene_coverage_profile_Normalised.txt:md5,05618239e3183e8f312552676c4256b5",
"qualimap_rnaseq_cov_hist.txt:md5,ea4b7cf07e5c7590ded5af29161db405",
Expand Down Expand Up @@ -1440,9 +1440,9 @@
]
],
"meta": {
"nf-test": "0.9.0",
"nextflow": "24.10.2"
"nf-test": "0.9.2",
"nextflow": "24.10.4"
},
"timestamp": "2024-12-03T17:25:47.179044158"
"timestamp": "2025-01-28T13:38:00.766111515"
}
}
}
28 changes: 14 additions & 14 deletions tests/hisat2.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,10 +99,10 @@
]
],
"meta": {
"nf-test": "0.9.0",
"nextflow": "24.10.2"
"nf-test": "0.9.2",
"nextflow": "24.10.4"
},
"timestamp": "2024-12-03T17:39:53.585492305"
"timestamp": "2025-01-28T13:46:14.476085587"
},
"Params: --aligner hisat2": {
"content": [
Expand Down Expand Up @@ -834,10 +834,10 @@
"multiqc/hisat2/multiqc_report_data/multiqc_samtools_stats.txt",
"multiqc/hisat2/multiqc_report_data/multiqc_software_versions.txt",
"multiqc/hisat2/multiqc_report_data/multiqc_sources.txt",
"multiqc/hisat2/multiqc_report_data/picard_MarkIlluminaAdapters_histogram.txt",
"multiqc/hisat2/multiqc_report_data/picard_MeanQualityByCycle_histogram.txt",
"multiqc/hisat2/multiqc_report_data/picard_QualityScoreDistribution_histogram.txt",
"multiqc/hisat2/multiqc_report_data/picard_deduplication.txt",
"multiqc/hisat2/multiqc_report_data/picard_histogram.txt",
"multiqc/hisat2/multiqc_report_data/picard_histogram_1.txt",
"multiqc/hisat2/multiqc_report_data/picard_histogram_2.txt",
"multiqc/hisat2/multiqc_report_data/qualimap_gene_coverage_profile_Counts.txt",
"multiqc/hisat2/multiqc_report_data/qualimap_gene_coverage_profile_Normalised.txt",
"multiqc/hisat2/multiqc_report_data/qualimap_genomic_origin.txt",
Expand Down Expand Up @@ -1293,11 +1293,11 @@
"multiqc_cutadapt.txt:md5,583b7b9ba76b26162bb9610ed746454b",
"multiqc_fastqc_fastqc_raw.txt:md5,81c3c1a2575a1891a7f2a9637a0f2cc0",
"multiqc_fastqc_fastqc_trimmed.txt:md5,54743154d0e8858980acffeb5b6f6a97",
"multiqc_featurecounts_biotype_plot.txt:md5,d784fc4b2dce860c8f88cc3e936e589e",
"multiqc_featurecounts_biotype_plot.txt:md5,6d251bec0231afccd0682e454a3b17bc",
"multiqc_samtools_idxstats.txt:md5,66a8e8aecb6233f5a3521151b1ce8d49",
"picard_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_histogram_1.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_histogram_2.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_MarkIlluminaAdapters_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_MeanQualityByCycle_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"picard_QualityScoreDistribution_histogram.txt:md5,d41d8cd98f00b204e9800998ecf8427e",
"qualimap_gene_coverage_profile_Counts.txt:md5,f3b9bb1902af4c7b5e04c0830ded2c02",
"qualimap_gene_coverage_profile_Normalised.txt:md5,ebcedec8e5c959414a4e89ccae3fc07e",
"qualimap_rnaseq_cov_hist.txt:md5,b78ff616f267f9b061b3297b767e88fb",
Expand Down Expand Up @@ -1340,9 +1340,9 @@
]
],
"meta": {
"nf-test": "0.9.0",
"nextflow": "24.10.2"
"nf-test": "0.9.2",
"nextflow": "24.10.4"
},
"timestamp": "2024-12-03T17:38:50.917761241"
"timestamp": "2025-01-28T13:45:28.520702162"
}
}
}
Loading