Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding cellrangermulti subworkflow #276

Merged
merged 160 commits into from
May 22, 2024
Merged
Show file tree
Hide file tree
Changes from 148 commits
Commits
Show all changes
160 commits
Select commit Hold shift + click to select a range
94397e5
Add cellranger multi testing assets
fmalmeida Nov 7, 2023
e6a54bb
allow cellrangermulti option
fmalmeida Nov 7, 2023
ae20b9e
include cellrangermulti testing conf/profile
fmalmeida Nov 7, 2023
5863d2c
allow cellrangermulti option
fmalmeida Nov 7, 2023
631a980
fix example samplesheet
fmalmeida Nov 8, 2023
70fff60
fixed samplesheet for cellranger multi
fmalmeida Nov 8, 2023
f97e430
don't get cellrangemulti metadata if not needed
fmalmeida Nov 8, 2023
ff66e97
fix check_samplesheet script to be more generic
fmalmeida Nov 9, 2023
0af2761
update input_check for cellranger multi
fmalmeida Nov 9, 2023
8e7d436
avoid renaming sample ids in input check
fmalmeida Nov 9, 2023
97281aa
generate a parsed input channel for cellrangermulti sub-workflow
fmalmeida Nov 9, 2023
8b19f48
defined cellrangemulti sub-workflow and parsed input channel for exec…
fmalmeida Nov 9, 2023
6f494fe
included gex (normal) reference building and updated cellranger modules
fmalmeida Nov 10, 2023
215226b
include mkvdjref
fmalmeida Nov 13, 2023
7b86f80
refactored sample mapping
fmalmeida Nov 14, 2023
e6dc3b2
finally cellranger multi running, with errors, but now can be debugged
fmalmeida Nov 14, 2023
7c66115
not finding samples in data directory
fmalmeida Nov 14, 2023
abb0e3c
saving quick changes for shifting development workspace
fmalmeida Nov 15, 2023
5e27a4c
include option for unzipping reference files
Nov 16, 2023
e00a78d
First successfull run of cellranger multi with renaming module
fmalmeida Nov 17, 2023
ead6462
add whiteline
fmalmeida Nov 20, 2023
ab4425e
Testing github traffic
Nov 21, 2023
2cfb148
Remove file used for testing
Nov 21, 2023
4c275b4
input dataset parsing refactored and fixed
Nov 22, 2023
a8d2702
include cellrangermulti outputs in mqc channel
Nov 22, 2023
0d8be69
include option for cellrangermulti in mtx conversion modules
Nov 23, 2023
4d75c83
add files filter for cellranger multi outputs
Nov 23, 2023
e4e37b5
include cellranger multi outputs to mtx conversion subworkflow
Nov 23, 2023
132e247
update changelog
Nov 23, 2023
0ec7019
remove unused file
Nov 23, 2023
6c7e550
update comment
Nov 23, 2023
a0d666b
remove unused params
fmalmeida Nov 23, 2023
d5c24c3
update nextflow schema
fmalmeida Nov 23, 2023
9206c59
update version of cellranger multi module
Feb 23, 2024
9764f3a
delete unnecessary module
Feb 23, 2024
df89a9b
update modules.config
Feb 23, 2024
85136a2
include newly required parameters
Feb 23, 2024
2f77b4d
remove unnecessary module
Feb 23, 2024
32e480b
remove unwanted args
Feb 23, 2024
3aeaeb8
Merge branch 'dev' of https://github.com/nf-core/scrnaseq into 247-su…
Feb 26, 2024
c85c6dd
add new params
Feb 26, 2024
b38ef0f
update modules due linting
Feb 26, 2024
38060f0
include new columns in samplesheet checker
Feb 26, 2024
aa5e77a
add docker image workaround
Feb 26, 2024
c8b86e4
fix linting
Feb 26, 2024
cc4f0c0
update nextflow schema
fmalmeida Feb 27, 2024
56123e2
applied prettier changes
fmalmeida Feb 27, 2024
e98f444
make param.fasta and params.gtf optional again
fmalmeida Feb 27, 2024
98e92bb
Merge remote-tracking branch 'origin/dev' into 247-support-for-10x-ff…
grst Mar 7, 2024
0e7efe2
update publishDir path
Mar 11, 2024
d432d42
Merge branch '247-support-for-10x-ffpe-scrna' of https://github.com/n…
Mar 11, 2024
b41d89b
Merge branch 'dev' of https://github.com/nf-core/scrnaseq into 247-su…
fmalmeida Mar 19, 2024
912fc73
add sample headers to schema
fmalmeida Mar 19, 2024
f8c65ba
add missing modules
fmalmeida Mar 19, 2024
a5009f8
remove debugging .view()
fmalmeida Mar 19, 2024
53bd304
update cellrager/count
fmalmeida Mar 19, 2024
176060b
small dataset cannot run emptydrops
Mar 19, 2024
04009ff
also run for cellranger/multi
Mar 19, 2024
43f1374
fix white space
Mar 19, 2024
9ff70c6
do not run emptydrops for cellranger arc, and update ch_matrices filt…
Mar 19, 2024
87abd86
add cellrangermulti in aligners options for conversion and add * to m…
Mar 20, 2024
614537e
parse cellrangermulti matrix outputs to filter between raw / filtered…
Mar 20, 2024
ceafcdf
fixing filtering option and using correct cellranger-multi mtxs
Mar 20, 2024
31ff8f4
add nf-test for cellranger multi
fmalmeida Mar 20, 2024
4ed087b
also test cellrangermulti
fmalmeida Mar 20, 2024
5807fd2
revert cellranger modules to latest, without the multi-out-channels a…
Mar 21, 2024
8d396d5
update modules to latest version
Mar 25, 2024
bb8d909
add a parser for raw/filtered results
Mar 25, 2024
84f781d
update comment line
Mar 25, 2024
f86a8d2
add lint fix
fmalmeida Apr 2, 2024
8d30a6f
fixed with new from template using lint
fmalmeida Apr 2, 2024
3b33ded
fix changelog.
fmalmeida Apr 8, 2024
db13a48
update assets for subworkflow
Apr 8, 2024
f9e5017
start removing channels related to "deprecated" additional csvs
Apr 8, 2024
d044f51
fix projectDir
Apr 10, 2024
c8ac4cf
add a parser for frna/cmo data from customised, unified, barcodes sam…
Apr 10, 2024
d81062d
add comment line
Apr 10, 2024
c6e7bfc
fix namings
Apr 10, 2024
319092b
fix if-else and .join() operations
Apr 10, 2024
ac9e5b6
fix selected data
Apr 10, 2024
a92e697
fix testing size
Apr 10, 2024
03e9e82
change to workDir
Apr 10, 2024
51c8577
avoid always writing to allow caching
Apr 10, 2024
62a396e
starting conversion as module
Apr 10, 2024
58466c9
converted parsing and split to a module
Apr 11, 2024
4634a4a
fix code for ensuring FIFO
Apr 11, 2024
07c399a
fixed cellranger-multi input channel logic
Apr 11, 2024
4750ebc
solved cellranger multi parsing and pipeline execution
Apr 12, 2024
aeaea61
add first cellranger-multi try-out bugfixes
Apr 15, 2024
0973363
Fix renaming logic
grst Apr 15, 2024
fa05a9b
fix variable
Apr 16, 2024
a6f46e6
added frna probeset subset reference and include parsing in module
Apr 17, 2024
7ef943c
frna runs also generate raw data per sample
Apr 17, 2024
3b90ddd
use shared nf-core test-datasets
Apr 18, 2024
53f5dc2
update cellranger multi module
Apr 18, 2024
b2f434c
add options-gex meta parsing
Apr 18, 2024
524eedb
Merge branch 'dev' of https://github.com/nf-core/scrnaseq into 247-su…
Apr 18, 2024
555ede3
update last cellranger module
Apr 18, 2024
a0f4ebf
adjust mkvdjref inputs
Apr 18, 2024
92e9d22
fix double comma
Apr 18, 2024
39a953c
update 'channel checkings'
Apr 18, 2024
1841682
add new parameter
Apr 18, 2024
3069563
fix schema
Apr 18, 2024
b4d8d45
also save config per sample
Apr 19, 2024
fc16327
add todo
Apr 19, 2024
2881f09
add fb reference example
Apr 19, 2024
e849278
fix file saving and remove outdated workaround on mqc
Apr 22, 2024
2b68ac8
add an universal key in cellranger-multi data options map so that par…
Apr 22, 2024
acd15cf
update comment
Apr 22, 2024
0d8bb2f
add new inputs to nf-test
Apr 22, 2024
54b1fa6
commit latest editions required in nf-core/module ( must be added in …
Apr 22, 2024
9f0d8db
try test update
Apr 29, 2024
118d754
change global variable used
Apr 30, 2024
b64fa2d
update test to use chr14
fmalmeida Apr 30, 2024
e97e1c0
change reference and resources used
fmalmeida Apr 30, 2024
37a9bb6
update number of tasks
fmalmeida Apr 30, 2024
619d921
Make it work without specifying GTF file
grst Apr 30, 2024
c51d0f6
fix pre-commit
grst Apr 30, 2024
81650b1
set working nf-test for cellranger multi
fmalmeida May 1, 2024
b4204ff
add .clone() method
fmalmeida May 1, 2024
ae6c561
decrease asked memory
fmalmeida May 1, 2024
f6b0f92
make sure ArrayBag is cloned to avoid input modification
fmalmeida May 1, 2024
f8865e6
change name for a better explanation
fmalmeida May 1, 2024
5b68077
modified by prettier
fmalmeida May 1, 2024
32a070b
update resources in test profile
fmalmeida May 1, 2024
97f22ae
make sure the workflow can work with new version of module that does …
fmalmeida May 1, 2024
02dc61d
fix mkgtf module "lint"
fmalmeida May 1, 2024
0361375
updated cellranger multi via nf-core tools
fmalmeida May 1, 2024
04492b1
force deletion for lint
fmalmeida May 1, 2024
2194739
manually download correct images
fmalmeida May 1, 2024
b660d19
Merge branch '247-support-for-10x-ffpe-scrna' into fix-without-gtf
fmalmeida May 2, 2024
5cdb691
Merge pull request #322 from nf-core/fix-without-gtf
fmalmeida May 2, 2024
2b79067
adjust parse to be the same for raw/filtered matrices
fmalmeida May 6, 2024
e8e14de
fix fastqc channel naming
fmalmeida May 6, 2024
714db1b
update nf-tests
fmalmeida May 6, 2024
87f38d0
update name
fmalmeida May 6, 2024
273ddd9
use pre-made fastqc_multiqc channel
fmalmeida May 6, 2024
aa1733f
nf-core lint fix
fmalmeida May 6, 2024
7da5644
flatten channel
fmalmeida May 6, 2024
889353c
update subworkflow
fmalmeida May 6, 2024
3aa7278
add missing .mix() operator
fmalmeida May 7, 2024
844fbc3
include cellrangermulti raw matrices for custom emptydrops filtering
May 10, 2024
9a3e529
correct indentation
May 10, 2024
f842cba
starting documentation on cellranger multi
fmalmeida May 10, 2024
adfda0f
continue documentation
fmalmeida May 10, 2024
0fce1c8
update documentation
fmalmeida May 14, 2024
dc63ae3
add section in outputs
fmalmeida May 14, 2024
9bff0b5
Merge remote-tracking branch 'origin/dev' into 247-support-for-10x-ff…
grst May 15, 2024
92502c9
Update usage
grst May 15, 2024
c22ad6f
Fixed 'file-path' in nextflow schema
grst May 15, 2024
c95b11c
Update output documentation
grst May 15, 2024
f4304ad
Update nextflow_schema.json
grst May 15, 2024
03a38cd
remove gex_barcode_sample_assignment parameter
fmalmeida May 16, 2024
e22f986
add note
fmalmeida May 16, 2024
0d0275e
Update nextflow schema documentation
grst May 17, 2024
882812c
Revert "remove gex_barcode_sample_assignment parameter"
grst May 17, 2024
708c903
Revert "add note"
grst May 17, 2024
b3afdb7
Merge branch 'dev' into 247-support-for-10x-ffpe-scrna
maxulysse May 22, 2024
d497ca8
update file
maxulysse May 22, 2024
4d9f17e
update file better
maxulysse May 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:
NXF_VER:
- "23.04.0"
- "latest-everything"
profile: ["alevin", "cellranger", "kallisto", "star"]
profile: ["alevin", "cellranger", "cellrangermulti", "kallisto", "star"]

steps:
- name: Disk space cleanup
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## v2.7.0dev

- Add cellranger multi subworkflow ([#247](https://github.com/nf-core/scrnaseq/issues/247))

## v2.6.0 - 2024-04-16

- Update cellranger to v8.0.0 ([#317](https://github.com/nf-core/scrnaseq/pull/317))
Expand Down
1 change: 1 addition & 0 deletions assets/EMPTY
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

7 changes: 7 additions & 0 deletions assets/cellranger_barcodes_samplesheet.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
sample,multiplexed_sample_id,probe_barcode_ids,cmo_ids,description
PBMC_10K_CMO,PBMC_10K_CMO_PBMCs_human_1,,CMO301,PBMCs_human_1
PBMC_10K_CMO,PBMC_10K_CMO_PBMCs_human_2,,CMO302,PBMCs_human_2
4PLEX_HUMAN,Liver_BC1,BC001,,Healthy liver dissociated using the Miltenyi FFPE Tissue Dissociation Kit
4PLEX_HUMAN,Ovarian_BC2,BC002,,Ovarian cancer dissociated using the Miltenyi FFPE Dissociation Kit
4PLEX_HUMAN,Colorectal_BC3,BC003,,Colorectal cancer dissociated using the Miltenyi FFPE Dissociation Kit
4PLEX_HUMAN,Pancreas_BC4,BC004,,Healthy pancreas dissociated using the Miltenyi FFPE Tissue Dissociation Kit
13 changes: 13 additions & 0 deletions assets/cellrangermulti_samplesheet.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
sample,fastq_1,fastq_2,feature_type,expected_cells
PBMC_10K,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/5gex/5gex/subsampled_sc5p_v2_hs_PBMC_10k_5gex_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/5gex/5gex/subsampled_sc5p_v2_hs_PBMC_10k_5gex_S1_L001_R2_001.fastq.gz,gex,1000
PBMC_10K,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/bcell/subsampled_sc5p_v2_hs_PBMC_10k_b_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/bcell/subsampled_sc5p_v2_hs_PBMC_10k_b_S1_L001_R2_001.fastq.gz,vdj,1000
PBMC_10K,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/5gex/5fb/subsampled_sc5p_v2_hs_PBMC_10k_5fb_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc/fastqs/5gex/5fb/subsampled_sc5p_v2_hs_PBMC_10k_5fb_S1_L001_R2_001.fastq.gz,ab,1000
PBMC_10K_CMO,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc_cmo/fastqs/gex_1/subsampled_SC3_v3_NextGem_DI_CellPlex_Human_PBMC_10K_1_gex_S2_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc_cmo/fastqs/gex_1/subsampled_SC3_v3_NextGem_DI_CellPlex_Human_PBMC_10K_1_gex_S2_L001_R2_001.fastq.gz,gex,1000
PBMC_10K_CMO,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc_cmo/fastqs/cmo/subsampled_SC3_v3_NextGem_DI_CellPlex_Human_PBMC_10K_1_multiplexing_capture_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/10k_pbmc_cmo/fastqs/cmo/subsampled_SC3_v3_NextGem_DI_CellPlex_Human_PBMC_10K_1_multiplexing_capture_S1_L001_R2_001.fastq.gz,cmo,1000
PBMC_10K_CMV,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/gex_1/subsampled_5k_human_antiCMV_T_TBNK_connect_GEX_1_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/gex_1/subsampled_5k_human_antiCMV_T_TBNK_connect_GEX_1_S1_L001_R2_001.fastq.gz,gex,1000
PBMC_10K_CMV,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/ab/subsampled_5k_human_antiCMV_T_TBNK_connect_AB_S2_L004_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/ab/subsampled_5k_human_antiCMV_T_TBNK_connect_AB_S2_L004_R2_001.fastq.gz,ab,1000
PBMC_10K_CMV,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/vdj/subsampled_5k_human_antiCMV_T_TBNK_connect_VDJ_S1_L001_R1_001.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/5k_cmvpos_tcells/fastqs/vdj/subsampled_5k_human_antiCMV_T_TBNK_connect_VDJ_S1_L001_R2_001.fastq.gz,vdj,1000
4PLEX_HUMAN,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/4plex_scFFPE/4plex_human_liver_colorectal_ovarian_panc_scFFPE_multiplex_S1_L001_R1_001.subsampled.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/4plex_scFFPE/4plex_human_liver_colorectal_ovarian_panc_scFFPE_multiplex_S1_L001_R2_001.subsampled.fastq.gz,gex,
4PLEX_HUMAN,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/4plex_scFFPE/4plex_human_liver_colorectal_ovarian_panc_scFFPE_multiplex_S1_L002_R1_001.subsampled.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/4plex_scFFPE/4plex_human_liver_colorectal_ovarian_panc_scFFPE_multiplex_S1_L002_R2_001.subsampled.fastq.gz,gex,
4PLEX_HUMAN,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/4plex_scFFPE/4plex_human_liver_colorectal_ovarian_panc_scFFPE_multiplex_S1_L003_R1_001.subsampled.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/4plex_scFFPE/4plex_human_liver_colorectal_ovarian_panc_scFFPE_multiplex_S1_L003_R2_001.subsampled.fastq.gz,gex,
4PLEX_HUMAN,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/4plex_scFFPE/4plex_human_liver_colorectal_ovarian_panc_scFFPE_multiplex_S1_L004_R1_001.subsampled.fastq.gz,https://raw.githubusercontent.com/nf-core/test-datasets/modules/data/genomics/homo_sapiens/10xgenomics/cellranger/4plex_scFFPE/4plex_human_liver_colorectal_ovarian_panc_scFFPE_multiplex_S1_L004_R2_001.subsampled.fastq.gz,gex,
12 changes: 12 additions & 0 deletions assets/fb_reference.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
id,name,read,pattern,sequence,feature_type
CD3,CD3,R2,^NNNNNNNNNN(BC)NNNNNNNNN,CTCATTGTAACTCCT,Antibody Capture
CD4,CD4,R2,^NNNNNNNNNN(BC)NNNNNNNNN,TGTTCCCGCTCAACT,Antibody Capture
CD8,CD8,R2,^NNNNNNNNNN(BC)NNNNNNNNN,GCGCAACTTGATGAT,Antibody Capture
CD11c,CD11c,R2,^NNNNNNNNNN(BC)NNNNNNNNN,TACGCCTATAACTTG,Antibody Capture
CD14,CD14,R2,^NNNNNNNNNN(BC)NNNNNNNNN,TCTCAGACCTCCGTA,Antibody Capture
CD16,CD16,R2,^NNNNNNNNNN(BC)NNNNNNNNN,AAGTTCACTCTTTGC,Antibody Capture
CD19,CD19,R2,^NNNNNNNNNN(BC)NNNNNNNNN,CTGGGCAATTACTCG,Antibody Capture
CD56,CD56,R2,^NNNNNNNNNN(BC)NNNNNNNNN,TCCTTTCCTGATAGG,Antibody Capture
CD45,CD45,R2,^NNNNNNNNNN(BC)NNNNNNNNN,TCCCTTGCGATTTAC,Antibody Capture
IgG1,IgG1_control_TotalSeqC,R2,^NNNNNNNNNN(BC)NNNNNNNNN,GCCGGACGACATTAA,Antibody Capture
A0201_NLVPMVATV_CMV_TCR-1,NLVPMVATV_CMV_TCR-1,R2,^NNNNNNNNNN(BC)NNNNNNNNN,GGCCTCGGTCCTAGG,Antibody Capture
Loading
Loading