Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DSL2: Starting subworkflow for circularmapper #1076

Merged
merged 66 commits into from
Aug 16, 2024
Merged
Changes from 1 commit
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
010ffcd
Starting subworkflow for circularmapper
jbv2 Jun 14, 2024
3bbb28c
adding parameters and outputs. Missing to add subworkflow to the main…
jbv2 Jun 21, 2024
74b620e
Merge branch 'dev' into circularmapper
jfy133 Jun 22, 2024
25ec1da
adding circularmapper to map.nf
jbv2 Jun 28, 2024
68f7243
removing local path
jbv2 Jun 28, 2024
1a5a148
Merge branch 'dev' into circularmapper
jbv2 Jun 28, 2024
3fac93d
fixing schema
jbv2 Jun 28, 2024
ed9e948
Revert "fixing schema"
TCLamnidis Jul 5, 2024
050acfd
fix schema
TCLamnidis Jul 5, 2024
d7a2d92
WIP
TCLamnidis Jul 5, 2024
6caacef
fix swf emissions
TCLamnidis Jul 5, 2024
670771d
add elongate_reference swf
TCLamnidis Jul 12, 2024
21093ac
wip n CM mapping
TCLamnidis Jul 12, 2024
a6a145a
add elongated fasta and fai input
scarlhoff Jul 12, 2024
cd86749
attempt to add CM SWF
TCLamnidis Jul 12, 2024
e05cc42
index output bams
TCLamnidis Jul 15, 2024
1764505
add validation and address comments
scarlhoff Jul 16, 2024
530cf7f
Merge branch 'circularmapper' into dsl2_circularmappe_refsheet
scarlhoff Jul 16, 2024
203b9e5
Merge pull request #2 from scarlhoff/dsl2_circularmappe_refsheet
jbv2 Jul 16, 2024
dff6208
update CM modules
TCLamnidis Jul 19, 2024
c140062
updates to reference elongation SWF
TCLamnidis Jul 19, 2024
4cd7a19
Add unzipping in ELONGATE_REFERENCE SWF
TCLamnidis Jul 19, 2024
de747de
unzip, elongate and index reference when CM is used
TCLamnidis Jul 19, 2024
ff2ad9a
add original reference input
TCLamnidis Jul 19, 2024
b48e80f
i broke something Q_Q
TCLamnidis Jul 19, 2024
7e311f4
bracket swap
TCLamnidis Jul 19, 2024
159a48a
fix syntax errors.
TCLamnidis Jul 19, 2024
61c5a88
fix cardinality in reference maps
TCLamnidis Jul 19, 2024
9054dd7
add third input channel, and dummy output for CM for testing.
TCLamnidis Jul 19, 2024
d656966
WIP
TCLamnidis Jul 19, 2024
073b8fd
module updates on elongate_reference modules
TCLamnidis Jul 22, 2024
bce305a
actually run when the reference is not zipped
TCLamnidis Jul 22, 2024
20f8ccf
correct cardinality of circular reference channel
TCLamnidis Jul 25, 2024
c73e14f
output elongated reference in reference dirs when requested
TCLamnidis Jul 25, 2024
1121cbf
no publishing of lane bams in CM
TCLamnidis Jul 25, 2024
cb20652
this works now
TCLamnidis Jul 25, 2024
224994f
add circularmapper. fix issue with channel join before flagstat for l…
TCLamnidis Jul 25, 2024
b306e74
Merge branch 'dev' into circularmapper
TCLamnidis Jul 25, 2024
63d0888
remove completed TODOs
TCLamnidis Jul 25, 2024
36fc7c6
add CM to CI
TCLamnidis Jul 25, 2024
2f78995
minor tweak to avoid `null` in file names
TCLamnidis Jul 26, 2024
edccda3
typo
TCLamnidis Jul 26, 2024
1979181
add manual tests
TCLamnidis Jul 26, 2024
585de6b
incorporate review suggestions. put mapping output within tool subdir…
TCLamnidis Aug 2, 2024
ab2d40e
update output.md
TCLamnidis Aug 2, 2024
c028b6a
add mapping tool subdirectory within mapping
TCLamnidis Aug 2, 2024
bdf0d80
mapped bams/bais/csi/flagstat all in tool subdir
TCLamnidis Aug 2, 2024
ed4d753
force bwa indexing of original ref when circularmapper is used
TCLamnidis Aug 2, 2024
ef32d61
Rename Mapped Flagstat for consistency with other modules and outputs…
TCLamnidis Aug 2, 2024
e7388e8
Rename CM parameters
TCLamnidis Aug 2, 2024
3b3d1d0
Add suggestions from review
TCLamnidis Aug 2, 2024
d03b7c7
linting
TCLamnidis Aug 5, 2024
f72531b
rename elongated ref channel for clarity with its purpose
TCLamnidis Aug 5, 2024
8fe564b
add mapping_circularmapper_circularfilter option
TCLamnidis Aug 5, 2024
2a98bd9
update manual tests
TCLamnidis Aug 5, 2024
6a076c2
update CM modules
TCLamnidis Aug 9, 2024
81d79f8
add elongated chromosome list needed for circularmapper filtering
TCLamnidis Aug 9, 2024
f901063
start adding manual chrom_list creation
TCLamnidis Aug 9, 2024
0b0bcf1
create elongated_chr_list from circular target
TCLamnidis Aug 12, 2024
7063fd2
Circular target required regardless of elongated fasta
TCLamnidis Aug 12, 2024
5d8476f
remove dumps
TCLamnidis Aug 12, 2024
302acf3
update manual tests
TCLamnidis Aug 12, 2024
c89ef24
fix module configuration application
TCLamnidis Aug 12, 2024
32fa8b1
update parameter name in CI
TCLamnidis Aug 12, 2024
753994c
Apply suggestions from code review
jfy133 Aug 16, 2024
81784cb
Fix linting
jfy133 Aug 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
create elongated_chr_list from circular target
TCLamnidis committed Aug 12, 2024
commit 0b0bcf1bd02d365b9c97d452bdf6d30a3dfe3874
40 changes: 27 additions & 13 deletions subworkflows/local/elongate_reference.nf
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@
include { GUNZIP as GUNZIP_ELONGATED_FASTA } from '../../modules/nf-core/gunzip/main'
include { CIRCULARMAPPER_CIRCULARGENERATOR } from '../../modules/nf-core/circularmapper/circulargenerator/main'
include { BWA_INDEX as BWA_INDEX_CIRCULARISED } from '../../modules/nf-core/bwa/index/main'
// include { addNewMetaFromAttributes } from '../../subworkflows/local/utils_nfcore_eager_pipeline/main'
include { addNewMetaFromAttributes } from '../../subworkflows/local/utils_nfcore_eager_pipeline/main'

workflow ELONGATE_REFERENCE {
take:
@@ -62,8 +62,20 @@ workflow ELONGATE_REFERENCE {
needs_elongation: circularmapper_elongated_fasta == "" && circular_target != ""
}

// References that are already elongated, need ch_elongated_chr to be created from the circular target information
ch_needs_elongated_chr_list = ch_circulargenerator_input.ready
/* References that are already elongated, need ch_elongated_chr to be created from the circular target information
1) Get the reference information ready for joinin with the new channel.
2) Take all subchannels from the multiMap that do not go through CircularGenerator (.ready,.needs_index) and infer the name of the elongated_chr_list expected by RealignSAMFile
3) Put the circular target in a file of that name FOR EACH REFERENCE. The resulting channel has no meta, so we need to add it.
4) Add meta, and use to merge back to the reference channel. This way we can take the original reference's meta.

This is a bit convoluted, but it should work. Would be simpler if I could create the meta within collectFile, but I did not find a way to do that.
*/
ch_ref_for_chr_list = ch_reference
.map {
addNewMetaFromAttributes( it, "id", "id", false )
}

ch_chr_list_for_already_elongated_ref = ch_circulargenerator_input.ready
.mix( ch_circulargenerator_input.needs_index )
.join( ch_reference )
.map {
@@ -74,15 +86,17 @@ workflow ELONGATE_REFERENCE {
meta, fasta, circular_target ->
[ "${fasta.name}_500_elongated", circular_target + '\n' ]
}
/* The above gets the right information into the created files, but the channel then also needs a meta, which collectFile doesn't seem able to handle.
TODO Proposed solution:
- Use a map to infer the meta.id fromt he file name (i.e. file name without the suffix. since everything by now is unzipped, it should work).
- Then pull that info out of the ch_reference meta with addNewMetaFromAttributes.
- Join the channels by this meta
- Use a map to give the collected file the meta of the reference.

This is a bit convoluted, but it should work. Would be simpler if I could create the meta within collectFile.
*/
.map {
file ->
def id = file.getSimpleName()
[ [id: id ], file ]
}
.join(ch_ref_for_chr_list)
.map {
ignore_me, chr_list, meta, fasta, fai, dict, mapindex ->
[ meta, chr_list ]
}
.dump(tag: "collected_files", pretty:true)

// Elongate references that need it
// Join the original references to the branch of needs_elongation, to get the original fasta files, and elongate them.
@@ -103,7 +117,7 @@ workflow ELONGATE_REFERENCE {
ch_references_to_elongate.elongation_factor,
ch_references_to_elongate.target
)
ch_elongated_chr = CIRCULARMAPPER_CIRCULARGENERATOR.out.elongated
ch_elongated_chr = ch_chr_list_for_already_elongated_ref.mix(CIRCULARMAPPER_CIRCULARGENERATOR.out.elongated)
ch_versions = ch_versions.mix( CIRCULARMAPPER_CIRCULARGENERATOR.out.versions.first() )

// Collect newly generated circular references and provided ones without an index, and index them.