Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Draft PR for porting RAW and CLEAN to ONT data. #121

Draft
wants to merge 99 commits into
base: master
Choose a base branch
from
Draft

Draft PR for porting RAW and CLEAN to ONT data. #121

wants to merge 99 commits into from

Conversation

simonleandergrimm
Copy link
Collaborator

@simonleandergrimm simonleandergrimm commented Dec 9, 2024 •
edited
Loading

Open questions:

  • Rename n_read_pairs in summarize-multiqc-pair.R to n_sequences`?
  • Removing polya tails with cutadapt?
  • Handling files that are empty after cleaning (currently those trip up MultiQC and are likely to cause errors in later stages of the pipeline.)
  • Double check if fastqc is appropriate for ONT data.

@simonleandergrimm
Adding single read option to raw/main.nf
15354f6
@simonleandergrimm
Adding WIP version of run.nf to enable testing raw and clean versions…
ad2115d 
… for single read analyses.
@simonleandergrimm
Created separate versions of summarize-multiqc-single.R and summarize…
03ee37a 
…-multiqc-paired.R
@simonleandergrimm
Split processes in fastp to a single read and paired-end read version.
b517340
@simonleandergrimm
Split processes in MultiQC to a single read and paired-end read versi…
01ea0c5 
…on. Renamed Multiqc to not be confusing regardings its naming as "Single"
@simonleandergrimm
Deleted summarizeMultiqcSingle, which was superseded by summarizeMultiqc
ad8faf9
@simonleandergrimm
Split processes in truncateConcat to a single read and paired-end rea…
ef0e9c8 
…d version
@simonleandergrimm
Created a single_end if clause in Clean to either use the single read…
2535ccd 
… or paired-end read version of fastp
@simonleandergrimm
Created a single_end if clause in hv_screen to either use the single …
cbcb109 
…read or paired-end read version of fastp
@simonleandergrimm
Created a single_end if clause in qc to either use the single read or…
c7f8c83 
… paired-end read version of SUMMARIZE_MULTIQC
@simonleandergrimm
Renamed test dir to test-paired-end. Added clause in nextflow.config …
ff0a8be 
…for paired-end vs single read runs.
@simonleandergrimm
Edited gitignore to leave out test-paired-end and test-single-read ru…
6048dd3 
…n outputs
@simonleandergrimm
Fixed name of test-single-end dir to test-single-read
92270e5
@simonleandergrimm
Created a version of test dir that allows the run of single-read data.
b13ac94
@simonleandergrimm
Added script to quickly download the s3 output of test single read an…
dff2302 
…d paired end read runs
@simonleandergrimm
Added nextflow config for test paired and test single read.
64bb7f4
@simonleandergrimm
Fixed if clause in main.nf
5bd1aec
@simonleandergrimm
Updated gen samplesheet scripts to pull in data from s3://nao-mgs-sim…
c8fd3ac 
…on/*/raw
@simonleandergrimm
Updated gitignore
578fde0
@simonleandergrimm
Activated CLEAN subworkflow in run.nf
59218b9
@simonleandergrimm
Starting to adapt Will's https://data.securebio.org/wills-public-note…
fd9dc1e 
…book/notebooks/2024-10-17_crits-christoph-2-4-0.html to create analyses of single and paired-end read data.
@simonleandergrimm
Adding ignoring mgs-results to gitignore
81ff0ba
@simonleandergrimm
Adding Will's auxiliary scripts to run his quarto notebooks.
590b2c3
@simonleandergrimm
Merge branch 'master' into single-read-raw
6a650b4
@simonleandergrimm
Amended qmd somewhat so data imports work.
9f1eb03
@simonleandergrimm
Added a flag to summarize-multiqc-single.R that provides info on the …
9622004 
…samples being paired-end or not.
@simonleandergrimm
Amended logic of split_sample, so it does not split and pull out read…
c61ed0c 
… pair information for single read data. Also dropped some code which combines values across read pairs, for single read data.

I dropped the renaming of tab_tsv to tab_tsv_2 for paired end data, so I didn't have to create two different versions of the combine step at the end of the subscript.
```
 tab <- tab_json %>% inner_join(tab_tsv, by="sample")
```
@simonleandergrimm
Deleting seperate version of summarize-multiqc I created for paired r…
f8d9c28 
…ead data, as I instead amended the existing script to be able to handle both single read paired end data.
@simonleandergrimm
Revert "Split processes in MultiQC to a single read and paired-end re…
8e1c7b5 
…ad version. Renamed Multiqc to not be confusing regardings its naming as "Single""

This reverts commit 01ea0c5.
@simonleandergrimm
Revert "Deleted summarizeMultiqcSingle, which was superseded by summa…
0ba0552 
…rizeMultiqc"

This reverts commit ad8faf9.
simonleandergrimm and others added 30 commits November 24, 2024 01:52
@simonleandergrimm
Changed test dirs to only have one dir for run_dev_se.
4fd3ce6
@simonleandergrimm
Adding WIP progress
d460813
@simonleandergrimm
Merge branch 'dev' into single-read-raw-clean
f412b07
@simonleandergrimm
Fixing single_end being unbound.
3d10bb0
@simonleandergrimm
Merge branch 'dev' into single-read-raw-clean
7899979
@simonleandergrimm
Took into account new testing setup
dd942fa
@simonleandergrimm
adding single end info to config
50c2edc
@simonleandergrimm
Moved single end eval from config to run files
61ea369
@harmonbhasin
Changed version of nextflow, hopefully this works
c71289d
@harmonbhasin
Updated label to be correct
8d40459
@harmonbhasin
Merge branch 'master' of https://github.com/naobservatory/mgs-workflow …
222946d 
…into harmon_fix_gh_actions_test
@simonleandergrimm
Update nextflow.config
ad640c6
@simonleandergrimm
Merge remote-tracking branch 'origin/harmon_fix_gh_actions_test' into…
e85dd45 
… single-read-raw-clean
@harmonbhasin
Splitting up tests
ef3a1bb
@harmonbhasin
Didn't know that I had an out of date verison of this branch.
67742e8
@harmonbhasin
Forgot to remove validation from run test, hopefully this will test t…
8ae0a93 
…he timeout as well.
@simonleandergrimm
Merge remote-tracking branch 'origin/harmon_fix_gh_actions_test' into…
3a6f6b5 
… single-read-raw-clean
@simonleandergrimm
Put single_end into profiles.config
a0f5f32
@simonleandergrimm
fixed run-dev-se config in tests
d14da14
@simonleandergrimm
Creating a new config for read_type flag.
3fe2bd2
@simonleandergrimm
added run dev se to end-to-end yml
d0375ab
@simonleandergrimm
Made rundevse index and outputs look the same as run.nf
f5cf80a
@simonleandergrimm
Fixing setup of run_dev_se test config.
3dc323e
@simonleandergrimm
Update .gitignore (dropped new line)
1904931
@simonleandergrimm
Setting profiles.config back to original
e24d79e
@simonleandergrimm
Updated comments in main.nf to represent the posiblity of not not ala…
b38b93d 
…ysing read pairs!
@simonleandergrimm
Added variable that checks if data is ONT, based on fastq file name.
4fbed83
@simonleandergrimm
Adding the filtlong process and docker image,
71757bc
@simonleandergrimm
Added code to pull out legnth data from multiqc.
66569ad
@simonleandergrimm
Adding filtlong to the Cleaning step.
09eaf8f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@simonleandergrimm @harmonbhasin