Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Update and test fastx-tools #375

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

WIP: Update and test fastx-tools #375

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
a81f7d5
Test and update fastx tools
bgruening May 23, 2016
a554313
Upgrade to newer version of fastx
bgruening May 23, 2016
459ff1c
Add tests for the remaining fastx tools
mvdbeek Sep 17, 2016
8865acd
Explicitly set `-Q` for all non-cs fastq variants
mvdbeek Sep 18, 2016
0c1f99d
Close command tag
mvdbeek Sep 18, 2016
f4e5d27
Fix command tag ... again
mvdbeek Sep 18, 2016
f206706
Fix requirements
mvdbeek Sep 18, 2016
361f286
Don't fail with gnuplot warnings
mvdbeek Sep 18, 2016
0024992
Use simsize comparison for png
mvdbeek Sep 18, 2016
3fdc28d
Retab certain fastx tools, remove duplicate macro imports
mvdbeek Jan 1, 2017
d55db72
Retab more fastx tools
mvdbeek Jan 1, 2017
f9a5168
Replace '.ext ==' pattern with is_of_type
Jan 20, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 0 additions & 16 deletions .tt_blacklist
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,22 +129,6 @@ tool_collections/vcftools/vcftools_merge
tool_collections/vcftools/vcftools_subset
tool_collections/vcftools/vcftools_annotate
tool_collections/vcftools/vcftools_isec
tool_collections/fastx_toolkit/fastq_quality_converter
tool_collections/fastx_toolkit/fasta_clipping_histogram
tool_collections/fastx_toolkit/fastx_nucleotides_distribution
tool_collections/fastx_toolkit/fastx_trimmer
tool_collections/fastx_toolkit/fastx_clipper
tool_collections/fastx_toolkit/fastx_artifacts_filter
tool_collections/fastx_toolkit/fastx_barcode_splitter
tool_collections/fastx_toolkit/fastq_quality_filter
tool_collections/fastx_toolkit/fastx_renamer
tool_collections/fastx_toolkit/fastx_quality_statistics
tool_collections/fastx_toolkit/fasta_formatter
tool_collections/fastx_toolkit/fastq_quality_boxplot
tool_collections/fastx_toolkit/fasta_nucleotide_changer
tool_collections/fastx_toolkit/fastq_to_fasta
tool_collections/fastx_toolkit/fastx_collapser
tool_collections/fastx_toolkit/fastx_reverse_complement
tool_collections/gatk/variant_annotator
tool_collections/gatk/variant_apply_recalibration
tool_collections/gatk/count_covariates
Expand Down
21 changes: 14 additions & 7 deletions tool_collections/fastx_toolkit/fasta_clipping_histogram/fasta_clipping_histogram.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,25 @@
<tool id="cshl_fasta_clipping_histogram" name="Length Distribution" version="1.0.0">
<tool id="cshl_fasta_clipping_histogram" name="Length Distribution" version="@VERSION@">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this would cause problems, unless the version lineage code recognizes that 0.0.14 > 1.0.0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes, this is true. Too bad. Are you working on this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was planning on moving it to IUC in galaxyproject/tools-iuc#1413, and of course I can merge in your changes there to avoid reinventing effort.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, please go ahead and close this when you are ready.

<description>chart</description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<command>fasta_clipping_histogram.pl $input $outfile</command>

<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command><![CDATA[
command -v perl5.22.0 &&
perl5.22.0 `command -v fasta_clipping_histogram.pl` '$input' '$outfile' ||
fasta_clipping_histogram.pl '$input' '$outfile'
]]></command>
<inputs>
<param format="fasta" name="input" type="data" label="Library to analyze" />
</inputs>

<outputs>
<data format="png" name="outfile" metadata_source="input" />
</outputs>
<tests>
<test>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bgruening unfortunately this test only passes if I set galaxy's conda version to 4.2.7 :(.
Otherwise I'm getting fastx_toolkit==0.0.14-2 instead of -3. See https://travis-ci.org/galaxyproject/tools-devteam/jobs/160679791#L355

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Urgs, this offline bug needs to be fixed. This is really killing us.

<param name="input" value="input.fasta" ftype="fasta"/>
<output name="outfile" file="histogram.pdf" />
</test>
</tests>
<help>
**What it does**
Expand Down
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fasta_clipping_histogram/fastx_macros.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 6 additions & 0 deletions tool_collections/fastx_toolkit/fasta_clipping_histogram/test-data/input.fasta
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
>sequence1
AGTAGTAGGTGATGTAGAGAGAGAGAGAGTAG
>sequence2
GTGTGTGTGGGAAGTTGACACAGTA
>sequence3
CCTTGAGATTAACGCTAATCAAGTAAAC
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fasta_clipping_histogram/tool_dependencies.xml
View file
Open in desktop

This file was deleted.

15 changes: 8 additions & 7 deletions tool_collections/fastx_toolkit/fasta_formatter/fasta_formatter.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
<tool id="cshl_fasta_formatter" version="1.0.0" name="FASTA Width">
<tool id="cshl_fasta_formatter" version="@VERSION@" name="FASTA Width">
<description>formatter</description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<!--
Note:
fasta_formatter also has a tabular output mode (-t),
Expand All @@ -13,9 +14,9 @@
FASTA file.
-->
<command>
<![CDATA[
zcat -f < '$input' | fasta_formatter -w $width -o '$output'
]]>
<![CDATA[
zcat -f < '$input' | fasta_formatter -w $width -o '$output'
]]>
</command>
<inputs>
<param format="fasta" name="input" type="data" label="Library to re-format" />
Expand Down
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fasta_formatter/fastx_macros.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fasta_formatter/tool_dependencies.xml
View file
Open in desktop

This file was deleted.

15 changes: 8 additions & 7 deletions tool_collections/fastx_toolkit/fasta_nucleotide_changer/fasta_nucleotide_changer.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
<tool id="cshl_fasta_nucleotides_changer" version="1.0.0" name="RNA/DNA" >
<tool id="cshl_fasta_nucleotides_changer" version="@VERSION@" name="RNA/DNA" >
<description>converter</description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
<![CDATA[
zcat -f < '$input' | fasta_nucleotide_changer -$mode -v -o '$output'
]]>
<![CDATA[
zcat -f < '$input' | fasta_nucleotide_changer -$mode -v -o '$output'
]]>
</command>
<inputs>
<param format="fasta" name="input" type="data" label="Library to convert" />
Expand Down
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fasta_nucleotide_changer/fastx_macros.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
36 changes: 22 additions & 14 deletions tool_collections/fastx_toolkit/fastq_quality_boxplot/fastq_quality_boxplot.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,25 @@
<tool id="cshl_fastq_quality_boxplot" name="Draw quality score boxplot" version="1.0.0">
<description></description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<command>fastq_quality_boxplot_graph.sh -t '$input.name' -i $input -o $output</command>

<inputs>
<param format="txt" name="input" type="data" label="Statistics report file" help="output of 'FASTQ Statistics' tool" />
</inputs>

<outputs>
<data format="png" name="output" metadata_source="input" />
</outputs>
<tool id="cshl_fastq_quality_boxplot" name="Draw quality score boxplot" version="@VERSION@">
<description></description>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
fastq_quality_boxplot_graph.sh -t '$input.name' -i '$input' -o '$output'
</command>
<inputs>
<param format="txt" name="input" type="data" label="Statistics report file" help="output of 'FASTQ Statistics' tool" />
</inputs>

<outputs>
<data format="png" name="output" metadata_source="input" />
</outputs>
<tests>
<test>
<param name="input" value="fastq_stats1.out" ftype="txt"></param>
<output name="output" file="quality_boxplot_out.png"></output>
</test>
</tests>
<help>

**What it does**
Expand Down
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastq_quality_boxplot/fastx_macros.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
37 changes: 37 additions & 0 deletions tool_collections/fastx_toolkit/fastq_quality_boxplot/test-data/fastq_stats1.out
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
column count min max sum mean Q1 med Q3 IQR lW rW A_Count C_Count G_Count T_Count N_Count Max_count
1 9 23 34 288 32.00 33 33 33 0 33 33 3 1 4 1 0 9
2 9 28 33 287 31.89 31 33 33 2 28 33 3 3 2 1 0 9
3 9 13 34 268 29.78 28 33 33 5 21 34 5 1 0 3 0 9
4 9 17 33 261 29.00 30 33 33 3 26 33 1 2 3 3 0 9
5 9 22 33 269 29.89 30 33 33 3 26 33 3 3 3 0 0 9
6 9 22 33 277 30.78 30 33 33 3 26 33 5 3 0 1 0 9
7 9 21 33 258 28.67 24 33 33 9 21 33 4 1 3 1 0 9
8 9 12 33 263 29.22 32 33 33 1 31 33 2 1 1 5 0 9
9 9 29 33 290 32.22 33 33 33 0 33 33 3 3 2 1 0 9
10 9 23 33 277 30.78 32 33 33 1 31 33 1 4 2 2 0 9
11 9 12 33 245 27.22 21 31 33 12 12 33 5 2 1 1 0 9
12 9 13 33 214 23.78 15 24 33 18 13 33 2 4 2 1 0 9
13 9 5 33 249 27.67 29 31 33 4 23 33 2 1 1 5 0 9
14 9 5 33 233 25.89 24 33 33 9 11 33 3 3 2 1 0 9
15 9 15 33 251 27.89 24 33 33 9 15 33 5 1 1 2 0 9
16 9 23 34 269 29.89 24 33 33 9 23 34 3 1 2 3 0 9
17 9 13 34 266 29.56 33 33 33 0 33 33 2 3 1 3 0 9
18 9 21 34 272 30.22 31 33 33 2 28 34 0 5 1 3 0 9
19 9 5 34 244 27.11 27 30 33 6 18 34 4 4 1 0 0 9
20 9 11 34 241 26.78 23 32 33 10 11 34 3 4 2 0 0 9
21 9 13 33 240 26.67 24 27 33 9 13 33 1 4 0 4 0 9
22 9 5 33 190 21.11 13 21 33 20 5 33 1 4 0 3 1 9
23 9 5 33 205 22.78 16 26 33 17 5 33 4 4 1 0 0 9
24 9 5 33 247 27.44 28 31 33 5 21 33 1 5 1 2 0 9
25 9 11 34 241 26.78 24 33 33 9 11 34 3 4 0 2 0 9
26 9 5 33 212 23.56 18 31 33 15 5 33 0 6 0 3 0 9
27 9 5 33 227 25.22 21 26 33 12 5 33 3 4 1 1 0 9
28 9 21 33 255 28.33 24 31 33 9 21 33 2 4 3 0 0 9
29 9 5 33 228 25.33 21 30 33 12 5 33 2 4 1 2 0 9
30 9 10 33 213 23.67 16 28 33 17 10 33 3 4 2 0 0 9
31 9 5 33 236 26.22 21 31 33 12 5 33 1 4 1 3 0 9
32 9 5 33 210 23.33 12 29 33 21 5 33 3 3 0 3 0 9
33 9 5 33 183 20.33 9 21 33 24 5 33 1 4 2 2 0 9
34 9 5 33 150 16.67 7 17 22 15 5 33 3 4 1 1 0 9
35 9 13 33 217 24.11 21 24 29 8 13 33 1 4 1 3 0 9
36 9 5 33 195 21.67 18 21 32 14 5 33 3 2 1 3 0 9
Binary file added ...llections/fastx_toolkit/fastq_quality_boxplot/test-data/quality_boxplot_out.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastq_quality_boxplot/tool_dependencies.xml
View file
Open in desktop

This file was deleted.

9 changes: 5 additions & 4 deletions tool_collections/fastx_toolkit/fastq_quality_converter/fastq_quality_converter.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
<tool id="cshl_fastq_quality_converter" name="Quality format converter" version="1.0.0">
<tool id="cshl_fastq_quality_converter" name="Quality format converter" version="@VERSION@">
<description>(ASCII-Numeric)</description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
<![CDATA[
zcat -f < '$input' | fastq_quality_converter $QUAL_FORMAT -o '$output' -Q $offset
Expand Down
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastq_quality_converter/fastx_macros.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastq_quality_converter/tool_dependencies.xml
View file
Open in desktop

This file was deleted.

24 changes: 14 additions & 10 deletions tool_collections/fastx_toolkit/fastq_quality_filter/fastq_quality_filter.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,19 @@
<tool id="cshl_fastq_quality_filter" version="1.0.0" name="Filter by quality">
<tool id="cshl_fastq_quality_filter" version="@VERSION@" name="Filter by quality">
<description></description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
<![CDATA[
zcat -f < '$input' | fastq_quality_filter -q $quality -p $percent -v -o '$output'
#if $input.ext == "fastqsanger":
-Q 33
#end if
]]>
<![CDATA[
zcat -f < '$input' | fastq_quality_filter -q $quality -p $percent -v -o '$output'
## fastqsanger uses offset 33, illumina and solexa offset 64 (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2847217/table/T1/)
#if $input.is_of_type(fastqsanger):
-Q 33
#elif $input.is_of_type(fastqillumina" or $input.is_of_type(fastqsolexa):
-Q 64
#end if
]]>
</command>

<inputs>
Expand Down
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastq_quality_filter/fastx_macros.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastq_quality_filter/tool_dependencies.xml
View file
Open in desktop

This file was deleted.

97 changes: 51 additions & 46 deletions tool_collections/fastx_toolkit/fastq_to_fasta/fastq_to_fasta.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,49 +1,54 @@
<tool id="cshl_fastq_to_fasta" name="FASTQ to FASTA" version="1.0.0">
<description>converter from FASTX-toolkit</description>
<requirements>
<requirement type="package" version="0.0.13">fastx_toolkit</requirement>
</requirements>
<command>gunzip -cf $input | fastq_to_fasta $SKIPN $RENAMESEQ -o $output -v
#if $input.ext == "fastqsanger":
-Q 33
#end if
</command>

<inputs>
<param format="fastqsanger,fastqsolexa,fastqillumina" name="input" type="data" label="FASTQ Library to convert" />

<param name="SKIPN" type="select" label="Discard sequences with unknown (N) bases ">
<option value="">yes</option>
<option value="-n">no</option>
</param>

<param name="RENAMESEQ" type="select" label="Rename sequence names in output file (reduces file size)">
<option value="-r">yes</option>
<option value="">no</option>
</param>

</inputs>

<tests>
<test>
<!-- FASTQ-To-FASTA, keep N, don't rename -->
<param name="input" value="fastq_to_fasta1.fastq" ftype="fastqsolexa" />
<param name="SKIPN" value=""/>
<param name="RENAMESEQ" value=""/>
<output name="output" file="fastq_to_fasta1a.out" />
</test>
<test>
<!-- FASTQ-To-FASTA, discard N, rename -->
<param name="input" value="fastq_to_fasta1.fastq" ftype="fastqsolexa" />
<param name="SKIPN" value="no"/>
<param name="RENAMESEQ" value="yes"/>
<output name="output" file="fastq_to_fasta1b.out" />
</test>
</tests>

<outputs>
<data format="fasta" name="output" metadata_source="input" />
</outputs>
<tool id="cshl_fastq_to_fasta" name="FASTQ to FASTA" version="@VERSION@">
<description>converter from FASTX-toolkit</description>
<expand macro="requirements" />
<macros>
<import>fastx_macros.xml</import>
</macros>
<command>
<![CDATA[
gunzip -cf $input | fastq_to_fasta $SKIPN $RENAMESEQ -o $output -v
#if $input.is_of_type('fastqsanger'):
-Q 33
#elif $input.is_of_type('fastqillumina') or $input.is_of_type('fastqsolexa'):
-Q 64
#end if
]]>
</command>
<inputs>
<param format="fastqsanger,fastqsolexa,fastqillumina" name="input" type="data" label="FASTQ Library to convert" />

<param name="SKIPN" type="select" label="Discard sequences with unknown (N) bases ">
<option value="">yes</option>
<option value="-n">no</option>
</param>

<param name="RENAMESEQ" type="select" label="Rename sequence names in output file (reduces file size)">
<option value="-r">yes</option>
<option value="">no</option>
</param>

</inputs>

<tests>
<test>
<!-- FASTQ-To-FASTA, keep N, don't rename -->
<param name="input" value="fastq_to_fasta1.fastq" ftype="fastqsolexa" />
<param name="SKIPN" value=""/>
<param name="RENAMESEQ" value=""/>
<output name="output" file="fastq_to_fasta1a.out" />
</test>
<test>
<!-- FASTQ-To-FASTA, discard N, rename -->
<param name="input" value="fastq_to_fasta1.fastq" ftype="fastqsolexa" />
<param name="SKIPN" value="no"/>
<param name="RENAMESEQ" value="yes"/>
<output name="output" file="fastq_to_fasta1b.out" />
</test>
</tests>

<outputs>
<data format="fasta" name="output" metadata_source="input" />
</outputs>

<help>

Expand Down
1 change: 1 addition & 0 deletions tool_collections/fastx_toolkit/fastq_to_fasta/fastx_macros.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../fastx_macros.xml
6 changes: 0 additions & 6 deletions tool_collections/fastx_toolkit/fastq_to_fasta/tool_dependencies.xml
View file
Open in desktop

This file was deleted.

Loading