Skip to content

Commit

Permalink
Merge pull request #4432 from galaxyproject/zebu-bicolor
Browse files Browse the repository at this point in the history 
add zenodo api url linting, fix broken zenodo links
  • Loading branch information
@shiltemann
shiltemann authored Oct 18, 2023
2 parents 5b4db2f + 19b22b6 commit 2752a66
Show file tree
Hide file tree
Showing 19 changed files with 152 additions and 137 deletions.
18 changes: 17 additions & 1 deletion bin/lint.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -684,6 +684,21 @@ def self.check_bolded_heading(contents)
end
end

def self.zenodo_api(contents)
find_matching_texts(contents, /(zenodo\.org\/api\/files\/)/)
.map do |idx, _text, selected|
ReviewDogEmitter.error(
path: @path,
idx: idx,
match_start: selected.begin(1),
match_end: selected.end(1) + 1,
replacement: nil,
message: 'The Zenodo.org/api URLs are not stable, you must use a URL of the format zenodo.org/record/..., apologies we cannot fix automatically.',
code: 'GTN:032'
)
end
end

def self.fix_md(contents)
[
*fix_notoc(contents),
Expand All @@ -707,7 +722,8 @@ def self.fix_md(contents)
*check_useless_box_prefix(contents),
*check_bad_heading_order(contents),
*check_bolded_heading(contents),
*snippets_too_close_together(contents)
*snippets_too_close_together(contents),
*zenodo_api(contents),
]
end

Expand Down
6 changes: 3 additions & 3 deletions topics/admin/tutorials/data-library/tutorial.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -206,18 +206,18 @@ We've included this file in the example data libraries repository we cloned at t
> description: some data
> synopsis: samples collected from somewhere
> items:
> - url: https://zenodo.org/api/files/287966da-5411-4f79-8cfb-0ffa84d0d6cc/wildtype.fna
> - url: https://zenodo.org/records/582600/files/wildtype.fna
> src: url
> ext: fasta
> info: https://doi.org/10.5281/zenodo.582600
> - name: A directory
> description: Exome sequencing means that all protein-coding genes in a genome are
> items:
> - url: https://zenodo.org/api/files/287966da-5411-4f79-8cfb-0ffa84d0d6cc/mutant_R1.fastq
> - url: https://zenodo.org/records/582600/files/mutant_R1.fastq
> src: url
> ext: fastqsanger
> info: https://doi.org/10.5281/zenodo.582600
> - url: https://zenodo.org/api/files/287966da-5411-4f79-8cfb-0ffa84d0d6cc/mutant_R2.fastq
> - url: https://zenodo.org/records/582600/files/mutant_R2.fastq
> src: url
> ext: fastqsanger
> info: https://doi.org/10.5281/zenodo.582600
Expand Down
10 changes: 5 additions & 5 deletions topics/assembly/tutorials/flye-assembly/tutorial.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: tutorial_hands_on

title: 'Genome assembly using PacBio data'
zenodo_link: 'https://zenodo.org/record/5702408#.YZUb5uvjIiU'
zenodo_link: 'https://zenodo.org/record/5702408'
tags:
- assembly
- pacbio
Expand Down Expand Up @@ -58,12 +58,12 @@ We will use long reads sequencing data: CLR (continuous long reads) from PacBio
> <hands-on-title>Data upload from Zenodo</hands-on-title>
>
> 1. Create a new history for this tutorial
> 2. Import the files from [Zenodo](https://zenodo.org/record/5702408)
> 2. Import the files from [Zenodo]({{ page.zenodo_link }})
>
> ```
> https://zenodo.org/api/files/d010d8f1-a1fd-4366-991f-916c2f0c55db/SRR8534473_subreads.fastq.gz
> https://zenodo.org/api/files/d010d8f1-a1fd-4366-991f-916c2f0c55db/SRR8534474_subreads.fastq.gz
> https://zenodo.org/api/files/d010d8f1-a1fd-4366-991f-916c2f0c55db/SRR8534475_subreads.fastq.gz
> https://zenodo.org/records/5702408/files/SRR8534473_subreads.fastq.gz
> https://zenodo.org/records/5702408/files/SRR8534474_subreads.fastq.gz
> https://zenodo.org/records/5702408/files/SRR8534475_subreads.fastq.gz
> ```
>
> {% snippet faqs/galaxy/datasets_import_via_link.md %}
Expand Down
2 changes: 1 addition & 1 deletion topics/contributing/tutorials/create-new-tutorial-technical/tutorial.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ items:
description: latest
items:
- info: https://doi.org/10.5281/zenodo....
url: https://zenodo.org/api/files/URL/to/the/input/file
url: https://zenodo.org/records/URL/files/path/to/input
ext: galaxy-datatype
src: url
```
Expand Down
52 changes: 26 additions & 26 deletions topics/galaxy-interface/tutorials/upload-rules/tutorial.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ redirect_from:
- /topics/galaxy-data-manipulation/tutorials/upload-rules/tutorial

title: "Rule Based Uploader"
zenodo_link: ""
zenodo_link: "https://zenodo.org/records/3263975"
level: Intermediate
questions:
- "How to use the rule based uploader to create complex collections"
Expand Down Expand Up @@ -86,12 +86,12 @@ You can select all the data below and copy them to your clipboard.

```
study_accession sample_accession experiment_accession fastq_ftp
PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000770.fastqsanger.gz
PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000771.fastqsanger.gz
PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000772.fastqsanger.gz
PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000773.fastqsanger.gz
PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000774.fastqsanger.gz
PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000775.fastqsanger.gz
PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/records/3263975/files/DRR000770.fastqsanger.gz
PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/records/3263975/files/DRR000771.fastqsanger.gz
PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/records/3263975/files/DRR000772.fastqsanger.gz
PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/records/3263975/files/DRR000773.fastqsanger.gz
PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/records/3263975/files/DRR000774.fastqsanger.gz
PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/records/3263975/files/DRR000775.fastqsanger.gz
```
{:#example-1-metadata}

Expand All @@ -111,12 +111,12 @@ PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/api/files/c07c0fbd-c578-4c8
>
> ```
> study_accession sample_accession experiment_accession fastq_ftp
> PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000770.fastqsanger.gz
> PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000771.fastqsanger.gz
> PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000772.fastqsanger.gz
> PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000773.fastqsanger.gz
> PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000774.fastqsanger.gz
> PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000775.fastqsanger.gz
> PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/records/3263975/files/DRR000770.fastqsanger.gz
> PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/records/3263975/files/DRR000771.fastqsanger.gz
> PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/records/3263975/files/DRR000772.fastqsanger.gz
> PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/records/3263975/files/DRR000773.fastqsanger.gz
> PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/records/3263975/files/DRR000774.fastqsanger.gz
> PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/records/3263975/files/DRR000775.fastqsanger.gz
> ```
>
> ![screenshot](../../images/rules/rules_example_1_2_paste.png)
Expand Down Expand Up @@ -191,12 +191,12 @@ This example will demonstrate using such history datasets as the source for coll
>
> ```
> study_accession sample_accession experiment_accession fastq_ftp
> PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000770.fastqsanger.gz
> PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000771.fastqsanger.gz
> PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000772.fastqsanger.gz
> PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000773.fastqsanger.gz
> PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000774.fastqsanger.gz
> PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/api/files/c07c0fbd-c578-4c8c-989d-5181f8824773/DRR000775.fastqsanger.gz
> PRJDA60709 SAMD00016379 DRX000475 https://zenodo.org/records/3263975/files/DRR000770.fastqsanger.gz
> PRJDA60709 SAMD00016383 DRX000476 https://zenodo.org/records/3263975/files/DRR000771.fastqsanger.gz
> PRJDA60709 SAMD00016380 DRX000477 https://zenodo.org/records/3263975/files/DRR000772.fastqsanger.gz
> PRJDA60709 SAMD00016378 DRX000478 https://zenodo.org/records/3263975/files/DRR000773.fastqsanger.gz
> PRJDA60709 SAMD00016381 DRX000479 https://zenodo.org/records/3263975/files/DRR000774.fastqsanger.gz
> PRJDA60709 SAMD00016382 DRX000480 https://zenodo.org/records/3263975/files/DRR000775.fastqsanger.gz
> ```
>
> 2. **Open** the `Rule-based` upload tab again, but this time:
Expand Down Expand Up @@ -248,13 +248,13 @@ For this next example we will again use ENA data, this time corresponding to the
>
> ```
> study_accession sample_accession experiment_accession fastq_ftp
> PRJDB3920 SAMD00034150 DRX036147 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036147_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036147_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036148 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036148_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036148_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036149 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036149_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036149_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036150 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036150_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036150_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036151 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036151_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036151_2.fastq.gz
> PRJDB3920 SAMD00034153 DRX036152 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036152_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036152_2.fastq.gz
> PRJDB3920 SAMD00034152 DRX036164 https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036164_1.fastq.gz;https://zenodo.org/api/files/0c8d6a74-23dd-4d36-a32d-950eb038dacc/DRX036164_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036147 https://zenodo.org/records/3263975/files/DRX036147_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036147_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036148 https://zenodo.org/records/3263975/files/DRX036148_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036148_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036149 https://zenodo.org/records/3263975/files/DRX036149_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036149_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036150 https://zenodo.org/records/3263975/files/DRX036150_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036150_2.fastq.gz
> PRJDB3920 SAMD00034150 DRX036151 https://zenodo.org/records/3263975/files/DRX036151_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036151_2.fastq.gz
> PRJDB3920 SAMD00034153 DRX036152 https://zenodo.org/records/3263975/files/DRX036152_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036152_2.fastq.gz
> PRJDB3920 SAMD00034152 DRX036164 https://zenodo.org/records/3263975/files/DRX036164_1.fastq.gz;https://zenodo.org/records/3263975/files/DRX036164_2.fastq.gz
> ```
>
>
Expand Down
6 changes: 3 additions & 3 deletions topics/genome-annotation/tutorials/apollo-euk/tutorial.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -110,9 +110,9 @@ In this tutorial we use the same data as in the [Funannotate](../funannotate/tut
> -> `{{ page.title }}`):
>
> ```
> https://zenodo.org/api/files/042ff43f-cd53-45ec-90cf-781fe53e3771/genome.fasta
> https://zenodo.org/api/files/042ff43f-cd53-45ec-90cf-781fe53e3771/annotation.gff3
> https://zenodo.org/api/files/042ff43f-cd53-45ec-90cf-781fe53e3771/rnaseq.bam
> https://zenodo.org/record/3270822/files/genome.fasta
> https://zenodo.org/record/3270822/files/annotation.gff3
> https://zenodo.org/record/3270822/files/rnaseq.bam
> ```
>
> {% snippet faqs/galaxy/datasets_import_via_link.md %}
Expand Down
32 changes: 16 additions & 16 deletions topics/genome-annotation/tutorials/apollo/tutorial.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: tutorial_hands_on

title: Refining Genome Annotations with Apollo (prokaryotes)
zenodo_link: https://zenodo.org/record/3270822
zenodo_link: https://zenodo.org/record/4889110
tags:
- prokaryote
questions:
Expand Down Expand Up @@ -106,21 +106,21 @@ In this tutorial we have obtained some data from NCBI related to [*Escherichia c
> 3. Copy & Paste the following table into the Rule-based uploader textbox:
>
> ```
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/augustus.gff3 Augustus gff3
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/blastp_vs_swissprot_2018-01-22.blastxml Blastp vs swissprot blastxml
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/BWA-MEM_K12_Coverage.bigwig BWA-MEM K12 Coverage bigwig
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/BWA-MEM_K12_Mapping.bam BWA-MEM K12 Mapping bam
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/BWA-MEM_O104_Coverage.bigwig BWA-MEM O104 Coverage bigwig
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/BWA-MEM_O104_Mapping.bam BWA-MEM O104 Mapping bam
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/E._coli_str_K-12_substr_MG1655_100kb_subset.fasta Genome fasta
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/K12_Variants.vcf K12 Variants vcf
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/NCBI_AnnotWriter_Genes.gff3 NCBI AnnotWriter Genes gff3
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/O104_H4_LASTZ_Alignment.bed O104 H4 LASTZ Alignment bed
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/O104_Variants.vcf O104 Variants vcf
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/TopHat_SRR1927169_rep1.bam TopHat SRR1927169 rep1 bam
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/TopHat_SRR1927169_rep1_Coverage.bigwig TopHat SRR1927169 rep1 Coverage bigwig
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/TopHat_SRR1927170_rep2.bam TopHat SRR1927170 rep2 bam
> https://zenodo.org/api/files/6080d6d6-68ec-4ade-afef-bbd83f9b5e2b/TopHat_SRR1927170_rep2_Coverage.bigwig TopHat SRR1927170 rep2 Coverage bigwig
> https://zenodo.org/records/4889110/files/augustus.gff3 Augustus gff3
> https://zenodo.org/records/4889110/files/blastp_vs_swissprot_2018-01-22.blastxml Blastp vs swissprot blastxml
> https://zenodo.org/records/4889110/files/BWA-MEM_K12_Coverage.bigwig BWA-MEM K12 Coverage bigwig
> https://zenodo.org/records/4889110/files/BWA-MEM_K12_Mapping.bam BWA-MEM K12 Mapping bam
> https://zenodo.org/records/4889110/files/BWA-MEM_O104_Coverage.bigwig BWA-MEM O104 Coverage bigwig
> https://zenodo.org/records/4889110/files/BWA-MEM_O104_Mapping.bam BWA-MEM O104 Mapping bam
> https://zenodo.org/records/4889110/files/E._coli_str_K-12_substr_MG1655_100kb_subset.fasta Genome fasta
> https://zenodo.org/records/4889110/files/K12_Variants.vcf K12 Variants vcf
> https://zenodo.org/records/4889110/files/NCBI_AnnotWriter_Genes.gff3 NCBI AnnotWriter Genes gff3
> https://zenodo.org/records/4889110/files/O104_H4_LASTZ_Alignment.bed O104 H4 LASTZ Alignment bed
> https://zenodo.org/records/4889110/files/O104_Variants.vcf O104 Variants vcf
> https://zenodo.org/records/4889110/files/TopHat_SRR1927169_rep1.bam TopHat SRR1927169 rep1 bam
> https://zenodo.org/records/4889110/files/TopHat_SRR1927169_rep1_Coverage.bigwig TopHat SRR1927169 rep1 Coverage bigwig
> https://zenodo.org/records/4889110/files/TopHat_SRR1927170_rep2.bam TopHat SRR1927170 rep2 bam
> https://zenodo.org/records/4889110/files/TopHat_SRR1927170_rep2_Coverage.bigwig TopHat SRR1927170 rep2 Coverage bigwig
> ```
> 4. Click **Build**
>
Expand Down
Loading

0 comments on commit 2752a66

Please sign in to comment.