[WIP] Integration of ProteoRE galaxy components in Galaxy-P #535

vloux · 2021-01-06T17:37:56Z

Hello Galaxy-P team,
all the best for this new year.
As discussed in usegalaxy-eu/usegalaxy-eu-tools#289 , I transfered proteore components from http://github.com/ifb-git/ProteoRE to the Galaxy-P repository.

This consists in 19 tools :

proteore_id_converter
proteore_filter_keywords_values
proteore_venn_diagram
proteore_heatmap_visualization
proteore_prot_features_3orga
proteore_prot_features
proteore_expression_levels_by_tissue
proteore_expression_rnaseq_abbased
proteore_tissue_specific_expression_data
proteore_ms_observation_pepatlas
proteore_get_unique_peptide_srm_method
proteore_goprofiles
proteore_clusterprofiler
proteore_go_terms_enrich_comparison
proteore_topgo
proteore_reactome
proteore_maps_visualization
proteore_kegg_pathways_coverage
proteore_build_protein_interaction_maps

and one data_manager :

proteore_data_manager

I made one "big" PR with all the components, but tell me if you prefer one PR by tool.
Thank you for the review !

big modif for the make_dotplot function import of 2 functions from clusterProfiler . Goal : length of the y-legend of the dotplot is ok

UserDoc section now written - please visually check format and read carefully content - before deploiement on dev-migale

… keys

transmb domains nb in nextprot 2019 was wrong. correction og this, see "get all tm domain" in the code

version number update caus change in the pyhton associated script

ERROR: Invalid XML found in fille: ERROR:SCHEMASV:SCHEMAV_ELEMENT_CONTENT: Element 'requirement': This element is not expected.]

bgruening

Hi all!

What a great start. I have included some very high level and general comments inline.

A few more suggestions and hints here:

are those args.rds files needed?
tools/proteore_clusterprofiler/test-data/GGO_MF_bar-plot --> what are those files?
tools/proteore_clusterprofiler/tool_test_output.html --> should be removed.
tools/proteore_filter_keywords_values/tool_test_output.json --> should be removed
maybe it's better to create separate PRs for every tool? That could help to review and getting single tools accepted sooner.

Thanks a lot,
Bjoern

bgruening · 2021-01-07T14:16:29Z

tools/proteore_data_manager/tool_dependencies.xml

@@ -0,0 +1,3 @@
+<?xml version="1.0"?>
+<tool_dependency>


this should be removed I think

bgruening · 2021-01-07T14:17:01Z

tools/proteore_expression_levels_by_tissue/.shed.yml

+long_description: A tool for finding tissue in which your proteins are expressed (or not) - Human Protein Atlas
+name: proteore_expression_levels_by_tissue
+owner: proteore
+include:


I think you can remove the include. Or is there any need for it?

bgruening · 2021-01-07T14:18:23Z

tools/proteore_expression_levels_by_tissue/get_expression_profiles.xml

+        <requirement type="package" version="3.6.1">r-base</requirement>
+    <requirements>
+    </requirements>
+    <stdio>


this simple stdio should be relaced by detect_errors on the command tag.

bgruening · 2021-01-07T14:18:50Z

tools/proteore_expression_levels_by_tissue/get_expression_profiles.xml

+    <stdio>
+        <exit_code range="1:" />
+    </stdio>
+    <command interpreter="Rscript">


interpreter is deprecated, you should use Rscript in the command section.

bgruening · 2021-01-07T14:19:26Z

tools/proteore_expression_levels_by_tissue/get_expression_profiles.xml

+        <exit_code range="1:" />
+    </stdio>
+    <command interpreter="Rscript">
+        $__tool_directory__/get_expression_profiles.R


single quotes around all paths, e.g. '$__tool_directory__/get_expression_profiles.R'

bgruening · 2021-01-07T14:48:57Z

tools/proteore_prot_features/add_protein_features.xml

+      </param>
+    </when>
+    <when value="file">
+      <param name="genelist" type="data" format="txt,tabular" label="Select your file" help=""/>


labels should be improved ... like Please select your genelist file

bgruening · 2021-01-07T14:49:34Z

tools/proteore_prot_features/add_protein_features.xml

+    </when>
+    <when value="file">
+      <param name="genelist" type="data" format="txt,tabular" label="Select your file" help=""/>
+      <param name="column" type="text" label="Column IDs (e.g : Enter c1 for column n°1)" value="c1">


can this be of type="data_column" ?

bgruening · 2021-01-07T14:50:16Z

tools/proteore_reactome/reactome_analysis.xml

+    <description>[Reactome]</description>
+    <requirements>
+    </requirements>
+    <stdio>


use detect_errors

bgruening · 2021-01-07T14:50:25Z

tools/proteore_reactome/reactome_analysis.xml

+    </stdio>
+    <command><![CDATA[
+
+        python $__tool_directory__/reactome_analysis.py 


single-quotes

bgruening · 2021-01-07T14:52:00Z

tools/proteore_venn_diagram/venn_diagram.py

@@ -0,0 +1,187 @@
+#!/usr/bin/env python2.7


python 2 is depreated ...

Can we replace this tool wth https://github.com/peterjc/pico_galaxy/blob/master/tools/venn_list/venn_list.xml?

bernt-matthias · 2021-01-11T10:05:16Z

I would suggest to put all those tools in a single directory. Makes it easier to use macros and also simplifies the shed.yml file.

bernt-matthias

I would also prefer a tool by tool review. This gigantic effort seem hard to manage for reviewers. I guess we can move forward faster and more focused this way.

bernt-matthias · 2021-01-13T14:42:21Z

tools/proteore_build_protein_interaction_maps/build_protein_interaction_maps.xml

@@ -0,0 +1,311 @@
+<tool id="build_protein_interaction_maps" name="Build Protein interaction network" version="2020.02.06">


Please use a recent profile version of the tools, e.g. profile="20.01"

bernt-matthias · 2021-01-13T14:44:00Z

tools/proteore_build_protein_interaction_maps/build_protein_interaction_maps.xml

+            <when value="file" >
+                <param name="file" type="data" format="txt,tabular" label="Select your file" help="" />
+                <param name="header" type="boolean" checked="true" truevalue="true" falsevalue="false" label="Does file contain header?" />
+                <param name="ncol" type="text" value="c1" label="Column number of IDs to map" help='For example, fill in "c1" if it is the first column, "c2" if it is the second column and so on'>


This could be a type="data_column" parameter.

bernt-matthias · 2021-01-13T14:56:42Z

tools/proteore_build_protein_interaction_maps/build_protein_interaction_maps.xml

+            <conditional name="database">
+                <param name="ref" value="bioplex"/>
+                <param name="id_type" value="UniProt-AC"/>
+                <param name="ref_file" value="tool-data/Human_bioplex_2019-03-01"/>


You should restructure the loc files:

Instead of the columns: id, release, name, species, value you should rename id to value (guessing that id is a unique identifier) and value to path. Then in cheetah you can access the path with $ref_file.fields.path and in the test you can specify the id.

Also the test data (Human_bioplex_2019-03-01, ...) shoud be in test-data. tool-data should contain loc.sample files

bernt-matthias · 2021-01-13T15:00:31Z

tools/proteore_data_manager/data_manager/resource_building.xml

+    <data name="output" format="data_manager_json"/>
+</outputs>
+
+<tests>


Tests are needed.

vloux · 2021-01-13T16:14:23Z

Thank you Bjoern and Mathias for your helpful comments !
I realize that we have quite a lot of polishing to do on all these tools.
Indeed I will withdraw this PR and, after correction, manage to do one PR for each tool.
This will be really simpler to manage for all.

bernt-matthias · 2021-01-13T16:18:04Z

after correction, manage to do one PR for each tool

please feel free to just open the PRs before polishing .. I guess we can help a bit with early comments.

Please note that I'm working on kegg_pathway_visualizations .. because one of my users just sent in an error report :) I will PR to your repo.

bernt-matthias · 2021-01-13T17:35:00Z

@vloux are you on gitter? I would have qustions regarding kegg_pathway_visualizations

foellmelanie · 2021-06-25T15:00:48Z

Great to see that you started migrating your tools :)

For the following tools you could consider using already existing and installed on EU server Galaxy tools:

proteore_id_converter: https://toolshed.g2.bx.psu.edu/repository?repository_id=c8774310981b07c5

proteore_filter_keywords_values: https://usegalaxy.eu/root?tool_id=Grep1 and https://usegalaxy.eu/root?tool_id=Filter1, I cannot find them in the toolshed but I think they come automatically with each Galaxy installation.

proteore_venn_diagram: https://toolshed.g2.bx.psu.edu/repository?repository_id=ccb47ef9d02bfc83

HTH,
Melanie

yvandenb and others added 30 commits March 7, 2019 14:23

Update add_expression_data.xml

3c43825

modif for the make_dotplot function

b07edb1

big modif for the make_dotplot function import of 2 functions from clusterProfiler . Goal : length of the y-legend of the dotplot is ok

Merge branch 'master' of https://github.com/ifb-git/ProteoRE

be4872f

typo

29742cd

add expression data: copy paste ids fix

fd41122

typo again

70623e3

Merge branch 'master' of https://github.com/ifb-git/ProteoRE

77365da

add expression data: small fix

dc43865

Update build_protein_interaction_maps.xml

ef1fc6c

UserDoc section now written - please visually check format and read carefully content - before deploiement on dev-migale

build protein interaction: humap dictionary modified

6a1da10

data manager: humap dictionary now incorporate interactant A and B as…

a6106dd

… keys

Merge branch 'master' of https://github.com/ifb-git/ProteoRE

723ec8e

build protein interaction maps: add tool-data

fe64568

data manager: nextprot ref file for add protein features

c9adb94

add protein features: handle data manager update

7e06dde

add protein features: loc file corrected

c0511de

filter tool: add error message for wrong column

7a401bd

add protein features: nextprotID removed fixed

f010543

add protein features: nextprot ref of 2018

8e7f43d

data manager: small fix

ca41f64

Update build_protein_interaction_maps.xml

07862d0

build protein interaction maps: protein name column for nodes file added

a78dfe4

protein name dictionary added for humap dictionary

54c8cac

data manager documentation

251507c

get expression profiles: loc file corrected

b280b5a

version number

fe9749e

filter tool: output 'discarded' en deuxième position

1befc97

venn diagram: force user to give list names

b0d4004

Bug correction of transmb domains nb

e841cdb

transmb domains nb in nextprot 2019 was wrong. correction og this, see "get all tm domain" in the code

Update resource_building.xml

55ac698

version number update caus change in the pyhton associated script

vloux and others added 18 commits January 6, 2021 16:45

rename proteore_prot_features

9f06aa7

rename proteore_expression_levels_by_tissue

13c8332

rename proteore_expression_rnaseq_abbased

f17ade3

rename proteore_tissue_specific_expression_data

68b80a2

rename proteore_ms_observation_pepatlas

2f6ad8d

rename proteore_get_unique_peptide_srm_method

2f9b4e7

rename proteore_goprofiles

ecae379

rename proteore_clusterprofiler

c3948d5

rename proteore_go_terms_enrich_comparison

245b5e6

rename proteore_reactome

bba4f4a

rename proteore_maps_visualization

1fee028

rename proteore_kegg_pathways_coverage

2b3f2c9

rename proteore_build_protein_interaction_maps

e1aafc0

rename proteore_data_manager

b47b48d

rename proteore_topgo

0864679

cleaning

fc91623

Merge branch 'ProteoRE'

69df8ca

merge ProteoRE

9463d32

vloux self-assigned this Jan 6, 2021

Update get_expression_profiles.xml

0e700e5

ERROR: Invalid XML found in fille: ERROR:SCHEMASV:SCHEMAV_ELEMENT_CONTENT: Element 'requirement': This element is not expected.]

vloux changed the title ~~Integration of ProteoRE galaxy components in Galaxy-P~~ [WIP] Integration of ProteoRE galaxy components in Galaxy-P Jan 7, 2021

vloux added the wip label Jan 7, 2021

bgruening reviewed Jan 7, 2021

View reviewed changes

bernt-matthias reviewed Jan 13, 2021

View reviewed changes

combesf mentioned this pull request May 10, 2021

[WIP] ProteoRE 'filter kw and values' tool - Version 2021.06.01 #590

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Integration of ProteoRE galaxy components in Galaxy-P #535

[WIP] Integration of ProteoRE galaxy components in Galaxy-P #535

vloux commented Jan 6, 2021

bgruening left a comment

bgruening Jan 7, 2021

bgruening Jan 7, 2021

bgruening Jan 7, 2021

bgruening Jan 7, 2021

bgruening Jan 7, 2021

bgruening Jan 7, 2021

bgruening Jan 7, 2021

bgruening Jan 7, 2021

bgruening Jan 7, 2021

bgruening Jan 7, 2021

bernt-matthias commented Jan 11, 2021

bernt-matthias left a comment

bernt-matthias Jan 13, 2021

bernt-matthias Jan 13, 2021

bernt-matthias Jan 13, 2021

bernt-matthias Jan 13, 2021

bernt-matthias Jan 13, 2021

vloux commented Jan 13, 2021

bernt-matthias commented Jan 13, 2021

bernt-matthias commented Jan 13, 2021

foellmelanie commented Jun 25, 2021

		@@ -0,0 +1,311 @@
		<tool id="build_protein_interaction_maps" name="Build Protein interaction network" version="2020.02.06">

[WIP] Integration of ProteoRE galaxy components in Galaxy-P #535

Are you sure you want to change the base?

[WIP] Integration of ProteoRE galaxy components in Galaxy-P #535

Conversation

vloux commented Jan 6, 2021

bgruening left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bernt-matthias commented Jan 11, 2021

bernt-matthias left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vloux commented Jan 13, 2021

bernt-matthias commented Jan 13, 2021

bernt-matthias commented Jan 13, 2021

foellmelanie commented Jun 25, 2021