Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[24.1] xsd: allow change_format and actions also in statically defined collection elements, and break recursion #18605

Merged
merged 11 commits into from
Jul 31, 2024
Merged

[24.1] xsd: allow change_format and actions also in statically defined collection elements, and break recursion #18605

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
c74194c
allow change_format and actions in all data elements
bernt-matthias Jul 28, 2024
3b3a65d
extend tool test for change_format in statically defined collection e…
bernt-matthias Jul 29, 2024
8b8791a
xsd: add OutputDiscoverDatasetsCommon
bernt-matthias Jul 29, 2024
bc8d646
XSD: break recursion in collection data
bernt-matthias Jul 29, 2024
001faf7
also check schema doc template
bernt-matthias Jul 29, 2024
85d4b42
add test for output actions for collection elements
bernt-matthias Jul 29, 2024
7b09725
add test for filters in statically defined collection elements
bernt-matthias Jul 29, 2024
eeab749
Revert "add test for filters in statically defined collection elements"
bernt-matthias Jul 30, 2024
3cdbb8f
test actions in statically defined collection elements
bernt-matthias Jul 30, 2024
8310bf2
xsd: do not allow filter and discover_datasets in statically defined …
bernt-matthias Jul 30, 2024
262b4b6
remove test for actions from collection_creates_pair_format
bernt-matthias Jul 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/schema_template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ $tag:tool|outputs|data|actions|action://complexType[@name='Action']
$tag:tool|outputs|data|actions|action|option://complexType[@name='ActionsOption']
$tag:tool|outputs|data|discover_datasets://complexType[@name='OutputDiscoverDatasets']
$tag:tool|outputs|collection://complexType[@name='OutputCollection']
$tag:tool|outputs|collection|data://complexType[@name='OutputCollectionDataElement']
$tag:tool|outputs|collection|data://complexType[@name='OutputCollectionData']
$tag:tool|outputs|collection|filter://complexType[@name='OutputFilter']
$tag:tool|outputs|collection|discover_datasets://complexType[@name='OutputCollectionDiscoverDatasets']
$tag:tool|tests://complexType[@name='Tests']
Expand Down
99 changes: 34 additions & 65 deletions lib/galaxy/tool_util/xsd/galaxy.xsd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5827,15 +5827,24 @@ on Human (hg18)``.
<xs:attributeGroup ref="OutputDataAttributes"/>
</xs:complexType>

<!-- Allowed tags included in collection-data -->
<xs:group name="OutputCollectionDataElement">
<xs:choice>
<xs:element name="actions" type="Actions" />
<xs:element name="change_format" type="ChangeFormat" />
</xs:choice>
</xs:group>

<!-- Allowed tags included in collection -->
<xs:group name="OutputCollectionElement">
<xs:choice>
<xs:element name="data" type="OutputCollectionDataElement" />
<xs:element name="data" type="OutputCollectionData" />
<xs:element name="discover_datasets" type="OutputCollectionDiscoverDatasets" />
<xs:element name="filter" type="OutputFilter" />
</xs:choice>
</xs:group>

<xs:complexType name="OutputCollectionDataElement">
<xs:complexType name="OutputCollectionData">
<xs:annotation>
<xs:documentation xml:lang="en"><![CDATA[

Expand All @@ -5844,7 +5853,7 @@ define the elements of a collection statically. See also [Planemo's documentatio
]]></xs:documentation>
</xs:annotation>
<xs:sequence>
<xs:group ref="OutputCollectionElement" minOccurs="0" maxOccurs="unbounded" />
<xs:group ref="OutputCollectionDataElement" minOccurs="0" maxOccurs="unbounded" />
</xs:sequence>
<xs:attributeGroup ref="OutputCommon" />
<xs:attributeGroup ref="OutputDataAttributes" />
Expand Down Expand Up @@ -6002,23 +6011,7 @@ Therefore a filter for such a variable looks like the following example.
</xs:simpleContent>
</xs:complexType>

<xs:complexType name="OutputDiscoverDatasets">
<xs:annotation>
<xs:documentation xml:lang="en"><![CDATA[

Describe datasets to dynamically collect after the job complete.

There are many simple tools with examples of this element distributed with
Galaxy, including:

* [multi_output.xml](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/multi_output.xml)
* [multi_output_assign_primary.xml](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/multi_output_assign_primary.xml)
* [multi_output_configured.xml](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/multi_output_configured.xml)

More information can be found on Planemo's documentation for
[multiple output files](https://planemo.readthedocs.io/en/latest/writing_advanced.html#multiple-output-files).
]]></xs:documentation>
</xs:annotation>
<xs:attributeGroup name="OutputDiscoverDatasetsCommon">
<xs:attribute name="from_provided_metadata" type="xs:boolean" use="optional">
<xs:annotation>
<xs:documentation xml:lang="en">Indicate that dataset filenames should simply be read from the provided metadata file (e.g. galaxy.json). If this is set - pattern and sort must not be set.</xs:documentation>
Expand Down Expand Up @@ -6069,6 +6062,26 @@ More information can be found on Planemo's documentation for
<xs:documentation xml:lang="en">Indication if this dataset is visible in output history. This defaults to ``false``, but probably shouldn't - be sure to set to ``true`` if that is your intention.</xs:documentation>
</xs:annotation>
</xs:attribute>
</xs:attributeGroup>

<xs:complexType name="OutputDiscoverDatasets">
<xs:annotation>
<xs:documentation xml:lang="en"><![CDATA[

Describe datasets to dynamically collect after the job complete.

There are many simple tools with examples of this element distributed with
Galaxy, including:

* [multi_output.xml](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/multi_output.xml)
* [multi_output_assign_primary.xml](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/multi_output_assign_primary.xml)
* [multi_output_configured.xml](https://github.com/galaxyproject/galaxy/blob/dev/test/functional/tools/multi_output_configured.xml)

More information can be found on Planemo's documentation for
[multiple output files](https://planemo.readthedocs.io/en/latest/writing_advanced.html#multiple-output-files).
]]></xs:documentation>
</xs:annotation>
<xs:attributeGroup ref="OutputDiscoverDatasetsCommon"/>
<xs:attribute name="assign_primary_output" type="xs:boolean" use="optional">
<xs:annotation>
<xs:documentation xml:lang="en">Replace the primary dataset described by the parameter ``data`` parameter with the first output discovered.</xs:documentation>
Expand All @@ -6092,51 +6105,7 @@ Galaxy, including:

]]></xs:documentation>
</xs:annotation>
<xs:attribute name="from_provided_metadata" type="xs:boolean" use="optional">
<xs:annotation>
<xs:documentation xml:lang="en">Indicate that dataset filenames should simply be read from the provided metadata file (e.g. galaxy.json). If this is set - pattern and sort_by must not be set.</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="pattern" type="xs:string" use="optional">
<xs:annotation>
<xs:documentation xml:lang="en">Regular expression used to find filenames and parse dynamic properties.</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="directory" type="xs:string" use="optional">
<xs:annotation>
<xs:documentation xml:lang="en">Directory (relative to working directory) to search for files.</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="recurse" type="xs:boolean" use="optional">
<xs:annotation>
<xs:documentation xml:lang="en">Indicates that the specified directory should be searched recursively for matching files.</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="match_relative_path" type="xs:boolean" use="optional">
<xs:annotation>
<xs:documentation xml:lang="en">Indicates that the entire path of the discovered dataset relative to the specified directory should be available for matching patterns.</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="format" type="xs:string" use="optional">
<xs:annotation>
<xs:documentation xml:lang="en">Format (or datatype) of discovered datasets (an alias with ``ext``).</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="ext" type="xs:string" use="optional">
<xs:annotation>
<xs:documentation xml:lang="en">Format (or datatype) of discovered datasets (an alias with ``format``).</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="sort_by" type="xs:string" use="optional">
<xs:annotation>
<xs:documentation xml:lang="en">A string `[reverse_][SORT_COMP_]SORTBY` describing the desired sort order of the collection elements. `SORTBY` can be `filename`, `name`, `designation`, `dbkey` and the optional `SORT_COMP` can be either `lexical` or `numeric`. Default is lexical sorting by filename.</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attribute name="visible" type="xs:boolean" use="optional">
<xs:annotation>
<xs:documentation xml:lang="en">Indication if this dataset is visible in the history. This defaults to ``false``, but probably shouldn't - be sure to set to ``true`` if that is your intention.</xs:documentation>
</xs:annotation>
</xs:attribute>
<xs:attributeGroup ref="OutputDiscoverDatasetsCommon"/>
</xs:complexType>
<xs:complexType name="Actions">
<xs:annotation>
Expand Down
44 changes: 42 additions & 2 deletions test/functional/tools/collection_creates_pair_format.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,28 @@
</command>
<inputs>
<param name="input1" type="data" label="Input" help="Input to be split." />
<param name="out_format" type="select" label="Output data type">
<option value="">do not change</option>
<option value="interval">interval</option>
</param>
</inputs>
<outputs>
<!-- format is specified separately for the elements, i.e. default format (fasta in this case) is ignored -->
<collection name="paired_output" format="fasta" type="paired" label="Split Pair">
<data name="forward" format="txt" />
<data name="forward" format="txt">
<change_format>
<when input="out_format" value="interval" format="interval" />
</change_format>
</data>
<data name="reverse" format_source="input1" from_work_dir="reverse.txt" />
</collection>
<!-- no format is specified separately for the elements, i.e. default format (fasta in this case) is used -->
<collection name="paired_output_default_format" format="fasta" type="paired" label="Split Pair">
<data name="forward" />
<data name="forward">
<change_format>
<when input="out_format" value="interval" format="interval" />
</change_format>
</data>
<data name="reverse" from_work_dir="reverse.txt" />
</collection>
</outputs>
Expand Down Expand Up @@ -48,5 +60,33 @@
</element>
</output_collection>
</test>
<test>
<param name="input1" ftype="bed" value="simple_lines_interleaved.txt" />
<param name="out_format" value="interval"/>
<output_collection name="paired_output" type="paired">
<element name="forward" ftype="interval">
<assert_contents>
<has_text_matching expression="^This is a line of text.\nThis is a line of text.\n$" />
</assert_contents>
</element>
<element name="reverse" ftype="bed">
<assert_contents>
<has_text_matching expression="^This is a different line of text.\nThis is a different line of text.\n$" />
</assert_contents>
</element>
</output_collection>
<output_collection name="paired_output_default_format" type="paired">
<element name="forward" ftype="interval">
<assert_contents>
<has_text_matching expression="^This is a line of text.\nThis is a line of text.\n$" />
</assert_contents>
</element>
<element name="reverse" ftype="fasta">
<assert_contents>
<has_text_matching expression="^This is a different line of text.\nThis is a different line of text.\n$" />
</assert_contents>
</element>
</output_collection>
</test>
</tests>
</tool>
24 changes: 23 additions & 1 deletion test/functional/tools/metadata_column_names.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
<tool id="metadata_columns" name="metadata_columns" version="1.0.0">
<description>Tests whether metadata is being set correctly.</description>
<command><![CDATA[
cp '$input' '$output'
cp '$input' '$output';
cp '$input' '$paired_output.forward' ;
cp '$input' '$paired_output.reverse' ;
]]></command>
<inputs>
<param name="input" type="data" multiple="false" />
Expand All @@ -12,13 +14,33 @@
<action name="column_names" type="metadata" default="First,${input.name}" />
</actions>
</data>
<collection name="paired_output" format="tabular" type="paired" label="Split Pair">
<data name="forward">
<actions>
<action name="dbkey" type="metadata" default="hg38" />
</actions>
</data>
<data name="reverse">
<actions>
<action name="column_names" type="metadata" default="A,B,C" />
</actions>
</data>
</collection>
</outputs>
<tests>
<test>
<param name="input" value="2.tabular" />
<output name="output">
<metadata name="column_names" value="First,2.tabular"/>
</output>
<output_collection name="paired_output" type="paired">
<element name="forward" ftype="tabular" value="2.tabular">
<metadata name="dbkey" value="hg38"/>
</element>
<element name="reverse" ftype="tabular" value="2.tabular">
<metadata name="column_names" value="A,B,C"/>
</element>
</output_collection>
</test>
</tests>
</tool>
Loading