Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update SVG task dataset ids #908

Merged
merged 4 commits into from
Oct 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
## Minor changes

- Add the CELLxGENE immune cell atlas dataset as a common test resource (PR #907)
- Update `dataset_id` for `tenx_visium`, `zenodo_spatial`, `zenodo_spatial_slidetags` datasets and use `mouse_brain_coronal` as a test resource in the `spatially_variable_genes` task (PR #908)

# openproblems v2.0.0

Expand All @@ -25,7 +26,7 @@ Most relevant parts of the overall structure:
- `openproblems_neurips2022_pbmc`: Fetch a dataset from the OpenProblems NeurIPS2022 competition
- `openproblems_v1`: Fetch a legacy OpenProblems v1 dataset
- `openproblems_v1_multimodal`: Fetch a legacy OpenProblems v1 multimodal dataset
- `tenx_vision`: Fetch a and convert 10x Visium dataset
- `tenx_visium`: Fetch a and convert 10x Visium dataset
- `zenodo_spatial`: Fetch and process an Anndata file containing DBiT seq, MERFISH, seqFISH, Slide-seq v2, STARmap, and Stereo-seq data from Zenodo.
- `zenodo_spatial_slidetags`: Download a compressed file containing gene expression matrix and spatial locations from zenodo.

Expand Down
2 changes: 1 addition & 1 deletion src/datasets/loaders/tenx_visium/script.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
par = {
"input_expression": "https://cf.10xgenomics.com/samples/spatial-exp/2.0.0/CytAssist_FFPE_Mouse_Brain_Rep1/CytAssist_FFPE_Mouse_Brain_Rep1_filtered_feature_bc_matrix.h5",
"input_spatial": "https://cf.10xgenomics.com/samples/spatial-exp/2.0.0/CytAssist_FFPE_Mouse_Brain_Rep1/CytAssist_FFPE_Mouse_Brain_Rep1_spatial.tar.gz",
"dataset_id": "tenx_visium/mouse_brain_coronal_section1_visium",
"dataset_id": "mouse_brain_coronal",
"dataset_name": "Mouse Brain Coronal Section 1 (FFPE)",
"dataset_url": "https://www.10xgenomics.com/datasets/mouse-brain-coronal-section-1-ffpe-2-standard",
"dataset_summary": "Gene expression library of Mouse Brain (CytAssist FFPE) using the Mouse Whole Transcriptome Probe Set",
Expand Down
2 changes: 1 addition & 1 deletion src/datasets/loaders/tenx_visium/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

input_expression ="https://cf.10xgenomics.com/samples/spatial-exp/2.0.0/CytAssist_FFPE_Mouse_Brain_Rep1/CytAssist_FFPE_Mouse_Brain_Rep1_filtered_feature_bc_matrix.h5"
input_spatial = "https://cf.10xgenomics.com/samples/spatial-exp/2.0.0/CytAssist_FFPE_Mouse_Brain_Rep1/CytAssist_FFPE_Mouse_Brain_Rep1_spatial.tar.gz"
dataset_id = "10x_visium/mouse_brain_coronal_section1"
dataset_id = "mouse_brain_coronal"
dataset_name = "Mouse Brain Coronal Section 1 (FFPE)"
dataset_url = "https://www.10xgenomics.com/datasets/mouse-brain-coronal-section-1-ffpe-2-standard"
dataset_summary = "Gene expression library of Mouse Brain (CytAssist FFPE) using the Mouse Whole Transcriptome Probe Set"
Expand Down
281 changes: 134 additions & 147 deletions src/datasets/resource_scripts/tenx_visium.sh

Large diffs are not rendered by default.

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions src/datasets/resource_scripts/zenodo_spatial_slidetags.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

cat > "/tmp/params.yaml" << 'HERE'
param_list:
- id: zenodo_spatial_slidetags/human_cortex_slidetags
- id: zenodo_spatial_slidetags/slidetags/human_cortex
input_data: "https://zenodo.org/records/12785822/files/slidetag_human_cortex.tar.gz?download=1"
dataset_name: Slide-tags - Human Cortex
dataset_url: "https://www.nature.com/articles/s41586-023-06837-4"
Expand All @@ -14,7 +14,7 @@ param_list:
gene_filter_min_spots: 50
remove_mitochondrial: true

- id: zenodo_spatial_slidetags/human_skin_melanoma_slidetags
- id: zenodo_spatial_slidetags/slidetags/human_skin_melanoma
input_data: "https://zenodo.org/records/12785822/files/slidetag_human_skin_melanoma.tar.gz?download=1"
dataset_name: Slide-tags - Human Skin Melanoma
dataset_url: "https://www.nature.com/articles/s41586-023-06837-4"
Expand All @@ -26,7 +26,7 @@ param_list:
gene_filter_min_spots: 50
remove_mitochondrial: true

- id: zenodo_spatial_slidetags/human_tonsil_slidetags
- id: zenodo_spatial_slidetags/slidetags/human_tonsil
input_data: "https://zenodo.org/records/12785822/files/slidetag_human_tonsil.tar.gz?download=1"
dataset_name: Slide-tags - Human Tonsil
dataset_url: "https://www.nature.com/articles/s41586-023-06837-4"
Expand All @@ -38,7 +38,7 @@ param_list:
gene_filter_min_spots: 50
remove_mitochondrial: true

- id: zenodo_spatial_slidetags/mouse_embryo_slidetags
- id: zenodo_spatial_slidetags/slidetags/mouse_embryo
input_data: "https://zenodo.org/records/12785822/files/slidetag_mouse_embryo.tar.gz?download=1"
dataset_name: Slide-tags - Mouse Embryo
dataset_url: "https://www.nature.com/articles/s41586-023-06837-4"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ set -e

cat > /tmp/params.yaml << 'HERE'
param_list:
- id: mouse_brain_coronal_section1
- id: mouse_brain_coronal
input_expression: "https://cf.10xgenomics.com/samples/spatial-exp/2.0.0/CytAssist_FFPE_Mouse_Brain_Rep1/CytAssist_FFPE_Mouse_Brain_Rep1_filtered_feature_bc_matrix.h5"
input_spatial: "https://cf.10xgenomics.com/samples/spatial-exp/2.0.0/CytAssist_FFPE_Mouse_Brain_Rep1/CytAssist_FFPE_Mouse_Brain_Rep1_spatial.tar.gz"
dataset_name: Mouse Brain Coronal Section 1 (FFPE)
dataset_name: 10X Visium - Mouse Brain Coronal
dataset_url: "https://www.10xgenomics.com/datasets/mouse-brain-coronal-section-1-ffpe-2-standard"
dataset_summary: Gene expression library of Mouse Brain (CytAssist FFPE) using the Mouse Whole Transcriptome Probe Set
dataset_description: "FFPE Mouse Brain tissue blocks sectioned as described in Visium CytAssist Spatial Gene Expression for FFPE - Tissue Preparation Guide Demonstrated Protocol. The H&E stained glass slide with tissue section was processed via Visium CytAssist instrument to transfer analytes to a Visium CytAssist Spatial Gene Expression slide. The probe extension and library construction steps follow the standard Visium for FFPE workflow outside of the instrument. The H&E image was acquired using Olympus VS200 Slide Scanning Microscope. Sequencing depth was 53,497 reads per spot. Sequencing configuration: 28bp read 1 (16bp Visium spatial barcode, 12bp UMI), 90bp read 2 (transcript), 10bp i7 sample barcode and 10bp i5 sample barcode. Key metrics include: 2,310 spots detected under tissue; 6,736 median genes per spot; 24,862 median UMI counts per spot."
Expand Down
10 changes: 5 additions & 5 deletions src/tasks/spatially_variable_genes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ flowchart LR
A subset of the common dataset.

Example file:
`resources_test/common/mouse_brain_coronal_section1/dataset.h5ad`
`resources_test/common//dataset.h5ad`

Format:

Expand Down Expand Up @@ -142,7 +142,7 @@ Arguments:
The dataset without spatially variable genes.

Example file:
`resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad`
`resources_test/spatially_variable_genes//dataset.h5ad`

Format:

Expand Down Expand Up @@ -177,7 +177,7 @@ Slot description:
Anndata with true spatial variability.

Example file:
`resources_test/spatially_variable_genes/mouse_brain_coronal_section1/solution.h5ad`
`resources_test/spatially_variable_genes//solution.h5ad`

Description:

Expand Down Expand Up @@ -274,7 +274,7 @@ Arguments:
Anndata with estimate spatial variability.

Example file:
`resources_test/spatially_variable_genes/mouse_brain_coronal_section1/output.h5ad`
`resources_test/spatially_variable_genes//output.h5ad`

Description:

Expand Down Expand Up @@ -309,7 +309,7 @@ Slot description:
Metric score file.

Example file:
`resources_test/spatially_variable_genes/mouse_brain_coronal_section1/score.h5ad`
`resources_test/spatially_variable_genes//score.h5ad`

Format:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,6 @@ functionality:
path: /src/common/comp_tests/check_method_config.py
- type: python_script
path: /src/common/comp_tests/run_and_check_adata.py
- path: /resources_test/spatially_variable_genes/mouse_brain_coronal_section1
dest: resources_test/spatially_variable_genes/mouse_brain_coronal_section1
- path: /resources_test/spatially_variable_genes/mouse_brain_coronal
dest: resources_test/spatially_variable_genes/mouse_brain_coronal
- path: /src/common/library.bib
4 changes: 2 additions & 2 deletions src/tasks/spatially_variable_genes/api/comp_method.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@ functionality:
path: /src/common/comp_tests/check_method_config.py
- type: python_script
path: /src/common/comp_tests/run_and_check_adata.py
- path: /resources_test/spatially_variable_genes/mouse_brain_coronal_section1
dest: resources_test/spatially_variable_genes/mouse_brain_coronal_section1
- path: /resources_test/spatially_variable_genes/mouse_brain_coronal
dest: resources_test/spatially_variable_genes/mouse_brain_coronal
- path: /src/common/library.bib
4 changes: 2 additions & 2 deletions src/tasks/spatially_variable_genes/api/comp_metric.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ functionality:
path: /src/common/comp_tests/check_metric_config.py
- type: python_script
path: /src/common/comp_tests/run_and_check_adata.py
- path: /resources_test/spatially_variable_genes/mouse_brain_coronal_section1
dest: resources_test/spatially_variable_genes/mouse_brain_coronal_section1
- path: /resources_test/spatially_variable_genes/mouse_brain_coronal
dest: resources_test/spatially_variable_genes/mouse_brain_coronal
- path: /src/common/library.bib

Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,5 @@ functionality:
test_resources:
- type: python_script
path: /src/common/comp_tests/run_and_check_adata.py
- path: /resources_test/common/mouse_brain_coronal_section1
dest: resources_test/common/mouse_brain_coronal_section1
- path: /resources_test/common/mouse_brain_coronal
dest: resources_test/common/mouse_brain_coronal
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
type: file
example: "resources_test/common/mouse_brain_coronal_section1/dataset.h5ad"
example: "resources_test/common/mouse_brain_coronal/dataset.h5ad"
info:
label: "Common Dataset"
summary: A subset of the common dataset.
Expand Down
2 changes: 1 addition & 1 deletion src/tasks/spatially_variable_genes/api/file_dataset.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
type: file
example: "resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad"
example: "resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad"
info:
label: "Dataset"
summary: The dataset without spatially variable genes.
Expand Down
2 changes: 1 addition & 1 deletion src/tasks/spatially_variable_genes/api/file_output.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
type: file
example: "resources_test/spatially_variable_genes/mouse_brain_coronal_section1/output.h5ad"
example: "resources_test/spatially_variable_genes/mouse_brain_coronal/output.h5ad"
info:
label: Output
summary: "Anndata with estimate spatial variability."
Expand Down
2 changes: 1 addition & 1 deletion src/tasks/spatially_variable_genes/api/file_score.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
type: file
example: "resources_test/spatially_variable_genes/mouse_brain_coronal_section1/score.h5ad"
example: "resources_test/spatially_variable_genes/mouse_brain_coronal/score.h5ad"
info:
label: "Score"
summary: Metric score file.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
type: file
example: "resources_test/spatially_variable_genes/mouse_brain_coronal_section1/simulated_dataset.h5ad"
example: "resources_test/spatially_variable_genes/mouse_brain_coronal/simulated_dataset.h5ad"
info:
label: "Common Dataset"
summary: A subset of the common dataset.
Expand Down
2 changes: 1 addition & 1 deletion src/tasks/spatially_variable_genes/api/file_solution.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
type: file
example: "resources_test/spatially_variable_genes/mouse_brain_coronal_section1/solution.h5ad"
example: "resources_test/spatially_variable_genes/mouse_brain_coronal/solution.h5ad"
info:
label: Solution
summary: "Anndata with true spatial variability."
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_solution': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/solution.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'input_solution': 'resources_test/spatially_variable_genes/mouse_brain_coronal/solution.h5ad',
'output': 'output.h5ad'
}
meta = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_solution': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/solution.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'input_solution': 'resources_test/spatially_variable_genes/mouse_brain_coronal/solution.h5ad',
'output': 'output.h5ad'
}
meta = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ dest <- getwd()

# VIASH START
par <- list(
"input_data" = "resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad",
"input_data" = "resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad",
"output" = "output.h5ad",
"n_iter" = 10
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'output': 'output.h5ad',
'n_features': 120
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'output': 'output.h5ad',
'coord_type_moran_i': 'generic'

Expand Down
2 changes: 1 addition & 1 deletion src/tasks/spatially_variable_genes/methods/nnsvg/script.R
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ suppressMessages(library(dplyr))

# VIASH START
par = list(
'input_data' = 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_data' = 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'output' = 'output.h5ad'
)
meta = list(
Expand Down
2 changes: 1 addition & 1 deletion src/tasks/spatially_variable_genes/methods/scgco/script.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'output': 'output.h5ad'
}
meta = {
Expand Down
2 changes: 1 addition & 1 deletion src/tasks/spatially_variable_genes/methods/sepal/script.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'output': 'output.h5ad',
'coord_type_sepal': 'grid',
'max_neighs_sepal': 6,
Expand Down
2 changes: 1 addition & 1 deletion src/tasks/spatially_variable_genes/methods/somde/script.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'output': 'output.h5ad'
}
meta = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'output': 'output.h5ad'
}
meta = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'output': 'output.h5ad'
}
meta = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'output': 'output.h5ad'
}
meta = {
Expand Down
2 changes: 1 addition & 1 deletion src/tasks/spatially_variable_genes/methods/spark/script.R
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ suppressMessages(library(anndata))

# VIASH START
par <- list(
"input_data" = "resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad",
"input_data" = "resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad",
"output" = "output.h5ad"
)
meta <- list(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ suppressMessages(library(anndata))

# VIASH START
par <- list(
"input_data" = "resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad",
"input_data" = "resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad",
"output" = "output.h5ad"
)
meta <- list(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'output': 'output.h5ad'
}
meta = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

# VIASH START
par = {
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/dataset.h5ad',
'input_data': 'resources_test/spatially_variable_genes/mouse_brain_coronal/dataset.h5ad',
'output': 'output.h5ad'
}
meta = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@

## VIASH START
par = {
'input_method': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/output.h5ad',
'input_solution': 'resources_test/spatially_variable_genes/mouse_brain_coronal_section1/solution.h5ad',
'input_method': 'resources_test/spatially_variable_genes/mouse_brain_coronal/output.h5ad',
'input_solution': 'resources_test/spatially_variable_genes/mouse_brain_coronal/solution.h5ad',
'output': 'score.h5ad'
}
meta = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ functionality:
- type: python_script
path: script.py
test_resources:
- path: /resources_test/common/mouse_brain_coronal_section1
dest: resources_test/common/mouse_brain_coronal_section1
- path: /resources_test/common/mouse_brain_coronal
dest: resources_test/common/mouse_brain_coronal
- type: python_script
path: /src/common/comp_tests/run_and_check_adata.py
platforms:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

### VIASH START
par = {
"input": "resources_test/common/mouse_brain_coronal_section1/dataset.h5ad",
"input": "resources_test/common/mouse_brain_coronal/dataset.h5ad",
"input_layer": "normalized",
"output": "reference_dataset.h5ad",
"num_features": 50,
Expand Down
Loading