Bug in the MERFISH Technology tutorial

[Marius1311]

Describe the bug
Hi there, I'm running the public MERFISH tutorial using spatialdata 0.2.2 and spatialdata_plot 0.2.4 (https://spatialdata.scverse.org/en/stable/tutorials/notebooks/notebooks/examples/technology_merfish.html) and when I execute the final cell containing

import matplotlib.pyplot as plt

fig, ax = plt.subplots(ncols=1, figsize=(4, 4))

(
    merfish_sdata.pp.get_elements(["anatomical", "rasterized", "single_molecule"])
    .pl.render_images()
    .pl.render_points(color="cell_type")
    .pl.render_shapes(fill_alpha=0.5, outline=True)
    .pl.show(ax=ax)
)

I get

[/cluster/project/treutlein/USERS/mlange/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata/_core/_elements.py:116](http://localhost:56467/lab/tree/github/spatial_analysis/analysis/experiments_and_tutorials/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata/_core/_elements.py#line=115): UserWarning: Key `table` already exists. Overwriting it in-memory.
  self._check_key(key, self.keys(), self._shared_keys)
[/cluster/project/treutlein/USERS/mlange/mambaforge/envs/spatialdata/lib/python3.11/site-packages/anndata/_core/anndata.py:430](http://localhost:56467/lab/tree/github/spatial_analysis/analysis/experiments_and_tutorials/mambaforge/envs/spatialdata/lib/python3.11/site-packages/anndata/_core/anndata.py#line=429): FutureWarning: The dtype argument is deprecated and will be removed in late 2024.
  warnings.warn(
[/cluster/project/treutlein/USERS/mlange/mambaforge/envs/spatialdata/lib/python3.11/site-packages/anndata/_core/aligned_df.py:67](http://localhost:56467/lab/tree/github/spatial_analysis/analysis/experiments_and_tutorials/mambaforge/envs/spatialdata/lib/python3.11/site-packages/anndata/_core/aligned_df.py#line=66): ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)
[/cluster/project/treutlein/USERS/mlange/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata/_core/_elements.py:106](http://localhost:56467/lab/tree/github/spatial_analysis/analysis/experiments_and_tutorials/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata/_core/_elements.py#line=105): UserWarning: Key `single_molecule` already exists. Overwriting it in-memory.
  self._check_key(key, self.keys(), self._shared_keys)
[/cluster/project/treutlein/USERS/mlange/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata_plot/pl/utils.py:719](http://localhost:56467/lab/tree/github/spatial_analysis/analysis/experiments_and_tutorials/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata_plot/pl/utils.py#line=718): FutureWarning: The default value of 'ignore' for the `na_action` parameter in pandas.Categorical.map is deprecated and will be changed to 'None' in a future version. Please set na_action to the desired value to avoid seeing this warning
  color_vector = color_source_vector.map(color_map)
[/cluster/project/treutlein/USERS/mlange/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata_plot/pl/render.py:470](http://localhost:56467/lab/tree/github/spatial_analysis/analysis/experiments_and_tutorials/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata_plot/pl/render.py#line=469): UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap', 'norm' will be ignored
  _cax = ax.scatter(
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[26], line 10
      1 import matplotlib.pyplot as plt
      3 fig, ax = plt.subplots(ncols=1, figsize=(4, 4))
      5 (
      6     sdata.pp.get_elements(["anatomical", "rasterized", "single_molecule"])
      7     .pl.render_images()
      8     .pl.render_points(color="cell_type")
      9     .pl.render_shapes(fill_alpha=0.5, outline=True)
---> 10     .pl.show(ax=ax)
     11 )

File [/cluster/project/treutlein/USERS/mlange/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata_plot/pl/basic.py:906](http://localhost:56467/lab/tree/github/spatial_analysis/analysis/experiments_and_tutorials/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata_plot/pl/basic.py#line=905), in PlotAccessor.show(self, coordinate_systems, legend_fontsize, legend_fontweight, legend_loc, legend_fontoutline, na_in_legend, colorbar, wspace, hspace, ncols, frameon, figsize, dpi, fig, title, share_extent, pad_extent, ax, return_ax, save)
    901     wanted_elements, wanted_points_on_this_cs, wants_points = _get_wanted_render_elements(
    902         sdata, wanted_elements, params_copy, cs, "points"
    903     )
    905     if wanted_points_on_this_cs:
--> 906         _render_points(
    907             sdata=sdata,
    908             render_params=params_copy,
    909             coordinate_system=cs,
    910             ax=ax,
    911             fig_params=fig_params,
    912             scalebar_params=scalebar_params,
    913             legend_params=legend_params,
    914         )
    916 elif cmd == "render_labels" and has_labels:
    917     wanted_elements, wanted_labels_on_this_cs, wants_labels = _get_wanted_render_elements(
    918         sdata, wanted_elements, params_copy, cs, "labels"
    919     )

File [/cluster/project/treutlein/USERS/mlange/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata_plot/pl/render.py:495](http://localhost:56467/lab/tree/github/spatial_analysis/analysis/experiments_and_tutorials/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata_plot/pl/render.py#line=494), in _render_points(sdata, render_params, coordinate_system, ax, fig_params, scalebar_params, legend_params)
    492 else:
    493     palette = ListedColormap(dict.fromkeys(color_vector[~pd.Categorical(color_source_vector).isnull()]))
--> 495 _ = _decorate_axs(
    496     ax=ax,
    497     cax=cax,
    498     fig_params=fig_params,
    499     adata=adata,
    500     value_to_plot=col_for_color,
    501     color_source_vector=color_source_vector,
    502     palette=palette,
    503     alpha=render_params.alpha,
    504     na_color=render_params.cmap_params.na_color,
    505     legend_fontsize=legend_params.legend_fontsize,
    506     legend_fontweight=legend_params.legend_fontweight,
    507     legend_loc=legend_params.legend_loc,
    508     legend_fontoutline=legend_params.legend_fontoutline,
    509     na_in_legend=legend_params.na_in_legend,
    510     colorbar=legend_params.colorbar,
    511     scalebar_dx=scalebar_params.scalebar_dx,
    512     scalebar_units=scalebar_params.scalebar_units,
    513 )

File [/cluster/project/treutlein/USERS/mlange/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata_plot/pl/utils.py:891](http://localhost:56467/lab/tree/github/spatial_analysis/analysis/experiments_and_tutorials/mambaforge/envs/spatialdata/lib/python3.11/site-packages/spatialdata_plot/pl/utils.py#line=890), in _decorate_axs(ax, cax, fig_params, value_to_plot, color_source_vector, adata, palette, alpha, na_color, legend_fontsize, legend_fontweight, legend_loc, legend_fontoutline, na_in_legend, colorbar, scalebar_dx, scalebar_units, scalebar_kwargs)
    887     palette = None if isinstance(palette, list) and palette[0] else palette
    888     palette = _get_palette(
    889         adata=adata, cluster_key=value_to_plot, categories=clusters, palette=palette, alpha=alpha
    890     )
--> 891     _add_categorical_legend(
    892         ax,
    893         color_source_vector,
    894         palette=palette,
    895         legend_loc=legend_loc,
    896         legend_fontweight=legend_fontweight,
    897         legend_fontsize=legend_fontsize,
    898         legend_fontoutline=path_effect,
    899         na_color=[na_color],
    900         na_in_legend=na_in_legend,
    901         multi_panel=fig_params.axs is not None,
    902     )
    903 elif colorbar:
    904     # TODO: na_in_legend should have some effect here
    905     plt.colorbar(cax, ax=ax, pad=0.01, fraction=0.08, aspect=30)

File [/cluster/project/treutlein/USERS/mlange/mambaforge/envs/spatialdata/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py:1129](http://localhost:56467/lab/tree/github/spatial_analysis/analysis/experiments_and_tutorials/mambaforge/envs/spatialdata/lib/python3.11/site-packages/scanpy/plotting/_tools/scatterplots.py#line=1128), in _add_categorical_legend(ax, color_source_vector, palette, legend_loc, legend_fontweight, legend_fontsize, legend_fontoutline, multi_panel, na_color, na_in_legend, scatter_array)
   1127 if legend_loc == "right margin":
   1128     for label in cats:
-> 1129         ax.scatter([], [], c=palette[label], label=label)
   1130     ax.legend(
   1131         frameon=False,
   1132         loc="center left",
   (...)
   1135         fontsize=legend_fontsize,
   1136     )
   1137 elif legend_loc == "on data":
   1138     # identify centroids to put labels

KeyError: 'VISp_I'

To Reproduce
Run the public MERFISH tutorial

Desktop (optional):

• OS: Ubuntu 22.04

[LucaMarconato]

Hi, thanks for reporting. I have tried running the notebook on macOS using spatialdata==0.2.2 and spatialdata-plot==0.2.4 and I could not reproduce the bug. I also tried using the latest dev version and I could not reproduce.

My guesses:

• It could be a OS problem but I don't think this is the case because before release we tested that notebook on a linux machine.
• As a double check, I would try to visualize the data using napari-spatialdata with the following code

from napari_spatialdata import Interactive

Interactive(merfish_sdata)

and see if you can reproduce also in this case.

• If you can reproduce with napari, it could be a problem with the data. I'd try to regenerate the sdata object by running the lightweight scripts download.py and to_zarr.py for here: https://github.com/giovp/spatialdata-sandbox/tree/main/merfish. If this fixes the issue it could be that there has been a problem with the pipeline we use to regenerate the data.