New reader for G4X datasets (Singular Genomics)

[ckmah]

👋 Hello @scverse/spatialdata and community,

I would like to contribute the initial version of a spatialdata-io reader for Singular Genomics G4X datasets that I recently developed for internal use (I work at Singular), and now for the spatial community. It is still experimental and not fully battle-tested, but I tried to keep the API consistent with the other readers as much as possible. However, there are few key additions I made to streamline use with our datasets:

Notable features

• Incremental I/O of elements (images, tables etc.) G4X datasets can get pretty large since they are multimodal. Therefore, we made sure the reader saves elements as soon as they are converted to reduce memory Reduce readers' memory consumption #229 and mitigate data loss. This is handled via the g4x(..., mode="append") parameter. The user can also choose mode="overwrite" to turn this off. The constructed SpatialData is also re-read from disk automatically to fully take advantage of lazy loading.
• Read one or more samples at once. This corresponds to our assay design and enables converting an entire experiment with a single function call. The reader will then return a single SpatialData object or a list of them accordingly.

Additional Dependencies

• Some of our images are encoded in the Jpeg2000 (.jp2, j2k) format and require the glympur package to read

Misc.

• The reader corrects the Offset when rasterizing data spatialdata#165 for now. This can be removed when ready.
• Sample dataset PRed to https://github.com/giovp/spatialdata-sandbox/

Are there any other pieces I should have in this PR? Devs please let me know, I'm happy to add them. Here are relevant ones I can think of:

• Documentation/tutorial notebook? (also not sure if I used @injectdocs decorator properly)
• Parse experimental metadata: sample names, positions, acquisition info etc.
• spatialdata-io CLI compatibility

[codecov-commenter]

Codecov Report

❌ Patch coverage is 17.19298% with 236 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.41%. Comparing base (296d9a5) to head (0333b29).
⚠️ Report is 232 commits behind head on main.

Files with missing lines	Patch %	Lines
src/spatialdata_io/readers/g4x.py	10.26%	236 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #281      +/-   ##
==========================================
+ Coverage   39.16%   46.41%   +7.24%     
==========================================
  Files          26       27       +1     
  Lines        2663     2995     +332     
==========================================
+ Hits         1043     1390     +347     
+ Misses       1620     1605      -15     

Files with missing lines	Coverage Δ
src/spatialdata_io/__init__.py	100.00% <100.00%> (ø)
src/spatialdata_io/_constants/_constants.py	100.00% <100.00%> (ø)
src/spatialdata_io/readers/g4x.py	10.26% <10.26%> (ø)

... and 10 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[floRaths]

Hi @LucaMarconato, I would like to pick up this PR and get it past the pre-commit checks. How do you suggest me continuing? Should I create a fresh clone of the feature on the current main branch?

[LucaMarconato]

Hi @floRaths, thanks for reaching out! You could open a PR against this PR, or if you want merge this PR in a new PR (from you) against main. As you prefer.

We recently added a contributed guide for spatialdata-io that will make is easier to develop PRs requiring minimal review. I kindly ask you to consult it if you want to contribute the reader: https://github.com/scverse/spatialdata-io/blob/main/docs/contributing.md.

Also, in case you want to discuss via chat or setup a meeting, we offer community meetings every 2 weeks (covering different time zones): #spatial > spatialdata meetings @ 💬.