Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update precompressed tile documentation #390

Merged
merged 14 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion sphinx/users/comlinetools/conversion.rst
Original file line number Diff line number Diff line change
Expand Up @@ -301,10 +301,12 @@ correctly. So in Windows, the above example would read::

.. code-block::

bfconvert -noflat -precompressed -compression JPEG CMU-1.svs CMU-1.dcm
bfconvert -noflat -precompressed CMU-1.svs CMU-1.dcm

.. versionadded:: 7.1.0

.. seealso:: :doc:`Summary of the precompressed tiles feature <precompressed>`

.. option:: -extra-metadata METADATA FILE

Enables the writing of additional supplemental metadata from an external file during the conversion process .
Expand Down
52 changes: 52 additions & 0 deletions sphinx/users/comlinetools/precompressed.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
"Precompressed" tiles
=====================

Some formats allow reading and/or writing "precompressed" tiles.
This feature is primarily intended for whole slide images, each of which may have thousands of tiles.
Data representing these tiles may be compressed, either lossily or losslessly, depending on the
file format, imaging modality, or acquisition configuration.

When reading, this allows for retrieving compressed bytes that are as close as possible
to what is actually stored in the file. Decompression (if necessary) is the responsibility
of the consumer.

When writing, compressed bytes instead of raw bytes can be provided to the writer,
such that the writer will just save the provided compressed bytes instead of recompressing.

This feature is primarily exposed in the :program:`bfconvert` command, using the `-precompressed` option.
When the `-precompressed` option is used, best results are obtained by:

* using the `-noflat` option
* _not_ using the `-compression` option (this can work, but may force tile recompression)
melissalinkert marked this conversation as resolved.
Show resolved Hide resolved
* _not_ using the `-tilex` or `-tiley` options (this can work, but may force tile recompression)
melissalinkert marked this conversation as resolved.
Show resolved Hide resolved

There are several advantages to using the "precompressed" feature for formats that support it:

* faster tile read and write times, as no tile compression/decompression needs to be performed
* no change in compression quality
* very little change in file sizes between input and output data

There are also a few disadvantages:

* tile sizes cannot be changed during conversion
* compression type cannot be changed during conversion
* input and output format must support same compression type and tile size
* not all formats supported by Bio-Formats currently allow reading/writing precompressed tiles
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first read, this felt slightly redundant with the previous bullet point even though the message is different.

Also maybe this needs to be stated as in. the input format must have precompressed tiles reading support and the output format must have precompressed tiles writing support with a reference to the paragraph about the supported formats

* tools other than :program:`bfconvert` do not currently make use of the precompressed tile API
melissalinkert marked this conversation as resolved.
Show resolved Hide resolved

Formats that support precompressed tile reading:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming additional formats get added to this list mid-term, https://bio-formats.readthedocs.io/en/latest/formats/index.html would probably the natural section to keep a reference of all readers/writers supporting precompressed tile reading/writing. If a developer page gets introduced about precompressed tile reading/writing, this would also be more natural as a link rather than adding a reference to a subsection of a command-line utility page.

Reviewing the few available options:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to split into two pages - one under /formats/ that has the high-level summary and list of supported readers/writers, and the original one under /users/comlinetools/ that's more bfconvert-specific. Open to other thoughts though if that doesn't make sense.


* SVS (since 7.1.0)
* NDPI (forthcoming: https://github.com/ome/bioformats/pull/4181)
* DICOM (forthcoming: https://github.com/ome/bioformats/pull/4190)

Formats that support precompressed tile writing:

* DICOM (since 7.1.0)
* TIFF (forthcoming: https://github.com/ome/bioformats/pull/4190)
* OME-TIFF (forthcoming: https://github.com/ome/bioformats/pull/4190)

For additional discussion of the "precompressed" tiles feature, see:

* https://forum.image.sc/t/vsi-to-dcm-file-conversion/98249/5
* https://forum.image.sc/t/exporting-a-dicom-stack/99400
1 change: 1 addition & 0 deletions sphinx/users/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ for carrying out a variety of tasks:
:includehidden:

comlinetools/index
melissalinkert marked this conversation as resolved.
Show resolved Hide resolved
comlinetools/precompressed

.. seealso::
:doc:`/formats/options`
Expand Down