Decoder (#126)

* Inject Decoder to decode frame data to Image * Move create method to Decoder * Add image codecs decoders * Use decoder to chech if transfer syntax is supported * Simplify create_instance_dataset() * Add missing properties * Fix decoding of color image * Simplify image codecs decoder * Config for what decoder to use * Use pydicom config to get pixel data handlers * Check if image codecs are available * Use jpeg codecs directly * Change supported transfer syntaxes to dict * Add option to provide a session to the web client (#117) * Add option to provide web client a session We would like to create the `requests.Session` object on our own and pass it into `WsiDicomWebClient`, since we are not using an object derived from `AuthBase` to create it. This PR adds support to pass a session to `WsiDicomWebClient` directly. Signed-off-by: Patrick Avery <[email protected]> * Refactor __init__ method of WsiDicomWebClient The `__init__` method now accepts a `DICOMwebClient` object. The previous `__init__` method was moved into a `create_client()` class method. Signed-off-by: Patrick Avery <[email protected]> * Remove `WsiDicomFileClient` and update README.md Signed-off-by: Patrick Avery <[email protected]> --------- Signed-off-by: Patrick Avery <[email protected]> * Dicomweb get multiple frames (#121) * Get multiple frames from dicom web * Allow loading multiple series with DICOMweb (#122) * Allow loading multiple series with DICOMweb `WsiDicom.open_web()` was modified to be able to either take a single series uid (as before) or a list of series uids. If a list of series uids is passed, their instances are all loaded together. This can be useful for cases where, for instance, different optical paths are located in different series. This also loosens the UID matching to only check the frame of references. This also adds `TotalPixelMatrixOriginSequence` matching when comparing datasets. Fixes: #118 Signed-off-by: Patrick Avery <[email protected]> * Fix `isinstance()` check for series uids Signed-off-by: Patrick Avery <[email protected]> * Fix logic in `SlideUids.matches()` Signed-off-by: Patrick Avery <[email protected]> * Remove unused import Signed-off-by: Patrick Avery <[email protected]> --------- Signed-off-by: Patrick Avery <[email protected]> * Add property to count number of tile frames (#116) * Add property to count number of tile frames The "NumberOfFrames" attribute on a dataset takes into account not just the number of frames corresponding to tiles, but also the number of focal planes and optical paths (for example, see [here](https://github.com/imi-bigpicture/wsidicom/blob/774fe3ca00096e3fd2556fc956520dbfabc81311/wsidicom/instance/dataset.py#L651-L655)). When I was trying to view a dataset with multiple optical paths, I would encounter errors in the `image_size` property since it did not distinguish frames that came from optical paths and tiles. This fixes viewing [this example](https://imagingdatacommons.github.io/slim/studies/2.25.23897195960526781231486877255455994829/series/2.25.83282858720704132758110891374375550907) via DICOMweb in [large_image](https://github.com/girder/large_image). I am new to the field, so please feel free to correct any naming issues or misconceptions... Signed-off-by: Patrick Avery <[email protected]> * Add option to provide a session to the web client (#117) * Add option to provide web client a session We would like to create the `requests.Session` object on our own and pass it into `WsiDicomWebClient`, since we are not using an object derived from `AuthBase` to create it. This PR adds support to pass a session to `WsiDicomWebClient` directly. Signed-off-by: Patrick Avery <[email protected]> * Refactor __init__ method of WsiDicomWebClient The `__init__` method now accepts a `DICOMwebClient` object. The previous `__init__` method was moved into a `create_client()` class method. Signed-off-by: Patrick Avery <[email protected]> * Remove `WsiDicomFileClient` and update README.md Signed-off-by: Patrick Avery <[email protected]> --------- Signed-off-by: Patrick Avery <[email protected]> * Dicomweb get multiple frames (#121) * Get multiple frames from dicom web * Allow loading multiple series with DICOMweb (#122) * Allow loading multiple series with DICOMweb `WsiDicom.open_web()` was modified to be able to either take a single series uid (as before) or a list of series uids. If a list of series uids is passed, their instances are all loaded together. This can be useful for cases where, for instance, different optical paths are located in different series. This also loosens the UID matching to only check the frame of references. This also adds `TotalPixelMatrixOriginSequence` matching when comparing datasets. Fixes: #118 Signed-off-by: Patrick Avery <[email protected]> * Fix `isinstance()` check for series uids Signed-off-by: Patrick Avery <[email protected]> * Fix logic in `SlideUids.matches()` Signed-off-by: Patrick Avery <[email protected]> * Remove unused import Signed-off-by: Patrick Avery <[email protected]> --------- Signed-off-by: Patrick Avery <[email protected]> * Remove unsafe number_of_focal_planes and number_of_optical_paths properties * FIx if * Skip frame count check for concatenated instances --------- Signed-off-by: Patrick Avery <[email protected]> Co-authored-by: Erik O Gabrielsson <[email protected]> Co-authored-by: Erik O Gabrielsson <[email protected]> * Add encoder * Move decoder and encoder to codec module * Test decoder against prepared test files * Add decode test for pillow * Tests for encoder * Add property of samples per pixel * use pyjpeg-rle for rle encoding * Combine similar settings * Fix encoding of uncompressed data * Fix parameter order * Add init for Rle settings * Fix writing tag length * Add pillow decoder * Check if decoder is avaiable * Make codec packages optional * Use codec * Make settings dataclass-y * Skip test if no test data * Determine transfer syntax for dicom web through optional qido field * Try transfer syntax until supported * Skip image test if no test image * Skip test if optional package * Take sequence of requested transfer syntaxes * Remove wsi-standard options from codec * Open dicom web with threads * Update python version * Use pixel data handles directly * Update shapely * Fix typing * Use test data package * Rle codec based on image codecs PackBits * Update packages * Fix jpeg2k_encode import * Add checks if codec available, skip tests if not * Run ci with extras * Update changelog * Run codespell * Fix spelling * Update version * Add license * blank_color can be grey level * Relax requirement for optical dataset * Use `x-dicom-rle` for rle as that is what dicomwebclient supports * Try transfer syntaxes that has been detected for other instances first * Try-load Lut * add __all__ * Update changelog * Update readme * Change default subsampling to 422 * Test using scaled images * More compact selection for encoder * Changelog for release 0.14.0 --------- Signed-off-by: Patrick Avery <[email protected]> Co-authored-by: Patrick Avery <[email protected]>
imi-bigpicture · Nov 29, 2023 · 0e0db82 · 0e0db82
1 parent 5ffaa8b
commit 0e0db82
Show file tree

Hide file tree

Showing 44 changed files with 4,134 additions and 613 deletions.
diff --git a/.github/workflows/python-package.yml b/.github/workflows/python-package.yml
@@ -33,7 +33,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, macos-latest, windows-latest]
-        python-version: ["3.8", "3.9", "3.10", "3.11"]
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
     steps:
       - name: Prevent cache-miss on windows
         run: |
@@ -67,7 +67,7 @@ jobs:
           python -m pip install flake8 pytest
           python -m pip install poetry
       - name: Install Application
-        run: python -m poetry install
+        run: python -m poetry install --all-extras
       - name: Lint with flake8
         run: |
           # stop the build if there are Python syntax errors or undefined names

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,10 +7,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.14.0] - 2023-11-29
+
 ### Added
 
+- Support for additional transfer syntaxes both for reading and writing, using pydicom pixel handlers, (optional) image codecs, and (optional) pylibjpeg-rle.
+- Additional decoders and encoders to Pillow. Decoder and encoder is selected automatically. Decoder to use can be overridden with the `prefered_decoder`-setting.
+- Detection of suitable available transfer syntax when opening slide with DICOM web.
 - Support for opening DICOMDIR files using `open_dicomdir()`.
 
+### Changed
+
+- Opening DICOM web instances in parallel. Configurable with `open_web_theads`-setting.
+- `open_web()` now takes a list of requested transfer syntaxes to test if available from the server.
+- When fetching multiple frames from DICOM web, fetch multiple frames per request instead of making several requests.
+- Renamed `OffsetTableType` option `NONE` to `EMPTY`, and added type `NONE` for use with unencapsulated data.
+
+### Removed
+
+- Support for Python 3.8.
+
+### Fixed
+
+- Loading annotation instances using DICOM web.
+
 ## [0.13.0] - 2023-11-11
 
 ### Added

diff --git a/README.md b/README.md
@@ -26,15 +26,36 @@ Please note that this is an early release and the API is not frozen yet. Functio
 
 ## Requirements
 
-*wsidicom* uses pydicom, numpy, Pillow (with jpeg and jpeg2000 plugins), and dicomweb-client.
+*wsidicom* uses pydicom, numpy, Pillow, and dicomweb-client. Imagecodecs and pylibjpeg-rle can be installed as optionals to support additional transfer syntaxes.
 
 ## Limitations
 
-Levels are required to have (close to) 2 factor scale and same tile size.
+- Levels are required to have (close to) 2 factor scale and same tile size.
 
-Only JPEGBaseline8Bit, JPEG2000 and JPEG2000Lossless transfer syntax is supported.
+- Only 8 bits per sample is supported for color images, and 8 and 16 bits for grayscale images.
 
-Optical path identifiers needs to be unique across file set.
+- Without optional dependencies, the following transfer syntaxes are supported:
+
+  - JPEGBaseline8Bit
+  - JPEG2000
+  - JPEG2000Lossless
+  - ImplicitVRLittleEndian
+  - ExplicitVRLittleEndian
+  - ExplicitVRBigEndian
+
+- With imagecodecs, the following transfer syntaxes are additionally supported:
+
+  - JPEGExtended12Bit
+  - JPEGLosslessP14
+  - JPEGLosslessSV1
+  - JPEGLSLossless
+  - JPEGLSNearLossless
+
+- With pylibjpeg-rle RLELossless is additionally supported.
+
+- Optical path identifiers needs to be unique across instances.
+
+- Only one pyramid (i.e. offset from slide corner) per frame of reference is supported.
 
 ## Basic usage
 
@@ -231,7 +252,7 @@ point_annotation_with_measurment = Annotation(Point(10.0, 20.0), [measurement])
 
 ```python
 from wsidicom import PointAnnotationGroup
-# The 222 suplement requires groups to have a label, a category and a type
+# The 222 supplement requires groups to have a label, a category and a type
 group = PointAnnotationGroup(
     annotations=[point_annotation, point_annotation_with_measurment],
     label='group label',

diff --git a/poetry.lock b/poetry.lock
diff --git a/pyproject.toml b/pyproject.toml
@@ -1,11 +1,12 @@
 [tool.poetry]
 name = "wsidicom"
-version = "0.13.0"
+version = "0.14.0"
 description = "Tools for handling DICOM based whole scan images"
 authors = ["Erik O Gabrielsson <[email protected]>"]
 license = "Apache-2.0"
 readme = "README.md"
 repository = "https://github.com/imi-bigpicture/wsidicom"
+packages = [{include = "wsidicom"}]
 keywords = ["whole slide image", "digital pathology", "annotations", "dicom"]
 classifiers = [
     "Development Status :: 4 - Beta",
@@ -17,21 +18,28 @@ classifiers = [
 ]
 
 [tool.poetry.dependencies]
-python = "^3.8"
+python = "^3.9"
 numpy = "^1.22.0"
 pydicom = ">=2.1.0"
-Pillow = ">=9.1.1, <11.0.0"
+Pillow = ">=9.1.1"
 dicomweb-client = "^0.59.1"
+imagecodecs = { version = "^2023.9.18", optional = true }
+pylibjpeg-rle = { version = "^1.3.0", optional = true }
+
+[tool.poetry.extras]
+imagecodecs = ["imagecodecs"]
+rle = ["pylibjpeg-rle"]
 
 [tool.poetry.group.dev.dependencies]
 pytest = "^7.2.0"
 pytest-watch = "^4.2.0"
 xmltodict = "^0.12.0"
-shapely = "^1.7.0"
+shapely = "^2.0.2"
 pycodestyle = "^2.8.0"
 black = "^23.1.0"
 flake8 = "^4.0.1"
 codespell = "^2.2.5"
+wsidicom-data =  "^0.2.0"
 
 [build-system]
 requires = ["poetry-core>=1.2.0"]

diff --git a/tests/codec/conftest.py b/tests/codec/conftest.py
@@ -0,0 +1,30 @@
+#    Copyright 2023 SECTRA AB
+#
+#    Licensed under the Apache License, Version 2.0 (the "License");
+#    you may not use this file except in compliance with the License.
+#    You may obtain a copy of the License at
+#
+#        http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS,
+#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#    See the License for the specific language governing permissions and
+#    limitations under the License.
+
+import pytest
+from wsidicom_data import EncodedTestData, TestData
+
+from wsidicom.codec import Settings
+
+
+@pytest.fixture
+def image(settings: Settings):
+    return TestData.image(settings.bits, settings.samples_per_pixel)
+
+
+@pytest.fixture
+def encoded(settings: Settings):
+    file_path = EncodedTestData.get_filepath_for_encoder_settings(settings)
+    with open(file_path, "rb") as file:
+        return file.read()