JPEG2000 decoding failure

[hayatoikoma]

After upgrading to v2024.6.1, I'm sometimes encountering a decoding failure of JPEG2000 with imagecodecs._jpeg2k.Jpeg2kError: opj_decode or opj_end_decompress failed.

As it fails only in a long script and I'm using imagecodecs from zarr, it is difficult to share a reproducible example. (I pickled the input, but it didn't fail in a minimal example.) However, I found that it is outputting some warnings which were not seen in v2024.1.1. Do you think that it is the cause of the failure?

This is the snippet for reproducing the warnings.

import imagecodecs
import numpy as np

image_data = np.ones((11, 146, 2960), dtype=np.uint16)
for i in range(image_data.shape[0]):
    print(i)
    print("encoding")
    encoded = imagecodecs.jpeg2k_encode(image_data[i], level=0, verbose=3)
    print("decoding")
    decoded = imagecodecs.jpeg2k_decode(encoded, verbose=3)

With v2024.6.1

0
encoding
JPEG2K info: tile number 1 / 1
decoding
JPEG2K info: Start to read j2k main header (85).
JPEG2K info: Main header has been correctly decoded.
JPEG2K info: Header of tile 1 / 1 has been read.
JPEG2K info: Stream reached its end !
mean 1.0
1
encoding
JPEG2K info: tile number 1 / 1
decoding
JPEG2K info: Start to read j2k main header (85).
JPEG2K info: Main header has been correctly decoded.
JPEG2K info: Header of tile 1 / 1 has been read.
JPEG2K info: Stream reached its end !
JPEG2K warning: Problem with skipping JPEG2000 box, stream error
mean 1.0
2
encoding
JPEG2K info: tile number 1 / 1
decoding
JPEG2K info: Start to read j2k main header (85).
JPEG2K info: Main header has been correctly decoded.
JPEG2K info: Header of tile 1 / 1 has been read.
JPEG2K info: Stream reached its end !
JPEG2K warning: Problem with skipping JPEG2000 box, stream error
mean 1.0

With v2024.1.1

0
encoding
JPEG2K info: tile number 1 / 1
decoding
JPEG2K info: Start to read j2k main header (85).
JPEG2K info: Main header has been correctly decoded.
JPEG2K info: Header of tile 1 / 1 has been read.
JPEG2K info: Stream reached its end !
mean 1.0
1
encoding
JPEG2K info: tile number 1 / 1
decoding
JPEG2K info: Start to read j2k main header (85).
JPEG2K info: Main header has been correctly decoded.
JPEG2K info: Header of tile 1 / 1 has been read.
JPEG2K info: Stream reached its end !
mean 1.0
2
encoding
JPEG2K info: tile number 1 / 1
decoding
JPEG2K info: Start to read j2k main header (85).
JPEG2K info: Main header has been correctly decoded.
JPEG2K info: Header of tile 1 / 1 has been read.
JPEG2K info: Stream reached its end !
mean 1.0

Environment

python                    3.12.3          hab00c5b_0_cpython    conda-forge
openjpeg                  2.5.2                h488ebb8_0    conda-forge
imagecodecs               2024.6.1        py312h083dece_0    conda-forge

Anyway, thank you for maintaining a fantastic package!

[cgohlke]

Thanks for reporting the issue. I can reproduce the warnings and traced them to the latest changes in the encoder of _jpeg2k.pyx, not due to any OpenJPEG changes.

[cgohlke]

Ouch. Turns out the encoded buffer is not cropped correctly in some cases and trailed with uninitialized memory, which sometimes interferes with decoding. Not sure how the tests could pass on several platforms...