Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add HTJ2K Compressor #1

Closed
wants to merge 10 commits into from
Closed

Conversation

palemieux
Copy link

@palemieux palemieux commented Oct 12, 2024

WIP DO NOT MERGE

This patch fulfills my action item from the September 19 TSC meeting.

Introduction

This patch proposes to add support for High-Throughput JPEG 2000 (HTJ2K) compression to OpenEXR -- HTJ2K is JPEG 2000 with the HT block coder standardized in Rec. ITU-T T.814 | ISO/IEC 15444-15. As detailed at Evaluating HTJ2K as a compressor option for OpenEXR, HTJ2K demonstrates significant improvements in both speed and file size over other OpenEXR compressors. Furthermore, HTJ2K is a worldwide standard that benefits from a diversity of implementations, builds on JPEG 2000, which is in broad use in studio workflows, is appropriate for long-term preservation, has both lossy and lossless modes and supports integer and floating point samples. Finally, the unique features of HTJ2K could be powerful additions to high-performance OpenEXR workflows where the full image may not always be viewed at its full resolution.

The patch currently defines four compressors:

  • HT: Lossless coding using HTJ2K. Each chunk is the full image frame and the open-source OpenJPH library is used.
  • HT256: Lossless coding using HTJ2K. Each chunk is 256 lines and the open-source OpenJPH library is used.
  • HTK: Lossless coding using HTJ2K. Each chunk is the full image frame and the commercial KDU library is used.
  • HTK256: Lossless coding using HTJ2K. Each chunk is 256 lines and the commercial KDU library is used.

Questions

  • can a compressor store configuration information once in a file, so that the same information is available for each chunk to both the compressor and decompressor?

Notes

  • HT and HTK Compressors are identical and use the open-source OpenJPH library is the Kakadu SDK is not found
  • Ultimately there will be a single lossless HTJ2K Compressor. The HT and HTK Compressors are currently offered to ensure interop, and facilitate comparisons, between OSS and commercial options.
  • The full-frame and 256-line chunk options are currently offered to evaluate the impact of sub-frame chunking on coding efficiency performance.

Todo

  • add support for 32-bit samples to OpenJPH
  • explore adding a progressive decoding API to OpenEXR
  • explore tradeoff of chunk-size (currently set to 256) on compression efficiency, single-threaded/multi-threaded throughput and partial decoding (low-resolution and/or region of interest)
    • can chunking can be replaced with full-frame coding and J2K accessibility features? This is more important in lossy coding to avoid edge effects.
    • select a chunk size, i.e., is 256-line a reasonable value?
    • can J2K tiles be used instead of independent codestreams to remove JPEG 2000 codestream header overhead?

@palemieux palemieux changed the title Feature/add ht support rebase Add HTJ2K Compressor Oct 12, 2024
@palemieux palemieux marked this pull request as draft October 12, 2024 07:55
CompressionDesc (
"ht",
"High-Throughtput JPEG 2000 (OpenJPH, full frame)",
16000,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
16000,
16384,

16384 lines?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

16000 is just a placeholder to ensure that the full-frame is encoded in one chunk and thus understand the impact of full-frame vs chunk encoding on coding efficiency.

@TodicaIonut
Copy link

Add Compression Level Quality is controlled by setting. The "base-error" is just Compression Level / 100000.

@palemieux palemieux marked this pull request as ready for review October 13, 2024 18:51
@palemieux
Copy link
Author

Replaced with AcademySoftwareFoundation#1883

@palemieux palemieux closed this Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants