Color conversion with ICC profiles #1567
Conversation
|
I wonder why the test |
|
@brianpopow It'll be an accuracy issue most likely. (I hope it's not a JIT issue). It should be possible to inspect the result and see. |
Codecov Report
@@ Coverage Diff @@
## main #1567 +/- ##
======================================
Coverage ? 87%
======================================
Files ? 1023
Lines ? 55212
Branches ? 7052
======================================
Hits ? 48227
Misses ? 5768
Partials ? 1217
Flags with carried forward coverage won't be shown. Click here to find out more. 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
The issue only happens with a Release build. I think i found the reason, but it seems very weird. This can be seen with the testoutput:
|
@brianpopow Woah! That's bonkers! |
I have reported this issue: dotnet/runtime#55623 They confirmed the issue, but they say its unlikely to be fixed because netcore2.1 is out of support in august. |
|
@JimBobSquarePants It would be really nice, if we could bring this PR forward. This would be a good addition to ImageSharp. I thought, I may ask you, if you know what the PR needs (besides tests) to be finished? What would be the right way to apply an embedded color profile? My first attempt was: Here is an example image with adobe rgb color profile:
This does not seems to work, the colors seem to be wrong. Here are more example images |
|
@brianpopow Honestly..... I don't know. I was hoping the OP would come back to finish things off. I've just kept things updated over the years and hadn't gotten involved at all in the implementation as yet. Judging from the old comments in the previous PR I believe the code is based somewhat based on the following https://github.com/InternationalColorConsortium/DemoIccMAX/tree/master/IccProfLib As for accuracy. That result looks like it's just spitting out the sRGB values again. I do agree that it would be an awesome addition to the library and would save a ton of headaches. I hoped we'd get it in V3 but that's a mighty big ask. |
|
I think we definitely need a reference implementation to compare the results against. I tried BABL which gnome is using, but i could not get it to work on windows. I will take a look at DemoIccMAX |
Apologies for the slow response and thank you for your investigation. Yes...That makes perfect sense.... 😞 Our code is assuming that the converters are operating in a manner that can handle the input Vector using an RGBA layout with values ranging from 0-1. Of course, this isn't the case and is a huge bug. When converting from a This is going to get messy given that our color-space converter expects a generic type. |
|
The fix for this (dear reader) will be to update If anyone has time to make a start on the first part I can run with the second. |
This sounds like something I'm conceptually familiar with so I'll aim to find some time over the holidays (though if anyone else is keen, please go ahead) |
|
@waacton if you can please do. 👍 |
|
@JimBobSquarePants I've been exploring the The kind of things lurking in my mind are:
|
Hi @waacton Thank you for having a look at this and sorry for the slow reply. My computer time is limited currently while I recover from a few "long term super bad posture and I really should know better" injuries. I had a flash of inspiration the last night and figured out a way to extend the The gist of it is as follows:
The type Instead, the plan is to pass any ICC profiles to the converter when we do any color space conversion using it. This code, unfortunately, is scattered through the various formats. I also need to massively simplify the JPEG colorspace conversion code to use only (hopefully I had a look at the UniColor code earlier and I cannot seem to understand how you normalize values for conversion? It's making me very much doubt myself that my approach is sound. Using the library would be a very good idea for test references. On tests, we need to test a lot of things. The more test data we can compare against the better. I just had to disable some tests CC @brianpopow |
Not a problem in the slightest I'm just trying to get a head start, and I feel a bit bad that you have to keep getting so heavily involved 😅
Yeah it's not particularly obvious. You can't simply give the ICC LUTs actual LAB values (sometimes PCS adjustment is needed, need to map to [0, 1], and potentially a correction for ICC v2 LAB values) or actual XYZ values (sometimes PCS adjustment is needed, need to map to [0, ~0.5] 😕) so I've tried to isolate everything ICC-related from the pure unrestricted maths of the non-ICC colour spaces, which ends up looking something like:
So I think the normalisation you're looking for is in point 2, and is a layer that you might be missing. Basically, all the glue I use between standard colour spaces and the ICC profile is in Looking at your latest changes, that was the direction I was trying to go, I just hadn't worked out how - still getting accustomed to the codebase! For what it's worth, I at least had an If this is already able to perform ICC conversions would you object if I
|
|
One additional question @JimBobSquarePants - the latest update to |
|
Hi @waacton was just about to reply to your message.
I'm not sure we're talking about normalization in the same regard. I've covered the LAB/XYZ PCS normalization (I used your code as a reference) but I also meant that the to convert from the data to PCS you would need to normalize to [0-1] scaling. For example, if my RGB component values in the image data are scaled [0-255] I would have to first convert them to [0-1]
I think 1. is a definite requirement. You've solved the problem for your needs so we should compare to an accessible working solution. I think 2 has already been covered in the code I've written.
That's something I never actually considered. Given the type parameter constraints I think we'd have to write multiple methods to allow the conversion which might get complicated fast. Would complete the API though nicely. Perhaps a nice to have which can be added down the line. What do you think of the approach I've developed for |
I noticed that a few moments after I sent that message! 🤦
Ah yes I see, in that case that goes back to my comment that Unicolour funnels every ICC conversion through my tried-and-tested XYZ colour space (even if it means a redundant conversion, e.g. LAB ⇒ XYZ in
As a newcomer I found it a bit confusing but it's slowly clicking into place. Things like With regards to ICC ⇒ non-ICC, that makes things much easier - and is a big part of what I was struggling with! I agree it's a nice-to-have for a more complete API later on though no need to further complicate matters right now. I've got a wee demo testing values converting from FOGRA39 to SWOP2006 (and various permutations) using |
This is absolutely the BEST news!! I've sent you an invite. You should be able to push directly to this branch once accepted. |
|
|
|
Doesn't feel like enough code for all the discussion around it 😄 - let me know if there's anything I need to change to meet the project's standards and best practices. Otherwise if you give the thumbs up, and any opinions about what kind of values to test, I'll expand the tests. Dare I ask about profiles with non-CMYK data colour spaces? For instance, I've tested Unicolour with a 7-channel profile representing CMYKOGV but that seems pretty niche. But are "input" profiles for things like cameras and scanners something that ImageSharp is likely to encounter? |
|
@waacton I did some cleanup and added ICC profiles to LFS since they're quite chunky. It's great to see tests passing! I think we should try to expand on these tests to cover as many conversion types as possible (e.g CMYK to RGB is important) and then once confident it works, I can start wiring up the transforms to the individual formats.
This is something that @saucecontrol has brought up before. I don't actually know what we might encounter. AFAIK image formats are very unlikely to contain non-standard profiles but that doesn't mean we should not potentially consider them. At the moment, as you've spotted, we're limited to 4 channels since we use |
|
@JimBobSquarePants the expanded tests uncovered some tricky things around perceptual rendering intent for v2 profiles (which seems to be the most common type of profile) that I've tried to address. I plan to work on it more later this week but I've focused on getting tests passing over optimised code for now. Notably I've introduced an extra step in the ICC conversion, where all conversions go via XYZ - even if both profiles have LAB PCS - in case a PCS adjustment needs to be made (performed in XYZ space). Shouldn't be too hard to bypass some of the work when unnecessary if the additional overhead is problematic, just more branching surely. I also feel like I've butchered the use of vectors a bit, would appreciate a review there and a point in the right direction 🙏 |
| @@ -33,64 +34,103 @@ internal static TTo ConvertUsingIccProfile<TFrom, TTo>(this ColorProfileConverte | |||
| MemoryAllocator = converter.Options.MemoryAllocator, | |||
|
|
|||
| // TODO: Double check this but I think these are normalized values. | |||
| SourceWhitePoint = CieXyz.FromScaledVector4(new(converter.Options.SourceIccProfile.Header.PcsIlluminant, 1F)), | |||
| TargetWhitePoint = CieXyz.FromScaledVector4(new(converter.Options.TargetIccProfile.Header.PcsIlluminant, 1F)), | |||
| SourceWhitePoint = new CieXyz(converter.Options.SourceIccProfile.Header.PcsIlluminant), | |||
There was a problem hiding this comment.
Are the illuminant values not given using the ICC scaling? I would have assumed they were given we need to pass them as such.
There was a problem hiding this comment.
This has sent me down a rabbit hole and I don't feel any closer to understanding why, but the illuminant values are stored in the header in the range [0, 1], no scaling needed.
I can't find solid information why the scaling is even needed for XYZ LUTs other than "that's what the DemoIccMAX code does". The closest thing I can find in the v4 spec itself is this footnote in Annex F.3 page 102:
NOTE A three-component Matrix-based model can alternatively be represented in a lutAToBType tag with M curves, a matrix with zero offsets, and identity B curves. While the M curves are set to the corresponding TRC curves, matrix values from the three-component Matrix-based model need to be scaled by (32 768/65 535) before being stored in the lutAToBType matrix in order to produce equivalent PCS values. (32 768/65 535) represents the encoding factor for the PCS PCSXYZ encoding.
(The spec is so cumbersome, the information I'm looking for could easily be buried elsewhere...)
At this point I'm assuming either
- XYZ LUT data is in [0, ~0.5] by convention (or by something in the spec I can't find)
- XYZ LUT data range is profile-specific, and I've not encountered one that isn't [0, ~0.5] (or DemoIccMAX doesn't account for the possibility)
🤕
One other note, as far as I understand the PCS illuminant must be D50 (in case that enables any further optimisation)
7.2.16 PCS illuminant field (Bytes 68 to 79)
The PCS illuminant field shall contain the nCIEXYZ values of the illuminant of the PCS, encoded as an
XYZNumber. The value, when rounded to four decimals, shall be X = 0,9642, Y = 1,0 and Z = 0,8249.
There was a problem hiding this comment.
Thinking about it a bit more, it's going to be related to LUTs storing uInt16 [0, 65535] but XYZ values being encoded as s15Fixed16 [-32768, ~32768], and needing to account for that.
| IccVersion targetVersion = targetHeader.Version; | ||
|
|
||
| // all conversions are funnelled through XYZ in case PCS adjustments need to be made | ||
| CieXyz xyz; |
There was a problem hiding this comment.
Reading below we only need this if adjustPcsForPerceptual is true. I'd rather avoid the overhead of additional conversions when not necessary. We'll be using this code in our decoder which must be fast.
There was a problem hiding this comment.
I'll take a shot at avoiding the overhead when unnecessary. I expect it will result in functions that look very similar like ConvertIcc() and ConvertIccWithPerceptualAdjustment() - I can't see a natural if (adjustmentNeeded) { PerformExtraStep() } at the moment
| { | ||
| // Convert from Lab v4 to Lab v2. | ||
| pcs = LabToLabV2(pcs); | ||
| Vector3 iccXyz = xyz.ToScaledVector4().AsVector128().AsVector3(); |
There was a problem hiding this comment.
It looks like we're mixing up normalized and standard values here and it's a little confusing.
// We use the original ref values here...
Vector3 scale = Vector3.One - Vector3.Divide(refBlack.ToVector3(), refWhite.ToVector3());
// But scale them here?
Vector3 offset = refBlack.ToScaledVector4().AsVector128().AsVector3();I would extract the methods out with an explanation of the theory behind them also. For example, I don't understand why the math for source and targeted PCS adjustments is different. We're going to need to vectorize these also. (Which may mean providing your reference colors as Vector4)
There was a problem hiding this comment.
Yep, happy to refactor to methods with explanations. I think I need to do some reading on best practices regarding Vectors etc.
| CieLab lab = pcsConverter.Convert<CieXyz, CieLab>(in xyz); | ||
| pcs = lab.ToScaledVector4(); | ||
| case IccColorSpaceType.CieLab: | ||
| if (sourceConverter.Is16BitLutEntry) |
There was a problem hiding this comment.
Why is a 16bit LUT calculator treated differently and why is that not version specific?
There was a problem hiding this comment.
LAB encodings only changed for 16-bit LUTs from v2 to v4.
The LAB encodings themselves:
- 16-bit max values for 100 & 127 were
FF00in v2 and becameFFFFin v4 - 8-bit max values for 100 & 127 were
FFin v2 and stayedFFin v4
But for the LUTs, the 16-bit type continues to use the legacy encoding:
For colour values that are in the PCSLAB colour space on the PCS side of the tag, this tag uses the legacy 16-
bit PCSLAB encoding defined in Tables 42 and 43, not the 16-bit PCSLAB encoding defined in 6.3.4.2. This
encoding is retained for backwards compatibility with profile version 2.
Prerequisites
Description
Note: This is a replacement for the original PR #273 from @JBildstein that was automatically closed by our Git LFS history rewrite. Individual commits have unfortunately been lost in the process. Help is very much needed to complete the work.
As the title says, this adds methods for converting colors with an ICC profile.
Architecturally, the idea is that the profile is checked once for available and appropriate conversion methods and a then a delegate is stored that only takes the color values to convert and returns the calculated values. The possible performance penalty for using a delegate is far smaller than searching through the profile for every conversion. I'm open for other suggestions though.
There are classes to convert from the profile connection space (=PCS, can be XYZ or Lab) to the data space (RGB, CMYK, etc.) and vice versa. There are also classes to convert from PCS to PCS and Data to Data but they are only used for special profiles and are not important for us now but I still added them for completeness sake.
A challenge here is writing tests for this because of the complexity of the calculations and the big amount of different possible conversion paths. This is a rough list of the paths that exist:
The three main approaches in that list are
The most used approaches are Color Trc for RGB profiles and LutAToB/LutBToA for CMYK profiles.
Todo list:
Help and suggestions are very welcome.