Suggestion: "Factory method" to be able to inspect image and choose encoder

[mortb]

Hi,

Thanks for the great library!

I'm using your lib to resize images that our users upload to our web site.
The images are used as avatars and may (as I imagine) be either photos, or more of a drawn logotype kind.
Therefore I reason that "logotypes" should be compressed lossless and images lossy so I choose png as lossless output and jpeg as lossy.
While it is easy to know what encoder to use when the input content types are bmp, png, gif or jpeg, it is less easy when images are in the newer webp and heic formats that may encapsulate either lossless or lossy pictures. It would also be useful to know if the image contains animation. Therefore I'm suggesting some kind of factory method that may be used to load enough of the image to determine properties like lossless / lossy state, still or animated image and maybe some other properties like if image has indexed palette and color depth? The factory method could be used to return IEncoderOptions suited to the input

"Sketchy pseudo code" definitions

class ImageProperties
{
      public string OriginalContentType { get; set; } // image/jpeg, image/webp etc
      public bool Lossless { get; set }
      public bool IsAnimation { get; set; }
      public bool IsIndexed { get; set; }
}
MagicImageProcessor.AnalyzeChooseEncoder(Stream stream, string originalContentType, Func<ImageProperties, IEncoderOptions>factoryMethod);

usage

MagicImageProcessor.AnalyzeChooseEncoder(myStream, "image/webp", imgProps => imgProps.IsAnimation? new GifEncoderOptions { ... } ? imgProps.Lossless ? new PngEncoderOptions {...} : new JpegEncoderOptions {...} );

I don't know if the above is something that is achievable in your current API or if you would consider it suitable for adding as a feature.

[saucecontrol]

Thanks for the suggestion! I was just thinking of something similar a few days ago. The current API has ImageFileInfo.Load, which will give you the original format and can be used to detect animation by checking frame count. However, that's more expensive that it needs to be since e.g. in GIF and WebP, the frame count can only be retrieved by parsing the entire file. And as you point out, some of the more modern formats like JXL and WebP support both lossy and lossless, which would be good to know sometimes.

I am looking at overhauling the way basic image metadata is exposed after I get the remaining base set of standard codecs integrated. I'll give this some thought and see what I can come up with.

[mortb]

Thanks! I'll look into ImageFileInfo.Load. If calls don't take seconds to complete I may use it for now. Basic metadata image metadat could also expose if the image features transparency and pallette size if possible. It might be useful to know when trying to match the image with an encoder. I guess an indexed png takes less space than nonindexed, so knowing if the image palette size is "small" an indexed encoder might be used

[saucecontrol]

Transparency is already available as ImageFrameInfo.HasAlpha if that's useful to you.

The idea of using the properties of the source image to customize the output is good in general, but it also requires an assumption that the source image was encoded sensibly, which unfortunately is often not true. We've all seen the terrible JPEG-compressed line art or text examples, and we just as often see PNG-compressed photos because someone has taken a screenshot of a JPEG. 🤦‍♂️

The other issue is that when you resize an image, you change its characteristics. The animated pixel art WebP you shared earlier in the week is a good example of this. After resize with the default settings, the encoded image is larger than the original because it introduces false edges. Those can be addressed by resizing with NearestNeighbor interpolation and disabling sharpening, but you have to know what the content is to know that's necessary. Similarly, an indexed color image, after resize, may have significantly more colors than the original because dithered patterns blur into the colors they are supposed to appear to be. Perhaps there's no good way of dealing with these things, but it's something to think about.

[mortb]

We've all seen the terrible JPEG-compressed line art or text examples, and we just as often see PNG-compressed photos because someone has taken a screenshot of a JPEG. 🤦‍♂️

Yeah, I know garbage in => garbage out. I guess we've all seen nice vector based logos that have been compressed to crap with jpeg. I'm trying to give the user a possibility to have a "fair" compression, the compressed output will be displayed to the user after their upload. However, the best compression will always be chosen by hand (or a sufficiently well trained AI in close future?)