PyTorch Image Quality (PIQ) v0.8.0 Release Notes
New Features
CLIP-IQA
With this release, we introduce implementation of CLIP-IQA. In contrast to other available implementations, our apprach is standalone (say no to any additional dependencies) and delivers the estimations, which match the official implementation.
The image quality is measured based on a general notion of text-to-image similarity learned by the CLIP model during its large-scale pre-training on a large dataset with paired texts and images.
Such approach follows the idea that two antonyms (“Good photo” and “Bad photo”) can be used as anchors in the text embedding space representing good and bad images in terms of their image quality.
After the anchors are defined, one can use them to determine the quality of a given image in the following way:
- Compute the image embedding of the image of interest using the pre-trained CLIP model;
- Compute the text embeddings of the selected anchor antonyms;
- Compute the angle (cosine similarity) between the image embedding (1) and both text embeddings (2);
- Compute the Softmax of cosine similarities (3) -> CLIP-IQA score.
Fixes
- Added proper meshgrid indexing (#359);
- Fixed usage of metrics on GPU (#364);
- Added documentation for new measures (#368);
Full Changelog: v0.7.1...v0.8.0
