Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the code of computing PSNR in the Disco repository is wrong #20

Open
Delicious-Bitter-Melon opened this issue Mar 25, 2024 · 2 comments

Comments

@Delicious-Bitter-Melon
Copy link

The Disco code of computing PSNR as follows is wrong:
https://github.com/Wangt-CN/DisCo/blob/8538889c9ee9edd8dd43ffee182d1a91ce7a9828/tool/metrics/ssim_l1_lpips_psnr.py#L13.

image

As pointed out in Wangt-CN/DisCo#86, the accurate code is mse = np.mean((original/1.0 - compressed/1.0) ** 2) instead of mse = np.mean((original - compressed) ** 2) , because original and compressed images are uint8 in their code, and (original - compressed) * * 2 will cause numerical overflow.

If you use their evaluation code of computing PSNR, please update your results.

@Delicious-Bitter-Melon
Copy link
Author

Delicious-Bitter-Melon commented Mar 25, 2024

In some cases, the wrong computation manner will increase PSNR but in some cases it decreases PSNR.

image

@Delicious-Bitter-Melon
Copy link
Author

Besides, I am certain that the FID calculation method used by Disco is correct, but I am not sure if other metrics such as FVD are correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant