Add perceptual loss from pre-trained CNN

[RobinCSIRO]

Hi, @Avhirup , thank you very much for your project. I am new to the task of 'image inpainting', and I noticed that most of papers of this project adopt another loss, called perceptual loss by calculating the difference of reconstructed image and original image with the feature vector extracted from a pre-trained CNN (e.g., VGG16). Could you please show me how to add such loss into your model or recommend me some other implementation you know to realize such tasks? Thank you very much~~

[avhirupc]

Hey @RobinCSIRO, Unfortunately, i have not read any paper follow the method you are describing. The current SOTA uses two discriminators. Trained to minimize adversarial loss. Also a weighted reconstruction loss is applied just to the generator.(I guess this is similar to what you are talking about). In simple form reconstruction, the loss could be just the mean of the difference between your GT and generators output.
Try looking at this paper,
http://hi.cs.waseda.ac.jp/~iizuka/projects/completion/en/

Also it would be great if you could revert me with the paper you are talking about

[RobinCSIRO]

Hi, @Avhirup , thank you for your reply! It should be noticed that only part of papers adopt such perceptual loss calculated with a pre-trained classification model (e.g., VGG16), and results are proved to be a little bit better than the one without it. Some relevant papers are: https://arxiv.org/abs/1611.09969 https://arxiv.org/abs/1711.09345 and https://arxiv.org/abs/1801.00289. Thx~