SCC5830: Image processing, first semester of 2020 at University of São Paulo (USP), Brazil.
Students:
- Amanda Carrijo Viana Figur (8937736) - Amanda GitHub profile.
- Luiz Augusto Vieira Manoel (8937308) - Luiz GitHub profile.
Abstract:
The aim of this project is to "erase" the eyeglasses of photos by filling removed pixels from people's faces. This technique is known as Inpainting and we made it by using Color Processing, Histogram Analysis, Image Enhancement, Morphology and Segmentation. Our project receives a RGB image of a person's face wearing eyeglasses, extracts the eyeglasses mask and returns the respective inpaited image without the eyeglasses. We used Python 3
to make all our codes and Adobe Photoshop to obtain some of our images. This process might help with facial recognition problems.
Keywords: Inpainting; Faces; Mask; Extraction; Segmentation.
The aim of this project is to "erase" the eyeglasses of photos by filling masked pixels from people's faces. We also implemented a program that generates such mask of pixels. Every code at this project is implemented using Python 3
.
Link for a brief video explaining our work (in Portuguese): https://youtu.be/5S9GbPym8l4
Example:
Input Image | Aim Result |
---|---|
![]() |
![]() |
![]() |
![]() |
Images Source: acquired by the students (Illustrative)
Here you can find the following inpainting codes:
-
inpanting_popular
: the simplest implementation of a inpainting algorithm you can find. Lacks a visual descriptor, only replaces the pixel in the mask area by a pixel outside the mask area; -
inpanting_revolution
: the above algorithm with a visual descriptor. Searches and replaces near pixels; -
inpanting_pyheal
: We used the PyHeal code, a pure Python implementation of Telea article on FMM inpainting to compare its results to our own algorithms. It uses an Image Inpainting Technique Based on the Fast Marching Method (FMM). The idea behind such method is to replace the eyeglasses estimating the image smoothness as a weighted average over a known neighborhood of the pixel to inpaint. The article can be found here: "An Image Inpainting Technique Based on the Fast Marching Method", by Alexandru Telea at the Eindhoven University of Technology, DOI: 10.1080/10867651.2004.10487596.
You can also find the additional codes:
-
mask_generator
: a code that receives an image and a seed point inside the glasses and generates a mask that can be used on any of the inpainting algorithms above; -
plot_compare
: a function you can use to compare two images side by side while running any of the algorithms.
Try to run our demo.py
file and use Final_Project.ipynb
as a reference for understanding all the code implementations. This last code describe with details and compares the results we obtained.
You can find most of the images used in this project (the input, outputs, examples, etc.) on the images folder.
On the table bellow we show the details of each image we used as input on this project. The original Angela Davis and Fidel Castro images can be accessed on Wikimedia and lie under a Creative Commons license. We altered them in order to satisfy our needs.
Image name | Image format | Dimensions | Color Model | Channel Size | Description |
---|---|---|---|---|---|
original_Fidel | .PNG | 436 x 458 | RGB | 8 bit | The original image without eyeglasses (for comparison) |
oculus_Fidel | .PNG | 436 x 458 | RGB | 8 bit | The input Fidel image with artificial eyeglasses |
manual_mask_Fidel | .PNG | 436 x 458 | RGB | 8 bit | The manual eyeglasses mask for Fidel |
original_Angela | .PNG | 436 x 458 | RGB | 8 bit | Angela Davis wearing eyeglasses |