Final Project - Oculus Extinguisher

SCC5830: Image processing, first semester of 2020 at University of São Paulo (USP), Brazil.

Students:

• Amanda Carrijo Viana Figur (8937736) - Amanda GitHub profile.
• Luiz Augusto Vieira Manoel (8937308) - Luiz GitHub profile.

---

Abstract:

The aim of this project is to "erase" the eyeglasses of photos by filling removed pixels from people's faces. This technique is known as Inpainting and we made it by using Color Processing, Histogram Analysis, Image Enhancement, Morphology and Segmentation. Our project receives a RGB image of a person's face wearing eyeglasses, extracts the eyeglasses mask and returns the respective inpaited image without the eyeglasses. We used Python 3 to make all our codes and Adobe Photoshop to obtain some of our images. This process might help with facial recognition problems.

Keywords: Inpainting; Faces; Mask; Extraction; Segmentation.

---

Project Description

The aim of this project is to "erase" the eyeglasses of photos by filling masked pixels from people's faces. We also implemented a program that generates such mask of pixels. Every code at this project is implemented using Python 3.

Link for a brief video explaining our work (in Portuguese): https://youtu.be/5S9GbPym8l4

Example:

Input Image	Aim Result

Images Source: acquired by the students (Illustrative)

Here you can find the following inpainting codes:

• inpanting_popular: the simplest implementation of a inpainting algorithm you can find. Lacks a visual descriptor, only replaces the pixel in the mask area by a pixel outside the mask area;

• inpanting_revolution: the above algorithm with a visual descriptor. Searches and replaces near pixels;

• inpanting_pyheal: We used the PyHeal code, a pure Python implementation of Telea article on FMM inpainting to compare its results to our own algorithms. It uses an Image Inpainting Technique Based on the Fast Marching Method (FMM). The idea behind such method is to replace the eyeglasses estimating the image smoothness as a weighted average over a known neighborhood of the pixel to inpaint. The article can be found here: "An Image Inpainting Technique Based on the Fast Marching Method", by Alexandru Telea at the Eindhoven University of Technology, DOI: 10.1080/10867651.2004.10487596.

You can also find the additional codes:

• mask_generator: a code that receives an image and a seed point inside the glasses and generates a mask that can be used on any of the inpainting algorithms above;

• plot_compare: a function you can use to compare two images side by side while running any of the algorithms.

Try to run our demo.py file and use Final_Project.ipynb as a reference for understanding all the code implementations. This last code describe with details and compares the results we obtained.

You can find most of the images used in this project (the input, outputs, examples, etc.) on the images folder.

Input images

On the table bellow we show the details of each image we used as input on this project. The original Angela Davis and Fidel Castro images can be accessed on Wikimedia and lie under a Creative Commons license. We altered them in order to satisfy our needs.

Image name	Image format	Dimensions	Color Model	Channel Size	Description
original_Fidel	.PNG	436 x 458	RGB	8 bit	The original image without eyeglasses (for comparison)
oculus_Fidel	.PNG	436 x 458	RGB	8 bit	The input Fidel image with artificial eyeglasses
manual_mask_Fidel	.PNG	436 x 458	RGB	8 bit	The manual eyeglasses mask for Fidel
original_Angela	.PNG	436 x 458	RGB	8 bit	Angela Davis wearing eyeglasses