EmotionUI

Software for Multimodalty 2D+3D Facial Expression Recognition (FER) UI.

demo screenshot

(with RealSense)

required packages

• Python >= 3.6
• numpy >= 1.19.5
• Opencv-python >= 4.5
• PySide6 >= 6.2.1
• PyTorch >= 1.10
• TorchVision >= 0.11

hardware

1. usb camera
2. Intel RealSense (optional for depth imaging)
3. NV GPU (optional, CPU version also works)

loding weights

Two ways, try them, depending on your Internet speed.

1. Manually download. Download vgg16 weights from:

   https://drive.google.com/file/d/1f-tKgovJ54l9xR3oIZ6gy77NdPirUddr/view?usp=sharing

   Then, move the weights to "./weights" folder, and rename it with "ui_weights.pth".

or

2. Run script. Open terminal, run:

   > cd weights
   > python download_weights.py

train your own model

> cd train/dataset
> python main_train.py

This will also allow you to collect your own expression dataset before the network training.

After the training, remember to move the new weights to "weights" folder, and rename it with the "ui_weights.pth".

P.S.

The current FER network is implemented by vgg16 for simplicity. Current predictions are based on 2D (RGB texture) result.

One can easily change the network as you wish.