Overview

---

Lightweight face detection and recognition library, which can be recognized in real time on jetson/pc devices. The algorithm uses mature MTCNN/Ultra-Light-Fast-Generic-Face-Detector-1MB face detection+InsightFace(Arcface+cosface head Face recognition with +MobileFacenet, resnet backbone), the recognition range is limited to a closed set range. This project also adds Servo tracking face position function,opencv-cuda acceleration + voice recognition broadcast + Bluetooth control. Among them, "matlabPro/" is a functional module independent of other python/c++, equivalent to face detection + tracking + HC08 Bluetooth module transmit signal + voice broadcast and other functions.

轻量级的人脸检测和识别库，可在jetson/pc设备上实时识别，算法采用成熟的MTCNN/Ultra-Light-Fast-Generic-Face-Detector-1MB人脸检测+InsightFace(Arcface+cosface head和+MobileFacenet、resnet backbone)进行人脸识别，识别范围限定为闭集范围内。 本项目还增加舵机跟踪人脸位置功能，opencv-cuda加速+语音识别播报+蓝牙控制。其中，"matlabPro/"是独立与其他python/c++的功能模块，等价的人脸检测+追踪+HC08蓝牙模块发射信号+语音播报等功能。

software main requirements

for pc train and inference

• numpy
• pre-build opencv-python or opencv with cuda support
• pytorch
• tensorRT
• pyserial
• pyaudio
• arduino IDE
• matlab2020a+(and arduino support package)

for jetson inference

• numpy
• pre-build opencv-python or opencv with cuda support
• pytorch
• tensorRT
• pyserial

hardware main requirements

• jetson(nano,tx2,nx,...)
• arduino(nano,mega2560,...)
• 2 high precision digit servo
• rotating platform(steering gear bracket)
• high fps usb camera or ordinary camera
• HC-08 bluetooth module,reference product or tao bao
• serial port voice broadcast module,reference productor tao bao
• arduino voice recording module,reference product or tao bao
• breadboard
• dupont Line

datasets

train datasets （pc上完成）

参考其中项目InsightFace_Pytorch数据集,数据量太大，这里只给出了101人的人脸图像.
百度网盘，提取码：nfy3
google drive
训练数据集目录结构如下，命名无要求。

yourTrainDataFolder/
        ---> train_name1/
            ---> id1_1.jpg
			---> ...
        ---> train_name2/
            ---> id2_1.jpg
			---> ...
        ---> train_name3/
            ---> id3_1.jpg
           ---> id3_2.jpg
		   ---> ...

test datasets (pc或者jetson上完成)

在当前项目目录下的文件夹 ./dataSets/facebank/准备自己需要识别的人脸图像,目录结构也类似上面排布，命名无要求，文件夹名与姓名同名最佳。

./dataSets/facebank/
        ---> zhangsan/
            ---> id1_1.jpg
			---> ...
        ---> lisi/
            ---> id2_1.jpg
			---> ...
        ---> wangwu/
            ---> id3_1.jpg
           ---> id3_2.jpg
		   ---> ...
		---> ...
		...
		facebank.pth
		names.npy

提供已知的人脸图像还可以使用prepare_getFaces.py摄像头在线采集，图像会自动存储到上述路径。若有多幅图像，则特征取平均值。

How to recognize face (use camera to inference,for pc and jetson)

1. Prepare the data set and model: The data set is the above test datasets and needs to recognize the known face images, and the model is the appropriate one under the folder ./models/;
   
2. Prepare the face database: python prepare_faceDatabase.py stores the features and names of the face database, and will automatically generate facebank.pth and names.npy in the above directory structure;
   
3. Online camera recognition: python camera_faceRec.py
   

---

1、准备数据集和模型:数据集为上述test datasets自己需要识别已知的人脸图像，模型为./models/文件夹下选择适合的;
2、准备人脸数据库：python prepare_faceDatabase.py进行人脸数据库特征和姓名存储，会自动生成上述目录结构中的facebank.pth和names.npy；
3、摄像头在线识别：python camera_faceRec.py

How to train your model (for pc)

准备好上述的train datasets，设置好一些arguments，执行train.py即可。

models

包含人脸检测和识别模型（IR-SE50，Mobilefacenet），caffemodel,onnx,pth类型.
百度网盘，提取码f3v6
google drive

performance

model	LFW(%)	CFP-FF(%)	CFP-FP(%)	AgeDB-30(%)	calfw(%)	cplfw(%)	vgg2_fp(%)
ir_se50	0.9952	0.9962	0.9504	0.9622	0.9557	0.9107	0.9386
mobilenet-face	0.9918	0.9891	0.8986	0.9347	0.9402	0.866	0.9100

reference statistics

jetson-opencv-cuda编译

jetson-opencv-cuda编译

References

• Arcface(paper)
• Insightface(github)
• InsightFace_Pytorchgithub
• Ultra-Light-Fast-Generic-Face-Detector-1MB
• MTCNN_Pytorch
• onnx转tensorRT-linux
• pytorch quantization
• onnxruntime quantization
• MobileFacenet(paper)
• jetson_Nano