FACENET

A TensorFlow backed FaceNet implementation for Node.js, which can solve face verification, recognition and clustering problems.

FaceNet is a deep convolutional network designed by Google, trained to solve face verification, recognition and clustering problem with efficiently at scale.

directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity.
optimize the embedding face recognition performance using only 128-bytes per face.
achieves accuracy of 99.63% on Labeled Faces in the Wild (LFW) dataset, and 95.12% on YouTube Faces DB.

INSTALL

$ npm install facenet numjs flash-store

Peer Dependencies

numjs
flash-store

EXAMPLE

The follow examples will give you some intuitions for using the code.

demo exmaple will show you how to do align for face alignment and embedding to get face feature vector.
visualize example will calculate the similarity between faces and draw them on the photo.

1. Demo for API Usage

TL;DR: Talk is cheap, show me the code!

import { Facenet } from 'facenet'

const facenet = new Facenet()

// Do Face Alignment, return faces
const imageFile = `${__dirname}/../tests/fixtures/two-faces.jpg`
const faceList = await facenet.align(imageFile)

for (const face of faceList) {
  console.log('bounding box:',  face.boundingBox)
  console.log('landmarks:',     face.facialLandmark)

  // Calculate Face Embedding, return feature vector
  const embedding = await facenet.embedding(face)
  console.log('embedding:', embedding)
}
faceList[0].embedding = await facenet.embedding(faceList[0])
faceList[1].embedding = await facenet.embedding(faceList[1])
console.log('distance between the different face: ', faceList[0].distance(faceList[1]))
console.log('distance between the same face:      ', faceList[0].distance(faceList[0]))

Full source code can be found at here: https://github.com/zixia/node-facenet/blob/master/examples/demo.ts

The output should be something like:

image file: /home/zixia/git/facenet/examples/../tests/fixtures/two-faces.jpg
face file: 1-1.jpg
bounding box: {
  p1: { x: 360, y: 95 }, 
  p2: { x: 589, y: 324 } 
}
landmarks: { 
  leftEye:  { x: 441, y: 181 },
  rightEye: { x: 515, y: 208 },
  nose:     { x: 459, y: 239 },
  leftMouthCorner:  { x: 417, y: 262 },
  rightMouthCorner: { x: 482, y: 285 } 
}
embedding: array([ 0.02453, 0.03973, 0.05397, ..., 0.10603, 0.15305,-0.07288])

face file: 1-2.jpg
bounding box: { 
  p1: { x: 142, y: 87 }, 
  p2: { x: 395, y: 340 } 
}
landmarks: { 
  leftEye:  { x: 230, y: 186 },
  rightEye: { x: 316, y: 197 },
  nose:     { x: 269, y: 257 },
  leftMouthCorner:  { x: 223, y: 273 },
  rightMouthCorner: { x: 303, y: 281 } 
}
embedding: array([ 0.03241, -0.0737,  0.0475, ..., 0.07235, 0.12581,-0.00817])

2. Visualize for Intuition

Face is in the green rectangle.
Similarity(distance) between faces showed as a number in the middle of the line.
To identify if two faces belong to the same person, we could use an experiential threshold of distance: 0.75.

$ git clone git@github.com:zixia/node-facenet.git
$ cd facenet
$ npm install
$ npm run example:visualize

01:15:43 INFO CLI Visualized image saved to:  facenet-visulized.jpg

3. Get the diffence of two face

Get the two face's distance, the smaller the number is, the similar of the two face

import { Facenet } from 'facenet'

const facenet = new Facenet()
const imageFile = `${__dirname}/../tests/fixtures/two-faces.jpg`

const faceList = await facenet.align(imageFile)
faceList[0].embedding = await facenet.embedding(faceList[0])
faceList[1].embedding = await facenet.embedding(faceList[1])
console.log('distance between the different face: ', faceList[0].distance(faceList[1]))
console.log('distance between the same face:      ', faceList[0].distance(faceList[0]))

Output:
distance between the different face: 1.2971515811057608
distance between the same face: 0

In the example,
faceList[0] is totally the same with faceList[0], so the number is 0
faceList[1] is different with faceList[1], so the number is big.
If the number is smaller than 0.75, maybe they are the same person.

Full source code can be found at here: https://github.com/zixia/node-facenet/blob/master/examples/distance.ts

4. Save the face picture from a picture

Recognize the face and save the face to local file.

import { Facenet } from 'facenet'

const facenet = new Facenet()
const imageFile = `${__dirname}/../tests/fixtures/two-faces.jpg`

const faceList = await facenet.align(imageFile)
for (const face of faceList) {
  await face.save(face.md5 + '.jpg')
  console.log(`save face ${face.md5} successfuly`)
}
console.log(`Save ${faceList.length} faces from the imageFile`)

Full source code can be found at here: https://github.com/zixia/node-facenet/blob/master/examples/get-face.ts

FACENET MANAGER

UNDER HEAVY DEVELOPMENT NOW

Roadmap: release facenet-manager on version 0.8

The above ascii recording is just for demo purpose. Will replace it with facenet-manager later.

DOCUMENT

See auto generated docs

INSTALL & REQUIREMENT

$ npm install facenet

OS

Supported:

Linux
Mac
Windows

Dependency

Node.js >= 7 (8 is recommend)
Tensorflow >= 1.2
Python3 >=3.5 (3.6 is recommend)

Make sure you run those commands under Ubuntu 17.04:

sudo apt install python3-pip
pip3 install setuptools --upgrade

Ram

Neural Network Model	Task	Ram
MTCNN	Facenet#align()	100MB
Facenet	Facenet#embedding()	2GB

If you are dealing with very large images(like 3000x3000 pixels), there will need additional 1GB of memory.

So I believe that Facenet will need at least 2GB memory, and >=4GB is recommended.

API

Neural Network alone is not enough. It's Neural Network married with pre-trained model, married with easy to use APIs, that yield us the result that makes our APP sing.

Facenet is designed for bring the state-of-art neural network with bleeding-edge technology to full stack developers.

Facenet

import { Facenet } from 'facenet'

const facenet = new Facenet()
facenet.quit()

1. Facenet#align(filename: string): Promise<Face[]>

Do face alignment for the image, return a list of faces.

2. Facenet#embedding(face: Face): Promise<FaceEmbedding>

Get the embedding for a face.

face.embedding = await facenet.embedding(face)

Face

Get the 128 dim embedding vector for this face.(After alignment)

import { Face } from 'facenet'

console.log('bounding box:',  face.boundingBox)
console.log('landmarks:',     face.facialLandmark)
console.log('embedding:',     face.embedding)

ENVIRONMENT VARIABLES

FACENET_MODEL

FaceNet neural network model files, set to other version of model as you like.

Default is set to models/ directory inside project directory. The pre-trained models is come from 20170512-110547, 0.992, MS-Celeb-1M, Inception ResNet v1, which will be download & save automatically by postinstall script.

$ pwd
/home/zixia/git/node-facenet

$ ls models/
20170512-110547.pb
model-20170512-110547.ckpt-250000.index
model-20170512-110547.ckpt-250000.data-00000-of-00001
model-20170512-110547.meta

DOCKER

DEVELOP

$ git clone git@github.com:zixia/node-facenet.git
$ cd facenet
$ npm install
$ npm test

COMMAND LINE INTERFACES

align

Draw a rectangle with five landmarks on all faces in the input_image, save it to output_image.

$ ./node_modules/.bin/ts-node bin/align.ts input_image output_image

embedding

Output the 128 dim embedding vector of the face image.

$ ./node_modules/.bin/ts-node bin/embedding.ts face_image

RESOURCES

Machine Learning

Machine Learning is Fun! Part 4: Modern Face Recognition with Deep Learning
Face recognition using Tensorflow
Google: Our new system for recognizing faces is the best one ever
A tensorflow implementation of "Deep Convolutional Generative Adversarial Networks
What does Locality Sensitive Hashing Forests do? · maheshakya/my_first_project Wiki
Average Face : OpenCV ( C++ / Python ) Tutorial

Python3

Google Python Style Guide
PyLint, PyChecker or PyFlakes?
Useful Python Modules: Flake8
PEP 8 - Style Guide for Python Code
Python 3.6 venv — Creation of virtual environments

1. Typing

Mypy syntax cheat sheet (Python 3)
Python 3 Type Hints and Static Analysis
typing — Support for type hints

1. NumJS

Stackoverflow: numpy-like package for node
Read/manipulate/display images using NumJs
Numjs - Like NumPy, in JavaScript
ndarray - Modular multidimensional arrays for JavaScript

Dataset

LFW - Labeled Faces in the Wild

TODO

NPM Module: facenet
Docker Image: zixia/facenet
Examples
- API Usage Demo
- Triple Distance Visulization Demo
- Performance Test(Align/Embedding/Batch)
- Validation Test(LFW Accuracy)
Neural Network Models
- Facenet
- Mtcnn
- Batch Support
~~Python3 async & await~~
Divide Different Neural Network to seprate class files(e.g. Facenet/Mtcnn)
K(?)NN Alghorithm Chinese Whispers
TensorFlow Sereving
OpenAPI Specification(Swagger)

INSPIRATION

This repository is heavily inspired by the following implementations:

FaceNet by David Sandberg @davidsandberg
OpenFace by CMU Satya Lab @cmusatyalab

CREDITS

Face alignment using MTCNN: Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks
Face embedding using FaceNet: FaceNet: A Unified Embedding for Face Recognition and Clustering
TensorFlow implementation of the face recognizer: Face recognition using Tensorflow

CONTRIBUTE

FaceNet Badge

[![Powered by FaceNet](https://img.shields.io/badge/Powered%20By-FaceNet-green.svg)](https://github.com/zixia/node-facenet)

CHANGELOG

Master

Added facenet-manager command line tool for demo/validate/sort photos
Switch to FlashStore npm module as key-value database

v0.3 Sep 2017

Added three cache classes: AlignmentCache & EmbeddingCache & FaceCache.
Added cache manager utilities: embedding-cache-manager & alignment-cache-manager & face-cache-manager
Added Dataset manager utility: lfw-manager (should be dataset-manager in future)
BREAKING CHANGE: Face class refactoring.

v0.2 Aug 2017 (BREAKING CHANGES)

Facenet#align() now accept a filename string as parameter.
BREAKING CHANGE: FaceImage class had been removed.
BREAKING CHANGE: Face class refactoring.

v0.1 Jul 2017

npm run demo to visuliaze the face alignment and distance(embedding) in a three people photo.
Facenet.align() to do face alignment
Facenet.embedding() to calculate the 128 dim feature vector of face
Initial workable version

TROUBLESHOOTING

Dependencies

OS	Command
os x	`brew install pkg-config cairo pango libpng jpeg giflib`
ubuntu	`sudo apt-get install libcairo2-dev libjpeg8-dev libpango1.0-dev libgif-dev build-essential g++`
fedora	`sudo yum install cairo cairo-devel cairomm-devel libjpeg-turbo-devel pango pango-devel pangomm pangomm-devel giflib-devel`
solaris	`pkgin install cairo pango pkg-config xproto renderproto kbproto xextproto`
windows	instructions on our wiki

more os see node-canvas Wiki.

FAQ

facenet-manager display not right under Windows

See: Running Terminal Dashboards on Windows

Error when install: No package 'XXX' found

It's related with the NPM module canvas.

Error messages:

No package 'pixman-1' found
No package 'cairo' found
No package 'pangocairo' found

Solution for Ubuntu 17.04:

sudo apt install -y libpixman-1-dev
sudo apt-get install -y libcairo2-dev
sudo apt-get install -y libpango1.0-dev

Solution for Mac:

brew install python3
brew install pkg-config
brew install cairo
brew install pango
brew install libpng
brew install libjpeg

Error when install: fatal error: jpeglib.h: No such file or directory