This is a Tensorflow implementation of the object detector described by the paper "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks" by Ren et al. (NIPS2015) and "Deep Residual Learning for Image Recognition" by He et al. (CVPR2016). Given an image, it predicts the bounding box and label of each object in the image. It uses a Region Proposal Network (RPN) to find a set of rectangular cadidate regions, and uses a Fast R-CNN to classify these regions. To improve the efficiency, the RPN and Fast R-CNN modules share their convolutional layers. These convolutional layers need to be initialized from VGG16, ResNet50, ResNet101 or ResNet152 net, which can be obtained by using Caffe-to-Tensorflow.

Note: I perform the RoI pooling layer by concatenating a RoI warping layer and a standard max pooling layer. The RoI warping layer is currently implemented in NumPy. It remains to add this layer to Tensorflow and achieve end-to-end training of the whole system.

References

• Fast R-CNN. Ross Girshick. ICCV 2015.
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun. NIPS 2015.
• Deep Residual Learning for Image Recognition. Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. CVPR 2016.
• Instance-aware Semantic Segmentation via Multi-task Network Cascades. Jifeng Dai, Kaiming He, Jian Sun. CVPR 2016.
• Faster R-CNN in Caffe (Matlab version)
• Faster R-CNN in Caffe (Python version)
• Microsoft COCO dataset
• PASCAL VOC dataset
• Caffe to Tensorflow