This branch of py-R-FCN extend py-R-FCN by integrating MKL DNN API accelerations and doing many extra OpenMP/Zero-Copy optimizations to make py-RFCN faster on Intel platforms. We can get 20x acceleration compared with Vanilla CPU Caffe in Pascal-VOC RFCN end2end case in Xeon E5 2699-v4.
- support fine tune models, I am fixing model parse issue in BN layer, you can run below cmd to test performance:
$ ./experiments/scripts/rfcn_end2end.sh 0 ResNet-101 pascal_voc
- implement softmax_loss_ohem_layer's CPU.
R-FCN: Object Detection via Region-based Fully Convolutional Networks
py-R-FCN now supports both joint training and alternative optimization.
The official R-FCN code (written in MATLAB) is available here.
py-R-FCN is modified from the offcial R-FCN implementation and py-faster-rcnn code, and the usage is quite similar to py-faster-rcnn.
There are slight differences between py-R-FCN and the official R-FCN implementation.
- py-R-FCN is ~10% slower at test-time, because some operations execute on the CPU in Python layers (e.g., 90ms / image vs. 99ms / image for ResNet-50)
- py-R-FCN supports both join training and alternative optimization of R-FCN.
The original py-faster-rcnn uses class-aware bounding box regression. However, R-FCN use class-agnostic bounding box regression to reduce model complexity. So I add a configuration AGNOSTIC into fast_rcnn/config.py, and the default value is False. You should set it to True both on train and test phase if you want to use class-agnostic training and test.
OHEM need all rois to select the hard examples, so I changed the sample strategy, set BATCH_SIZE: -1
for OHEM, otherwise OHEM would not take effect.
In conclusion:
AGNOSTIC: True
is required for class-agnostic bounding box regression
BATCH_SIZE: -1
is required for OHEM
And I've already provided two configuration files for you(w/ OHEM and w/o OHEM) under experiments/cfgs
folder, you could just use them and needn't change anything.
R-FCN is released under the MIT License (refer to the LICENSE file for details).
If you find R-FCN useful in your research, please consider citing:
@article{dai16rfcn,
Author = {Jifeng Dai, Yi Li, Kaiming He, Jian Sun},
Title = {{R-FCN}: Object Detection via Region-based Fully Convolutional Networks},
Journal = {arXiv preprint arXiv:1605.06409},
Year = {2016}
}
| training data | test data | [email protected] | time/img (Titian X)
-------------------|:-------------------:|:---------------------:|:-----:|:------------------:| R-FCN, ResNet-50 | VOC 07+12 trainval | VOC 07 test | 77.6% | 0.099sec | R-FCN, ResNet-101 | VOC 07+12 trainval | VOC 07 test | 79.4% | 0.136sec |
| training data | test data | mAP@[0.5:0.95] | time/img (Titian X)
-------------------|:-------------------:|:---------------------:|:-----:|:------------------:| R-FCN, ResNet-101 | COCO 2014 train | COCO 2014 val | 29.0% | 0.0138sec |
| training data | test data | [email protected] | time/img (Titian X)
-------------------|:-------------------:|:---------------------:|:-----:|:------------------:| R-FCN, ResNet-50 | VOC 07+12 trainval | VOC 07 test | 77.4%| 0.099sec | R-FCN, ResNet-101 | VOC 07+12 trainval | VOC 07 test | 79.4%| 0.136sec |
- This version contains a Caffe in it, you can read caffe folder's README.md to see how to build caffe for py-R-FCN.
- Python packages you might not have:
cython
,python-opencv
,easydict
- [Optional] MATLAB is required for official PASCAL VOC evaluation only. The code now includes unofficial Python evaluation code.
- Clone repository
git clone https://github.com/yao-matrix/py-R-FCN.git
We'll call the directory that you cloned R-FCN into RFCN_ROOT
- Build the Cython modules
cd $RFCN_ROOT/lib make
- Build Caffe and pycaffe
cd $RFCN_ROOT/caffe # Now follow the Caffe installation instructions here: # http://caffe.berkeleyvision.org/installation.html # If you're experienced with Caffe and have all of the requirements installed # and your Makefile.config in place, then simply do: make -j<core bumber> && make pycaffe
-
To use demo you need to download the pretrained R-FCN model, please download the model manually from OneDrive, and put it under
$RFCN/data
.Make sure it looks like this:
$RFCN/data/rfcn_models/resnet50_rfcn_final.caffemodel $RFCN/data/rfcn_models/resnet101_rfcn_final.caffemodel
-
To run the demo
$RFCN/tools/demo_rfcn.py
The demo performs detection using a ResNet-101 network trained for detection on PASCAL VOC 2007.
-
Download the training, validation, test data and VOCdevkit
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCdevkit_08-Jun-2007.tar wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
-
Extract all of these tars into one directory named
VOCdevkit
tar xvf VOCtrainval_06-Nov-2007.tar tar xvf VOCtest_06-Nov-2007.tar tar xvf VOCdevkit_08-Jun-2007.tar tar xvf VOCtrainval_11-May-2012.tar
-
It should have this basic structure
$VOCdevkit/ # development kit $VOCdevkit/VOCcode/ # VOC utility code $VOCdevkit/VOC2007 # image sets, annotations, etc. $VOCdevkit/VOC2012 # image sets, annotations, etc. # ... and several other directories ...
-
Since py-faster-rcnn does not support multiple training datasets, we need to merge VOC 2007 data and VOC 2012 data manually. Just make a new directory named
VOC0712
, put all subfolders exceptImageSets
inVOC2007
andVOC2012
intoVOC0712
(you'll merge some folders). I provide a merged-versionImageSets
folder for you, please put it intoVOCdevkit/VOC0712/
. -
Then the folder structure should look like this
$VOCdevkit/ # development kit
$VOCdevkit/VOCcode/ # VOC utility code
$VOCdevkit/VOC2007 # image sets, annotations, etc.
$VOCdevkit/VOC2012 # image sets, annotations, etc.
$VOCdevkit/VOC0712 # you just created this folder
# ... and several other directories ...
-
Create symlinks for the PASCAL VOC dataset
cd $RFCN_ROOT/data ln -s $VOCdevkit VOCdevkit0712
-
Please download ImageNet-pre-trained ResNet-50 and ResNet-100 model manually, and put them into
$RFCN_ROOT/data/imagenet_models
-
Then everything is done, you could train your own model.
To train and test a R-FCN detector using the approximate joint training method, use experiments/scripts/rfcn_end2end.sh
.
Output is written underneath $RFCN_ROOT/output
.
To train and test a R-FCN detector using the approximate joint training method with OHEM, use experiments/scripts/rfcn_end2end_ohem.sh
.
Output is written underneath $RFCN_ROOT/output
.
To train and test a R-FCN detector using the alternative optimization method with OHEM, use experiments/scripts/rfcn_alt_opt_5stage_ohem.sh
.
Output is written underneath $RFCN_ROOT/output
cd $RFCN_ROOT
./experiments/scripts/rfcn_end2end[_ohem].sh 0 [NET] [DATASET] [--set ...]
# NET in {ResNet-50, ResNet-101} is the network arch to use
# DATASET in {pascal_voc, coco} is the dataset to use(I only tested on pascal_voc)
# --set ... allows you to specify fast_rcnn.config options, e.g.
# --set EXP_DIR seed_rng1701 RNG_SEED 1701
Trained R-FCN networks are saved under:
output/<experiment directory>/<dataset name>/
Test outputs are saved under:
output/<experiment directory>/<dataset name>/<network snapshot name>/
py-faster-rcnn code can also work properly, but I do not add any other feature(such as ResNet and OHEM).