cannot reproduce faster rcnn mAP

[jiangxiaoyan]

i run bash script/vgg_voc07.sh 1,2,3, use three GPU cards， Tesla M40
but "root:Means AP=0.6860", not like 70.23

can you give the detail for experiment config

also, i test 2 GPU cards. the mAP is only 0.6303. i do not understand why this.

[mAtthEwwww]

i run
bash script/resnet_voc0712.sh 1, 2
but only got mAP 71%, far less than 79%
@precedenceguo

[ijkguo]

Please check if today's commit from mxnet pr #6849 fixed the problem.

[Zehaos]

@jiangxiaoyan @mAtthEwwww Can you reproduce the result now? Thanks.

[mAtthEwwww]

@Zehaos i have gave up

[jonbakerfish]

I can't reproduce the resnet101+voc2017+voc2012 either. I'm using MXNet 0.11.1.
The mAP I got is ~0.69.

[wassryan]

I met the same problem as @ @jiangxiaoyan @mAtthEwwww ,I got vgg+alternate by 68.3MAP.Another problem is that I train resnet101+alternate,I got the high train accuracy and low loss,but only got a low test accuracy which is only 28MAP.What happened to me,anyone help?

[ijkguo]

I find VGG still works by these steps. Results can fluctuate in 68 to 70 between experiments. Usually it is 69.xx or 70.xx.
Will look into resnet.

[ijkguo]

We did not fix random seed so results would vary between experiments. However, it is always around the reference on my end. Did you try to evaluate the released models?

[315386775]

@k-miracle i have the same problem. I train resnet101+end2end, I got the high train accuracy and low loss,but only got a low test accuracy and map. img_pixel_means = (0.0, 0.0, 0.0).

[ijkguo]

Please evaluate the released model. Let's look at inference stage first.