Question about Result of Metrics

[hustcc19860606]

I use the command: CUDA_VISIBLE_DEVICES=1 python -m object-locator.locate --imgsize 256X256 --dataset mall_dataset/frames/test/ --out mall_dataset/output/ --model checkpoints/mall,lambdaa=1,BS=32,Adam,LR1e-4.ckpt --evaluate, and the test set takes pictures from No.seq_001801.jpg to No.seq_002000.jpg. However, the results have large difference from your paper. Can you give me some suggestions? @javiribera

[javiribera]

The "results" you are trying to reproduce are these two plots? You seem to have posted the same plot repeated 3 times, by the way.

However, the take-away from the two plots is the same from the ones of the paper: (1) there are two clusters of pixels with tau values of 0 and 1, very close to the boundaries, (2) BMM does better than Otsu most of the time in terms of the F-score, and (3) optimizing for tau is not going to be significantly better than BMM or Otsu.

[hustcc19860606]

@javiribera Vertical coordinates of three plots are different. The F-score in the first plot only can achieve at about 22.5. However，the figure 6 in your paper can achieve at over 80. And the precision in the following plot is also much lower than the value in your paper. However，the estimated_maps are similar to the paper. I didn't modify your code, all plots are generated automatically by the code. Can your give me some suggestions to reproduce the values in your paper ?