dlib_mmod do not detect too small rectangles

[alphastint]

Hello,

I would like to train a detector for specific objects. I have a database of matching rectangles, and I am using the dlib::loss_mmod for the network. After a little manual optimization of the layer parameters, particularly the number of pyramid levels, I have managed to find topologies which detect the objects reasonably. The problem is, that the detector also returns countless false positives, but mostly rectangles which are too small to be an object.

What is the appropriate strategy in this case? Should I simply create a heuristic and throw away too small rectangles a posteriori? Should I collect negative detections and add them to the training manually? The training in the example program uses the cropper to collect negative samples. It is selecting rectangles around the object and labels them false. If so, it raises a question of how many false detections should I select, and in which proportion should I use them in the training?

Also the dlib::mmod_options::min_target_size does not work as I would expect it. However, even if it really filters the detections by size, it is not always appropriate, as I have elongated objects, where the short dimension for the true object could be the long dimension for the false detection.

[arrufat]

Some time ago, I played quite a bit with the MMOD loss. Without more information, it's hard for me to say what's going on.

Usually, I hardly ever touched the pyramid levels. You want to make sure that the difference between two levels is not too big so that the objects appear at the right scale somewhere.

Are you training on extreme aspect-ratios? Are you sure the training has converged? I find it surprising that you get false positives with the MMOD loss, since it uses all non-labeled image as the negative examples. When I played with it, I rarely got false positives. False negatives were way more likely.

What was your expected behavior for dlib::mmod_options::min_target_size?
Wasn't it the one described in http://dlib.net/dlib/dnn/loss_abstract.h.html#mmod_options?

The longest edge of each detector window is target_size pixels in
length, unless the window's shortest side would be less than
min_target_size pixels in length. In this case the shortest side
will be set to min_target_size length, and the other side sized to
preserve the aspect ratio of the window.

[alphastint]

Hello,

Are you training on extreme aspect-ratios?

I do not know if 1500x400 can be considered an "extreme" aspect ratio, but it is certainly not square-like.

Are you sure the training has converged?

As far as I can tell from the losses, it has converged.

all non-labeled image as the negative examples.

As far as I understand dlib::random_cropper, it takes the object and randomly picks some rectangles around the object as negative examples. I have verified this several times, and this is what happens. Far parts of the image do not / possibly not get into the training set.

What was your expected behavior for dlib::mmod_options::min_target_size?

The expected behavior would be that the detector would not not return results which are smaller than min_target_size in their shorter edge. What happens is I increase the min_target_size before training, and true positive detections also disappear. The detector is simply less sensitive. This may be related to the training, specifically the way samples are collected, but the results are different.

[arrufat]

I would would consider it as an extreme aspect ratio, since it's close to 4. Are you using bounding box regression? It might help.

Also, what's the object's minimum size that you care about and the object is still recognizable? That's what you should set your sizes in the mmod loss and cropper to.

Finally, for this kind of more extreme aspect ratios, I found the YOLO loss (also available in dlib) easier to get good results.
I have set up a repo that handles most (if not all?) options via de command line, and it's compatible with dlib datasets.

https://github.com/dlibml/yolo-object-detector
Might be worth trying.