# of classes in detection

[fjremnav]

How to change it to 13 which BDD100K has?

Thanks,

[thinkthinking]

How to change it to 13 which BDD100K has?

Thanks,

1. search nc=1, and change it to nc=13;
2. search single_cls = True, and change it to single_cls = False;
3. change [ [17, 20, 23], Detect, [1, [[3,9,5,11,4,20], [7,18,6,39,12,31], [19,50,38,81,68,157]], [128, 256, 512]]], #Detection head 24 in YOLOP.py to [ [17, 20, 23], Detect, [13, [[3,9,5,11,4,20], [7,18,6,39,12,31], [19,50,38,81,68,157]], [128, 256, 512]]], #Detection head 24.

[dinarkino]

How to change it to 13 which BDD100K has?
Thanks,

1. search nc=1, and change it to nc=13;
2. search single_cls = True, and change it to single_cls = False;
3. change [ [17, 20, 23], Detect, [1, [[3,9,5,11,4,20], [7,18,6,39,12,31], [19,50,38,81,68,157]], [128, 256, 512]]], #Detection head 24 in YOLOP.py to [ [17, 20, 23], Detect, [13, [[3,9,5,11,4,20], [7,18,6,39,12,31], [19,50,38,81,68,157]], [128, 256, 512]]], #Detection head 24.

Hey @thinkthinking, have you succeeded with multi-class detection training?
After making changes you advised the model couldn't train detection well in my experiments. At the same time, segmentation training is ok as well as multitask training for one detection class with original parameters

[ankur219]

@dinarkino What mAP did you get for multiple classes? My mAP after making the changes @thinkthinking mentioned is 0.25

[fjremnav]

Hi,

I am also keen to get mAP score of 13 classes.
Maybe it is a good idea to have a PR of 13 classes configuration if mAP is good.

Thanks,

Thanks,

[dinarkino]

@dinarkino What mAP did you get for multiple classes? My mAP after making the changes @thinkthinking mentioned is 0.25

I'm training the model on my own data, but I will also train it on the bdd100k. I will share metrics later.
Do your results look reasonable and do you know mAP per class? Also, it would be very helpful if you could provide some important hyperparameters.

For my own data I started with only car class, and now I do experiments for car and person classes. The behavior of training seems strange to me. For two classes segmentation metric fluctuates and it is lower, whereas detection metric is much lower, and it increases slowly. Maybe I use not the best hyperparameters such as lr and loss gains, or maybe I should just train the model for a much longer time. Here dark blue for one class training and light blue for two classes training

[dinarkino]

Hey @ankur219 I got training results. After 70 epochs I have such metrics (I did not train lane line segmentation task).

Epoch: [79] Loss(0.329)
Driving area Segment: Acc(0.964) IOU (0.816) mIOU(0.886)
Lane line Segment: Acc(0.522) IOU (0.007) mIOU(0.209)
Detect: P(0.648) R(0.320) [email protected](0.277) [email protected]:0.95(0.128)

For specific classes the results are the following:

So for car class it is around 57% of [email protected]. I wonder if it is ok or not? Authors in the paper have 76.5% of [email protected], but they combine car, bus, truck, and train in one car class.

[ankur219]

Hi @dinarkino Thank you very much for the results. This probably shows that either there is some other change that needs to be done for multiclass or the network is not capable enough.
Also, I trained YoloP for single class and the numbers are similar to the ones reported in the paper. So, the problem is only in the case of multiclass I suppose.

[dinarkino]

Yeah, it seems so. I started an experiment just for one raw car class on bdd data and got 60% of map after 30 epochs which is higher than for multiclass training. I suppose for longer training and after combining several classes in car class I will also get results as in the paper

[LuthraBhomik]

@dinarkino @thinkthinking @ankur219 @fjremnav Have you tried to take pre-trained model (trained on BDD, single class) and then fine tuned it on multiple class (lets say, nc = 13).

I am able to train the model from scratch on multiple class, but if I use pretrained model and change number of classes, its throwing an error. I think, its because we are changing [ [17, 20, 23], Detect, [1, [[3,9,5,11,4,20], [7,18,6,39,12,31], [19,50,38,81,68,157]], [128, 256, 512]]], #Detection head 24

Anyone tried this or has any solution ?

I am getting the following error :

Traceback (most recent call last):
File "tools/train.py", line 435, in
main()
File "tools/train.py", line 182, in main
model.load_state_dict(checkpoint['state_dict'])
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1223, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MCnet:
size mismatch for model.24.m.0.weight: copying a param with shape torch.Size([18, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([24, 128, 1, 1]).
size mismatch for model.24.m.0.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([24]).
size mismatch for model.24.m.1.weight: copying a param with shape torch.Size([18, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([24, 256, 1, 1]).
size mismatch for model.24.m.1.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([24]).
size mismatch for model.24.m.2.weight: copying a param with shape torch.Size([18, 512, 1, 1]) from checkpoint, the shape in current model is torch.Size([24, 512, 1, 1]).
size mismatch for model.24.m.2.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([24]).

[dinarkino]

Yes, you can do that, but you should change the code. You can find similar functionality in YOLOv5 repository. You need to drop the last layer of the detection head, add a new one with a new amount of classes and then finetune the network. You will lose metrics for classes from the original network, but you will reuse weights for other layers.

If you want just to change number of classes and train with such a value, you can change number of classes in detection head:
[ [17, 20, 23], Detect, [1 (this is number of classes), [[3,9,5,11,4,20], [7,18,6,39,12,31], [19,50,38,81,68,157]], [128, 256, 512]]]

[LuthraBhomik]

@dinarkino If I change the number of classes, the model doesnt get trained. I am able to fine-tune provided, the number of classes in pre-trained model are same.

Model trained on BDD single class - > Model fine-tuned on new data, single class only (OK)
Model trained on BDD (nc=3) - > Model fine-tuned on new data, (nc=3) (OK)
Model trained on BDD (nc=1) - > Model fine-tuned on new data, (nc=3) (Not working)

Error :
Traceback (most recent call last):
File "tools/train.py", line 435, in
main()
File "tools/train.py", line 182, in main
model.load_state_dict(checkpoint['state_dict'])
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1223, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MCnet:
size mismatch for model.24.m.0.weight: copying a param with shape torch.Size([18, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([24, 128, 1, 1]).
size mismatch for model.24.m.0.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([24]).
size mismatch for model.24.m.1.weight: copying a param with shape torch.Size([18, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([24, 256, 1, 1]).
size mismatch for model.24.m.1.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([24]).
size mismatch for model.24.m.2.weight: copying a param with shape torch.Size([18, 512, 1, 1]) from checkpoint, the shape in current model is torch.Size([24, 512, 1, 1]).
size mismatch for model.24.m.2.bias: copying a param with shape torch.Size([18]) from checkpoint, the shape in current model is torch.Size([24]).

[dinarkino]

@LuthraBhomik yes, that is exactly what I said. If you want to reuse weights with different number of classes, you should change the code. Or you can train the network from scratch with any number of classes

[soumya997]

I had the same problem, it was showing some size mismatch error. I had to change this line to this,

https://github.com/hustvl/YOLOP/blob/main/lib/models/YOLOP.py#L480
[ [17, 20, 23], Detect, [1, [[3,9,5,11,4,20], [7,18,6,39,12,31], [19,50,38,81,68,157]], [128, 256, 512]]], #Detection head 24

to this,

[ [17, 20, 23], Detect, [13, [[3,9,5,11,4,20], [7,18,6,39,12,31], [19,50,38,81,68,157]], [128, 256, 512]]], #Detection head 24

Here, 13 is number of classes.

and also do this, -> https://github.com/hustvl/YOLOP/blob/main/lib/models/YOLOP.py#L508
self.nc = 1 #nc is number of classes

to

self.nc = 13

[HunterShinobiTitan]

Hello @dinarkino I just want to ask for help how did you visualize the mAP, of the classes. Can you guide me??

Hey @ankur219 I got training results. After 70 epochs I have such metrics (I did not train lane line segmentation task).

Epoch: [79] Loss(0.329) Driving area Segment: Acc(0.964) IOU (0.816) mIOU(0.886) Lane line Segment: Acc(0.522) IOU (0.007) mIOU(0.209) Detect: P(0.648) R(0.320) [email protected](0.277) [email protected]:0.95(0.128)

For specific classes the results are the following:

So for car class it is around 57% of [email protected]. I wonder if it is ok or not? Authors in the paper have 76.5% of [email protected], but they combine car, bus, truck, and train in one car class.

[RahulRewale]

I created a fork and updated the code for BDD's 13 classes. Check it out here.