MobileNet implementation of MaskRCNN

[ha5463]

Can anyone please help on how to proceed for implementing this architecture using the MobileNet architecture. We are having some issues in implementing SeparableConv2D layer in this model. We wanted to train the model from scratch using random weights, hence it would be appreciable if anyone could help us.

[gustavz]

I am also interested / trying to implement a MobileNet Version of Mask R-CNN.
Is your project publich @ha5463 ?

[ha5463]

@gustavz The project is not complete yet, we are still working on the problem. Are you working on the same problem too, maybe we can work on it together?

[gustavz]

@ha5463 yes I am working since a while on the topic to get a mask network running on a jetson tx2
My current approach is to use mask rcnn with a mobilenet v2 or v2 backbone.
You can have a look at my repo github.com/GustavZ/Mobile_Mask_RCNN

Would be nice to work together / benefit of each other’s experience

[ha5463]

@gustavz What you have in your repo is nearly the same what we started with, now we want to use SeparableConv2D layer in model.py file so that the backbone still remains ResNet50 but with SeparableConv2D layer. We started off with this, but we were facing problems in random initialization at the start.
What you have in your repo is also quite helpful, if you have done some work in replacing the layers as I said above, please share.

[JonathanCMitchell]

Hi @gustavz @ha5463 I attempted to train with a MobileNetV2 version about a week ago, it did not train properly, but I was also modifying certain hyperparameters such as:

1. Training at a lower resolution
2. Modifying the backbone strides
3. No image padding
4. Using SeparableConv2D layers in the TimeDistributed fpn mask graph.

@gustavz Are you training MobilenetV2 from scratch or did you find the imagenet weights somewhere? Also, are you training at full res (1024, 1024, 3)?

It would be great to work together.

@gustavz Why the choice to make ResNet50 SeparableConv2D? It has been shown that MobileNet can be used effectively as a base in the MobileMaskRCNN implementation.

Feel free to email me [email protected]

[Cpruce]

@ha5463 what's the issue you're facing with random weight initialization?

[ha5463]

@Cpruce I am facing problem in initializing the model with coco dataset. With imagenet dataset it works fine but with coco dataset, it is giving some error. Do you know a way as to how to initialize the model with coco dataset?
The model I am talking about is with mobilenet backbone and not ResNet backbone.

[Cpruce]

@ha5463 you need to produce the coco weights yourself by either training from scratch or bootstrapping with the imagenet weights

[betterclever]

@Cpruce I am working with @ha5463

I initialized the with ImageNet weights and training was successful but the inference was not working.
Can you suggest what are we doing wrong?

Edit: As it turns out on the further evaluation that only "person" object is being masked, maybe I ran too few epochs. I am running more to see if anything improves.

[gustavz]

@ha5463 do i understand you right that you want to implement SeparableConv2D Layers into the standard ResNet backbone? Or where do you want to place these SeparableConv2D layers?

@JonathanCMitchell you say that it has been shown that MobileNet can be used effectivly. is that because of your and @Cpruce 's work or is there some official paper about it? Would love to see any literature blogs that go deeper into this.

Guys what do you think about starting a slack or something like that to get communication faster?

[ha5463]

@gustavz yes you got it right, we do want to implement ResNet using SeparableConv2D layers, but when I ran the code of @Cpruce from the repo https://github.com/Cpruce/Mask_RCNN which is mobilenet backbone, on coco dataset, it displayed some error as mentioned by @betterclever in the comment above.

Does anyone know what is the cause of this error?

[JonathanCMitchell]

@gustavz MobileNet is simply a feature extractor the same way ResNet is a feature extractor. It can be considered a "base network", and you can attach any head to it. For instance, InceptionV3 is also a base network, and so is VGG. You could use either of them in this case. I chose to use MobileNetV1 because of its speed. I have not run any successful experiments with MobileNetV2 yet, but if you do please be sure to let us know in this thread. If you start a slack channel I am down to join.

[gustavz]

@JonathanCMitchell I know the architecture of Mask R-CNN with the backbone structure. My question was if you know any literature or projects that already successfully included MobileNet as Base Arch into Mask R-CNN, because that's how it sounded and that's why I asked.
Also i wrote you an email as you proposed : =)

[ha5463]

@gustavz as you suggested, we should start a slack channel for faster communication. Also, do you know any possible solution to the problem which @betterclever posted above?

[JonathanCMitchell]

@betterclever I have the same issue when running the inspect_model.ipynb notebook that @betterclever has. Working on a solution now...

[Cpruce]

@ha5463 @betterclever I believe you guys are seeing the above error because no predictions/masks were produced. Can you please try to fine-tune with imagenet weights to produce coco weights, and then run the inspect_model.ipynb notebook? What about with other notebooks or just running detection on some test images? I'll look into this more later

@gustavz I hope to write a paper explaining why things work and possibly contribute something even more novel.

Everyone - By all means feel free to add and improve and im down for a slack channel

[betterclever]

@Cpruce Indeed. We trained just one epoch, and one epoch fine-tune to see just check results once when I posted the error. I ran over 6 epochs training on heads, and 6 epochs fine-tuning and results are better now. It is computing mAP for most cases except some in which it is not able to find mask.

[betterclever]

I created a Slack Channel. Please invite yourself here

[damienstanton]

I've joined the slack and am going to get myself caught up on this work, as I have the same goal (mask rcnn with mobilenet v2 backbone).

[mehditlili]

@betterclever your invitation link is not valid anymore. Can you please renew it?

[betterclever]

@mehditlili Even I am not able to login. I think there's a server error with Slack. Check here: https://status.slack.com/ . Will check back later. Also, the link redirects to signup portal but it fails after that, so I think link it still valid.

[mehditlili]

@betterclever I get "This invite link is no longer active."
try running it from incognito mode, your browser might be remembering you.

[zhoujinhai]

@ha5463 yes I am working since a while on the topic to get a mask network running on a jetson tx2
My current approach is to use mask rcnn with a mobilenet v2 or v2 backbone.
You can have a look at my repo github.com/GustavZ/Mobile_Mask_RCNN

Would be nice to work together / benefit of each other’s experience

hello，Do you have the pretrained weigths of movilenetV2 mask rcnn