Automatic segmentation

[nikolaydyankov]

I have a generic question about automatic segmentation. In my code I'm fine-tuning SAM by providing a bbox for a prompt, representing the bbox of the ground truth. During inference it works well if you provide accurate boxes as prompts, but I wanted to try your approach of training with a bbox of the full image.

In my code I'm not changing anything, except manually setting the box to the image size:

bbox = [x * ratio_w, y * ratio_h, (x + w) * ratio_w, (y + h) * ratio_h] # Old
bbox = [0, 0, 1024, 1024] # New

However, my model breaks and I'm literally getting IoU = 0 in my testing. I tried looking into the auto segmentation notebook, but I can't see anything unusual in the training process. Did you have to make any other adjustments besides setting the bbox to the image size?

[JunMa11]

Hi @nikolaydyankov ,

Sorry for my late reply. i didn't check the discussions page.

Could you please let me know what target you want to segment?

If the target is tiny, I didn't obtain good results in my past experiments. Basically, for tiny segmentation targets, the challenge is how to detect them rather than how to segment them.

[nikolaydyankov]

Hello,
If you imagine a photo of a building I’m trying to segment each floor individually. With other models I’m getting IoU around 0.65.

[JunMa11]

MedSAM's training set didn't include the building photos. The default SAM could do a better job for such images.

[nikolaydyankov]

I'm not using your trained model for my task, you misunderstood my question. Anyway, thanks you for taking the time to answer.