2 issues

[NerdCat822]

hello i want to get a instance segmentation results.
but this only give the results of semantic segmentation and panoptic segmentation.
can you add the command for instance segmentation please?

And i cannot get the pretrained weight of [FoodSeg103-SETR-MLA].
please add the another link.
Thank you.

[starhiking]

Hi,

panoptic segmentation result contains instance result and other non-food results.

We have added the auxiliary link in install.md , and #3 may help you

[NerdCat822]

Hello, i download the 3 weight file succeesful.
but this code not works now..
here is the error message.
Namespace(SAM_checkpoint='ckpts/sam_vit_h_4b8939.pth', ann_dir='ann_dir/test', area_thr=0, aug_test=False, box_nms_thresh=None, category_txt='FoodSAM/FoodSAM_tools/category_id_files/foodseg103_category_id.txt', color_list_path='FoodSAM/FoodSAM_tools/color_list.npy', confidence_threshold=0.5, crop_n_layers=None, crop_n_points_downscale_factor=None, crop_nms_thresh=None, crop_overlap_ratio=None, data_root='dataset/FoodSeg103/Images', detection_config='configs/Unified_learned_OCIM_RS200_6x+2x.yaml', device='cuda', eval=False, eval_options=None, img_dir='img_dir/test', img_path=None, min_mask_region_area=None, model_type='vit_h', num_class=104, options=None, opts=['MODEL.WEIGHTS', 'ckpts/Unified_learned_OCIM_RS200_6x+2x.pth'], output='Output/Panoramic_Results', points_per_batch=None, points_per_side=None, pred_iou_thresh=None, ratio_thr=0.5, semantic_checkpoint='ckpts/SETR_MLA/iter_80000.pth', semantic_config='configs/SETR_MLA_768x768_80k_base.py', stability_score_offset=None, stability_score_thresh=None, top_k=80)
Create Logger success in Output/Panoramic_Results/sam_process.log
[2023-11-30 03:34:41,727] [panoptic.py:285] [INFO] running sam!
[2023-11-30 03:35:12,269] [panoptic.py:303] [INFO] Processing 'dataset/FoodSeg103/Images/img_dir/test/00004990.jpg'...
Traceback (most recent call last):
File "FoodSAM/panoptic.py", line 335, in
main(args)
File "FoodSAM/panoptic.py", line 309, in main
masks = generator.generate(image)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/segment_anything/automatic_mask_generator.py", line 163, in generate
mask_data = self._generate_masks(image)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/segment_anything/automatic_mask_generator.py", line 206, in _generate_masks
crop_data = self._process_crop(image, crop_box, layer_idx, orig_size)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/segment_anything/automatic_mask_generator.py", line 245, in _process_crop
batch_data = self._process_batch(points, cropped_im_size, crop_box, orig_size)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/segment_anything/automatic_mask_generator.py", line 283, in _process_batch
return_logits=True,
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/segment_anything/predictor.py", line 234, in predict_torch
multimask_output=multimask_output,
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/segment_anything/modeling/mask_decoder.py", line 98, in forward
dense_prompt_embeddings=dense_prompt_embeddings,
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/segment_anything/modeling/mask_decoder.py", line 132, in predict_masks
hs, src = self.transformer(src, pos_src, tokens)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/segment_anything/modeling/transformer.py", line 96, in forward
key_pe=image_pe,
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/segment_anything/modeling/transformer.py", line 178, in forward
attn_out = self.cross_attn_image_to_token(q=k, k=q, v=queries)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/rlagusrb/miniconda3/envs/FoodSAM/lib/python3.7/site-packages/segment_anything/modeling/transformer.py", line 231, in forward
attn = q @ k.permute(0, 1, 3, 2) # B x N_heads x N_tokens x N_tokens
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasSgemmStridedBatched( handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)

what should do i do?

[starhiking]

Your environments (torch, cudatoolkit) may have issues.