My implementation of the SLiMe (Segment Like Me), a pretty cool paper I read. I did only single class segmentation, not multi-class.
I also tried to add on instance segmentation, but it didn't work too well. The attention maps don't seem very "instance-focused", but maybe I didn't try hard enough.
Uses cross attention and self attention maps of Stable Diffusion to do few-shot segmentation. It learns a handful of weights and one Stable Diffusion language embedding vector per class.
i.e. from two labelled pictures of a dog, you can learn a vector that'll highlight dogs in a new image.
Given this labelled image (and one other):
It semi-precisely labels new images of that cat. Here are some of its test-set predictions.
