Did you remove the RefCOCO test images from the COCO training set?

[pvoigtlaender]

Hi,

thx for releasing this, this is really great work!

I recently discovered that the RefCOCO annotations, even validation and test, are all done on the training set of the original COCO dataset, which I think was done quite badly for the annotations, but now we have to live with that.
This means, that the COCO training set overlaps with the RefCOCO test set.

Did you filter out those overlapping images from the training set? If not, then the scores for RefCOCO (also + and g) would be too optimistic, because the instances that are refferred to have already been seen during instance segmentation training.
For a fair comparison to previous work, it would be necessary to re-train with those images removed (if you didn't remove those images).

[qinliuliuqin]

Grounding DINO mentioned this data leak issue in Table 5.

I have an additional question: What's the data split used for RefCOCO? 'unc', 'google', or 'umd' as mentioned here?