Skip to content

Latest commit

 

History

History
626 lines (552 loc) · 21.5 KB

MODEL_ZOO.md

File metadata and controls

626 lines (552 loc) · 21.5 KB
Raw

DVIS++ Model Zoo

Introduction

This file documents a collection of trained DVIS++ and OV-DVIS++ models. The "Config" column contains a link to the config file.

Weights

The weights for all the following can be found on (HuggingFace)[https://huggingface.co/zhangtao-whu/DVIS_Plus/tree/main/DVIS%2B%2B], and you can also download them from there.

Pretrained segmenter

Model Backbone Train datasets Used for Download
VIT-L(DINOv2) VIT-L - - baidupan
Mask2Former(instance) R50 COCO OVIS,YTVIS19&21 link
Mask2Former(instance) VIT-L COCO OVIS,YTVIS19&21 baidupan
Mask2Former(panoptic) R50 COCO VSPW,VIPSeg link
Mask2Former(panoptic) VIT-L COCO VSPW,VIPSeg baidupan
FC-CLIP R50 COCO OVIS,YTVIS19&21,VSPW,VIPSeg link
FC-CLIP ConvNext-L COCO OVIS,YTVIS19&21,VSPW,VIPSeg link

Finetuned segmenter

Model Backbone Train datasets Config Download
Mask2Former R50 COCO+OVIS yaml baidupan
Mask2Former R50 COCO+YTVIS19 yaml baidupan
Mask2Former R50 COCO+YTVIS21 yaml baidupan
Mask2Former R50 VIPSeg yaml baidupan
Mask2Former R50 VSPW yaml baidupan
Mask2Former VIT-L COCO+OVIS yaml baidupan
Mask2Former VIT-L COCO+YTVIS19 yaml baidupan
Mask2Former VIT-L COCO+YTVIS21 yaml baidupan
Mask2Former VIT-L VIPSeg yaml baidupan
Mask2Former VIT-L VSPW yaml baidupan
FC-CLIP R50 COCO+OVIS+YTVIS19&21+VIPSeg yaml baidupan
FC-CLIP ConvNext-L COCO+OVIS+YTVIS19&21+VIPSeg yaml baidupan

Close-vocabulary (DVIS++)

OVIS

Model Backbone Queries Video AP AP50 AP75 Config Download
Online R50 100 480P 37.2 62.8 37.3 yaml baidupan
Offline R50 100 480P 41.2 68.9 40.9 yaml baidupan
Online VIT-L 200 480P 49.6 72.5 55.0 yaml baidupan
Offline VIT-L 200 480P 53.4 78.9 58.5 yaml baidupan

YouTubeVIS 2019

Model Backbone Queries Video AP AP50 AP75 Config Download
Online R50 100 480P 55.5 80.2 60.1 yaml baidupan
Offline R50 100 480P 56.7 81.4 62.0 yaml baidupan
Online VIT-L 200 480P 67.7 88.8 75.3 yaml baidupan
Offline VIT-L 200 480P 68.3 90.3 76.1 yaml baidupan

YouTubeVIS 2021

Model Backbone Queries Video AP AP50 AP75 Config Download
Online R50 100 480P 50.0 72.2 54.5 yaml baidupan
Offline R50 100 480P 52.0 75.4 57.8 yaml baidupan
Online VIT-L 200 480P 62.3 82.7 70.2 yaml baidupan
Offline VIT-L 200 480P 63.9 86.7 71.5 yaml baidupan

VIPSeg

Model Backbone Queries Video VPQ VPQthing VPQstuff Config Download
Online R50 100 720P 41.9 41.0 42.7 yaml baidupan
Offline R50 100 720P 44.2 44.5 43.9 yaml baidupan
Online VIT-L 200 720P 56.0 58.0 54.3 yaml baidupan
Offline VIT-L 200 720P 58.0 61.2 55.2 yaml baidupan

VSPW

Model Backbone Queries Video VC8 VC16 mIOU Config Download
Online R50 100 720P 92.3 91.1 46.9 yaml baidupan
Offline R50 100 720P 93.4 92.4 48.6 yaml baidupan
Online VIT-L 200 720P 95.0 94.2 62.8 yaml baidupan
Offline VIT-L 200 720P 95.7 95.1 63.8 yaml baidupan

Open-vocabulary (OV-DVIS++)

Model Backbone Training datasets Video AP(OVIS) AP(YTVIS19) AP(YTVIS21) mIOU(VSPW) VPQ(VIPSeg) Config Download
Online R50 COCO 480P 14.8 34.5 30.9 27.6 24.4 yaml baidupan
Offline R50 COCO 480P 13.0 34.4 31.0 28.4 23.8 yaml baidupan
Online ConvNext-L COCO 480P 24.0 48.8 44.5 34.3 28.9 yaml baidupan
Offline ConvNext-L COCO 480P 21.6 48.7 44.2 34.1 30.4 yaml baidupan
Online ConvNext-L COCO+OVIS+YTVIS19&21+VIPSeg 480P 38.9 60.1 56.0 53.3 49.7 yaml baidupan
Offline ConvNext-L COCO+OVIS+YTVIS19&21+VIPSeg 480P 40.6 61.1 56.7 56.4 51.7 yaml baidupan