Skip to content

Latest commit

 

History

History
87 lines (68 loc) · 5.83 KB

README.md

File metadata and controls

87 lines (68 loc) · 5.83 KB
Raw

DVIS++: Improved Decoupled Framework for Universal Video Segmentation

Tao Zhang, XingYe Tian, Yikang Zhou, ShunPing Ji, Xuebo Wang, Xin Tao,

Yuan Zhang, Pengfei Wan, Zhongyuan Wang and Yu Wu

PWC PWC PWC PWC PWC PWC

News

  • DVIS-DAQ achieves 57.1 AP on the OVIS dataset and also sets a new SOTA performance on YTVIS19/21 and VIPSeg. The code will be released in this repository and DAQ-VS. The paper is available at DVIS-DAQ: Improving Video Segmentation via Dynamic Anchor Queries and the project page can be found in project page.
  • DVIS and DVIS++ achieved 1st place in the VPS Track of the PVUW challenge at CVPR 2023. 2023.5.25
  • DVIS and DVIS++ achieved 1st place in the VIS Track of the 5th LSVOS challenge at ICCV 2023. 2023.8.15

Features

  • DVIS++ is a universal video segmentation framework that supports VIS, VPS and VSS.
  • DVIS++ can run in both online and offline modes.
  • DVIS++ achieved SOTA performance on YTVIS 2019&2021&2022, OVIS, VIPSeg and VSPW datasets.
  • OV-DVIS++ is the first open-vocabulary video universal segmentation framework with powerful zero-shot segmentation capability.

Demos

VIS

VSS

VPS

Open-vocabulary demos

Installation

See Installation Instructions.

Getting Started

See Preparing Datasets for DVIS++.

See Getting Started with DVIS++.

Model Zoo

Trained models are available for download in the DVIS++ Model Zoo.

Citing DVIS and DVIS++

@article{zhang2023dvis,
  title={DVIS: Decoupled Video Instance Segmentation Framework},
  author={Zhang, Tao and Tian, Xingye and Wu, Yu and Ji, Shunping and Wang, Xuebo and Zhang, Yuan and Wan, Pengfei},
  journal={arXiv preprint arXiv:2306.03413},
  year={2023}
}

@article{zhang2023dvisplus,
  title={DVIS++: Improved Decoupled Framework for Universal Video Segmentation}, 
  author={Tao Zhang and Xingye Tian and Yikang Zhou and Shunping Ji and Xuebo Wang and Xin Tao and Yuan Zhang and Pengfei Wan and Zhongyuan Wang and Yu Wu},
  journal={arXiv preprint arXiv:2312.13305},
  year={2023},
}

@article{dvisdaq,
  title={DVIS-DAQ: Improving Video Segmentation via Dynamic Anchor Queries}, 
  author={Yikang Zhou and Tao Zhang and Shunping Ji and Shuicheng Yan and Xiangtai Li},
  journal={arXiv},
  year={2024},
}

Acknowledgement

This repo is largely based on Mask2Former, MinVIS, VITA, CTVIS, FC-CLIP and DVIS. Thanks for their excellent works.