This repository makes available the source code, datasets, pre-trained weights for the work, "One-shot Learning-based Animal Video Segmentation", which is currently in review.
Our proposed approach achieved mean intersection-over-union score of 89.2% on the DAVIS 2016 animal dataset [1] with only one labeled frame each video.
The source code is licensed under CC BY 4.0 license. The contents of this repository are released under an Apache 2 license.
The required python packages are listed in requirements.txt
Our repository has three stages as described in paper: base training, objectness training, fine-tuning. Details will be introudced in the following.
You can download the selection results in bubbleNet_data.zip (72 MB), unzip it in "datasets" folder. Noticeably, the BubbleNet that runs on the testing set does not need any labels.
Download ImageNet pretrained model for XceptionNet imagenet_pretrain_weights.zip (585 MB), unzip it in "weights" folder
Download Pascal and its extension datasets pascal_extension_dataset.zip (1 GB), unzip it in "datasets" folder
Then you can run train_pascal_base.py to start base training. Noticeably, the dataset here is large, which requires you to have a GPU with 11GB memory or more. You can reduce the number of training images by delete items in "datasets/pretrain_benchmark_reduced.txt".
"pretrain_benchmark_and_pascal.txt" represents the full dataset and "test_algorithm_pretrain_benchmark_reduced.txt" is used to debug fast.
You can also download our pre-trained weights on pascal and its extension pascal_base_train_weights.zip (121 MB), then unzip it in "weights" folder.
Download Davis 2016 training datasets DAVIS2016_train_dataset.zip (254 MB), unzip it in "datasets" folder
Before running train_davis_objectness.py, you should ensure that pre-trained weights of pascal and its extension dataset are in "weights/pascal_base_train_weights" folder, and the iterations are matched with program here.
You can also download our pre-trained weights on DAVIS 2016 training set objectness_weights.zip (121 MB), then unzip it in "weights" folder.
Download DAVIS animal testing datasets (fine-tune and test) finetune_test_dataset.zip (71 MB), unzip it in "datasets" folder
Similar to objectness training, please note that the pretrained weights in "weights/objectness_weights" should be matched with the one in program here. After running finetune_test.py, you can find the segmentation results in "results/segmentation" folder.
If you wanna evaluate the results quantatively as we did in paper, you can refer to the DAVIS official code https://davischallenge.org/davis2016/code.html
Our paper is in review currently. If you use our work, please cite this github website address in your code.
[1] Perazzi, F., Pont-Tuset, J., McWilliams, B., Van Gool, L., Gross, M. and Sorkine-Hornung, A., 2016. A benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 724-732). https://davischallenge.org/davis2016/code.html