Skip to content

Latest commit

 

History

History
170 lines (123 loc) · 9.48 KB

README.md

File metadata and controls

170 lines (123 loc) · 9.48 KB
Raw



CerberusDet: Unified Multi-Dataset Object Detection

[Paper]

The code is based on:

Install

Python>=3.8.0 is required.

$ git clone
$ pip install -e .

Docker

Run the docker:

sudo docker-compose up -d
sudo docker attach cerberusdet_cerber_1

Data

  • Use script voc.py to download VOC dataset

For information about the VOC dataset and its creators, visit the PASCAL VOC dataset website.

  • Use script objects365_part.py to download subset of Objects365 dataset with 19 animals categories:
['Monkey', 'Rabbit', 'Yak', 'Antelope', 'Pig',  'Bear', 'Deer', 'Giraffe', 'Zebra', 'Elephant',
'Lion', 'Donkey', 'Camel', 'Jellyfish', 'Other Fish', 'Dolphin', 'Crab', 'Seal', 'Goldfish']

Along with Objects365 subset with 12 tableware categories:

  [ 'Cup', 'Plate', 'Wine Glass', 'Pot', 'Knife', 'Fork', 'Spoon', 'Chopsticks',
    'Cutting/chopping Board', 'Tea pot', 'Kettle', 'Tong']

To download full Objects365 dataset, set DOWNLOAD_SUBSETS = False in the script objects365_part.py.

The Objects365 dataset is available for the academic purpose only. For information about the dataset and its creators, visit the Objects365 dataset website.

Train

  • Download pretrained on COCO yolov8 weights
  • Run train process with 1 GPU
$ python3 cerberusdet/train.py \
--img 640 --batch 32 \
--data data/voc_obj365.yaml \
--weights pretrained/yolov8x_state_dict.pt \
--cfg cerberusdet/models/yolov8x_voc_obj365.yaml \
--hyp data/hyps/hyp.cerber-voc_obj365.yaml \
--name voc_obj365_v8x --device 0
  • OR run train process with several GPUs:
$ CUDA_VISIBLE_DEVICES="0,1,2,3" \
python -m torch.distributed.launch --nproc_per_node 4 cerberusdet/train.py \
--img 640 --batch 32 \
--data data/voc_obj365.yaml \
--weights pretrained/yolov8x_state_dict.pt \
--cfg cerberusdet/models/yolov8x_voc_obj365.yaml \
--hyp data/hyps/hyp.cerber-voc_obj365.yaml \
--name voc_obj365_v8x \
--sync-bn

By default logging will be done with tensorboard, but you can use mlflow if set --mlflow-url, e.g. --mlflow-url localhost.

CerberusDet model config details

Example of the model's config for 2 tasks: yolov8x_voc_obj365.yaml

  • The model config is based on yolo configs, except that the head is divided into two sections (neck and head)
  • The layers of the neck section can be shared between tasks or be unique
  • The head section defines what the head will be for all tasks, but each task will always have its own unique parameters
  • The from parameter of the first neck layer must be a positive ordinal number, specifying from which layer, starting from the beginning of the entire architecture, to take features.
  • The cerber section is optional and defines the architecture configuration for determining the neck layers to be shared among tasks. If not specified, all layers will be shared among tasks, and only the heads will be unique.
  • The CerberusDet configuration is constructed as follows:
    cerber: List[OneBranchConfig], where
      OneBranchConfig = List[cerber_layer_number, SharedTasksConfig], where
          cerber_layer_number - the layer number (counting from the end of the backbone) after which branching should occur
          SharedTasksConfig = List[OneBranchGroupedTasks], where
                OneBranchGroupedTasks = [number_of_task1_head, number_of_task2_head, ...] - the task head numbers (essentially task IDs) that should be in the same branch and share layers thereafter

    The head numbers will correspond to tasks according to the sequence in which they are listed in the data configuration.

    Example for YOLO v8x:
    [[2, [[15], [13, 14]]], [6, [[13], [14]]]] - configuration for 3 tasks. Task id=15 will have all task-specific layers, starting from the 3rd. Tasks id=13, id=14 will share layers 3-6, then after the 6th, they will have their own separate branches with all layers.

Evaluation

Inference

  • Download CerberusDet checkpoint trianed on VOC and part of Objects 365 datasets (see below)
  • Run script bash_scripts/detect.sh

Pretrained Checkpoints

Model Train set size
(pixels)		 mAPval
50-95		 mAPval
50		 Speed
V100 b32, fp16
(ms)		 params
(M)		 FLOPs
@640 (B)
YOLOv8x VOC 640 0.758 0.916 5.6 68 257.5
YOLOv8x Objects365_animals 640 0.43 0.548 5.6 68 257.5
YOLOv8x Objects365_tableware 640 0.56 0.68 5.6 68 257.5
YOLOv8x Objects365_full 640 0.291 0.381 5.6 70 267.0
CerberusDet_v8x VOC, Objects365_animals 640 0.751, 0.432 0.918, 0.556 7.2 105 381.3
CerberusDet_v8x VOC, Objects365_animals, Objects365_tableware 640 0.762, 0.421, 0.56 0.927, 0.541, 0.68 10 142 505.1
CerberusDet_v8x VOC, Objects365_full 640 0.767, 0.355 0.932, 0.464 7.2 107 390.8

YOLOv8x models were trained with the commit: https://github.com/ultralytics/ultralytics/tree/2bc36d97ce7f0bdc0018a783ba56d3de7f0c0518

Hyperparameter Evolution

See the launch example in the bash_scripts/evolve.sh.

Notes
  • To evolve hyperparameters specific to each task, specify initial parameters separately per task and append --evolve_per_task
  • To evolve specific set of hyperparameters, specify their names separated by comma via the --params_to_evolve argument, e.g. --params_to_evolve 'box,cls,dfl'
  • Use absolute paths to configs.
  • Specify search algorith via --evolver. You can use the search algorithms of the ray library (see available values here: predefined_evolvers.py), or 'yolov5'

License

CerberusDet is released under the GNU AGPL v.3 license.

See the file LICENSE for more details.

Citing

If you use our models, code or dataset, we kindly request you to cite our paper and give repository a ⭐

@article{cerberusdet,

   Author = {Irina Tolstykh,Michael Chernyshov,Maksim Kuprashevich},

   Title = {CerberusDet: Unified Multi-Dataset Object Detection},

   Year = {2024},

   Eprint = {arXiv:2407.12632},

}