Implementation of Factor Transfer(Knowledge Distillation) on Yolov5

[ashymuzuro]

Hi everybody I'm new here , I want to use yolov5 models to implement knowledge distillation via method called factor transfer by I have no idea where to begin or how to do it so .Can anyone help me ? Any help will be appreciated whether its advice or references or starting point.

Thanks in advance.
@yoshitomo-matsubara,

[yoshitomo-matsubara]

Hi @ashymuzuro

To answer your question, please explain

1. what your student and teacher models are (YOLOv5 as student? or teacher?)
2. what dataset you are going to use (COCO?)

[ashymuzuro]

1. For the student model (Yolov5n/Yolov5nano) and for the teacher model (Yolov5l/Yolov5large).
2. For the dataset I'm going to use VisDrone.

Many thanks for considering my request.
@yoshitomo-matsubara,

[yoshitomo-matsubara]

You will first need to confirm the interface of the models and dataset to start with https://github.com/yoshitomo-matsubara/torchdistill/blob/main/examples/legacy/object_detection.py (this is legacy ver as I'm currently working for the next major release of torchdistill)

1. What are the inputs/outputs of the YOLOv5 models' forward function?
2. Is the annotation format of VisDrone dataset compatible with COCO format? (If not, you'll need to prepare evaluation scripts by yourself)

I also suggest that you take a closer look at existing, related discussions/issues such as #254

[ashymuzuro]

I will look into what you have said if it works I will update you or provide feedback.