Scheduler

[Adesh696]

How can I use a different scheduler like ReduceLROnPlateau?

[yoshitomo-matsubara]

Hi @Adesh696 ,

For torchdistill <= 0.33, you can use built-in PyTorch schedulers simply by specifying the class name and kwargs like this

e.g, MultiStepLR for final training densenet_bc_k12

  scheduler:
    type: 'MultiStepLR'
    params:
      milestones: [150, 225]
      gamma: 0.1

If you want to use ReduceLROnPlateau, then you will need to replace the scheduler entry with

  scheduler:
    type: 'ReduceLROnPlateau'
    params:
      mode: 'min'
      factor: 0.1
      # ... other kwargs for instantiating ReduceLROnPlateau (except optimizer) e.g., threshold, threshold_mode, cooldown, min_lr, eps, verbose, if necessary

[Adesh696]

Got key error, I am using torchdistill version '0.3.3'.

warnings.warn(_create_warning_msg(
2023/05/29 19:08:34 INFO torchdistill.models.util [student model]
2023/05/29 19:08:34 INFO torchdistill.models.util Using the original student model
2023/05/29 19:08:34 INFO torchdistill.core.training Loss = 1.0 * OrgLoss
2023/05/29 19:08:36 INFO torchdistill.misc.log Epoch: [0] [ 0/782] eta: 0:24:36 lr: 0.1 img/s: 59.38591832594536 loss: 2.3583 (2.3583) time: 1.8880 data: 0.8101 max mem: 2208
2023/05/29 19:08:50 INFO torchdistill.misc.log Epoch: [0] [100/782] eta: 0:01:52 lr: 0.1 img/s: 433.4700898314308 loss: 1.8272 (2.0299) time: 0.1476 data: 0.0001 max mem: 2208
2023/05/29 19:09:05 INFO torchdistill.misc.log Epoch: [0] [200/782] eta: 0:01:31 lr: 0.1 img/s: 436.82632588732866 loss: 1.6908 (1.8605) time: 0.1479 data: 0.0001 max mem: 2208
2023/05/29 19:09:20 INFO torchdistill.misc.log Epoch: [0] [300/782] eta: 0:01:14 lr: 0.1 img/s: 432.30080554508766 loss: 1.4343 (1.7610) time: 0.1480 data: 0.0001 max mem: 2208
2023/05/29 19:09:35 INFO torchdistill.misc.log Epoch: [0] [400/782] eta: 0:00:58 lr: 0.1 img/s: 429.4431350967399 loss: 1.3171 (1.6675) time: 0.1484 data: 0.0001 max mem: 2208
2023/05/29 19:09:50 INFO torchdistill.misc.log Epoch: [0] [500/782] eta: 0:00:42 lr: 0.1 img/s: 433.2706906236079 loss: 1.1854 (1.5868) time: 0.1483 data: 0.0001 max mem: 2208
2023/05/29 19:10:05 INFO torchdistill.misc.log Epoch: [0] [600/782] eta: 0:00:27 lr: 0.1 img/s: 430.89351401503114 loss: 1.1877 (1.5208) time: 0.1491 data: 0.0001 max mem: 2208
2023/05/29 19:10:19 INFO torchdistill.misc.log Epoch: [0] [700/782] eta: 0:00:12 lr: 0.1 img/s: 432.0322404125331 loss: 1.1551 (1.4642) time: 0.1484 data: 0.0001 max mem: 2208
2023/05/29 19:10:32 INFO torchdistill.misc.log Epoch: [0] Total time: 0:01:57
2023/05/29 19:10:33 INFO torchdistill.misc.log Validation: [ 0/79] eta: 0:01:32 acc1: 55.4688 (55.4688) acc5: 99.2188 (99.2188) time: 1.1666 data: 0.6698 max mem: 2208
2023/05/29 19:10:39 INFO torchdistill.misc.log Validation: Total time: 0:00:07
2023/05/29 19:10:39 INFO main * Acc@1 52.2400 Acc@5 94.9000

2023/05/29 19:10:39 INFO main Best top-1 accuracy: 0.0000 -> 52.2400
2023/05/29 19:10:39 INFO main Updating ckpt at ./resource/ckpt/cifar10/ce/cifar10-densenet_bc_k12_depth100-final_run.pt
Traceback (most recent call last):
File "/home/ubuntu/data/torchdistill/examples/legacy/image_classification.py", line 196, in
main(argparser.parse_args())
File "/home/ubuntu/data/torchdistill/examples/legacy/image_classification.py", line 177, in main
train(teacher_model, student_model, dataset_dict, ckpt_file_path, device, device_ids, distributed, config, args)
File "/home/ubuntu/data/torchdistill/examples/legacy/image_classification.py", line 134, in train
training_box.post_process()
File "/home/ubuntu/miniconda3/envs/adesh/lib/python3.9/site-packages/torchdistill/core/training.py", line 217, in post_process
metrics = kwargs['metrics']
KeyError: 'metrics'

[yoshitomo-matsubara]

@Adesh696

You need to change this line and add metrics=your_metrics as part of training_box.post_process()
e.g., training_box.post_process(metrics=-val_top1_accuracy)

Close this discussion and "Mark as answer" if it resolves the issue

[Adesh696]

Thanks a lot