CIFAR 10/100 KD wide_resnet40_1_from_wide_resnet40_4 doesn't work #229

rakovAndrew · 2022-06-07T10:13:37Z

rakovAndrew
Jun 7, 2022

Hi!
I'm trying to distill knowledge using the framework but I can't do this.
If I run mentioned code here in Colab , changing only the config for the dataset and the neural network

!python torchdistill/examples/image_classification.py --config torchdistill/configs/sample/cifar10/kd/wide_resnet40_1_from_wide_resnet40_4-hyperparameter_tuning.yaml --log log/cifar10/kd/wide_resnet40_1_from_wide_resnet40_4-hyperparameter_tuning.log

I get this ERROR

2022/06/07 09:46:50  INFO  torchdistill.common.main_util  Not using distributed mode
2022/06/07 09:46:50  INFO  main  Namespace(adjust_lr=False, config='torchdistill/configs/sample/cifar10/kd/wide_resnet40_1_from_wide_resnet40_4-hyperparameter_tuning.yaml', device='cuda', dist_url='env://', log='log/cifar10/kd/wide_resnet40_1_from_wide_resnet40_4-hyperparameter_tuning.log', log_config=False, seed=None, start_epoch=0, student_only=False, test_only=False, world_size=1)
2022/06/07 09:46:50  INFO  torchdistill.datasets.util  Loading dummy data
Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./resource/dataset/cifar10/cifar-10-python.tar.gz
170499072it [00:04, 38577939.73it/s]                   
Extracting ./resource/dataset/cifar10/cifar-10-python.tar.gz to ./resource/dataset/cifar10
2022/06/07 09:46:58  INFO  torchdistill.datasets.util  Splitting dummy dataset (50000 samples in total)
2022/06/07 09:46:58  INFO  torchdistill.datasets.util  new dataset_id: cifar10/train (45000 samples)
2022/06/07 09:46:58  INFO  torchdistill.datasets.util  new dataset_id: cifar10/val (5000 samples)
2022/06/07 09:46:58  INFO  torchdistill.datasets.util  dataset_id dummy: 8.389266729354858 sec
2022/06/07 09:46:58  INFO  torchdistill.datasets.util  Loading test data
Files already downloaded and verified
2022/06/07 09:46:59  INFO  torchdistill.datasets.util  dataset_id cifar10/test: 0.750535249710083 sec
Downloading: "https://github.com/yoshitomo-matsubara/torchdistill/releases/download/v0.1.1/cifar10-wide_resnet40_4.pt" to /root/.cache/torch/hub/checkpoints/cifar10-wide_resnet40_4.pt
100% 34.3M/34.3M [00:00<00:00, 106MB/s] 
2022/06/07 09:47:00  INFO  torchdistill.common.main_util  ckpt file is not found at ./resource/ckpt/cifar10/teacher/cifar10-wide_resnet40_4.pt
Traceback (most recent call last):
  File "torchdistill/examples/image_classification.py", line 186, in <module>
    main(argparser.parse_args())
  File "torchdistill/examples/image_classification.py", line 162, in main
    student_model = load_model(student_model_config, device, distributed)
  File "torchdistill/examples/image_classification.py", line 48, in load_model
    model = get_model(model_config['name'], repo_or_dir, **model_config['params'])
  File "/usr/local/lib/python3.7/dist-packages/torchdistill/models/registry.py", line 22, in get_model
    return MODEL_FUNC_DICT[model_name](**kwargs)
TypeError: wide_resnet() missing 1 required positional argument: 'progress'

If I put the file cifar10-wide_resnet40_4.pt here ./resource/ckpt/cifar10/teacher/ I get another ERROR message

2022/06/06 14:25:28  INFO  torchdistill.common.main_util  Not using distributed mode
2022/06/06 14:25:28  INFO  main  Namespace(adjust_lr=False, config='/content/torchdistill/configs/sample/cifar100/kd/wide_resnet40_1_from_wide_resnet40_4-hyperparameter_tuning.yaml', device='cuda', dist_url='env://', log='/content/torchdistill/log/cifar100/kd/wide_resnet40_1_from_wide_resnet40_4-hyperparameter_tuning.log', log_config=False, seed=None, start_epoch=0, student_only=False, test_only=False, world_size=1)
2022/06/06 14:25:29  INFO  torchdistill.datasets.util  Loading dummy data
Files already downloaded and verified
2022/06/06 14:25:30  INFO  torchdistill.datasets.util  Splitting dummy dataset (50000 samples in total)
2022/06/06 14:25:30  INFO  torchdistill.datasets.util  new dataset_id: cifar100/train (45000 samples)
2022/06/06 14:25:30  INFO  torchdistill.datasets.util  new dataset_id: cifar100/val (5000 samples)
2022/06/06 14:25:30  INFO  torchdistill.datasets.util  dataset_id dummy: 1.152956485748291 sec
2022/06/06 14:25:30  INFO  torchdistill.datasets.util  Loading test data
Files already downloaded and verified
2022/06/06 14:25:30  INFO  torchdistill.datasets.util  dataset_id cifar100/test: 0.7582111358642578 sec
2022/06/06 14:25:31  INFO  torchdistill.common.main_util  Loading model parameters only
Traceback (most recent call last):
  File "/content/torchdistill/examples/image_classification.py", line 186, in <module>
    main(argparser.parse_args())
  File "/content/torchdistill/examples/image_classification.py", line 162, in main
    student_model = load_model(student_model_config, device, distributed)
  File "/content/torchdistill/examples/image_classification.py", line 48, in load_model
    model = get_model(model_config['name'], repo_or_dir, **model_config['params'])
  File "/usr/local/lib/python3.7/dist-packages/torchdistill/models/registry.py", line 22, in get_model
    return MODEL_FUNC_DICT[model_name](**kwargs)
TypeError: wide_resnet() missing 1 required positional argument: 'progress'

I found in your code example that it's okay that app can't find teacher model cifar10-densenet_bc_k12_depth100.pt (I took logs from here as well)

2021/05/23 21:49:41	INFO	torchdistill.common.main_util	ckpt file is not found at `./resource/ckpt/cifar10/teacher/cifar10-densenet_bc_k12_depth100.pt`
2021/05/23 21:49:52	INFO	torchdistill.common.main_util	ckpt file is not found at `./resource/ckpt/cifar10/kd/cifar10-resnet20_from_densenet_bc_k12_depth100-hyperparameter_tuning.pt`

But in case of cifar10-wide_resnet40_4.pt it doesn't work

Help with this issue please

Regards,
Andrew

Answered by yoshitomo-matsubara

Jun 8, 2022

Hi @rakovAndrew

If I run mentioned code here in Colab , changing only the config for the dataset and the neural network

!python torchdistill/examples/image_classification.py --config torchdistill/configs/sample/cifar10/kd/wide_resnet40_1_from_wide_resnet40_4-hyperparameter_tuning.yaml --log log/cifar10/kd/wide_resnet40_1_from_wide_resnet40_4-hyperparameter_tuning.log
I get this ERROR

Could you share your config?
I need to take a look at your configs as my guess is that your model config has something wrong.

I found in your code example that it's okay that app can't find teacher model cifar10-densenet_bc_k12_depth100.pt (I took logs from here as well)

It's ok that ckpt file is not fou…

View full answer

yoshitomo-matsubara · 2022-06-08T01:31:46Z

yoshitomo-matsubara
Jun 8, 2022
Maintainer

Hi @rakovAndrew

If I run mentioned code here in Colab , changing only the config for the dataset and the neural network

!python torchdistill/examples/image_classification.py --config torchdistill/configs/sample/cifar10/kd/wide_resnet40_1_from_wide_resnet40_4-hyperparameter_tuning.yaml --log log/cifar10/kd/wide_resnet40_1_from_wide_resnet40_4-hyperparameter_tuning.log
I get this ERROR

Could you share your config?
I need to take a look at your configs as my guess is that your model config has something wrong.

I found in your code example that it's okay that app can't find teacher model cifar10-densenet_bc_k12_depth100.pt (I took logs from here as well)

It's ok that ckpt file is not found in that case because the config file I published is downloading a pretrained weight from GitHub unless you want to use your own ckpt in your local machine.

3 replies

rakovAndrew Jun 8, 2022
Author

Could you share your config?
I need to take a look at your configs as my guess is that your model config has something wrong.

I used the default config for the NN. I just changed epochs quantity to train the NN to 100

yoshitomo-matsubara Jun 8, 2022
Maintainer

Hi @rakovAndrew

Ok, I noticed that I had added progress as a kwarg after my experiments.
You can resolve the issue simply by adding either progress: True or progress: False like this.

Thank you for sharing that. Mark the comment above as answer if it resolves your issue.

rakovAndrew Jun 8, 2022
Author

Thanks @yoshitomo-matsubara !
It works great
Have a nice day

upd: and thanks for the quick answer ;)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CIFAR 10/100 KD wide_resnet40_1_from_wide_resnet40_4 doesn't work #229

{{title}}

Replies: 1 comment 3 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

CIFAR 10/100 KD wide_resnet40_1_from_wide_resnet40_4 doesn't work #229

rakovAndrew Jun 7, 2022

Replies: 1 comment · 3 replies

yoshitomo-matsubara Jun 8, 2022 Maintainer

rakovAndrew Jun 8, 2022 Author

yoshitomo-matsubara Jun 8, 2022 Maintainer

rakovAndrew Jun 8, 2022 Author

rakovAndrew
Jun 7, 2022

Replies: 1 comment 3 replies

yoshitomo-matsubara
Jun 8, 2022
Maintainer

rakovAndrew Jun 8, 2022
Author

yoshitomo-matsubara Jun 8, 2022
Maintainer

rakovAndrew Jun 8, 2022
Author