Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] MMDataParallel loads very slowly #1069

Open
3 tasks done
1shenhui opened this issue Sep 23, 2024 · 1 comment
Open
3 tasks done

[Bug] MMDataParallel loads very slowly #1069

1shenhui opened this issue Sep 23, 2024 · 1 comment

Comments

@1shenhui
Copy link

Prerequisite

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

master branch https://github.com/open-mmlab/mmrotate

Environment

/hdc/miniconda3/envs/LSKNet/lib/python3.8/site-packages/mmcv/init.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
warnings.warn(
fatal: not a git repository (or any of the parent directories): .git
sys.platform: linux
Python: 3.8.18 | packaged by conda-forge | (default, Oct 10 2023, 15:44:36) [GCC 12.3.0]
CUDA available: True
GPU 0,1,2,3: NVIDIA A40-48Q
CUDA_HOME: None
GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
PyTorch: 1.13.1+cu117
PyTorch compiling details: PyTorch built with:

  • GCC 9.3
  • C++ Version: 201402
  • Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  • Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  • OpenMP 201511 (a.k.a. OpenMP 4.5)
  • LAPACK is enabled (usually provided by MKL)
  • NNPACK is enabled
  • CPU capability usage: AVX2
  • CUDA Runtime 11.7
  • NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  • CuDNN 8.5
  • Magma 2.6.1
  • Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.14.1+cu117
OpenCV: 4.8.0
MMCV: 1.7.1
MMCV Compiler: GCC 9.3
MMCV CUDA Compiler: 11.6
MMRotate: 0.3.4+

Reproduces the problem - code sample

no

Reproduces the problem - command or script

CUDA_VISIBLE_DEVICES=0 python /home/shenhui/semi-objetc-detection/SOOD-main/train.py /home/shenhui/semi-objetc-detection/SOOD-main/configs/ssad_fcos/dense_teacher_fcos_dota15_10per.py --work-dir /hdc/workspace_sh/debug

Reproduces the problem - error message

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.neck.fpn_convs.4.conv.weight - torch.Size([256, 256, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.neck.fpn_convs.4.conv.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.0.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.0.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.0.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.1.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.1.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.1.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.2.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.2.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.2.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.3.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.3.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.cls_convs.3.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.0.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.0.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.0.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.1.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.1.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.1.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.2.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.2.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.2.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.3.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.3.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,734 - mmcv - INFO -
teacher.bbox_head.reg_convs.3.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.conv_cls.weight - torch.Size([16, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=-4.59511985013459

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.conv_cls.bias - torch.Size([16]):
NormalInit: mean=0, std=0.01, bias=-4.59511985013459

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.conv_reg.weight - torch.Size([4, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.conv_reg.bias - torch.Size([4]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.conv_centerness.weight - torch.Size([1, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.conv_centerness.bias - torch.Size([1]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.conv_angle.weight - torch.Size([1, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.conv_angle.bias - torch.Size([1]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.scales.0.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.scales.1.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.scales.2.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.scales.3.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.scales.4.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO -
teacher.bbox_head.scale_angle.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.conv1.weight - torch.Size([64, 3, 7, 7]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.bn1.weight - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.bn1.bias - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.layer1.0.conv1.weight - torch.Size([64, 64, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.layer1.0.bn1.weight - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.layer1.0.bn1.bias - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.layer1.0.conv2.weight - torch.Size([64, 64, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.layer1.0.bn2.weight - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.layer1.0.bn2.bias - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.layer1.0.conv3.weight - torch.Size([256, 64, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.layer1.0.bn3.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.layer1.0.bn3.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.layer1.0.downsample.0.weight - torch.Size([256, 64, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,735 - mmcv - INFO -
student.backbone.layer1.0.downsample.1.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.0.downsample.1.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.1.conv1.weight - torch.Size([64, 256, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.1.bn1.weight - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.1.bn1.bias - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.1.conv2.weight - torch.Size([64, 64, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.1.bn2.weight - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.1.bn2.bias - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.1.conv3.weight - torch.Size([256, 64, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.1.bn3.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.1.bn3.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.2.conv1.weight - torch.Size([64, 256, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.2.bn1.weight - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.2.bn1.bias - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.2.conv2.weight - torch.Size([64, 64, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.2.bn2.weight - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.2.bn2.bias - torch.Size([64]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.2.conv3.weight - torch.Size([256, 64, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.2.bn3.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer1.2.bn3.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer2.0.conv1.weight - torch.Size([128, 256, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer2.0.bn1.weight - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer2.0.bn1.bias - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer2.0.conv2.weight - torch.Size([128, 128, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer2.0.bn2.weight - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer2.0.bn2.bias - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer2.0.conv3.weight - torch.Size([512, 128, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer2.0.bn3.weight - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,736 - mmcv - INFO -
student.backbone.layer2.0.bn3.bias - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.0.downsample.0.weight - torch.Size([512, 256, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.0.downsample.1.weight - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.0.downsample.1.bias - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.1.conv1.weight - torch.Size([128, 512, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.1.bn1.weight - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.1.bn1.bias - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.1.conv2.weight - torch.Size([128, 128, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.1.bn2.weight - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.1.bn2.bias - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.1.conv3.weight - torch.Size([512, 128, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.1.bn3.weight - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.1.bn3.bias - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.2.conv1.weight - torch.Size([128, 512, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.2.bn1.weight - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.2.bn1.bias - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.2.conv2.weight - torch.Size([128, 128, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.2.bn2.weight - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.2.bn2.bias - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.2.conv3.weight - torch.Size([512, 128, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.2.bn3.weight - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.2.bn3.bias - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.3.conv1.weight - torch.Size([128, 512, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.3.bn1.weight - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.3.bn1.bias - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.3.conv2.weight - torch.Size([128, 128, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.3.bn2.weight - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.3.bn2.bias - torch.Size([128]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,737 - mmcv - INFO -
student.backbone.layer2.3.conv3.weight - torch.Size([512, 128, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer2.3.bn3.weight - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer2.3.bn3.bias - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.conv1.weight - torch.Size([256, 512, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.bn1.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.bn1.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.conv2.weight - torch.Size([256, 256, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.bn2.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.bn2.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.conv3.weight - torch.Size([1024, 256, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.bn3.weight - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.bn3.bias - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.downsample.0.weight - torch.Size([1024, 512, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.downsample.1.weight - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.0.downsample.1.bias - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.1.conv1.weight - torch.Size([256, 1024, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.1.bn1.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.1.bn1.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.1.conv2.weight - torch.Size([256, 256, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.1.bn2.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.1.bn2.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.1.conv3.weight - torch.Size([1024, 256, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.1.bn3.weight - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.1.bn3.bias - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.2.conv1.weight - torch.Size([256, 1024, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.2.bn1.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.2.bn1.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,738 - mmcv - INFO -
student.backbone.layer3.2.conv2.weight - torch.Size([256, 256, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.2.bn2.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.2.bn2.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.2.conv3.weight - torch.Size([1024, 256, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.2.bn3.weight - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.2.bn3.bias - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.3.conv1.weight - torch.Size([256, 1024, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.3.bn1.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.3.bn1.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.3.conv2.weight - torch.Size([256, 256, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.3.bn2.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.3.bn2.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.3.conv3.weight - torch.Size([1024, 256, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.3.bn3.weight - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.3.bn3.bias - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.4.conv1.weight - torch.Size([256, 1024, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.4.bn1.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.4.bn1.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.4.conv2.weight - torch.Size([256, 256, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.4.bn2.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.4.bn2.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.4.conv3.weight - torch.Size([1024, 256, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.4.bn3.weight - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.4.bn3.bias - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.5.conv1.weight - torch.Size([256, 1024, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.5.bn1.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.5.bn1.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.5.conv2.weight - torch.Size([256, 256, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.5.bn2.weight - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,739 - mmcv - INFO -
student.backbone.layer3.5.bn2.bias - torch.Size([256]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer3.5.conv3.weight - torch.Size([1024, 256, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer3.5.bn3.weight - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer3.5.bn3.bias - torch.Size([1024]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.conv1.weight - torch.Size([512, 1024, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.bn1.weight - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.bn1.bias - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.conv2.weight - torch.Size([512, 512, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.bn2.weight - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.bn2.bias - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.conv3.weight - torch.Size([2048, 512, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.bn3.weight - torch.Size([2048]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.bn3.bias - torch.Size([2048]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.downsample.0.weight - torch.Size([2048, 1024, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.downsample.1.weight - torch.Size([2048]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.0.downsample.1.bias - torch.Size([2048]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.1.conv1.weight - torch.Size([512, 2048, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.1.bn1.weight - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.1.bn1.bias - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.1.conv2.weight - torch.Size([512, 512, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.1.bn2.weight - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.1.bn2.bias - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.1.conv3.weight - torch.Size([2048, 512, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.1.bn3.weight - torch.Size([2048]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.1.bn3.bias - torch.Size([2048]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.2.conv1.weight - torch.Size([512, 2048, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.2.bn1.weight - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,740 - mmcv - INFO -
student.backbone.layer4.2.bn1.bias - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO -
student.backbone.layer4.2.conv2.weight - torch.Size([512, 512, 3, 3]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO -
student.backbone.layer4.2.bn2.weight - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO -
student.backbone.layer4.2.bn2.bias - torch.Size([512]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO -
student.backbone.layer4.2.conv3.weight - torch.Size([2048, 512, 1, 1]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO -
student.backbone.layer4.2.bn3.weight - torch.Size([2048]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO -
student.backbone.layer4.2.bn3.bias - torch.Size([2048]):
PretrainedInit: load from torchvision://resnet50

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.lateral_convs.0.conv.weight - torch.Size([256, 512, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.lateral_convs.0.conv.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.lateral_convs.1.conv.weight - torch.Size([256, 1024, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.lateral_convs.1.conv.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.lateral_convs.2.conv.weight - torch.Size([256, 2048, 1, 1]):
XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.lateral_convs.2.conv.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.fpn_convs.0.conv.weight - torch.Size([256, 256, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.fpn_convs.0.conv.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.fpn_convs.1.conv.weight - torch.Size([256, 256, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.fpn_convs.1.conv.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.fpn_convs.2.conv.weight - torch.Size([256, 256, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.fpn_convs.2.conv.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.fpn_convs.3.conv.weight - torch.Size([256, 256, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.fpn_convs.3.conv.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.fpn_convs.4.conv.weight - torch.Size([256, 256, 3, 3]):
XavierInit: gain=1, distribution=uniform, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO -
student.neck.fpn_convs.4.conv.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO -
student.bbox_head.cls_convs.0.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO -
student.bbox_head.cls_convs.0.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO -
student.bbox_head.cls_convs.0.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,741 - mmcv - INFO -
student.bbox_head.cls_convs.1.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,741 - mmcv - INFO -
student.bbox_head.cls_convs.1.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.cls_convs.1.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.cls_convs.2.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.cls_convs.2.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.cls_convs.2.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.cls_convs.3.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.cls_convs.3.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.cls_convs.3.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.0.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.0.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.0.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.1.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.1.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.1.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.2.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.2.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.2.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.3.conv.weight - torch.Size([256, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.3.gn.weight - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.reg_convs.3.gn.bias - torch.Size([256]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.conv_cls.weight - torch.Size([16, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=-4.59511985013459

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.conv_cls.bias - torch.Size([16]):
NormalInit: mean=0, std=0.01, bias=-4.59511985013459

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.conv_reg.weight - torch.Size([4, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.conv_reg.bias - torch.Size([4]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.conv_centerness.weight - torch.Size([1, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.conv_centerness.bias - torch.Size([1]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.conv_angle.weight - torch.Size([1, 256, 3, 3]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.conv_angle.bias - torch.Size([1]):
NormalInit: mean=0, std=0.01, bias=0

2024-09-23 11:46:08,742 - mmcv - INFO -
student.bbox_head.scales.0.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,743 - mmcv - INFO -
student.bbox_head.scales.1.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,743 - mmcv - INFO -
student.bbox_head.scales.2.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,743 - mmcv - INFO -
student.bbox_head.scales.3.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,743 - mmcv - INFO -
student.bbox_head.scales.4.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

2024-09-23 11:46:08,743 - mmcv - INFO -
student.bbox_head.scale_angle.scale - torch.Size([]):
The value is the same before and after calling init_weights of RotatedSSTGDenseTeacher

fatal: not a git repository (or any of the parent directories): .git

It was stuck here for a long time for more than half an hour and I found out by debugging that it's stuck here
model = MMDataParallel(
model.cuda(cfg.gpu_ids[0]), device_ids=cfg.gpu_ids)
and The rest of the training became very slow， Both the GPU and the CPU have 0 usage

Additional information

No response
@Leoeeeeeeea
Copy link

have you solved this ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Leoeeeeeeea @1shenhui