hydra.errors.HydraException: Error calling 'datasets.semseg.SemanticSegmentationDataset' : not available number labels, select from: 200, 200 #103
Comments
|
I encountered the same question! Did you solve it? |
|
Hi! Is it possible that your generated dataset is ScanNet200 and not ScanNet? Best, |
|
Your reply and help are greatly appreciated! and regenerated the scannet200 data by "python -m datasets.preprocessing.scannet_preprocessing preprocess
|
|
And when I run the train command as "python main_instance_segmentation.py" or "python main_instance_segmentation.py \ general.checkpoint='/home/mylabs/Mask3D/checkpoints/s3dis/from_scratch/area1.ckpt' |
not yet. |
|
I have the similar problem and hope the author could help to explain it. Many thanks! @JonasSchult Mask3D/main_instance_segmentation.py", line 79, in train runner = Trainer( File "/usr2/.local/lib/python3.10/site-packages/pytorch_lightning/utilities/argparse.py", line 345, in insert_env_defaults return fn(self, **kwargs) During handling of the above exception, another exception occurred: Mask3D/main_instance_segmentation.py", line 116, in |
|
It seems that the issue results from Lines 699 to 720 in 3db966d The label_database.yaml is the same for both ScanNet and ScanNet200 after preprocessing. |
|
Thank you for your reply, but I don't understand how to change the code to solve this problem. May I get some advice from you? |
|
You just need to rerun scannet data preprocessing with |
|
Thank you very much, I have solved the issue. However, I have got another problem with the flowing: Epoch 49: 100%|████| 1513/1513 [1:56:39<00:00, 4.63s/it, loss=43.9, v_num=TION] During handling of the above exception, another exception occurred: Traceback (most recent call last): I am not very clear about the problem above, may I get some help from you, if so, it would my best greatest. |
|
I have the same problem. Did you ever find solution for this problem. |
|
Hey, I just found that the function traceback.print_exception() has been changed in python v3.10, where the 'etype' parameter has been renamed to 'exc' and is now positional-only. So I rewrite the file "/Users/.../python3.10/site-packages/hydra/_internal/utils.py", line 267 from
to
And things go well. The package's change log can be found in website). |
when I run the flow command, I got the issue: hydra.errors.HydraException: Error calling 'datasets.semseg.SemanticSegmentationDataset' : not available number labels, select from: 200, 200
main_instance_segmentation.py general.experiment_name=test1_scannet_val_query_150_topk_500_dbscan_0.95 general.project_name=scannet_eval general.checkpoint='checkpoints/scannet/scannet_val.ckpt' general.train_mode=false general.eval_on_segments=true general.train_on_segments=true model.num_queries=150 general.topk_per_image=500 general.use_dbscan=true general.dbscan_eps=0.95
/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/utilities/seed.py:55: UserWarning: No seed found, seed set to 2801433411
rank_zero_warn(f"No seed found, seed set to {seed}")
Global seed set to 2801433411
EXPERIMENT ALREADY EXIST
{'target': 'pytorch_lightning.loggers.WandbLogger', 'project': '${general.project_name}', 'name': '${general.experiment_name}', 'save_dir': '${general.save_dir}', 'entity': 'manjusaka_labs', 'resume': 'allow', 'id': '${general.experiment_name}'}
wandb: Currently logged in as: bh_c (manjusaka_labs). Use
wandb login --reloginto force reloginwandb: wandb version 0.15.4 is available! To upgrade, please run:
wandb: $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.15.0
wandb: Run data is saved locally in saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95/wandb/run-20230609_012749-test1_scannet_val_query_150_topk_500_dbscan_0.95
wandb: Run
wandb offlineto turn off syncing.wandb: Resuming run test1_scannet_val_query_150_topk_500_dbscan_0.95
wandb: ⭐️ View project at https://wandb.ai/manjusaka_labs/scannet_eval
wandb: 🚀 View run at https://wandb.ai/manjusaka_labs/scannet_eval/runs/test1_scannet_val_query_150_topk_500_dbscan_0.95
2023-06-09 01:27:54.018 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:91 - Key not found, it will be initialized randomly: model.scene_min
2023-06-09 01:27:54.019 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:91 - Key not found, it will be initialized randomly: model.scene_max
2023-06-09 01:27:54.145 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:100 - criterion.empty_weight not in loaded checkpoint
2023-06-09 01:27:54.149 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:115 - excessive key: model.scene_min
2023-06-09 01:27:54.149 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:115 - excessive key: model.scene_max
[2023-06-09 01:27:54,238][main][INFO] - {'general_train_mode': False, 'general_task': 'instance_segmentation', 'general_seed': None, 'general_checkpoint': 'checkpoints/scannet/scannet_val.ckpt', 'general_backbone_checkpoint': None, 'general_freeze_backbone': False, 'general_linear_probing_backbone': False, 'general_train_on_segments': True, 'general_eval_on_segments': True, 'general_filter_out_instances': False, 'general_save_visualizations': False, 'general_visualization_point_size': 20, 'general_decoder_id': -1, 'general_export': False, 'general_use_dbscan': True, 'general_ignore_class_threshold': 100, 'general_project_name': 'scannet_eval', 'general_workspace': 'jonasschult', 'general_experiment_name': 'test1_scannet_val_query_150_topk_500_dbscan_0.95', 'general_num_targets': 19, 'general_add_instance': True, 'general_dbscan_eps': 0.95, 'general_dbscan_min_points': 1, 'general_export_threshold': 0.0001, 'general_reps_per_epoch': 1, 'general_on_crops': False, 'general_scores_threshold': 0.0, 'general_iou_threshold': 1.0, 'general_area': 5, 'general_eval_inner_core': -1, 'general_topk_per_image': 500, 'general_ignore_mask_idx': [], 'general_max_batch_size': 99999999, 'general_save_dir': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95', 'general_gpus': 1, 'data_train_mode': 'train', 'data_validation_mode': 'validation', 'data_test_mode': 'validation', 'data_ignore_label': 255, 'data_add_raw_coordinates': True, 'data_add_colors': True, 'data_add_normals': False, 'data_in_channels': 3, 'data_num_labels': 20, 'data_add_instance': True, 'data_task': 'instance_segmentation', 'data_pin_memory': False, 'data_num_workers': 4, 'data_batch_size': 5, 'data_test_batch_size': 1, 'data_cache_data': False, 'data_voxel_size': 0.02, 'data_reps_per_epoch': 1, 'data_cropping': False, 'data_cropping_args_min_points': 30000, 'data_cropping_args_aspect': 0.8, 'data_cropping_args_min_crop': 0.5, 'data_cropping_args_max_crop': 1.0, 'data_crop_min_size': 20000, 'data_crop_length': 6.0, 'data_cropping_v1': True, 'data_train_dataloader__target_': 'torch.utils.data.DataLoader', 'data_train_dataloader_shuffle': True, 'data_train_dataloader_pin_memory': False, 'data_train_dataloader_num_workers': 4, 'data_train_dataloader_batch_size': 5, 'data_validation_dataloader__target_': 'torch.utils.data.DataLoader', 'data_validation_dataloader_shuffle': False, 'data_validation_dataloader_pin_memory': False, 'data_validation_dataloader_num_workers': 4, 'data_validation_dataloader_batch_size': 1, 'data_test_dataloader__target_': 'torch.utils.data.DataLoader', 'data_test_dataloader_shuffle': False, 'data_test_dataloader_pin_memory': False, 'data_test_dataloader_num_workers': 4, 'data_test_dataloader_batch_size': 1, 'data_train_dataset__target_': 'datasets.semseg.SemanticSegmentationDataset', 'data_train_dataset_dataset_name': 'scannet', 'data_train_dataset_data_dir': 'data/processed/scannet', 'data_train_dataset_image_augmentations_path': 'conf/augmentation/albumentations_aug.yaml', 'data_train_dataset_volume_augmentations_path': 'conf/augmentation/volumentations_aug.yaml', 'data_train_dataset_label_db_filepath': 'data/processed/scannet/label_database.yaml', 'data_train_dataset_color_mean_std': 'data/processed/scannet/color_mean_std.yaml', 'data_train_dataset_data_percent': 1.0, 'data_train_dataset_mode': 'train', 'data_train_dataset_ignore_label': 255, 'data_train_dataset_num_labels': 20, 'data_train_dataset_add_raw_coordinates': True, 'data_train_dataset_add_colors': True, 'data_train_dataset_add_normals': False, 'data_train_dataset_add_instance': True, 'data_train_dataset_instance_oversampling': 0.0, 'data_train_dataset_place_around_existing': False, 'data_train_dataset_point_per_cut': 0, 'data_train_dataset_max_cut_region': 0, 'data_train_dataset_flip_in_center': False, 'data_train_dataset_noise_rate': 0, 'data_train_dataset_resample_points': 0, 'data_train_dataset_add_unlabeled_pc': False, 'data_train_dataset_cropping': False, 'data_train_dataset_cropping_args_min_points': 30000, 'data_train_dataset_cropping_args_aspect': 0.8, 'data_train_dataset_cropping_args_min_crop': 0.5, 'data_train_dataset_cropping_args_max_crop': 1.0, 'data_train_dataset_is_tta': False, 'data_train_dataset_crop_min_size': 20000, 'data_train_dataset_crop_length': 6.0, 'data_train_dataset_filter_out_classes': [0, 1], 'data_train_dataset_label_offset': 2, 'data_validation_dataset__target_': 'datasets.semseg.SemanticSegmentationDataset', 'data_validation_dataset_dataset_name': 'scannet', 'data_validation_dataset_data_dir': 'data/processed/scannet', 'data_validation_dataset_image_augmentations_path': None, 'data_validation_dataset_volume_augmentations_path': None, 'data_validation_dataset_label_db_filepath': 'data/processed/scannet/label_database.yaml', 'data_validation_dataset_color_mean_std': 'data/processed/scannet/color_mean_std.yaml', 'data_validation_dataset_data_percent': 1.0, 'data_validation_dataset_mode': 'validation', 'data_validation_dataset_ignore_label': 255, 'data_validation_dataset_num_labels': 20, 'data_validation_dataset_add_raw_coordinates': True, 'data_validation_dataset_add_colors': True, 'data_validation_dataset_add_normals': False, 'data_validation_dataset_add_instance': True, 'data_validation_dataset_cropping': False, 'data_validation_dataset_is_tta': False, 'data_validation_dataset_crop_min_size': 20000, 'data_validation_dataset_crop_length': 6.0, 'data_validation_dataset_filter_out_classes': [0, 1], 'data_validation_dataset_label_offset': 2, 'data_test_dataset__target_': 'datasets.semseg.SemanticSegmentationDataset', 'data_test_dataset_dataset_name': 'scannet', 'data_test_dataset_data_dir': 'data/processed/scannet', 'data_test_dataset_image_augmentations_path': None, 'data_test_dataset_volume_augmentations_path': None, 'data_test_dataset_label_db_filepath': 'data/processed/scannet/label_database.yaml', 'data_test_dataset_color_mean_std': 'data/processed/scannet/color_mean_std.yaml', 'data_test_dataset_data_percent': 1.0, 'data_test_dataset_mode': 'validation', 'data_test_dataset_ignore_label': 255, 'data_test_dataset_num_labels': 20, 'data_test_dataset_add_raw_coordinates': True, 'data_test_dataset_add_colors': True, 'data_test_dataset_add_normals': False, 'data_test_dataset_add_instance': True, 'data_test_dataset_cropping': False, 'data_test_dataset_is_tta': False, 'data_test_dataset_crop_min_size': 20000, 'data_test_dataset_crop_length': 6.0, 'data_test_dataset_filter_out_classes': [0, 1], 'data_test_dataset_label_offset': 2, 'data_train_collation__target_': 'datasets.utils.VoxelizeCollate', 'data_train_collation_ignore_label': 255, 'data_train_collation_voxel_size': 0.02, 'data_train_collation_mode': 'train', 'data_train_collation_small_crops': False, 'data_train_collation_very_small_crops': False, 'data_train_collation_batch_instance': False, 'data_train_collation_probing': False, 'data_train_collation_task': 'instance_segmentation', 'data_train_collation_ignore_class_threshold': 100, 'data_train_collation_filter_out_classes': [0, 1], 'data_train_collation_label_offset': 2, 'data_train_collation_num_queries': 150, 'data_validation_collation__target_': 'datasets.utils.VoxelizeCollate', 'data_validation_collation_ignore_label': 255, 'data_validation_collation_voxel_size': 0.02, 'data_validation_collation_mode': 'validation', 'data_validation_collation_batch_instance': False, 'data_validation_collation_probing': False, 'data_validation_collation_task': 'instance_segmentation', 'data_validation_collation_ignore_class_threshold': 100, 'data_validation_collation_filter_out_classes': [0, 1], 'data_validation_collation_label_offset': 2, 'data_validation_collation_num_queries': 150, 'data_test_collation__target_': 'datasets.utils.VoxelizeCollate', 'data_test_collation_ignore_label': 255, 'data_test_collation_voxel_size': 0.02, 'data_test_collation_mode': 'validation', 'data_test_collation_batch_instance': False, 'data_test_collation_probing': False, 'data_test_collation_task': 'instance_segmentation', 'data_test_collation_ignore_class_threshold': 100, 'data_test_collation_filter_out_classes': [0, 1], 'data_test_collation_label_offset': 2, 'data_test_collation_num_queries': 150, 'logging': [{'target': 'pytorch_lightning.loggers.WandbLogger', 'project': 'scannet_eval', 'name': 'test1_scannet_val_query_150_topk_500_dbscan_0.95', 'save_dir': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95', 'entity': 'manjusaka_labs', 'resume': 'allow', 'id': 'test1_scannet_val_query_150_topk_500_dbscan_0.95'}], 'model__target_': 'models.Mask3D', 'model_hidden_dim': 128, 'model_dim_feedforward': 1024, 'model_num_queries': 150, 'model_num_heads': 8, 'model_num_decoders': 3, 'model_dropout': 0.0, 'model_pre_norm': False, 'model_use_level_embed': False, 'model_normalize_pos_enc': True, 'model_positional_encoding_type': 'fourier', 'model_gauss_scale': 1.0, 'model_hlevels': [0, 1, 2, 3], 'model_non_parametric_queries': True, 'model_random_query_both': False, 'model_random_normal': False, 'model_random_queries': False, 'model_use_np_features': False, 'model_sample_sizes': [200, 800, 3200, 12800, 51200], 'model_max_sample_size': False, 'model_shared_decoder': True, 'model_num_classes': 19, 'model_train_on_segments': True, 'model_scatter_type': 'mean', 'model_voxel_size': 0.02, 'model_config_backbone__target_': 'models.Res16UNet34C', 'model_config_backbone_config_dialations': [1, 1, 1, 1], 'model_config_backbone_config_conv1_kernel_size': 5, 'model_config_backbone_config_bn_momentum': 0.02, 'model_config_backbone_in_channels': 3, 'model_config_backbone_out_channels': 20, 'model_config_backbone_out_fpn': True, 'metrics__target_': 'models.metrics.ConfusionMatrix', 'metrics_num_classes': 20, 'metrics_ignore_label': 255, 'optimizer__target_': 'torch.optim.AdamW', 'optimizer_lr': 0.0001, 'scheduler_scheduler__target_': 'torch.optim.lr_scheduler.OneCycleLR', 'scheduler_scheduler_max_lr': 0.0001, 'scheduler_scheduler_epochs': 601, 'scheduler_scheduler_steps_per_epoch': -1, 'scheduler_pytorch_lightning_params_interval': 'step', 'trainer_deterministic': False, 'trainer_max_epochs': 601, 'trainer_min_epochs': 1, 'trainer_resume_from_checkpoint': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95/last-epoch.ckpt', 'trainer_check_val_every_n_epoch': 50, 'trainer_num_sanity_val_steps': 2, 'callbacks': [{'target': 'pytorch_lightning.callbacks.ModelCheckpoint', 'monitor': 'val_mean_ap_50', 'save_last': True, 'save_top_k': 1, 'mode': 'max', 'dirpath': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95', 'filename': '{epoch}-{val_mean_ap_50:.3f}', 'every_n_epochs': 1}, {'target': 'pytorch_lightning.callbacks.LearningRateMonitor'}], 'matcher__target_': 'models.matcher.HungarianMatcher', 'matcher_cost_class': 2.0, 'matcher_cost_mask': 5.0, 'matcher_cost_dice': 2.0, 'matcher_num_points': -1, 'loss__target_': 'models.criterion.SetCriterion', 'loss_num_classes': 19, 'loss_eos_coef': 0.1, 'loss_losses': ['labels', 'masks'], 'loss_num_points': -1, 'loss_oversample_ratio': 3.0, 'loss_importance_sample_ratio': 0.75, 'loss_class_weights': -1}
/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py:446: LightningDeprecationWarning: Setting
Trainer(gpus=1)is deprecated in v1.7 and will be removed in v2.0. Please useTrainer(accelerator='gpu', devices=1)instead.rank_zero_deprecation(
/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py:52: LightningDeprecationWarning: Setting
Trainer(resume_from_checkpoint=)is deprecated in v1.5 and will be removed in v1.7. Please passTrainer.fit(ckpt_path=)directly instead.rank_zero_deprecation(
/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py:57: LightningDeprecationWarning: Setting
Trainer(weights_save_path=)has been deprecated in v1.6 and will be removed in v1.8. Please passdirpathdirectly to theModelCheckpointcallbackrank_zero_deprecation(
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/home/mylabs/Mask3D/datasets/semseg.py:696: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
file = yaml.load(f)
Traceback (most recent call last):
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/utils.py", line 63, in call
return _instantiate_class(type_or_callable, config, *args, **kwargs)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 500, in _instantiate_class
return clazz(*args, **final_kwargs)
File "/home/mylabs/Mask3D/datasets/semseg.py", line 218, in init
self._labels = self._select_correct_labels(labels, num_labels)
File "/home/mylabs/Mask3D/datasets/semseg.py", line 724, in _select_correct_labels
raise ValueError(msg)
ValueError: not available number labels, select from:
200, 200
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 347, in
lambda: hydra.run(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 107, in run
return run_job(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/core/utils.py", line 128, in run_job
ret.return_value = task_function(task_cfg)
File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 110, in main
test(cfg)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/main.py", line 27, in decorated_main
return task_function(cfg_passthrough)
File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 100, in test
runner.test(model)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 864, in test
return self._call_and_handle_interrupt(self._test_impl, model, dataloaders, ckpt_path, verbose, datamodule)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 911, in _test_impl
results = self._run(model, ckpt_path=self.ckpt_path)
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1097, in _run
self._data_connector.prepare_data()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/data_connector.py", line 120, in prepare_data
self.trainer._call_lightning_module_hook("prepare_data")
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1552, in _call_lightning_module_hook
output = fn(*args, **kwargs)
File "/home/mylabs/Mask3D/trainer/trainer.py", line 1269, in prepare_data
self.train_dataset = hydra.utils.instantiate(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/utils.py", line 70, in call
raise HydraException(f"Error calling '{cls}' : {e}") from e
hydra.errors.HydraException: Error calling 'datasets.semseg.SemanticSegmentationDataset' : not available number labels, select from:
200, 200
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 114, in
main()
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/main.py", line 32, in decorated_main
_run_hydra(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra
run_and_report(
File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 267, in run_and_report
print_exception(etype=None, value=ex, tb=final_tb) # type: ignore
TypeError: print_exception() got an unexpected keyword argument 'etype'
Can you tell me how to solve the problem, thank you very much!
The text was updated successfully, but these errors were encountered: