Issues about Training and Sampling

[Zhengwen-Lan]

Hello, I would like to know which dataset you used to train the SDM.
During training, was it conditioned generation or unconditional generation?
In the original SDM code, both the training and sampling commands involve the num-classes parameter. How can I address the issue where the number of classes in the training dataset does not match the number of classes in the sampling dataset?
Thanks

[hjoh01]

Hi,
I have trained SDM model for each dataset, respectively.
Therefore, in this work, I have different num-classes on each dataset.
For training, the ground truth label is given as condition for image generation.

[Zhengwen-Lan]

Thank you for your response.
Did you train the SDM using ConSeP and GlaS datasets from scratch? I noticed that both datasets are quite small. Is it possible to achieve good-quality synthetic images with such small datasets?
Cause I trained on another 3D dataset with only about 30 samples for 10k steps, and the generated images are close to noise-like images. What could be the possible reasons for this?

[hjoh01]

I agree that training SDMs typically requires larger datasets to generate high-quality synthetic images. In my work with the CoNSeP and GLySAC datasets, I was able to utilize a larger effective dataset compared to the 3D dataset with 30 samples you mentioned. This is because:

• The original patch size for CoNSeP and GLySAC is 1024x1024 pixels.
• I trained the SDM on 256x256 pixel patches, which allowed for multiple patches per original image.

Furthermore, to enhance the quality of the generated data, I employed a different sampling strategy:

• Instead of sampling from random noise, I sampled from noised versions of the original images.
  This method helps to have the generated images more closely to the characteristics of the real data, potentially improving the overall quality and relevance of the synthetic images.

[Zhengwen-Lan]

Hi!
In your image_train.py, whenever I see the definition related to model_kwargs, it seems that during the training process it is always set to model_kwargs = None. However, in image_sample.py,model_kwargs = preprocess_input(cond, num_classes=args.num_classes) is used to pass conditional labels. Does this mean your training process is unconditional, and the conditional labels are only used during sampling?

[hjoh01]

Both the training and sampling processes are conditional processes.
In image_train.py, the Train_loop class from guided_diffusion.train_util is used to provide label conditioning.
In the Train_loop class, you can find 'run_loop' in line 163, and it preprocess conditional input using 'preprocess_input'.
For easier implementation, I have uploaded the training script to this repository.

[Zhengwen-Lan]

I am debugging your training code, and this part of the code is running at line 264. Is this normal? In my understanding, the conditional information should be evaluated at line 262.

[Zhengwen-Lan]

我正在调试你的训练代码，这部分代码在第 264 行运行。这正常吗？根据我的理解，条件信息应该在第 262 行进行评估。

When debugging to line 737 in gaussian_diffusion.py, model_kwargs becomes None.

[hjoh01]

I'll provide a detailed reply by next Tuesday, as my servers are currently under inspection.

• As I haven't worked on this repository recently, I need some time to review the code. At this point, I believe there might be some differences between the guided_diffusion/ and guided_diffusion_sample/ directories, as these are used for training and sampling respectively. These differences could potentially be the source of the issue you're experiencing.

I'll investigate this further and provide a more comprehensive analysis ASAP. Thank you for your patience.

[Zhengwen-Lan]

Thank you very much