Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

During the training process, the loss has been rising #84

Open
amnesiac555 opened this issue Oct 4, 2020 · 8 comments
Open

During the training process, the loss has been rising #84

amnesiac555 opened this issue Oct 4, 2020 · 8 comments

Comments

@amnesiac555
Copy link

amnesiac555 commented Oct 4, 2020

Snipaste_2020-10-04_10-52-50
i use the mobilenet and fpn_inception, training data is Gopro, config is as follows:
anyone has trained successfully?
1
2

@amnesiac555
Copy link
Author

i didn't change training code, feel confused about it

@NoviceCrom
Copy link

NoviceCrom commented Oct 5, 2020

Maybe you can change the "disc_loss" in the config.yml to ragan_ls which is stated in the paper. By the way, could you share your Gopro dataset download from the link with me? I have no access to it for some reason.

@amnesiac555
Copy link
Author

Maybe you can change the "disc_loss" in the config.yml to ragan_ls which is stated in the paper.

OK THANKS ! i will try it immediately

@ldlshizhu
Copy link

请问你训练成功了吗

@ldlshizhu
Copy link

你好,我看到你train_batches_per_epoch: 1000
val_batches_per_epoch: 100这两行注释掉了,但是我下载的代码里没有注释这两行。请问是什么原因呢

@zmwang0574
Copy link

你好,我看到你train_batches_per_epoch: 1000
val_batches_per_epoch: 100这两行注释掉了,但是我下载的代码里没有注释这两行。请问是什么原因呢

Hi, I'm also wondering why you # these two lines. Are they unnecessary for the training? BTW, I'm also wondering what does "bounds" do, which is also #-ed by you.

@cianhwang
Copy link

你好,我看到你train_batches_per_epoch: 1000
val_batches_per_epoch: 100这两行注释掉了,但是我下载的代码里没有注释这两行。请问是什么原因呢

Hi, I'm also wondering why you # these two lines. Are they unnecessary for the training? BTW, I'm also wondering what does "bounds" do, which is also #-ed by you.

Commenting train/val_batches_per_epoch ensures each epoch the network sees all the training/validation samples. "bounds" s are intended to split the train/val set, which should all be set to [0.0, 1.0] if you specify separate training/validation sets.

@KMCzajkowski
Copy link

Have you solved this issue? I also have a similar problem. The loss is not improving during training regardless of what I do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants