Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What do the following parameters mean? #3

Open
littleTwelve opened this issue Sep 9, 2018 · 1 comment
Open

What do the following parameters mean? #3

littleTwelve opened this issue Sep 9, 2018 · 1 comment

Comments

@littleTwelve
Copy link

littleTwelve commented Sep 9, 2018

Could you explain what do 'lr_rampup_kimg', 'total_kimg', 'tick_kimg_default' and 'cur_nimg' mean in train.py respectively?
And what do 'resume_nimg' and 'cur_tick' mean in trainer.py?
And what is the function 'run' in trainer.py for?

@Michalaq
Copy link
Member

Hi,
to get the best grip, refer to the original implementation in theano. Basically, the original implementation instead of counting "iterations" or "epochs", only counts "nimg" or "kimg", which stand for number of images or thousands of images. So 1kimg = 1000nimg = 1000 images shown to the network during training.
That being said:

  1. lr_rampup_kimg: how many kimg should learning rate "ramp up"
  2. total_kimg for how many kimg in total should the network be trained
  3. tick_kimg_default, so, this is only for logging purposes/doing stuff periodically. You can think of a "tick" as a sort of "iteration": a log line will be printed every tick. tick_kimg_default says "how many kimgs to show the network before the next tick". It is useful, because showing 100kimg to a small network is much faster than showing 100kimg to a full network (after full progressive growing). So you may want to print loss every 40kimg in the beginning, but move down to every 2kimg later on.
  4. cur_nimg - well, current nimg.
  5. resume_nimg - what is the initial value for nimg (this is used only when you resume training)
  6. cur_tick - current tick
  7. run method of trainer: this is the entrypoint. I think now that i may be kind of misleading, but I think I was following the convention of what used to be PyTorch Trainer class (now removed from PyTorch). Basically, .train() trains on one batch, while .run(3000) will run the whole training for 3000kimg.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants