Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added some features to SCM #3

Open
author31 opened this issue Dec 1, 2022 · 2 comments
Open

Added some features to SCM #3

author31 opened this issue Dec 1, 2022 · 2 comments

Comments

@author31
Copy link

author31 commented Dec 1, 2022

Hi! I really like the your work and I tried that on my RTX 3060 12G, but it kept going NaN so I added lr scheduler and refactor the whole codebase. Here is the link https://github.com/author31/scm_nbdev. Once again, thanks for your work, keep doing.

@hbai98
Copy link
Owner

hbai98 commented Dec 2, 2022

Hi! Thanks for your attention.

Could you please specify where NaN is produced in your code?
Furthermore, can you briefly describe your plan to change the lr scheduler?

@author31
Copy link
Author

author31 commented Dec 2, 2022

In my cases, the NaN problem is because learning rates to large even is 3e-6. So, I added the lr scheduler to do the warmcos scheduling style which like this
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants