New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

loss一直为0 #293

Open

chk4991 opened this issue Aug 2, 2024 · 1 comment

chk4991 commented Aug 2, 2024

训练qwen1.5-14B，lora，参数都是项目默认配置，loss一直为0，请问这种情况有遇到吗，怎么解决？已经在json文件中设置bf16为true了。

Author

chk4991 commented Aug 2, 2024

搞定了，最大句长设至太少，导致所有target_mask都是0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment