Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

这道题为啥我认为是除以批量大小呢？ #47

Open
YUFEIFUT opened this issue Nov 19, 2023 · 2 comments
Open

这道题为啥我认为是除以批量大小呢？ #47

YUFEIFUT opened this issue Nov 19, 2023 · 2 comments

Comments

@YUFEIFUT
Copy link

image

这道题为啥我认为是除以批量大小呢？问了一下 GPT ，也是除以的

image
@Ethan-Chen-plus
Copy link
Contributor

梯度是累积的，貌似就该累积batch大小的学习率？

@YUFEIFUT
Copy link
Author

emmm，我发现这道题其实是逻辑问题，它是问总损失变成了平均损失，那么学习率应该怎么变吧，这个时候学习率应该除以批量的数量吧；

然后说到梯度是累计的，这个应该是指 Pytorch 中的梯度累计吧，这个应该跟这道题关系不大吧

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@YUFEIFUT @Ethan-Chen-plus