New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

为什么ppo model 需要接AutoModelForCausalLMWithValueHead呢？ #51

Open

jiahuanluo opened this issue Aug 29, 2023 · 1 comment

jiahuanluo commented Aug 29, 2023

感谢工作！
请问这里 ppo model 为什么要接一个valuehead 呢？
https://github.com/beyondguo/LLM-Tuning/blob/ed68123815bc0add9ad2d7ddc2a48dc584db2c94/RLHF/rl_training.py#L185C1-L185C11
这个head好像随机初始化的？

nghuyong commented Oct 10, 2023

因为还有一个cirtic model

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment