Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Request] Any plan to support BF16 inference #11

Open
zhouheyun opened this issue Sep 23, 2023 · 2 comments
Open

[Feature Request] Any plan to support BF16 inference #11

zhouheyun opened this issue Sep 23, 2023 · 2 comments

Comments

@zhouheyun
Copy link

zhouheyun commented Sep 23, 2023 •
edited
Loading

Any plan to support BF16 inference? Our model encountered fp16 overflow after deployment.
@zhouheyun zhouheyun changed the title [Feature Request] [Feature Request] Any plan to support BF16 inference Sep 23, 2023
@ZhangZhiPku
Copy link

啥家庭要跑BF16啊，啥算子啊

@zhouheyun
Copy link
Author

啥家庭要跑BF16啊，啥算子啊

就是用BF16的类型推理GPT模型啊，我理解涉及到的算子，都得支持BF16数据类型？具体来说是发现MLP的第二个FC FP16 上溢出了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ZhangZhiPku @zhouheyun