Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fm-equalize] Extend functionalities #14264

Open
mhs4670go opened this issue Oct 28, 2024 · 2 comments
Open

[fm-equalize] Extend functionalities #14264

mhs4670go opened this issue Oct 28, 2024 · 2 comments

Comments

@mhs4670go
Copy link
Contributor

What

Let's support more features for fm-equalize.

  • Parameterize alpha of SmoothQuant.
  • Think about how to apply the algorithm to Layernorm.

Why

Parameterize alpha of SmoothQuant.

Current implementation fixes alpha value of SmoothQuant. But, best alpha may vary depending on the models. Let's parameterize it. Users would be able to set their best alpha then.

Think about how to apply the algorithm to Layernorm.

Layernorm decomposes into Mul and Add. Therefore, if the circle to apply is optimized first, such mul and add are fused into other ops, which makes it impossible to apply SmoothQuant. Therefore, we need to consider the order of the sequences.

@jinevening
Copy link
Contributor

It seems that the two issues are different. Why not making separate issues ?

@mhs4670go
Copy link
Contributor Author

@jinevening This issue might be a parent issue. I'll split them when they should be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants