SGLang int8 kernels #2196
vadimkantorov
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I wonder if these Triton kernels are any relevant for wider torchao / pytorch usage (and if this Triton impl is also any portable for CPU):
and if not - I wonder why sglang does not use the quant triton kernels/bindings from ao?
(Also similar question on liger / unsloth kernels - including the notorious rmsnorm kernels - any plans to upstream their main components like linear + chunked cross entropy some place upstream?)
Beta Was this translation helpful? Give feedback.
All reactions