Skip to content

Commit

Permalink
[Bugfix] Fix compressed_tensors_moe bad config.strategy (vllm-project…
Browse files Browse the repository at this point in the history
…#9677)

Signed-off-by: Erkin Sagiroglu <[email protected]>
  • Loading branch information
mgoin authored and Erkin Sagiroglu committed Oct 26, 2024
1 parent 32908d1 commit bbd8c92
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@ def __init__(
config = self.quant_config.target_scheme_map["Linear"].get("weights")
self.num_bits = config.num_bits
self.packed_factor = 32 // config.num_bits
self.strategy = config.strategy.value
self.strategy = config.strategy
self.group_size = config.group_size
assert config.symmetric, (
"Only symmetric quantization is supported for MoE")
Expand Down

0 comments on commit bbd8c92

Please sign in to comment.