Skip to content

Commit 71bbb1b

Browse files
authored
Update derived_zero initialization to use device aligned with other i… (#14978)
Run into issues when trying to use GPU to quantize the model. Fix the device type for zero point. The log was like ``` input.device.type cuda scales.device.type cuda zero_points.device.type cpu ```
1 parent d382f6b commit 71bbb1b

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

backends/qualcomm/quantizer/qconfig.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,9 @@ def _derive_bias_qparams_fn(
5252
act_scale, weight_scale
5353
)
5454
derived_scale = (broadcast_act_scale * broadcast_weight_scale).to(torch.float32)
55-
derived_zero = torch.zeros(derived_scale.size()).to(torch.int32)
55+
derived_zero = torch.zeros(derived_scale.size(), device=weight_zp.device).to(
56+
torch.int32
57+
)
5658
if isinstance(weight_obs_or_fq, PerBlockParamObserver):
5759
# keep maximum scale of each channel for bias
5860
derived_scale = (

0 commit comments

Comments
 (0)