Enable fp16/bf16 absmax #1672
Enable fp16/bf16 absmax #1672
Conversation
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
jiqing-feng
force-pushed
the
absmax
branch
2 times, most recently
from
69b2146 to
50ee994
Compare
|
Hi @matthewdouglas . I kept cuda op the same as before, only enabled cpu/xpu absmax on half-precision. This PR could pass all cuda tests on A100 and all cpu tests on Intel Xeon node. For XPU, we have around 20 tests failed because of compile error but not introduced by this PR. So please review this PR. Thanks! |
Signed-off-by: jiqing-feng <[email protected]>
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| @@ -137,11 +135,10 @@ def test_dynamic_blockwise_quantization(self, device, dtype, nested, blocksize, | |||
| abserr = sum(diffs) / len(diffs) | |||
| relerr = sum(reldiffs) / len(reldiffs) | |||
| if signed: | |||
| threshold_abserr = 0.0036 if device in ("cpu", "xpu") and (F.ipex_cpu or F.ipex_xpu) else 0.0035 | |||
| assert abserr < 0.0036 | |||
There was a problem hiding this comment.
This is because threshold_abserr is not used.
| assert abserr < 0.0036 | ||
| assert relerr < 0.015 | ||
| else: | ||
| assert abserr < 0.00175 if device in ("cpu", "xpu") and (F.ipex_cpu or F.ipex_xpu) else 0.0023 | ||
| assert abserr < 0.0023 |
There was a problem hiding this comment.
We have no reason to have a tighter threshold for ipex, otherwise the half-precision check cannot pass.
|
Detect conflict with xpu sycl path, hold on this PR until xpu sycl path is merged. |
Hi @matthewdouglas , enable fp16/bf16 absmax on XPU could get 20% speed-up on our qlora case. Please review it. I am checking if there are any failed tests on CUDA, will let you know once it's completed. BTW, the tests are too much...