Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

quant: Quantization routines only support CPU processing #14

Open
cimes-isi opened this issue Sep 1, 2022 · 0 comments
Open

quant: Quantization routines only support CPU processing #14

cimes-isi opened this issue Sep 1, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@cimes-isi
Copy link
Collaborator

Quantization routines use unsigned 32 bit integers to pack quantized data. Since PyTorch doesn't support this datatype, we're use numpy arrays before casting back to a Tensor, which currently limits us to running on the CPU. This restriction means we are forced to move data to/from the CPU when sending/receiving data between stages, rather than a more efficient direct network transfer to/from GPUs and NICs.

@cimes-isi cimes-isi added the enhancement New feature or request label Sep 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant