Torch autograd support for dequantize methods

[yaldashbz]

Feature request

Hi,

Could you please also provide support codes for autograd in the dequantization function?

Motivation

It can be helpful for future research on the available quantization methods

Your contribution

I can help to write autograd Functions for backward if you could please guide me on the available forwards for dequantize_blockwise functions

[matthewdouglas]

Hi!

We do not have the capacity to prioritize this, but contributions are welcome! The backward would likely need to be written as a CUDA function for efficiency.

The dequantize_blockwise function is implemented in functional.py.

For more details on the dynamic data type, see (8-Bit Approximations for Parallelism in Deep Learning)[https://arxiv.org/abs/1511.04561].