Add support for non-CUDA architectures at the same time Bitsandbytes is doing it

[sealad886]

Feature request

Currently, the helper/setup functions explicitly check for CUDA support:

transformers/src/transformers/quantizers/quantizer_bnb_4bit.py

Lines 60 to 63 in 8685b3c

	def validate_environment(self, *args, **kwargs):
	if not torch.cuda.is_available():
	raise RuntimeError("No GPU found. A GPU is needed for quantization.")
	if not (is_accelerate_available() and is_bitsandbytes_available()):

BNB is currently doing a project to enable support for other GPU backends:
ALPHA TESTERS WANTED

Motivation

Apple MPS support is being added for so many major players, it'd be great for the biggest one of all to support it as its dependencies do. Also would be good to not hard-code this kind of limitation so that code updates aren't necessary as dependent libraries update themselves...

Your contribution

idea done

[amyeroberts]

cc @younesbelkada @SunMarc

[younesbelkada]

Thanks ! We could instead use an environment variable to temporarly bypass this check to easy let users experiment with transformers + bnb multi-backend refactor

See also: #31098

WDYT @Titus-von-Koeller ?

[Titus-von-Koeller]

cc for visibility @pnunna93 @Xia-Weiwen @jianan-gu @matthewdouglas for others involved in the multi backend refactor

[matthewdouglas]

I think we're probably going to need some changes in accelerate as well where there's similar device checks.

Something we could consider is the addition of an 'a' suffix on the version number in the refactor branch and check against that. An environment variable sounds reasonable too.

[Titus-von-Koeller]

Will be resolved by #31098. However, Apple Silicon support is still not implemented. For now we will work on enabling use of BNB AMD and Intel backends and once the Apple Silicon implementation will be provided by the community, we might add further necessary tweaks in a separate PR to Transformers.