Unsloth Crashes on Import When GPU Is Missing

[Axel-At-Apollo]

Hi! Is there a way to import unsloth without it throwing an error when a GPU is not available? This complicates unit testing setups since some of my tests will be run on a CPU-only machine. These tests subsequently fail, as they rely on functionality from files that import the library. Is there a workaround for this? If not, I suggest modifying unsloth so that it only raises an error when a GPU-dependent method is called, rather than at the import stage.

Current behaviour:

root@ef936059842d:/app# python -c "import unsloth"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.12/dist-packages/unsloth/__init__.py", line 120, in <module>
    major_version, minor_version = torch.cuda.get_device_capability()
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/cuda/__init__.py", line 509, in get_device_capability
    prop = get_device_properties(device)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/cuda/__init__.py", line 523, in get_device_properties
    _lazy_init()  # will define _get_device_properties
    ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/torch/cuda/__init__.py", line 319, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
root@ef936059842d:/app# pip show unsloth
Name: unsloth
Version: 2025.1.5
Summary: 2-5X faster LLM finetuning
Home-page: http://www.unsloth.ai
...

Proposed Behaviour:

root@ef936059842d:/app# python -c "import unsloth"
root@ef936059842d:/app# 

[danielhanchen]

Would a try except work? Unsloth for now requires a GPU