ImportError when importing FastLanguageModel from unsloth

[MurphyJUAN]

After updating unsloth to the latest version, I encountered an ImportError when attempting to import FastLanguageModel from the package. The error message is as follows:

ImportError: cannot import name 'Replicate' from 'torch.distributed.tensor'

This error occurs when executing the following code:

from unsloth import FastLanguageModel

Environment:
Python version: 3.10
PyTorch version: 2.4.1+cu121
unsloth version: unsloth-2024.11.7
CUDA version: 12.1
OS: Linux Ubuntu 20.04
Environment: Conda environment

Could you please provide guidance on how to resolve this issue or if there’s a specific version of torch that unsloth is currently compatible with?

Thank you!

[zhenduow]

Hi,

I had a similar error

ImportError: cannot import name 'patch_compiled_autograd' from 'unsloth_zoo.patching_utils'

when I ran the notebook , cell 2.

Thank you!

[danielhanchen]

@zhenduow You can try updating Unsloth-zoo via pip install --upgrade unsloth-zoo

[danielhanchen]

@MurphyJUAN That is a very weird error hmmm - maybe best to uninstall Unsloth entirely and reinstall it

[lukalafaye]

Same issue here, running cuda 12.5 on ubuntu if that can help.

Error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[22], line 2
      1 import torch
----> 2 import unsloth

File [~/miniconda/lib/python3.9/site-packages/unsloth/__init__.py:174](https://caiehack-yolo22.hf.space/lab/tree/~/miniconda/lib/python3.9/site-packages/unsloth/__init__.py#line=173)
    171     raise ImportError("Unsloth: Please install unsloth_zoo via `pip install unsloth-zoo`")
    172 pass
--> 174 from .models import *
    175 from .save import *
    176 from .chat_templates import *

File [~/miniconda/lib/python3.9/site-packages/unsloth/models/__init__.py:15](https://caiehack-yolo22.hf.space/lab/tree/~/miniconda/lib/python3.9/site-packages/unsloth/models/__init__.py#line=14)
      1 # Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.
      2 #
      3 # Licensed under the Apache License, Version 2.0 (the "License");
   (...)
     12 # See the License for the specific language governing permissions and
     13 # limitations under the License.
---> 15 from .loader  import FastLanguageModel, FastVisionModel
     16 from .llama   import FastLlamaModel
     17 from .mistral import FastMistralModel

File [~/miniconda/lib/python3.9/site-packages/unsloth/models/loader.py:15](https://caiehack-yolo22.hf.space/lab/tree/~/miniconda/lib/python3.9/site-packages/unsloth/models/loader.py#line=14)
      1 # Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.
      2 #
      3 # Licensed under the Apache License, Version 2.0 (the "License");
   (...)
     12 # See the License for the specific language governing permissions and
     13 # limitations under the License.
---> 15 from ._utils import is_bfloat16_supported, HAS_FLASH_ATTENTION, HAS_FLASH_ATTENTION_SOFTCAPPING
     16 from .llama   import FastLlamaModel, logger
     17 from .mistral import FastMistralModel

File [~/miniconda/lib/python3.9/site-packages/unsloth/models/_utils.py:79](https://caiehack-yolo22.hf.space/lab/tree/~/miniconda/lib/python3.9/site-packages/unsloth/models/_utils.py#line=78)
     74 from packaging.version import Version
     76 from unsloth_zoo.tokenizer_utils import (
     77     patch_tokenizer as _patch_tokenizer,
     78 )
---> 79 from unsloth_zoo.patching_utils import (
     80     patch_compiling_bitsandbytes,
     81     patch_layernorm,
     82     patch_torch_compile,
     83     patch_model_and_tokenizer,
     84     patch_compiled_autograd,
     85 )
     86 from unsloth_zoo.gradient_checkpointing import (
     87     Unsloth_Offloaded_Gradient_Checkpointer,
     88     unsloth_offloaded_gradient_checkpoint,
   (...)
     99     create_gradient_checkpointing_buffer,
    100 )
    101 from unsloth_zoo.loss_utils import (
    102     HAS_CUT_CROSS_ENTROPY,
    103     fused_linear_cross_entropy,
    104 )

ImportError: cannot import name 'patch_compiled_autograd' from 'unsloth_zoo.patching_utils' ([/home/user/miniconda/lib/python3.9/site-packages/unsloth_zoo/patching_utils.py](https://caiehack-yolo22.hf.space/home/user/miniconda/lib/python3.9/site-packages/unsloth_zoo/patching_utils.py))