ImportError: cannot import name 'LLaMAForCausalLM' from 'transformers'

[SeekPoint]

(gh_alpaca-lora-for-huggingface) ub2004@ub2004-B85M-A0:~/llm_dev/alpaca-lora-for-huggingface$ accelerate launch --config_file peft_config.yaml finetune.py
[09:06:17] WARNING The following values were not passed to accelerate launch and had defaults used instead: launch.py:887
--dynamo_backend was set to a value of 'no'
To avoid this warning pass in values for each of the problematic parameters or run accelerate config.
Traceback (most recent call last):
File "finetune.py", line 8, in
from transformers import AutoTokenizer, AutoConfig, LLaMAForCausalLM, LLaMATokenizer
ImportError: cannot import name 'LLaMAForCausalLM' from 'transformers' (/home/ub2004/.local/lib/python3.8/site-packages/transformers/init.py)
[09:06:22] ERROR failed (exitcode: 1) local_rank: 0 (pid: 17154) of binary: /usr/bin/python3 api.py:673
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/ub2004/.local/bin/accelerate:8 in │
│ │
│ 5 from accelerate.commands.accelerate_cli import main │
│ 6 if name == 'main': │
│ 7 │ sys.argv[0] = re.sub(r'(-script.pyw|.exe)?$', '', sys.argv[0]) │
│ ❱ 8 │ sys.exit(main()) │
│ 9 │
│ │
│ /home/ub2004/.local/lib/python3.8/site-packages/accelerate/commands/accelerate_cli.py:45 in main │
│ │
│ 42 │ │ exit(1) │
│ 43 │ │
│ 44 │ # Run │
│ ❱ 45 │ args.func(args) │
│ 46 │
│ 47 │
│ 48 if name == "main": │
│ │
│ /home/ub2004/.local/lib/python3.8/site-packages/accelerate/commands/launch.py:900 in │
│ launch_command │
│ │
│ 897 │ │ if mp_from_config_flag: │
│ 898 │ │ │ args.deepspeed_fields_from_accelerate_config.append("mixed_precision") │
│ 899 │ │ args.deepspeed_fields_from_accelerate_config = ",".join(args.deepspeed_fields_fr │
│ ❱ 900 │ │ deepspeed_launcher(args) │
│ 901 │ elif args.use_fsdp and not args.cpu: │
│ 902 │ │ multi_gpu_launcher(args) │
│ 903 │ elif args.use_megatron_lm and not args.cpu: │
│ │
│ /home/ub2004/.local/lib/python3.8/site-packages/accelerate/commands/launch.py:643 in │
│ deepspeed_launcher │
│ │
│ 640 │ │ ) │
│ 641 │ │ with patch_environment(**current_env): │
│ 642 │ │ │ try: │
│ ❱ 643 │ │ │ │ distrib_run.run(args) │
│ 644 │ │ │ except Exception: │
│ 645 │ │ │ │ if is_rich_available() and debug: │
│ 646 │ │ │ │ │ console = get_console() │
│ │
│ /home/ub2004/.local/lib/python3.8/site-packages/torch/distributed/run.py:753 in run │
│ │
│ 750 │ │ ) │
│ 751 │ │
│ 752 │ config, cmd, cmd_args = config_from_args(args) │
│ ❱ 753 │ elastic_launch( │
│ 754 │ │ config=config, │
│ 755 │ │ entrypoint=cmd, │
│ 756 │ )(*cmd_args) │
│ │
│ /home/ub2004/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py:132 in │
│ call │
│ │
│ 129 │ │ self._entrypoint = entrypoint │
│ 130 │ │
│ 131 │ def call(self, *args): │
│ ❱ 132 │ │ return launch_agent(self._config, self._entrypoint, list(args)) │
│ 133 │
│ 134 │
│ 135 def _get_entrypoint_name( │
│ │
│ /home/ub2004/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py:246 in │
│ launch_agent │
│ │
│ 243 │ │ │ # if the error files for the failed children exist │
│ 244 │ │ │ # @record will copy the first error (root cause) │
│ 245 │ │ │ # to the error file of the launcher process. │
│ ❱ 246 │ │ │ raise ChildFailedError( │
│ 247 │ │ │ │ name=entrypoint_name, │
│ 248 │ │ │ │ failures=result.failures, │
│ 249 │ │ │ ) │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ChildFailedError:

finetune.py FAILED

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2023-05-10_09:06:22
host : ub2004-B85M-A0
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 17154)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

(gh_alpaca-lora-for-huggingface) ub2004@ub2004-B85M-A0:~/llm_dev/alpaca-lora-for-huggingface$

[naem1023]

Hi, I think the version of transforemrs is the reason.

Main reason

• Current version's llama implementaion name: LlamaForCausalLM
• This repository's llama implementation name: LLaMAForCausalLM

Conclusion

I'm sorry to bother you, but there was no official Tokenizer and CausalLM for LLAMA when I worked on this. Therefore, I used custom version of transforemers for this. I will update the work soon.

Thank you very much for your interest!!