新人请教

[lingaoan]

Reminder

• I have read the README and searched the existing issues.

System Info

I7-27000,32G内存,RTX 3060显卡 12G专用+15G共享，系统win11
目前问题：
（1）训练后测试效果，和未训练一样。
（2）通过LLAMA BOARD加载模型非常慢（甚至轮数为10时，根本加载不了模型，直接崩溃），如果检查点路径为空时，非常快

1、训练模型llama-3-chinese-8b-instruct-v3
2、数据集是自己创建的json文件，内容如下
[
{
"instruction": "你是谁？",
"input": "",
"output": "您好，我是H-lin重新训练过的智慧小助手，有什么可以帮到您吗？"
},{
"instruction": "你是谁？",
"input": "",
"output": "您好，我是H-lin重新训练过的智慧小助手，很高兴认识您，有什么可以帮到您吗？"
}
]
3、学习率尝试过5e-5和1e-4,轮数：3或者10，计算类型：bf16
4、训练很快
{'train_runtime': 113.0455, 'train_samples_per_second': 0.265, 'train_steps_per_second': 0.088, 'train_loss': 0.6034916400909424, 'epoch': 10.0, 'num_input_tokens_seen': 2984}
5、加载模型非常慢，甚至有时无法加载，通过llamafactory-cli webui启动，因为加载失败直接退出服务，也未报错

请教大神，我的问题在哪儿，我就想尝试一次训练成功，让它接受我命令，请抽空指导一下

Reproduction

新手操作，全用的配置界面操作

Expected behavior

No response

Others

No response

[Nihaokai]

lora训练吗？是不是没加载lora。检查一下是不是加载到CPU上了

[lingaoan2024]

@Nihaokai 不是，确实是在GPU上，可能机器性能还是太差了

[Nihaokai]

lora权重有没有合并？

[lingaoan2024]

@Nihaokai 不知道要如何合并，我只选择了当前训练的检查点，然后就直接导出，看了导出大小，以为自动合并了

因为选择加载模型要等很久，所以我直接选择导出，不知道是否是我操作有误。目前还发现训练过的模型丢失了部分能力，不知道是不是这个原因

[Nihaokai]

@Nihaokai 不知道要如何合并，我只选择了当前训练的检查点，然后就直接导出，看了导出大小，以为自动合并了

因为选择加载模型要等很久，所以我直接选择导出，不知道是否是我操作有误。目前还发现训练过的模型丢失了部分能力，不知道是不是这个原因

训练集太小了，这样容易过拟合

[lingaoan2024]

@Nihaokai 谢谢，我之前一直是用的webui可视化界面操作，我尝试一下脚本，非常感谢指导

[linkeusen]

请问为什么会出现如下报错啊，5555
root@dsw-370537-84565fc77c-lttkb:/mnt/workspace/LLaMA-Factory# llamafactory-cli webui
2024-06-21 10:56:20.984131: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-06-21 10:56:22.025011: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Traceback (most recent call last):
File "/opt/conda/bin/llamafactory-cli", line 5, in
from llamafactory.cli import main
File "/mnt/workspace/LLaMA-Factory/src/llamafactory/init.py", line 17, in
from .cli import VERSION
File "/mnt/workspace/LLaMA-Factory/src/llamafactory/cli.py", line 21, in
from . import launcher
File "/mnt/workspace/LLaMA-Factory/src/llamafactory/launcher.py", line 15, in
from llamafactory.train.tuner import run_exp
File "/mnt/workspace/LLaMA-Factory/src/llamafactory/train/tuner.py", line 20, in
from ..data import get_template_and_fix_tokenizer
File "/mnt/workspace/LLaMA-Factory/src/llamafactory/data/init.py", line 16, in
from .data_utils import Role, split_dataset
File "/mnt/workspace/LLaMA-Factory/src/llamafactory/data/data_utils.py", line 20, in
from ..extras.logging import get_logger
File "/mnt/workspace/LLaMA-Factory/src/llamafactory/extras/logging.py", line 20, in
from .constants import RUNNING_LOG
File "/mnt/workspace/LLaMA-Factory/src/llamafactory/extras/constants.py", line 19, in
from peft.utils import SAFETENSORS_WEIGHTS_NAME as SAFE_ADAPTER_WEIGHTS_NAME
File "/opt/conda/lib/python3.10/site-packages/peft/init.py", line 22, in
from .auto import (
File "/opt/conda/lib/python3.10/site-packages/peft/auto.py", line 32, in
from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING
File "/opt/conda/lib/python3.10/site-packages/peft/mapping.py", line 22, in
from .mixed_model import PeftMixedModel
File "/opt/conda/lib/python3.10/site-packages/peft/mixed_model.py", line 26, in
from peft.tuners.mixed import COMPATIBLE_TUNER_TYPES
File "/opt/conda/lib/python3.10/site-packages/peft/tuners/init.py", line 21, in
from .lora import LoraConfig, LoraModel, LoftQConfig
File "/opt/conda/lib/python3.10/site-packages/peft/tuners/lora/init.py", line 20, in
from .model import LoraModel
File "/opt/conda/lib/python3.10/site-packages/peft/tuners/lora/model.py", line 50, in
from .awq import dispatch_awq
File "/opt/conda/lib/python3.10/site-packages/peft/tuners/lora/awq.py", line 26, in
from awq.modules.linear import WQLinear_GEMM
File "/opt/conda/lib/python3.10/site-packages/awq/init.py", line 2, in
from awq.models.auto import AutoAWQForCausalLM
File "/opt/conda/lib/python3.10/site-packages/awq/models/init.py", line 1, in
from .mpt import MptAWQForCausalLM
File "/opt/conda/lib/python3.10/site-packages/awq/models/mpt.py", line 1, in
from .base import BaseAWQForCausalLM
File "/opt/conda/lib/python3.10/site-packages/awq/models/base.py", line 11, in
from awq.quantize.quantizer import AwqQuantizer
File "/opt/conda/lib/python3.10/site-packages/awq/quantize/quantizer.py", line 11, in
from awq.modules.linear import WQLinear_GEMM, WQLinear_GEMV
File "/opt/conda/lib/python3.10/site-packages/awq/modules/linear.py", line 4, in
import awq_inference_engine # with CUDA kernels
ImportError: libcudart.so.11.0: cannot open shared object file: No such file or directory

[Nihaokai]

@linkeusen 报错显示你的CUDA没有安装，或者版本过低。建议使用conda虚拟环境，在官网获取安装pytorch的方法，会一同在虚拟环境中安装CUDA。

[linkeusen]

@linkeusen 报错显示你的CUDA没有安装，或者版本过低。建议使用conda虚拟环境，在官网获取安装pytorch的方法，会一同在虚拟环境中安装CUDA。

但是nvidia-smi显示的cuda为12.1

[Nihaokai]

@linkeusen nvcc -V显示的是真正版本，nvidia-smi上的是最高支持的版本。

[linkeusen]

@linkeusen nvcc -V显示的是真正版本，nvidia-smi上的是最高支持的版本。
请问这样对吗
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Mon_Apr__3_17:16:06_PDT_2023
Cuda compilation tools, release 12.1, V12.1.105
Build cuda_12.1.r12.1/compiler.32688072_0

[Nihaokai]

@linkeusen 是的，关于这个原因，可能是CUDA相关包的问题，我还是建议在conda环境中安装

[linkeusen]

@linkeusen 是的，关于这个原因，可能是CUDA相关包的问题，我还是建议在conda环境中安装

我确实是在conda环境中装的，谢谢您的帮助，我觉得会不会是awq的版本不对，我接下来尝试一下

[muliu]

@linkeusen 训练后测试效果，和未训练一样。

我也遇到这个问题了。请问你排查结果是什么，望分享下。