Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

训练RegressionTask的一些问题 #3260

Open
NicholasKX opened this issue Feb 25, 2025 · 1 comment
Open

训练RegressionTask的一些问题 #3260

NicholasKX opened this issue Feb 25, 2025 · 1 comment

Comments

@NicholasKX
Copy link

NicholasKX commented Feb 25, 2025

用ms-swift框架进行回归任务的训练,具体训练脚本如下:

swift sft     
--custom_register_path ../ms-swift/scripts/sft/custom/custom_dataset.py     
--model ../pretrain_model/Qwen2.5-0.5B-Instruct     
--dataset ../regression_normalized.json     
--train_type lora     
--lora_rank 8     
--lora_alpha 32    
--target_modules all-linear     
--logging_steps 10     
--torch_dtype bfloat16     
--learning_rate 2e-4     
--output_dir $output_dir     
--lazy_tokenize true     
--max_length 8192     
--save_steps 1000     
--eval_steps 1000     
--per_device_train_batch_size 1     
--per_device_eval_batch_size 1     
--gradient_checkpointing true     
--num_train_epochs 3     
--dataloader_num_workers 4     
--save_only_model true     
--warmup_ratio 0.05     
--gradient_accumulation_steps 4     
--deepspeed zero3     
--use_chat_template false      
--num_labels 1     
--metric regression

1.num_labels=1,会把这个任务识别为seq_cls任务,会有影响吗。
2.Config里会有一个label2id,会把label全部当作类别0吗。
Image
3.因为识别成seq_cls任务,数据集的label默认转成torch.long类型(修改为torch.float32解决)。
4.自定义的 regression metric 没有打印。
5.当label!=None,框架默认会计算compute_acc,不适用于回归任务(注释了该行代码解决)。

@NicholasKX
Copy link
Author

我想知道利用ms-swift进行regression task的正确方法是什么呢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant