Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

已解决:训练之后如何保存模型问题 #565

Open
CZ581 opened this issue Jan 6, 2024 · 3 comments
Open

已解决:训练之后如何保存模型问题 #565

CZ581 opened this issue Jan 6, 2024 · 3 comments

Comments

@CZ581
Copy link

CZ581 commented Jan 6, 2024

CJ模型,前4步尝试多次后正常运行,训练大概3h左右在TensorBoard调了大概满意的声音,Google Drive存副本之后就停止运行了。显示没有可用的GPU所以切换了一下,结果在第四步微调之后url无法显示,运行第五步也无法下载,请问是什么原因?还可以补救吗?
屏幕截图 2024-01-06 195553

屏幕截图 2024-01-06 195616
@CZ581 CZ581 changed the title 训练之后如何保存模型问题 已解决:训练之后如何保存模型问题 Jan 7, 2024
@CZ581
Copy link
Author

CZ581 commented Jan 7, 2024

已解决:Google的colab训练有时限,T4GPU一段时间会被禁用,这个时候不能切换环境否则会丢失数据,从新训练,大概1-2h左右,结束后再运行step5,或者等待直到可以重连T4环境即可。

@softbeast
Copy link

你好,你没有遇到step1环境搭建时候报错么?

@wodhei
Copy link

wodhei commented Jan 20, 2024

你好,你遇到这个问题吗?我看好多人都遇到了,我也是。
#571

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants