Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Weights from pretrained model not used #12

Open
AlanLinZhizhou opened this issue Sep 14, 2021 · 0 comments
Open

Weights from pretrained model not used #12

AlanLinZhizhou opened this issue Sep 14, 2021 · 0 comments

Comments

@AlanLinZhizhou
Copy link

Weights from pretrained model not used in BertForSequenceClassification: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.phrase_predictions.transform.dense.weight', 'cls.phrase_predictions.transform.dense.bias', 'cls.phrase_predictions.transform.LayerNorm.weight', 'cls.phrase_predictions.transform.LayerNorm.bias', 'cls.phrase_predictions.decoder.weight', 'cls.phrase_predictions.decoder.bias']
Are these weights essential during finetuning? Did you come across the same problem when implementing sentibert? If so, how did you solve it? The bin file and json file are downloaed from your link.
"这些参数没有加载成功是否会影响整体的准确率呀,您在实现的时候这些参数加载成功了么,模型文件和json文件是从您提供的链接下载的,应该没有问题"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant