Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DistilRoBERTa config #1534

Open
grohit1810 opened this issue Nov 9, 2023 · 2 comments
Open

DistilRoBERTa config #1534

grohit1810 opened this issue Nov 9, 2023 · 2 comments

Comments

@grohit1810
Copy link

grohit1810 commented Nov 9, 2023

Hi,
I was trying to convert DistilRoBERTa model and got the following error
ValueError: No conversion is registered for the model configuration RobertaConfig (supported configurations are: BartConfig, BertConfig, BloomConfig, CodeGenConfig, DistilBertConfig, FalconConfig, GPT2Config, GPTBigCodeConfig, GPTJConfig, GPTNeoXConfig, LlamaConfig, M2M100Config, MBartConfig, MPTConfig, MT5Config, MarianConfig, MixFormerSequentialConfig, OPTConfig, PegasusConfig, RWConfig, T5Config, WhisperConfig, XLMRobertaConfig)
I know that DistilRoBERTa is based on RobertaConfig, do we know if RobertaConfig will be supported in later version of the project?
There is support for XLM-RoBERTa which is also based on RobertaConfig I think, is there any way where I can convert DistilRoBERTa model

@mhillebrand
Copy link

mhillebrand commented Feb 18, 2024

XLM-Roberta uses XLMRobertaConfig, but it looks nearly identical to RobertaConfig under the hood.

@mhillebrand
Copy link

mhillebrand commented Feb 18, 2024

There's a PR for this. However, I just realized that this library won't let us do classification with these encoder models. What's the point of supporting them at all? Ugh.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
@mhillebrand @grohit1810 and others