Using FlauBert with ParlAI

[Cyril-Joubert]

Hello,
I need to use FlauBert (or other french pretrained models) which is available via Hugging Face's Transformer, in order to create a french dialogue system but I didn't find any way to use it with ParlAI.
Is it possible to use FlauBert with ParlAI ? and if yes, how to do it ?

[stephenroller]

Hi there,

FlauBERT was pretrained with a MLM objective, so it would need significant adjustment to be used as a generation model

You can check out our hugging_face folder for a variety of models we have brought in using HuggingFace transformers:

https://github.com/facebookresearch/ParlAI/tree/master/parlai/agents/hugging_face

Different models have different idiosyncrasies which makes it hard to do this in the "generic" case, but hopefully these examples can help you add a model of your choosing.

[Cyril-Joubert]

Thanks for your reply. I will take a look at it.