How to use arabert within Poly-encoder model???

[salama4]

i want to use poly-encoder model, so i need to use arabic (BERT,GPT,ALBERT,.......) inside it as a pretrained transformer, but i don't know what is the syntax for doing that.
could i get any help please?

poly-encoder model syntax:-
parlai train_model \ --init-model zoo:pretrained_transformers/poly_model_huge_reddit/model \ -t convai2 \ --model transformer/polyencoder --batchsize 256 --eval-batchsize 10 \ --warmup_updates 100 --lr-scheduler-patience 0 --lr-scheduler-decay 0.4 \ -lr 5e-05 --data-parallel True --history-size 20 --label-truncate 72 \ --text-truncate 360 --num-epochs 8.0 --max_train_time 200000 -veps 0.5 \ -vme 8000 --validation-metric accuracy --validation-metric-mode max \ --save-after-valid True --log_every_n_secs 20 --candidates batch --fp16 True \ --dict-tokenizer bpe --dict-lower True --optimizer adamax --output-scaling 0.06 \ --variant xlm --reduction-type mean --share-encoders False \ --learn-positional-embeddings True --n-layers 12 --n-heads 12 --ffn-size 3072 \ --attention-dropout 0.1 --relu-dropout 0.0 --dropout 0.1 --n-positions 1024 \ --embedding-size 768 --activation gelu --embeddings-scale False --n-segments 2 \ --learn-embeddings True --polyencoder-type codes --poly-n-codes 64 \ --poly-attention-type basic --dict-endtoken __start__ \ --model-file <YOUR MODEL FILE>
could i have any help please?

to make my question more deterministic , --init-model requires the path for arabert model files which contains model weights, to be used by poly-encoder as an initial start point or pretrained model to directly start fine-tuning on it

so, can i get the path for arabert model files which contains model weights???
like that

[WissamAntoun]

@salama4 Honestly I'm not sure, since I have experience in using ParlAI

[salama4]

you meant you have not experience in ParlAI, right ?
any way, thank you i appreciate you.

[salama4]

to make my question more deterministic , --init-model requires the path for arabert model files which contains model weights, to be used by poly-encoder as an initial start point or pretrained model to directly start fine-tuning on it

so, can i get the path for arabert model files which contains model weights???
like that

[WissamAntoun]

😂Yes i meant i don't** have any experience.

Maybe this library works with models trained by ParlAI. Check if it works with English BERT, if yes, then it should work with AraBERT