Details contrastive training

[boscoj2008]

During contrastive pre-training, do you also set batch_size to 1024? or is it the default 8? Thanks

[rpeeters85]

The batch size to be used during pre-training can be set manually (this is the batch size you define in the command line). The value used in the paper is 1024. On the other hand, the batch size for the fine-tuning step is set to 64, you can find this in the .sh script of the corresponding fine-tuning e.g. src/contrastive/lspc/run_finetune_siamese_frozen_roberta.sh and change it there if you want to change the batch size used for fine-tuning.