How to choose batch size?

[GitFTuan]

Dear Developer. I am currently trying to learn and replicate the "Mutate Everything" project. However, I noticed a discrepancy: in the article, the batch size is described as 3, while the GitHub README uses a batch size of 1. Additionally, when I set the batch size to a value greater than 1, I encounter an error in data.py at the line ret['one_letter_idx'] = one_letter_idx. This issue might be caused by the protein sequences having varying lengths, which are not aligned, leading to the error. Could you please suggest a solution for this problem?Thank you.

[jozhang97]

Hi,

Thanks for your interest. Below is the command copy pasted from the README. The per-GPU batch size is 1 but we are using 3 GPUs, thus the effective batch size is 3. We use openfold which only permits a batch size of 1.

Solution is to use 3 gpus. If only one GPU is available try batch size of 1, though I'm unsure you will get the same solution.

torchrun **--nproc_per_node=3** main_train.py \
    **--batch_size 1** \
    --finetune_backbone models/finetuning_ptm_2.pt \
    --data_path data/cdna/mutations/cdna_train.csv \
    --eval_data_paths data/s669/mutations/s669.csv \
    --dist_eval \
    --backbone af \
    --lambda_double 0. --lambda_single 1. \
    --eval_period 999 --epochs 20 --warmup_epochs 2 --save_period 20 \
    --disable_wandb \
    --output_dir logs/af_single