TypeError: can only concatenate str (not "list") to str

[Felixvillas]

Thanks for outstanding work!

However, I got the following error when I run it

• Error Message

TypeError: can only concatenate str (not "list") to str
[rank0]: Traceback (most recent call last):
[rank0]:   File "/nfs_global/S/tianzikang/rocky/projects/spatial_intelligence/open-r1-multimodal/src/open_r1/grpo.py", line 185, in <module>
[rank0]:     main(script_args, training_args, model_args)
[rank0]:   File "/nfs_global/S/tianzikang/rocky/projects/spatial_intelligence/open-r1-multimodal/src/open_r1/grpo.py", line 174, in main
[rank0]:     trainer.train()
[rank0]:   File "/lustre/S/tianzikang/rocky/miniconda3/envs/omnigibson/lib/python3.10/site-packages/transformers/trainer.py", line 2185, in train
[rank0]:     return inner_training_loop(
[rank0]:   File "/lustre/S/tianzikang/rocky/miniconda3/envs/omnigibson/lib/python3.10/site-packages/transformers/trainer.py", line 2491, in _inner_training_loop
[rank0]:     tr_loss_step = self.training_step(model, inputs, num_items_in_batch)
[rank0]:   File "/lustre/S/tianzikang/rocky/miniconda3/envs/omnigibson/lib/python3.10/site-packages/transformers/trainer.py", line 3609, in training_step
[rank0]:     loss = self.compute_loss(model, inputs, num_items_in_batch=num_items_in_batch)
[rank0]:   File "/nfs_global/S/tianzikang/rocky/projects/spatial_intelligence/open-r1-multimodal/./src/open_r1/trainer/grpo_trainer.py", line 345, in compute_loss
[rank0]:     prompts_text = [maybe_apply_chat_template(example, self.processing_class)["prompt"] for example in inputs]
[rank0]:   File "/nfs_global/S/tianzikang/rocky/projects/spatial_intelligence/open-r1-multimodal/./src/open_r1/trainer/grpo_trainer.py", line 345, in <listcomp>
[rank0]:     prompts_text = [maybe_apply_chat_template(example, self.processing_class)["prompt"] for example in inputs]
[rank0]:   File "/lustre/S/tianzikang/rocky/miniconda3/envs/omnigibson/lib/python3.10/site-packages/trl/data_utils.py", line 217, in maybe_apply_chat_template
[rank0]:     return apply_chat_template(example, tokenizer, tools)
[rank0]:   File "/lustre/S/tianzikang/rocky/miniconda3/envs/omnigibson/lib/python3.10/site-packages/trl/data_utils.py", line 102, in apply_chat_template
[rank0]:     prompt = tokenizer.apply_chat_template(
[rank0]:   File "/lustre/S/tianzikang/rocky/miniconda3/envs/omnigibson/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1695, in apply_chat_template
[rank0]:     rendered_chat = compiled_template.render(
[rank0]:   File "/lustre/S/tianzikang/rocky/miniconda3/envs/omnigibson/lib/python3.10/site-packages/jinja2/environment.py", line 1304, in render
[rank0]:     self.environment.handle_exception()
[rank0]:   File "/lustre/S/tianzikang/rocky/miniconda3/envs/omnigibson/lib/python3.10/site-packages/jinja2/environment.py", line 939, in handle_exception
[rank0]:     raise rewrite_traceback_stack(source=source)
[rank0]:   File "<template>", line 23, in top-level template code
[rank0]: TypeError: can only concatenate str (not "list") to str

• Command

export WANDB_MODE=offline
export CUDA_LAUNCH_BLOCKING=1

ARNOLD_WORKER_GPU=4
ARNOLD_WORKER_NUM=1
ARNOLD_ID=0
METIS_WORKER_0_HOST=127.0.0.1
port_in_cmd=12345

torchrun --nproc_per_node="${ARNOLD_WORKER_GPU}" \
    --nnodes="${ARNOLD_WORKER_NUM}" \
    --node_rank="${ARNOLD_ID}" \
    --master_addr="${METIS_WORKER_0_HOST}" \
    --master_port="${port_in_cmd}" \
    src/open_r1/grpo.py \
    --deepspeed local_scripts/zero3.json \
    --output_dir ./checkpoints/Qwen2.5-VL-2B-GRPO-8k \
    --model_name_or_path /nfs_global/S/tianzikang/LLMs/Qwen/Qwen-Qwen2.5-VL-7B-Instruct/ \
    --dataset_name lmms-lab/multimodal-open-r1-8k-verified \
    --max_prompt_length 8192 \
    --per_device_train_batch_size 1 \
    --gradient_accumulation_steps 1 \
    --logging_steps 1 \
    --bf16 \
    --report_to wandb \
    --gradient_checkpointing true \
    --attn_implementation flash_attention_2 \
    --max_pixels 2359296 \
    --save_total_limit 8 \
    --num_train_epochs 1 \
    --run_name Qwen2.5-VL-2B-GRPO-8k

The error is from the format of inputs in error line

the format of example in inputs is something like:

example = {
    "image": "123",
    "problem": "Based on the image, how many times larger is the area of Indonesia compared to Thailand?",
    "solution": "The area of Indonesia is 1.9 times larger than the area of Thailand.",
    "prompt": [
        {
            "content": [
                {
                    "type": "image",
                    "text": None
                },
                {
                    "type": "text",
                    "text": "Based on the image, how many times larger is the area of Indonesia compared to Thailand?  Output the thinking process in <think> </think> and final answer (number) in <answer> </answer> tags."
                }
            ],
            "role": "user"
        }
    ]
}

But maybe_apply_chat_template cannot handle the prompt with a list format content. The correct format of example should be:

example = {
    "image": "123",
    "problem": "Based on the image, how many times larger is the area of Indonesia compared to Thailand?",
    "solution": "The area of Indonesia is 1.9 times larger than the area of Thailand.",
    "prompt": [
        {
            "content": "Based on the image, how many times larger is the area of Indonesia compared to Thailand?  Output the thinking process in <think> </think> and final answer (number) in <answer> </answer> tags.",
            "role": "user"
        }
    ]
}

Is this a bug in the repository code, or a problem with my reproduction?

[Felixvillas]

I found that the bug was resolved after I replaced the base model with Qwen2-VL. It seems the issue was related to my use of Qwen2.5-VL. I speculate that the code was not fully compatible with Qwen 2.5-VL.