a logic error in _preprocess function of Qwen2VLImageProcessor Class

[InsaneGe]

System Info

in the _preprocess function of Qwen2VLImageProcessor Class(

transformers/src/transformers/models/qwen2_vl/image_processing_qwen2_vl.py

Line 278 in 348f328

repeats = np.repeat(patches[-1][np.newaxis], temporal_patch_size - 1, axis=0)

), it writes down as follows:

if patches.shape[0] % temporal_patch_size != 0:
    repeats = np.repeat(patches[-1][np.newaxis], temporal_patch_size - 1, axis=0)
    patches = np.concatenate([patches, repeats], axis=0)
grid_t = patches.shape[0] // temporal_patch_size

it should repeat temporal_patch_size - (patches.shape[0] % temporal_patch_size) instead of temporal_patch_size - 1, to make sure patches.shape[0] can be divisible by temporal_patch_size.

if patches.shape[0] % temporal_patch_size != 0:
    repeats = np.repeat(patches[-1][np.newaxis], temporal_patch_size- (patches.shape[0] % temporal_patch_size), axis=0)
    patches = np.concatenate([patches, repeats], axis=0)

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

nothing

Expected behavior

nothing

[zucchini-nlp]

@InsaneGe hey, I agree it is not very generalizable and seems that the prev PR was merged to fix only certain cases for num_frames. Feel free to make a PR :)

[LuHC409]

Hi, I've updated the code as suggested:

if patches.shape[0] % temporal_patch_size != 0:
    pad_len = temporal_patch_size - (patches.shape[0] % temporal_patch_size)
    repeats = np.repeat(patches[-1][np.newaxis], pad_len, axis=0)
    patches = np.concatenate([patches, repeats], axis=0)

I tested the updated code locally and confirmed that it passes all tests. The issue with incorrect padding is now resolved. Thanks!

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[zucchini-nlp]

Fixed by #38076