Does VideoDecoder supports customized filter graph descriptors?

[YANG-H]

🚀 The feature

For example, I'd like to resize the video frames before actually decoding them. The best way might be using a filter graph string in ffmpeg. What is the best practice for torchcodec? Does torchcodec supports customized filter graph strings in python?

Motivation, pitch

No response

[NicolasHug]

Hi @YANG-H , we don't support custom filters or resizing at the moment. We may support those in the future, but we'll want to be careful because models tend to be sensitive to resizing algorithm, i.e. it's very important for the resizing strat to be consistent between training and inference, and there many different resizing strategies: bilinear, bicubic, with/without antialiasing, on float/uint8, etc.

We'll want to make sure the resizing exposed by ffmpeg is/can be consistent with the ones we have in pytorch.

For now, the recommended solution is to rely on torchvision's transforms, which can natively handle NHWC videos.