[Bug] Out of memory running vsrpp, full build and env included

[rlewkowicz]

Prerequisite

• I have searched Issues and Discussions but cannot get the expected help.
• I have read the FAQ documentation but cannot get the expected help.
• The bug has not been fixed in the latest version (main) or latest version (0.x).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmagic

Environment

I can do you infinity better.
main...rlewkowicz:mmagic:main

I hope someone will take a look. I cannot give you my env much more aside from sending you my gpu.

python3 data/demo/mmagic_inference_demo.py --model-name basicvsr_pp --video data/bgs.mp4 --result-out-dir data/doot.mp4 --window-size 3

The mp4s are just test data

Reproduces the problem - code sample

python3 data/demo/mmagic_inference_demo.py --model-name basicvsr_pp --video data/bgs.mp4 --result-out-dir data/doot.mp4 --window-size 3

Reproduces the problem - command or script

python3 data/demo/mmagic_inference_demo.py --model-name basicvsr_pp --video data/bgs.mp4 --result-out-dir data/doot.mp4 --window-size 3

Reproduces the problem - error message

orch.OutOfMemoryError: CUDA out of memory. Tried to allocate 19.05 GiB. GPU 0 has a total capacity of 23.65 GiB of which 3.66 GiB is free. Process 7027 has 19.33 GiB memory in use. Of the allocated memory 19.07 GiB is allocated by PyTorch, and 3.48 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.01             Driver Version: 535.183.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3090 Ti     Off | 00000000:01:00.0  On |                  Off |
| 30%   28C    P8              20W / 450W |    575MiB / 24564MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
+---------------------------------------------------------------------------------------+

The cuda version in smi is unexpected, but everything should just be executing from the container.

Additional information

The build system is rough. This is the closest ive gotten this to being functional.