Utilizing NVJPEG to avoid CPU bottleneck during encoding and decoding

[darkcristle]

I noticed that my GPU was at a low Utilization often and that CPU spikes occurred sometimes. I believe that this is because neither png nor jpeg can be decoded or encoded on the GPU in most cases. Nvidia however has released a library called "NVJPEG" which does support this. It also appears that recent versions PyTorch also support this.

Is there any possibility that something like this could be used to overcome the bottleneck?

[TechnoMasterBoy]

Agreed, something must be done with the CPU bottleneck.
I have a shitty GTX 1080 and it's actually getting bottlenecked by my Ryzen 9 5900x which is just insane.
Like what? A GPU from 2016 is getting bottlenecked by a CPU made in 2020?
I thought upscaling was a GPU heavy process.
Which makes me think that upgrading to RTX 40xx when it comes out actually won't give any meaningful improvement in speed unless I upscale higher than 1440p, but I usually don't.
My GPU is currently averaging like 20% according to GPU-Z but CPU is sitting at 100% most of the time with some occasional dips to 80%, but I'd say average 90-95% CPU usage while upscaling from 628x480 to 2512x1920.
This is with the realesr-animevideov3 model at 4x and with 6 tasks running because that utilizes the most possible whereas just using 1 task takes more than double the upscale time.
Just look at this downtime where my GPU is doing NOTHING:

And then look at my CPU sweating its ass of:

[Aleonz00]

I was wondering what was happening... I noticed this when I was trying to upscale png to png. It would just completely stop after each image for a couple seconds. Horrible when trying to upscale tens of thousands of images.

png to jpg is extremely more consistent, never stopped once and stayed above 90% GPU. Topaz VEAI doesn't have this issue with png so it seems to be a bug with Real-ESRGAN.

[TechnoMasterBoy]

I was wondering what was happening... I noticed this when I was trying to upscale png to png. It would just completely stop after each image for a couple seconds. Horrible when trying to upscale tens of thousands of images.

png to jpg is extremely more consistent, never stopped once and stayed above 90% GPU. Topaz VEAI doesn't have this issue with png so it seems to be a bug with Real-ESRGAN.

One workaround I found was to set it to run several tasks at the same time. I usually have 6 tasks running as having just 1 task running makes it take almost 3 times longer due to the downtime.
Maybe that will help you a bit.