Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unfaithful to original model? #29

Open
UpscaleAnon opened this issue Aug 28, 2024 · 3 comments
Open

Unfaithful to original model? #29

UpscaleAnon opened this issue Aug 28, 2024 · 3 comments

Comments

@UpscaleAnon
Copy link

I've converted ESRGAN's AnimeVideoV3 to ONNX to use in VideoJaNai, however the output video is unfaithful compared to using the Windows build that uses NCNN aswell as the converted ONNX model if used in ChaiNNer.
VideoJaNai makes the models lose some detail that ChaiNNer would have otherwise kept.
I've tried everything I can think of. Switching from TensorRT to NCNN, Altering the TensorRT Engine parameters, choosing not to resize the video before or after upscaling, resizing the video manually beforehand with ffmpeg, encoding losslessly and using FP16 converted model instead of FP32.

Regular ESRGAN:
image

VideoJaNai:
image

Notice how at the top left, the black lines in the hair is blurred away in the VideoJaNai version.
The very little that was left of the nose almost vanished with VideoJaNai.
Even the included 2x_AnimeJaNai_HD_V3_Compact.onnx behaves differently in chaiNNer compared to VideoJaNai.

ChaiNNer:
AnimeJaNaiHDV3Compact

VideoJaNai:
image

Some weird colors using that in ChaiNNer for some reason, but either way it has the same issue, there is detail loss when using VideoJaNai, regardless of what setting I change in it.

@JossCamp
Copy link

JossCamp commented Nov 9, 2024

VideoJaNai uses vapoursynth to process videos, I imagine it must be a problem in the processing pipeline between the model and the frame processing, I thought I was the only one who seemed to have that problem.

@fideliochan
Copy link

@UpscaleAnon not related to topic what anime is this?

@UpscaleAnon
Copy link
Author

@UpscaleAnon not related to topic what anime is this?

Sorry for late response, but it's not anime, it's a hentai called Bible Black

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants
@UpscaleAnon @JossCamp @fideliochan and others