Ultra HFR support (real time 240fps, 360fps, 480fps on true 240Hz, 360Hz, 480Hz displays)

[mdrejhon]

Hello -- I'm the founder of Blur Busters and the creator of the world's most popular motion tests called Blur Busters TestUFO (www.testufo.com) which is commonly used by owners of high-Hz monitors.

Anybody who sees the UFO icon on a reviewer website, they're using the free tests that I invented -- very popular among high-Hz gamers.

I am looking for tools to help create Ultra HFR content (real time 240fps, 480fps and 1000fps on real-time 240Hz, 480Hz and 1000Hz displays). Traditionally, the method was to use a slo-mo camera, and speed up the footage using instructions in the Ultra HFR FAQ

I tried to research the DAIN tool to see if it will support UltraHFR, e.g. converting 24fps to 240fps video files (for 240Hz monitors), or 60fps to 360fps video files (for 360Hz monitors), but documentation prominently shows 60fps and 60Hz,

Doubling Hz halves motion blur without needing black frames (or strobing like a CRT), so Ultra HFR has had some recent buss, especially since the new Christie Digital Cinema Projector now supports 480fps 480Hz operation. If you've seen a 120Hz iPad, or played at 120Hz at full frame rate for an extended time period -- you already have seen the benefits.

I have an ASUS 360 Hz PG259QN gaming monitor, so I want to create test 240fps and 360fps video files out of existing 30fps and 60fps video recordings I have here, with far better quality than classic interpolation.

I wrote an article called Frame Rate Amplification Technology and I consider your DAIN within this universe. New improvements have occured as of late, so I'm writing a new article later this year to mention DAIN in a sequel to this article too.

BONUS feature request: Add support for DAIN to convert existing slo-mo files to real-time motion? For example converting a 120fps slo-mo file (in a 30fps file), and output at a custom frame rate (e.g. 240fps) for output to a 240Hz monitor. That way, the ffmpeg step can be skipped. I'm willing to do the ffmpeg step first though, as long as there is support for ultra-high output frame rates.

[mdrejhon]

Also posted at Bao Wenbo github baowenbo/DAIN#129

[n00mkrad]

DAIN is far too slow for anything realtime.

Check out RIFE which is ~25x faster.

[mdrejhon]

DAIN is far too slow for anything realtime.
Check out RIFE which is ~25x faster.

There are two useful use cases:

1. "real-time" parlance referring to non-slow-motion UltraHFR video, that is pre-processed
2. "real-time" parlance referring to real-time processing

I take this to mean that DAIN is capable of item 1, since obviously, it is extremely slow at this time, but more artifactless. I recently received instructions (elsewhere) how to use DAIN to pre-generate real-time 240fps+ files from non-240fps files.

RIFE sounds useful for item 2

[n00mkrad]

I'm still not sure what you want that you can't currently do with DAIN-App.

Speeding slowmo videos up to bring them back to realtime?

[mdrejhon]

Both, actually. Increase an intermediate HFR frame rate to Ultra HFR frame rates.

One problem with Ultra HFR is that cameras tend to record lower quality at higher frame rates. Loss of light, lower resolution, etc. Bypassing the problem of the degraded quality of high speed footage.

Even the GO PRO HERO8 running at 1080p 240fps looks lower quality for individual frames than 4K downsampled to 1080p.

So, a camera that has much higher quality HFR, such as 4K 120fps slo-mo, speed it up to 120fps realtime in post-process, then using DAIN to interpolate it to 4K 240fps or 4K 360fps, then downconvert it to 1080p 360Hz for the sharpest-possible, highest-quality, compression-artifactless Ultra HFR video, played back onto a high-quality IPS 240Hz or 360Hz gaming monitor.

The 2x oversampling makes a big difference (which is why 4K downconverted to 1080p often looks better than 1080p native). Plus using a lower frame rate HFR/UltraHFR material to create the final UltraHFR frame rate at better quality than natively with current high speed cameras.

There are commercial displays already available. The new Christie Digital Cinema Projector, for example, is currently capable of 1080p 480Hz today. Currently, they're working on 1000Hz commercialization.

One is 120fps HFR camera -> 240fps UltraHFR via DAIN
Future could be 240fps HFR camera -> 960fps UltraHFR via DAIN

This is only one possible example workflow; there's many others.

.....Also, from someone else, I found out that DAIN is already capable of doing what I need, through a roundabout process. Technically, this github item could be closed or reframed as a workflow simplification request (although Ultra HFR ability could be made slightly easier via command line options to reduce number of steps in the chain -- but right now this is still very early "pioneering" stuff.)