Process all subfiles or subfolders for export disparity

[3gMan]

Any chance we can get this option?

Right now with Flicker Reduction on, often the start of a new shot ends up with the whites of the depthmap crushed into the foreground. So what should be a gradual depth in the front becomes a flat wall of someone's face or the landscape that sharply and abruptly starts going back when the 100% white starts to go back to grayscale.

The white crush is an easy fix in sections just by restarting the depthmap render from the first frame of the new shot that messes up, so that it doesn't take the last shot into consideration. It would have to be done for every instance though.

Some of us are using Davinci Resolve to split a movie's shots into clips. If we can get an option for iw3 to process all these clips one after another (and still just put all the depthmap frames in the same folder in order like now), that would completely fix every instance of white crush without further intervention or any substantial addition of processing time. We could likely already do this for rendering the 3d within iw3 just by putting all the clips into separate folder, but there is no option in export disparity to do this. An option to do just all the clips in order would be great, but the same subfolders option would also work with just another step or two. Thanks for your hard work nagadomi!

[nagadomi]

About the feature/option in the title, it is just a matter of motivation.
There are no technical challenges. My only concern is that the output directory structure will be messy.
I will add this feature.

About the flicker reduction problem, it requires scene change detection and resetting the state.
Splitting a movie's shots into clips with Davinci Resolve is equivalent to that.
VideoDepthAnything also uses temporal smoothing/interpolation which may need scene change detection, so I think iw3 should have a scene detection feature.

[3gMan]

Nice! I am splitting scenes for Depthcrafter already for the same reason as VDA. While I am using Resolve, several people on the iw3 discord are using PySceneDetect and they say it is better than Resolve. Not sure if it could be or should be implemented into iw3?

https://github.com/Breakthrough/PySceneDetect/blob/main

As far as output directory structure, I think keeping it the same would be the best. Numbered from 00000000 and up, but you probably mean it will be difficult to implement in that way.

[nagadomi]

As far as output directory structure, it is the same as when you process one by one by manual operation.
I meant to say that since the export feature itself creates directories, the original directory structure and the directories created by the export will be mixed together.

---

I know about PySceneDetect, but I don't want to depend on OpenCV (CPU processing and numpy array), so I think I will develop code that can run on GPU.
In that case, I will use PySceneDetect method as a baseline so it shouldn't be any worse than that. It might be just a port of it.

I also plan to do scene detection and internal state resetting automatically.
So if you want to cut scenes more reliably, I think it is better to pre-split the video as you have been doing and turn off the automatic detection.

[3gMan]

Sounds good, thank you!

[nagadomi]

I have added the batch and recursive options to the dev branch.
It will be merged into the master branch eventually, but it may be a little later.

If you want to try it early, try the dev branch.
https://github.com/nagadomi/nunif/blob/dev/windows_package/docs/README.md#dev-branch
I have confirmed that the result is the same as the master branch, but the export code in the dev branch is changed a lot. So I do not recommend switching to the dev branch on a project you are working on.

[donrikk]

Nice! I am splitting scenes for Depthcrafter already for the same reason as VDA. While I am using Resolve, several people on the iw3 discord are using PySceneDetect and they say it is better than Resolve. Not sure if it could be or should be implemented into iw3?

https://github.com/Breakthrough/PySceneDetect/blob/main

As far as output directory structure, I think keeping it the same would be the best. Numbered from 00000000 and up, but you probably mean it will be difficult to implement in that way.

would creating a cache in memory of the last frame of each scene help inform the output of the number of frame its on that way it only is caching one frame in memory at a time only too inform what number frame the next scene starts at?

[nagadomi]

What problem are you talking about? I have no idea what you're talking about.

If it is about concatenating split scenes, frame numbers should be recorded when splitting.
It is not possible to know the frame numbers of the video before splitting from the splitted videos.

[donrikk]

What problem are you talking about? I have no idea what you're talking about.

If it is about concatenating split scenes, frame numbers should be recorded when splitting. It is not possible to know the frame numbers of the video before splitting from the splitted videos.

No problem I know u have it solved I just wanted more insight is all.

[nagadomi]

If you are concerned about file name sequences, you can maintain the original file names by outputting image sequences instead of video and then exporting(convert to depth).

EDIT: Sorry. This method does not work with Flicker Reduction (EMA normalize).

[nagadomi]

It is not possible to know the frame numbers of the video before splitting from the splitted videos.

@3gMan

The file name output by iw3 export is PTS (timestamp), so if PTS is not reset when splitting, the file name can be used as the sequence number throughout the entire video.
I have confirmed that it is possible with ffmpeg's segment option (-f segment without -reset_timestamps 1 option).

[nagadomi]

I fixed resume option for single video Export or Export disparity.
It was a GUI problem, Resume checkbox was disabled.

[nagadomi]

I have fixed a problem that the last frame of the video is not output.
This is not only a problem in the dev branch, the master branch has this bug that has not been fixed.

[nagadomi]

I added a CLI command to split the video by scene/shot.

1. Run nunif-prompt.bat
2. python -m nunif.cli.segment_video -i input.mp4 -o output_dir

videos split with this command will keep the original frame number as the file name when exporting.

TransNetV2 is used for scene detection.

example options,

use lossless codec(ffv1)

python -m nunif.cli.segment_video -i input.mp4 -o output_dir --codec ffv1

downscaling (If video is used only for depth estimation, higher resolution is not needed)

python -m nunif.cli.segment_video -i input.mp4 -o output_dir --max-height 720

reset timestamp (videos can be normally played on video player. note that frame number also reset. )

python -m nunif.cli.segment_video -i input.mp4 -o output_dir --reset-timestamp