An error: Segmentation fault

[yangqinhui0423]

Hey, authors! Thanks for your excellent work.
I followed the instructions in https://github.com/zju3dv/EasyVolcap#running-3dgst
But when I run "evc -t gui -c configs/exps/gaussiant/gaussiant_${expname}.yaml,configs/specs/superm.yaml", I got error as follows:

(easyvolcap1) ciciyang@DESKTOP-RVT20TR:~/code/EasyVolcap-main$ evc -t gui -c configs/exps/gaussiant/gaussiant_${expname}.yaml,configs/specs/superm.yaml
2024-01-20 11:24:44.302034 easyvolcap.models -> : Failed to import submodule init.py:9
volumetric_video_model.py:Zone.Identifier of
/home/ciciyang/code/EasyVolcap-main/easyvolcap/models/init.py
╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮
│ /home/ciciyang/code/EasyVolcap-main/easyvolcap/models/init.py:6 in │
│ │
│ ❱ 6 │ │ exec(f'from . import {module}') │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ from . import volumetric_video_model.py:Zone.Identifier │
│ ▲ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
SyntaxError: invalid syntax
......
2024-01-20 11:24:45.834482 easyvolcap.scripts.main -> preflight: Starting experiment: main.py:80
gaussiant_actor1_4_subseq, command: gui
2024-01-20 11:24:49.377692 easyvolcap.runners.visualizers.volumetric_video_visualizer volumetric_video_visualizer.py:78
-> init: Visualization output:
data/result/gaussiant_actor1_4_subseq/{RENDER,DEPTH,ALPHA}
2024-01-20 11:24:49.380925 easyvolcap.runners.recorders -> init: Saved config file to recorders.py:105
data/record/gaussiant_actor1_4_subseq/gaussiant_actor1_4_subseq_1705721089.y
aml
2024-01-20 11:24:58.594554 easyvolcap.utils.net_utils -> load_network: Loaded network net_utils.py:428
data/trained_model/gaussiant_actor1_4_subseq/latest.npz at epoch -1
Segmentation fault

I am confused about the "Segmentation fault", can you give me any advice?

[dendenxu]

Hi @yangqinhui0423 thanks for using our code!
Looks like this segmentation fault has similar origins to this issue in 4K4D.
Based on the log it should be related to OpenGL or glow's window creation process. But I haven't been able to reproduce this so I might need your assistance to further pinpoint the source of failure: Could you check:

1. Whether running the offline rendering works: evc -t test -c configs/exps/gaussiant/gaussiant_${expname}.yaml
2. Whether opening up a easyvolcap window without the network works: evc-gui
3. What's the error log if you replace evc with python -q -X faulthandler easyvolcap/scripts/main.py, which will turn your command into python -q -X faulthandler easyvolcap/scripts/main.py -c configs/exps/gaussiant/gaussiant_${expname}.yaml,configs/specs/superm.yaml and turn the window command into: python -q -X faulthandler easyvolcap/scripts/main.py -t gui -c configs/specs/gui.yaml

[yangqinhui0423]

Hi @yangqinhui0423 thanks for using our code! Looks like this segmentation fault has similar origins to this issue in 4K4D. Based on the log it should be related to OpenGL or glow's window creation process. But I haven't been able to reproduce this so I might need your assistance to further pinpoint the source of failure: Could you check:

1. Whether running the offline rendering works: evc -t test -c configs/exps/gaussiant/gaussiant_${expname}.yaml
2. Whether opening up a easyvolcap window without the network works: evc-gui
3. What's the error log if you replace evc with python -q -X faulthandler easyvolcap/scripts/main.py, which will turn your command into python -q -X faulthandler easyvolcap/scripts/main.py -c configs/exps/gaussiant/gaussiant_${expname}.yaml,configs/specs/superm.yaml and turn the window command into: python -q -X faulthandler easyvolcap/scripts/main.py -t gui -c configs/specs/gui.yaml

Thanks for your quick answer.
(1)Before I have run the command "evc -t test -c configs/exps/gaussiant/gaussiant${expname}.yaml_" and have no error.

(2)Then I follow your second advice "evc gui"，a black window quickly pops up and disappears, with the following error:

(3)Following the third advice, I run "python -q -X faulthandler easyvolcap/scripts/main.py -c configs/exps/gaussiant/gaussiant${expname}.yaml,configs/specs/superm.yaml_" and it seems that it just killed.

Then I turn to command "python -q -X faulthandler easyvolcap/scripts/main.py -t gui -c configs/specs/gui.yaml", meet the same——A black window pops up and disappears. The error as follows:

[yangqinhui0423]

By the way, I am using wsl2. I don't know if it may make any difference.

[dendenxu]

By the way, I am using wsl2. I don't know if it may make any difference.

@yangqinhui0423 Thanks for mentioning. That should be the culprit.

EasyVolcap supports running natively on Windows so you can just install PyTorch (and OpenGL related packages) to run on Windows. (Install PyTorch and run evc-gui, then install missing packages. Yes we will add a more concise requirements.txt for Windows to make this easier.)

Originally the GUI was developed for WSL2 (yes I was also using WSL2) but I found that Nvidia doesn't support CUDA-GL interop on WSL, which means we can't efficiently copy the PyTorch image tensors (on CUDA) onto the screen so I disabled supported for WSL2. I will add back the compatability mode (copying to to Numpy array (on main memory) and then upload to OpenGL) and put out a warning for users on WSL2.

[yangqinhui0423]

By the way, I am using wsl2. I don't know if it may make any difference.

@yangqinhui0423 Thanks for mentioning. That should be the culprit.

EasyVolcap supports running natively on Windows so you can just install PyTorch (and OpenGL related packages) to run on Windows. (Install PyTorch and run evc-gui, then install missing packages. Yes we will add a more concise requirements.txt for Windows to make this easier.)

Originally the GUI was developed for WSL2 (yes I was also using WSL2) but I found that Nvidia doesn't support CUDA-GL interop on WSL, which means we can't efficiently copy the PyTorch image tensors (on CUDA) onto the screen so I disabled supported for WSL2. I will add back the compatability mode (copying to to Numpy array (on main memory) and then upload to OpenGL) and put out a warning for users on WSL2.

Thanks for your answer, I will try on Windows instead.

[yangqinhui0423]

I tried on Windows anaconda3. I just follow the instruction here https://github.com/zju3dv/EasyVolcap/blob/main/docs/design/install.md
install Pytorch and pip install OpenGL and ImGUI related packages.
But when I run "evc-gui" to test, I get error follows:

I followed the solution in the similar question #13 but it doesn't work.

[dendenxu]

@yangqinhui0423 Hi, these kind of errors are usually caused by failed imports, could you check whether there are other exceptions thrown above?

[dendenxu]

@yangqinhui0423 I pushed an update for better support on WSL2.
Could you in the meantime try whether the commands mentioned in this comment can be run now?
Note that this update requires you to rerun the pip install -e . command
This update will also make your previous error on native Windows more easier to understand by making the stacktrace more consice.

Hi @yangqinhui0423 thanks for using our code! Looks like this segmentation fault has similar origins to this issue in 4K4D. Based on the log it should be related to OpenGL or glow's window creation process. But I haven't been able to reproduce this so I might need your assistance to further pinpoint the source of failure: Could you check:

1. Whether running the offline rendering works: evc -t test -c configs/exps/gaussiant/gaussiant_${expname}.yaml
2. Whether opening up a easyvolcap window without the network works: evc-gui
3. What's the error log if you replace evc with python -q -X faulthandler easyvolcap/scripts/main.py, which will turn your command into python -q -X faulthandler easyvolcap/scripts/main.py -c configs/exps/gaussiant/gaussiant_${expname}.yaml,configs/specs/superm.yaml and turn the window command into: python -q -X faulthandler easyvolcap/scripts/main.py -t gui -c configs/specs/gui.yaml

[yangqinhui0423]

@yangqinhui0423 Hi, these kind of errors are usually caused by failed imports, could you check whether there are other exceptions thrown above?

Yeah, thanks for your advice. I checked and installed the package I needed.
Now I can use gui to view!

[dendenxu]

Glad to have helped! :D
Is the WSL2 version working too?

[yangqinhui0423]

Glad to have helped! :D Is the WSL2 version working too?

I will try on WSL2 later and add answers.
Another question:
（1）Can I attach it to my pretrained model(not include in the static.md) and use it only when inference( to generate novel view)? If so, which file is needed? Is the “.npz” file necessary？（Can “.ply” file replace it？）
（2）Can I add more interactive operation on the GUI basis?
Looking forward your reply.

[dendenxu]

（1）Can I attach it to my pretrained model(not include in the static.md) and use it only when inference( to generate novel view)? If so, which file is needed? Is the “.npz” file necessary？（Can “.ply” file replace it？）

Are you refering to trained gaussian models? We currently only support pretrained models from EasyVolcap (.npz or .pt) since we use different activations than the original gaussian implementation.
But we're currently working on adding support for the original model's format (will update tonight if it goes well).

The gui options for adding such things are under the Meshes & points tab. Moreover, there's a button to reload the trained model from your trained model directory (under Model & network) if a training session is running.

（2）Can I add more interactive operation on the GUI basis?

All GUI controls are defined in this file: volumetric_video_viewer.py in the VolumetricVideoViewer class.
All methods starting with draw_ are for the ImGUI controls like buttons and sliders.
All mouse and keyboard callback functions can be found in methods starting with glfw_ like glfw_mouse_button_callback.

I've already implemented some basic mouse and botton controls (they're currently not documented since I update and expand them regularly) so it should be fairly simple to add your own control. In those methods, the camera can be accessed by self.camera and the network can be accessed by self.model.

[dendenxu]

Hi, @yangqinhui0423 I updated the model loader.

Now directly loading pretrained gaussians is supported: In the GUI, click Meshes & splats and Add gaussian splat from file then choose your ply file. The original gaussian paper's models can be downloaded from this link in their repo.

Here's a camera parameter that looks good enough on the Garden scene: just copy it and paste when you're focusing on the EasyVolcap window.

{"H":768,"W":1366,"K":[[1266.0,0.0,683.0],[0.0,1266.0,384.0],[0.0,0.0,1.0]],"R":[[-0.9540951251983643,-0.16322608292102814,0.2511170506477356],[0.1015067845582962,0.6126022934913635,0.7838568091392517],[-0.28178074955940247,0.7733640074729919,-0.5679122805595398]],"T":[[0.3130389451980591],[-1.5713047981262207],[8.376514434814453]],"n":0.019999999552965164,"f":20.0,"t":0.0,"v":0.0,"bounds":[[-10.0,-10.0,-10.0],[10.0,10.0,10.0]],"mass":0.10000000149011612,"moment_of_inertia":0.10000000149011612,"movement_force":1.0,"movement_torque":1.0,"movement_speed":1.0,"origin":[0.0,0.0,0.0],"world_up":[0.0,-1.0,-0.6499999761581421]}

[yangqinhui0423]

Glad to have helped! :D Is the WSL2 version working too?

I have tried WSL2 and it works well. Thanks for your update!

[yangqinhui0423]

（1）Can I attach it to my pretrained model(not include in the static.md) and use it only when inference( to generate novel view)? If so, which file is needed? Is the “.npz” file necessary？（Can “.ply” file replace it？）

Are you refering to trained gaussian models? We currently only support pretrained models from EasyVolcap (.npz or .pt) since we use different activations than the original gaussian implementation. But we're currently working on adding support for the original model's format (will update tonight if it goes well).

The gui options for adding such things are under the Meshes & points tab. Moreover, there's a button to reload the trained model from your trained model directory (under Model & network) if a training session is running.

（2）Can I add more interactive operation on the GUI basis?

All GUI controls are defined in this file: volumetric_video_viewer.py in the VolumetricVideoViewer class. All methods starting with draw_ are for the ImGUI controls like buttons and sliders. All mouse and keyboard callback functions can be found in methods starting with glfw_ like glfw_mouse_button_callback.

I've already implemented some basic mouse and botton controls (they're currently not documented since I update and expand them regularly) so it should be fairly simple to add your own control. In those methods, the camera can be accessed by self.camera and the network can be accessed by self.model.

Hey，I wonder if ".pth” file is useable. I just want to load my own pretrained model ".pth” file(which is got from my own method, not from the EasyVolcap instructions) but it seems to have failed.
I changed the "latest.pt" which got from the "Running 3DGS+T" instructions into "latest.pth" and it can view successfully. So I think it is caused by my own ".pth" file.
I see the error and pay attention to the "model" error, it seems that in volumetric_video_runner.py at line 179, the load_network function is trying to access the 'model' key from a dictionary but failed.
Here is the complete error:
2024-01-24 12:56:58.484484 easyvolcap.scripts.main -> preflight: Starting experiment: gaussiant_actor1_4_subseq, main.py:80
command: gui
2024-01-24 12:57:01.997297 easyvolcap.runners.visualizers.volumetric_video_visualizer -> volumetric_video_visualizer.py:78
init: Visualization output:
data/result\gaussiant_actor1_4_subseq{RENDER,DEPTH,ALPHA}
2024-01-24 12:57:02.001299 easyvolcap.runners.recorders -> init: Saved config file to recorders.py:105
data/record/gaussiant_actor1_4_subseq\gaussiant_actor1_4_subseq_1706072221.yaml
2024-01-24 12:57:02.119146 easyvolcap.utils.console_utils -> inner: Runtime exception: 'model' console_utils.py:341
╭─────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────╮
│ D:\cici\EasyVolcap-main\easyvolcap\utils\base_utils.py:99 in getitem │
│ │
│ ❱ 99 │ │ │ return dict.getitem(self, key) │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: 'model'

During handling of the above exception, another exception occurred:

╭─────────────────────────────────────────── Traceback (most recent call last) ────────────────────────────────────────────╮
│ D:\cici\EasyVolcap-main\easyvolcap\utils\console_utils.py:338 in inner │
│ │
│ ❱ 338 │ │ │ return func(*args, **kwargs) │
│ │
│ D:\cici\EasyVolcap-main\easyvolcap\engine\registry.py:56 in inner │
│ │
│ ❱ 56 │ │ return call_from_cfg(func, cfg) │
│ │
│ D:\cici\EasyVolcap-main\easyvolcap\engine\registry.py:47 in call_from_cfg │
│ │
│ ❱ 47 │ return func(**call_args) │
│ │
│ D:\cici\EasyVolcap-main\easyvolcap\scripts\main.py:119 in gui │
│ │
│ ❱ 119 │ viewer: "VolumetricVideoViewer" = RUNNERS.build(viewer_cfg, runner=runner) # will start the window │
│ │
│ D:\cici\EasyVolcap-main\easyvolcap\engine\registry.py:300 in build │
│ ❱ 300 │ │ return self.build_func(*args, **kwargs, registry=self) │
│ │
│ D:\cici\EasyVolcap-main\easyvolcap\engine\registry.py:131 in build_from_cfg │
│ │
│ ❱ 131 │ return call_from_cfg(obj_cls, args) │
│ │
│ D:\cici\EasyVolcap-main\easyvolcap\engine\registry.py:47 in call_from_cfg │
│ │
│ ❱ 47 │ return func(**call_args) │
│ │
│ D:\cici\EasyVolcap-main\easyvolcap\runners\volumetric_video_viewer.py:124 in init │
│ │
│ ❱ 124 │ │ self.epoch = self.runner.load_network() # load weights only (without optimizer states) │
│ │
│ D:\cici\EasyVolcap-main\easyvolcap\runners\volumetric_video_runner.py:179 in load_network │
│ │
│ ❱ 179 │ │ epoch = load_network(model=self.model, # only loading the network, without recorder? │
│ │
│ D:\cici\EasyVolcap-main\easyvolcap\utils\net_utils.py:395 in load_network │
│ │
│ ❱ 395 │ pretrained_model = pretrained['model'] │
│ │
│ D:\cici\EasyVolcap-main\easyvolcap\utils\base_utils.py:101 in getitem │
│ │
│ ❱ 101 │ │ │ raise AttributeError(e) │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'model'
*** 'model'
d:\cici\easyvolcap-main\easyvolcap\utils\base_utils.py(101)getitem()
-> raise AttributeError(e)
(Pdbr)