Error when running batch inference

[moaazassali]

Describe the bug

I am doing batch inference using the CLI and GUI and both aren't fully working for me.

Note: I tried running some files that crash the process individually and they seem to not work on their own either. Running svc infer on the file separately doesn't cause any error but the generated audio file is a small 4KB file that doesn't play anything. Those files that cause the crash also aren't corrupted, and I can play them just fine.

I guess a try except block somewhere would be nice to skip bad files. I checked the inference main.py file and tried putting try except block around the for loop running over the input files but that didn't work either.

Anyways, here are the errors:

I have a folder with 2052 files and here is the last output of the CLI after a few iterations when running svc infer -m D:\Projects\svc\logs\44k\G_4775.pth -c D:\Projects\svc\logs\44k\config.json -o output_folder ogg_files -re

ogg_files\0003027f.ogg:   0%|▏                                                        | 8/2052 [00:05<14:04,  2.42it/s]
          INFO     [18:40:27] Chunk: Chunk(Speech: True, 216128.0)                                          core.py:287
           WARNING  [18:40:27] Speaker None is not found. Use speaker 0 instead.                             core.py:210
           INFO     [18:40:27] F0 inference time:       0.125s, RTF: 0.021                                     f0.py:214
           INFO     [18:40:27] HuBERT inference time  : 0.021s, RTF: 0.004                                  utils.py:234
           INFO     [18:40:27] Inference time: 0.14s, RTF: 0.02                                              core.py:241
ogg_files\00034a4d.ogg:   0%|▎                                                        | 9/2052 [00:05<13:53,  2.45it/s]
          INFO     [18:40:27] Chunk: Chunk(Speech: True, 88200.0)                                           core.py:287
           WARNING  [18:40:27] Speaker None is not found. Use speaker 0 instead.                             core.py:210
           INFO     [18:40:27] F0 inference time:       0.066s, RTF: 0.022                                     f0.py:214
           INFO     [18:40:27] HuBERT inference time  : 0.019s, RTF: 0.006                                  utils.py:234
[18:40:28] INFO     [18:40:28] Inference time: 0.14s, RTF: 0.05                                              core.py:241
           INFO     [18:40:28] Chunk: Chunk(Speech: False, 17640.0)                                          core.py:287
           INFO     [18:40:28] Chunk: Chunk(Speech: True, 61740.0)                                           core.py:287
           WARNING  [18:40:28] Speaker None is not found. Use speaker 0 instead.                             core.py:210
           INFO     [18:40:28] F0 inference time:       0.036s, RTF: 0.015                                     f0.py:214
           INFO     [18:40:28] HuBERT inference time  : 0.009s, RTF: 0.004                                  utils.py:234
           INFO     [18:40:28] Inference time: 0.03s, RTF: 0.01                                              core.py:241
           INFO     [18:40:28] Chunk: Chunk(Speech: False, 17640.0)                                          core.py:287
           INFO     [18:40:28] Chunk: Chunk(Speech: True, 97020.0)                                           core.py:287
           WARNING  [18:40:28] Speaker None is not found. Use speaker 0 instead.                             core.py:210
           INFO     [18:40:28] F0 inference time:       0.070s, RTF: 0.022                                     f0.py:214
           INFO     [18:40:28] HuBERT inference time  : 0.015s, RTF: 0.005                                  utils.py:234
           INFO     [18:40:28] Inference time: 0.14s, RTF: 0.04                                              core.py:241
           INFO     [18:40:28] Chunk: Chunk(Speech: False, 8820.0)                                           core.py:287
           INFO     [18:40:28] Chunk: Chunk(Speech: True, 61740.0)                                           core.py:287
           WARNING  [18:40:28] Speaker None is not found. Use speaker 0 instead.                             core.py:210
           INFO     [18:40:28] F0 inference time:       0.037s, RTF: 0.015                                     f0.py:214
           INFO     [18:40:28] HuBERT inference time  : 0.006s, RTF: 0.003                                  utils.py:234
           INFO     [18:40:28] Inference time: 0.03s, RTF: 0.01                                              core.py:241
           INFO     [18:40:28] Chunk: Chunk(Speech: False, 17640.0)                                          core.py:287
           INFO     [18:40:28] Chunk: Chunk(Speech: True, 79380.0)                                           core.py:287
           WARNING  [18:40:28] Speaker None is not found. Use speaker 0 instead.                             core.py:210
           INFO     [18:40:28] F0 inference time:       0.036s, RTF: 0.013                                     f0.py:214
           INFO     [18:40:28] HuBERT inference time  : 0.015s, RTF: 0.005                                  utils.py:234
           INFO     [18:40:28] Inference time: 0.13s, RTF: 0.05                                              core.py:241
           INFO     [18:40:28] Chunk: Chunk(Speech: False, 8820.0)                                           core.py:287
           INFO     [18:40:28] Chunk: Chunk(Speech: True, 61740.0)                                           core.py:287
           WARNING  [18:40:28] Speaker None is not found. Use speaker 0 instead.                             core.py:210
           INFO     [18:40:28] F0 inference time:       0.037s, RTF: 0.015                                     f0.py:214
           INFO     [18:40:28] HuBERT inference time  : 0.007s, RTF: 0.003                                  utils.py:234
           INFO     [18:40:28] Inference time: 0.02s, RTF: 0.01                                              core.py:241
           INFO     [18:40:28] Chunk: Chunk(Speech: False, 4228.0)

With the GUI, I do the same thing and I get this error instead:

C:\<parent folder>\0002ef17.ogg:   0%|        | 8/2053 [00:05<15:40,  2.18it/s]
          INFO     [18:10:11] Chunk: Chunk(Speech: True, 504414.0)                                          core.py:287
[18:10:12] INFO     [18:10:12] F0 inference time:       0.545s, RTF: 0.047                                     f0.py:214
           INFO     [18:10:12] HuBERT inference time  : 0.021s, RTF: 0.002                                  utils.py:234
           INFO     [18:10:12] Inference time: 0.14s, RTF: 0.01                                              core.py:241
[18:10:17] INFO     [18:10:17] Finished inference for silence_removed                                         gui.py:151
           ERROR    [18:10:17] Error in realtime:                                                             gui.py:838
           ERROR    [18:10:17] Abnormal termination                                                           gui.py:839
                    Traceback (most recent call last):
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\site-packages\so_vits_svc_fork\gui.py",
                    line 836, in main
                        future.result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 451,
                    in result
                        return self.__get_result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 403,
                    in __get_result
                        raise self._exception
                    pebble.common.ProcessExpired: Abnormal termination
           ERROR    [18:10:17] Error in inference:                                                            gui.py:846
           ERROR    [18:10:17] Abnormal termination                                                           gui.py:847
                    Traceback (most recent call last):
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\site-packages\so_vits_svc_fork\gui.py",
                    line 844, in main
                        future.result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 451,
                    in result
                        return self.__get_result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 403,
                    in __get_result
                        raise self._exception
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\site-packages\so_vits_svc_fork\gui.py",
                    line 836, in main
                        future.result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 451,
                    in result
                        return self.__get_result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 403,
                    in __get_result
                        raise self._exception
                    pebble.common.ProcessExpired: Abnormal termination
           ERROR    [18:10:17] Error in realtime:                                                             gui.py:838
           ERROR    [18:10:17] Abnormal termination                                                           gui.py:839
                    Traceback (most recent call last):
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\site-packages\so_vits_svc_fork\gui.py",
                    line 836, in main
                        future.result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 451,
                    in result
                        return self.__get_result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 403,
                    in __get_result
                        raise self._exception
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\site-packages\so_vits_svc_fork\gui.py",
                    line 844, in main
                        future.result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 451,
                    in result
                        return self.__get_result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 403,
                    in __get_result
                        raise self._exception
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\site-packages\so_vits_svc_fork\gui.py",
                    line 836, in main
                        future.result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 451,
                    in result
                        return self.__get_result()
                      File "D:\miniconda3\envs\so-vits-svc-fork\lib\concurrent\futures\_base.py", line 403,
                    in __get_result
                        raise self._exception
                    pebble.common.ProcessExpired: Abnormal termination

To Reproduce

Not too sure if this is specific to my audio files or it's general. But I guess try running batch inference with hundreds of audio files and maybe include a corrupted one?

Additional context

No response

Version

4.1.7

Platform

Windows 11

Code of Conduct

• I agree to follow this project's Code of Conduct.

No Duplicate

• I have checked existing issues to avoid duplicates.