TypeError: cannot pickle '_thread.RLock' object on macOS

[madagiurgiu25]

Hi,

I have encountered the following error when running sniffles 2.4 (python 3.10) on a macos.

Error:

sniffles --threads 1  --input ../mapping/ngmlr.bam --minsupport 4 --vcf sniffles2.vcf  --allow-overwrite
2024-10-21 15:40:28,332 INFO sniffles.main (64570): Running Sniffles2, build 2.4
2024-10-21 15:40:28,332 INFO sniffles.main (64570):   Run Mode: call_sample
2024-10-21 15:40:28,332 INFO sniffles.main (64570):   Start on: 2024/10/21 15:40:28
2024-10-21 15:40:28,332 INFO sniffles.main (64570):   Working dir: /Users/madag/Projects/PhD/github/test/decoil-pre/tests/examples/sim_1502/sv
2024-10-21 15:40:28,332 INFO sniffles.main (64570):   Used command: /Users/madag/miniforge3/envs/envdecoil_18102024_6/bin/sniffles --threads 1 --input ../mapping/ngmlr.bam --minsupport 4 --vcf sniffles2.vcf --allow-overwrite
2024-10-21 15:40:28,332 INFO sniffles.main (64570): ==============================
2024-10-21 15:40:28,348 INFO sniffles.main (64570): Opening for reading: ../mapping/ngmlr.bam
2024-10-21 15:40:28,349 INFO sniffles.main (64570): Opening for writing: sniffles2.vcf (single-sample, sorted)
2024-10-21 15:40:28,353 INFO sniffles.main (64570): 
2024-10-21 15:40:28,353 INFO sniffles.main (64570): Analyzing 99548 alignments total...
2024-10-21 15:40:28,353 INFO sniffles.main (64570): 
2024-10-21 15:40:28,353 INFO sniffles.worker (64570): Starting worker 0
2024-10-21 15:40:28,363 ERROR sniffles.main (64570): Unhandled error while running sniffles.
Traceback (most recent call last):
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/bin/sniffles", line 524, in <module>
    Sniffles2_Main(processes)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/bin/sniffles", line 443, in Sniffles2_Main
    p.start()
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/site-packages/sniffles/parallel.py", line 496, in start
    self.process.start()
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.RLock' object

It seems that the issue is the start method for the multiprocessing module in Python, which is set by default on spawn in my case (macOS Ventura, platform.system()=='Darwin'). Changing this to fork fixes the error.

I got fixed by including in site-packages/sniffles/parallel.py:

from multiprocessing import set_start_method
set_start_method("fork")

[hermannromanek]

Hi @madagiurgiu25

Thank you! We're aware of this (see a4af992), and it will be fixed in the next release which we hope to get out really soon!

Can you confirm this version - i.e. with condition on "darwin" - also works for you? We dont have any Macs to test Sniffles on.

Thanks,
Hermann

[madagiurgiu25]

Hi @hermannromanek,

Yes, it also works like this:

import platform
if platform.system() == 'Darwin':
    from multiprocessing import set_start_method
    set_start_method("fork")

Best,
Madalina