macs3 function not works with multiple dataset

[uqnsarke]

data = snap.read_dataset("combined.h5ads")
data

AnnDataSet object with n_obs x n_vars = 277377 x 526765 backed at 'combined.h5ads'

obs: 'sample', 'leiden'
var: 'count', 'selected'
uns: 'spectral_eigenvalue', 'AnnDataSet', 'reference_sequences'
obsm: 'X_spectral_mnn', 'X_spectral', 'X_spectral_harmony', 'X_umap'
obsp: 'distances'

%%time
snap.tl.macs3(data, groupby='leiden', replicate='sample')

2024-09-01 21:26:50 - INFO - Exporting fragments...
thread 'thread 'thread '' panicked at src/call_peaks.rs:384' panicked at :src/call_peaks.rs73:
:called Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }384
:' panicked at note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
73:
src/call_peaks.rscalled Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }:
384:73:

[uqnsarke]

2024-09-02 15:41:03 - INFO - Exporting fragments...
thread '' panicked at src/call_peaks.rs:384:73:
called Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
thread '' panicked at src/call_peaks.rs:384:73:
called Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }
thread '' panicked at src/call_peaks.rs:384:73:
called Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }
thread '' panicked at src/call_peaks.rs:384:73:
called Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }
thread '' panicked at src/call_peaks.rs:384:73:
called Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }
thread '' panicked at src/call_peaks.rs:384:73:
thread 'called Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }
' panicked at src/call_peaks.rs:384:73:
called Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }
Traceback (most recent call last):
File "/data/sarkern2/multiome_liver/scanpy/snapatac/complete2/processed_data/sbatch_02/complete.py", line 11, in
snap.tl.macs3(data, groupby='leiden', replicate='sample')
File "/data/sarkern2/conda/envs/snap2/lib/python3.10/site-packages/snapatac2/tools/_call_peaks.py", line 138, in macs3
fragments = _snapatac2.export_tags(adata, tmpdirname, groupby, replicate, max_frag_size, selections)
pyo3_runtime.PanicException: called Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }

[kaizhang]

Try to find a computer with more memory OR set n_jobs=1 to reduce memory usage.

[uqnsarke]

2.5T memory also didn't work:

Script started at: Tue Sep 17 14:20:53 EDT 2024
Resource limits before running script:
core file size (blocks, -c) 1
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 6189869
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) 576716800
open files (-n) 8192
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) unlimited
cpu time (seconds, -t) unlimited
max user processes (-u) 4096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
2024-09-17 14:21:15 - INFO - Exporting fragments...
thread '' panicked at src/call_peaks.rs:384:73:
called Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
Memory usage after loading dataset: 1358.61 MB
AnnDataSet object with n_obs x n_vars = 277377 x 526765 backed at 'combined.h5ads'
contains 24 AnnData objects with keys: 'Geriatric_G1', 'Geriatric_G2', 'Geriatric_G3', 'Geriatric_G4', 'Geriatric_G5', 'Geriatric_G6', 'Geriatric_G7', 'Geriatric_G8', 'Old_O1', 'Old_O2', 'Old_O3', 'Old_O4', 'Old_O5', 'Old_O6', 'Old_O7', 'Old_O8', 'Young_Y1', 'Young_Y2', 'Young_Y3', 'Young_Y4', 'Young_Y5', 'Young_Y6', 'Young_Y7', 'Young_Y8'
obs: 'sample', 'leiden'
var: 'count', 'selected'
uns: 'AnnDataSet', 'reference_sequences', 'spectral_eigenvalue'
obsm: 'X_spectral_harmony', 'X_spectral_mnn', 'X_umap', 'X_spectral'
obsp: 'distances'
Traceback (most recent call last):
File "/data/sarkern2/multiome_liver/scanpy/snapatac/complete2/processed_data/sbatch_02/harmony.py", line 47, in
process_dataset(dataset_path)
File "/data/sarkern2/multiome_liver/scanpy/snapatac/complete2/processed_data/sbatch_02/harmony.py", line 26, in process_dataset
snap.tl.macs3(data, groupby='leiden', replicate='sample', n_jobs=1)
File "/data/sarkern2/conda/envs/snapatac3/lib/python3.10/site-packages/snapatac2/tools/_call_peaks.py", line 138, in macs3
fragments = _snapatac2.export_tags(adata, tmpdirname, groupby, replicate, max_frag_size, selections)
pyo3_runtime.PanicException: called Result::unwrap() on an Err value: Custom { kind: Other, error: "Allocation error : not enough memory" }
Script finished at: Tue Sep 17 14:57:02 EDT 2024
Resource usage after running script:
core file size (blocks, -c) 1
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 6189869
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) 576716800
open files (-n) 8192
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) unlimited
cpu time (seconds, -t) unlimited
max user processes (-u) 4096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

[qiuyugong-aifi]

I got same error. I have plenty of memory available. any solution to this?

Updates:

It seems the error occurred during this step::

counts.into_fragment_groups(|x| keys[x])
    .progress_with_style(style)
    .for_each(|vals| vals.into_iter().par_bridge().for_each(|(i, beds)| {
        if let Some((_, fl)) = files.get(&i) {
            let mut fl = fl.lock().unwrap();
            beds.into_iter().for_each(|bed| writeln!(fl, "{}", bed).unwrap());
        }
    }));

The codes works with 90 samples, but not working with 200 samples