Error unable to aggregate TsparseMatrix when integrating large datasets

[Ferris01]

Hi, I am trying to integrate large scRNAseq datasets with 40 samples (from 19 patients with both tumour tissues and normal tissues ), and I encountered the error below.

From #7311, it seems that I cannot integrate of more than 7 samples?

Thank you.

Integrating data
Merging dataset 7 2 20 38 4 1 31 9 8 11 24 30 32 35 37 27 13 15 18 into 21 28 3 10 25 40 22 39 33 6 5 19 17 23 29 26 12 14
Extracting anchors for merged samples
Finding integration vectors
Error in .M2C(newTMat(i = c(ij1[, 1], ij2[, 1]), j = c(ij1[, 2], ij2[,  : 
  unable to aggregate TsparseMatrix with 'i' and 'j' slots of length exceeding 2^31-1

anchors<-FindIntegrationAnchors(object.list = scRNA1, anchor.features = features, reduction="rpca")
scRNA1.integrated<-IntegrateData(anchorset = anchors,dims = 1:30)

sessionInfo()
R version 4.3.0 (2023-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.2 LTS

Matrix products: default
BLAS:   /ceph/package/u22/R-base/4.3.0/lib/R/lib/libRblas.so 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3;  LAPACK version 3.10.0

locale:
 [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C               LC_TIME=en_GB.UTF-8       
 [4] LC_COLLATE=en_GB.UTF-8     LC_MONETARY=en_GB.UTF-8    LC_MESSAGES=en_GB.UTF-8   
 [7] LC_PAPER=en_GB.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/London
tzcode source: system (glibc)

attached base packages:
[1] stats4    stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] BPCells_0.1.0               patchwork_1.1.2             Seurat_5.0.0               
 [4] SeuratObject_5.0.1          sp_2.1-1                    DropletUtils_1.20.0        
 [7] SingleCellExperiment_1.22.0 SummarizedExperiment_1.30.2 Biobase_2.60.0             
[10] GenomicRanges_1.52.0        GenomeInfoDb_1.36.1         IRanges_2.34.1             
[13] S4Vectors_0.38.1            BiocGenerics_0.46.0         MatrixGenerics_1.12.2      
[16] matrixStats_1.0.0          

loaded via a namespace (and not attached):
  [1] RColorBrewer_1.1-3        rstudioapi_0.14           jsonlite_1.8.5           
  [4] magrittr_2.0.3            spatstat.utils_3.0-3      zlibbioc_1.46.0          
  [7] vctrs_0.6.3               ROCR_1.0-11               spatstat.explore_3.2-1   
 [10] DelayedMatrixStats_1.22.1 RCurl_1.98-1.12           htmltools_0.5.5          
 [13] S4Arrays_1.2.0            Rhdf5lib_1.22.0           rhdf5_2.44.0             
 [16] sctransform_0.4.1         parallelly_1.36.0         KernSmooth_2.23-21       
 [19] htmlwidgets_1.6.2         ica_1.0-3                 plyr_1.8.8               
 [22] plotly_4.10.2             zoo_1.8-12                igraph_1.5.0             
 [25] mime_0.12                 lifecycle_1.0.3           pkgconfig_2.0.3          
 [28] Matrix_1.6-3              R6_2.5.1                  fastmap_1.1.1            
 [31] GenomeInfoDbData_1.2.10   fitdistrplus_1.1-11       future_1.32.0            
 [34] shiny_1.7.4               digest_0.6.32             colorspace_2.1-0         
 [37] tensor_1.5                dqrng_0.3.0               RSpectra_0.16-1          
 [40] irlba_2.3.5.1             beachmat_2.16.0           progressr_0.13.0         
 [43] spatstat.sparse_3.0-2     fansi_1.0.4               polyclip_1.10-4          
 [46] httr_1.4.6                abind_1.4-5               compiler_4.3.0           
 [49] BiocParallel_1.34.2       fastDummies_1.6.3         HDF5Array_1.28.1         
 [52] R.utils_2.12.2            MASS_7.3-60               DelayedArray_0.26.3      
 [55] tools_4.3.0               lmtest_0.9-40             httpuv_1.6.11            
 [58] future.apply_1.11.0       goftest_1.2-3             R.oo_1.25.0              
 [61] glue_1.6.2                nlme_3.1-162              rhdf5filters_1.12.1      
 [64] promises_1.2.0.1          grid_4.3.0                Rtsne_0.16               
 [67] cluster_2.1.4             reshape2_1.4.4            generics_0.1.3           
 [70] spatstat.data_3.0-1       gtable_0.3.3              R.methodsS3_1.8.2        
 [73] tidyr_1.3.0               data.table_1.14.8         utf8_1.2.3               
 [76] XVector_0.40.0            spatstat.geom_3.2-1       RcppAnnoy_0.0.20         
 [79] stringr_1.5.0             ggrepel_0.9.3             RANN_2.6.1               
 [82] pillar_1.9.0              spam_2.9-1                RcppHNSW_0.4.1           
 [85] limma_3.56.1              later_1.3.1               splines_4.3.0            
 [88] dplyr_1.1.2               lattice_0.21-8            deldir_1.0-9             
 [91] survival_3.5-5            tidyselect_1.2.0          locfit_1.5-9.7           
 [94] miniUI_0.1.1.1            scuttle_1.10.1            pbapply_1.7-2            
 [97] gridExtra_2.3             edgeR_3.42.2              scattermore_1.2          
[100] stringi_1.7.12            lazyeval_0.2.2            codetools_0.2-19         
[103] tibble_3.2.1              cli_3.6.1                 uwot_0.1.15              
[106] xtable_1.8-4              reticulate_1.34.0         munsell_0.5.0            
[109] Rcpp_1.0.10               spatstat.random_3.1-5     globals_0.16.2           
[112] png_0.1-8                 parallel_4.3.0            ellipsis_0.3.2           
[115] ggplot2_3.4.2             dotCall64_1.0-2           sparseMatrixStats_1.12.1 
[118] bitops_1.0-7              listenv_0.9.0             viridisLite_0.4.2        
[121] scales_1.2.1              ggridges_0.5.4            leiden_0.4.3             
[124] purrr_1.0.1               crayon_1.5.2              rlang_1.1.1              
[127] cowplot_1.1.1  

[PengpengGuan]

I have the same err, do you solver this problem？

[kenflab]

I have the same error.

The most likely cause of this error is that the internal sparse matrix exceeded the maximum allowable size of R's indexes due to trying to integrate a large amount of data.

R handles integer values in 32 bits, so it can only hold a maximum of 2^31-1.

[thomasrust18]

I had this error consistently (using Seurat v4) when performing CA integration. Here is the fix: #4380

I installed a modified version of Seurat used to handle large matrices, it fixed the issue.