-
Notifications
You must be signed in to change notification settings - Fork 126
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Strange SymEigsShiftSolver behavior when true eigenvalue is close to zero #126
Comments
I haven't tested the example in details, but theoretically the shift cannot be any of the true eigenvalues. If you know all the eigenvalues are nonnegative and you want to find the smallest one, then you can set the shift to be a small negative number, for example -0.001. That means you can change the line SymEigsShiftSolver<DenseSymShiftSolve<double>> eigs(op, 3, 6, 0.0); to SymEigsShiftSolver<DenseSymShiftSolve<double>> eigs(op, 3, 6, -0.001); |
Thanks @yixuan, using a negative value seems to do the trick. I think my input matrix is always positive semidefinite so it should be safe. It might be worth mentioning that caveat about the shift more prominently in the shift-and-invert documentation, its implications were not obvious. The negative sigma approach would also be useful to describe in the docs; I'd bet that most people looking for the smallest eigenvalues have positive semidefinite matrices as inputs. |
I see. I will improve the documentation in the future. Thanks for the feedback! |
Circling back to this. I just tried it out on a much larger matrix, and unfortunately, it didn't work out well. After some effort, I created a reproducible example at https://github.com/LTLA/DebuggingSpectra - the relevant part is in In Setting up the solver... # this part is for the standard solver
... still setting it up...
Initializing...
Computing...
Done!
0.00965459
0.00733508
0.00564922
3.36169e-17
Setting up the solver... # now trying the shift and invert
... still setting it up... # at this point it's been running for 10 minutes, steadily growing to 4-5 GB of RAM I should add that it doesn't matter whether I use a negative sigma or a sigma of zero, I still get this stall. (If you want some - mostly irrelevant - context about where this test data comes from, it's a graph Laplacian constructed from a symmetrized nearest-neighbors graph of the MNIST digits dataset. Some of the code to generate the matrix is provided in |
Just wanted to comment that I've been following this thread with interest, as I also struggled initially with setting the shift for eigenvalues close to zero (not realizing that the shift needed to be less than the desired target). Like @LTLA I was also studying graph Laplacians and qualitatively similar matrices. In my case, though, the poor conditioning of the matrices of interest seemed to be the root cause of slow convergence. While it doesn't address issues with using |
@LTLA I've checked the code and can confirm that the program stalls at the matrix factorization part, which calls the In this case I also recommend the approach suggested by @keevindoherty, which works quite well for the matrix in library(Matrix)
test <- readRDS("debug1.rds")
library(RSpectra)
emax <- eigs_sym(test, 1, opts = list(retvec = FALSE))
test_shift <- test - 2 * emax$values * Diagonal(nrow(test))
e <- eigs_sym(test_shift, 5)
e$values + 2 * emax$values Output:
|
Thanks both. It seems that the trick also reports the eigenvectors correctly; I'll try it out with some real data later this week. |
After implementing this, and then rereading the comments above, I just realized I managed to misunderstand the math. Instead of subtracting twice the max eigenvalue from the PSD and looking for the largest magnitude eigenvalues, I subtracted the PSD from the max eigenvalue and looked for the largest (signed) eigenvalues. Whoops. But in my defense, it still works: library(Matrix)
test <- readRDS("debug1.rds")
library(RSpectra)
emax <- eigs_sym(test, 1, opts = list(retvec = FALSE))
test_shift <- emax$values * Diagonal(nrow(test)) - test
e <- eigs_sym(test_shift, 5)
rev(emax$values - e$values)
## [1] 9.897154e-03 9.654586e-03 7.335084e-03 5.649224e-03 -8.215650e-15 I dunno if there are any implications from doing it this way (e.g., numerical stability, speed). But it is kind of interesting as it opens the door to using alternative algorithms that give the largest eigenvalues but not the largest magnitude eigenvalues. Regardless of the exact method, if the matrix shifting approach is the way to go, it would be good to throw that in the documentation somewhere, given that both @keevindoherty and I have struggled with the recommended shift-invert approach. |
@LTLA Yes, all these methods are valid in mathematics, but the stability and accuracy may depend on the distribution of all eigenvalues. I agree that it deserves to write a separate document to introduce these tricks. |
Sorry to bring this issue back from the dead but I have been in a similar situation to @LTLA (in fact looking at graph laplacians and often using the same MNIST digits as a test case as it reliably causes RSpectra to hang). I am basically using the shifted approach suggested by @keevindoherty to rule out as many other ways the eigendecomposition can fail, so I am dealing with a graph Laplacian which has been shifted by
|
I have an input matrix where one of the eigenvalues is very close to zero - in fact, I suspect its true value is actually zero - and I am trying to use the recommended shift-and-invert paradigm to identify the smallest eigenvalues. Unfortunately, it seems that this causes some problems with
SymEigsShiftSolver
. To illustrate, I'll repurpose the example from the link:Compiling this with Spectra 1.0.0 and running the subsequent executable gives me:
$ g++ -std=c++17 -I. blah.cpp # Exact C++ standard doesn't really matter $ ./a.out Eigenvalues found: 1 9.99978e-13 -0.00120755
I would have expected to get 1, 1 and some small value. In fact, this behavior gets worse as the off-diagonal element gets closer to 1. If I add a couple more 9's onto the end, e.g.,
M(1, 0) = 0.99999999999999;
, I get:Conversely, if I take a few 9's off, e.g.,
M(1, 0) = 0.999999;
, I get something more reasonable:What's the right behavior here? I guess that the shifted-and-inverted value is undefined when sigma and the eigenvalue is zero, so perhaps there is no solution; but I would have hoped for an explicit failure instead of this. Indeed, I get an actual error (
factorization failed with the given shift
) if I replace the off-diagonal element with 1.Interestingly enough, RSpectra 0.16-0 (which I assume is running an older version of Spectra) does not have any such problems:
Using the regular
SymEigsSolver
withSortRule::SmallestMagn
also seems to work fine.The text was updated successfully, but these errors were encountered: