pestpp glm upgrade vector long computation time #327
-
|
Hello, |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
the matrix that GLM needs to SVD factorize for each lambda is npar X npar, which is your case is about 10K X 10K, which isn't a small matrix. There is a chance you are thrashing that 16gb of memory with a matrix of this size. I think @doherty1234 has some linkages to the intel MKL shared libs for PEST_HP, but that won't help you on an M1. If you arent set on using derivatives, pestpp-ies only has to SVD factorize a matrix of npar X num_real, where you choose num_real (how many realizations - typically 100-500 is enough to get the inversion going), which results in a much smaller matrix to invert. But it looks like your parameters might have prior correlations (just based on the PI equations), so you'd probably need to gen up a prior parameter ensemble to expresses these correlations so that pestpp-ies is working in the parameter subspace that is plausible WRT these parameter correlations. All doable, either with the pest utilities or with pyEMU... |
Beta Was this translation helpful? Give feedback.
-
|
Thanks a lot for the answer, |
Beta Was this translation helpful? Give feedback.
the matrix that GLM needs to SVD factorize for each lambda is npar X npar, which is your case is about 10K X 10K, which isn't a small matrix. There is a chance you are thrashing that 16gb of memory with a matrix of this size.
I think @doherty1234 has some linkages to the intel MKL shared libs for PEST_HP, but that won't help you on an M1.
If you arent set on using derivatives, pestpp-ies only has to SVD factorize a matrix of npar X num_real, where you choose num_real (how many realizations - typically 100-500 is enough to get the inversion going), which results in a much smaller matrix to invert. But it looks like your parameters might have prior correlations (just based on the PI equatio…