Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eigenvalue and eigenvector deviation with GPU hardware #42

Open
AjinkyaBankar opened this issue Oct 12, 2021 · 2 comments
Open

Eigenvalue and eigenvector deviation with GPU hardware #42

AjinkyaBankar opened this issue Oct 12, 2021 · 2 comments

Comments

@AjinkyaBankar
Copy link

I have observed that the eigenvalues and eigenvectors change marginally if the underlying GPU or memory configurations are different. How to fix this issue? I am using random seed value to reproduce the results, but the hardware configuration change does not guarantee the exact eigenvalues and eigenvectors. It is affecting the accuracy of the algorithm. Can someone please help? Thank you.

@devansh20la
Copy link

Only setting seed does not guarantee deterministic behaviour on GPU. Have you tried using torch.use_deterministic_algorithms or torch.backends.cudnn.deterministic. Here is an article about it just in case you haven't seen it https://pytorch.org/docs/stable/notes/randomness.html

@AjinkyaBankar
Copy link
Author

Hi devansh20la,
I am already using the following:
random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants