OOM when using gradients_memory's list of checkpointed tensors

[IdoZach]

First, thanks very much for this contribution!

• I run my code using the gradients_memory() method, which indeed works.
• I then try to get the bottleneck tensors it found, by taking the "Checkpoint nodes used" list. I add them manually to a "checkpoints" collection, and then use gradients_collection(). Then, I get OOM.

I don't understand why that is so - if I take the same tensors, shouldn't the behaviour be the same?

[yaroslavvb]

Yes, behavior should be the same. If you notice the code after printing "Checkpoint nodes used" doesn't know whether checkpoints variable was specified manually, or automatically. So I would double-check that Checkpoint nodes used message prints the same message in both cases and debug from there