Explanation

[Ademord]

Hello!

On your blog post you write
"For AdamOptimizer, it shows two methods are effectively identical. This is what we would expect naturally unless there is a dynamic state of an optimizer, which changed during minibatch loop. It turns out this is the case for the three other optimizers we tested here. So we do expect red and blue not to completely overlap. "

I cannot seem to find any information online explaining what you write here. Could you detail? How does the minibatch size does not affect an adam optimizer?

Best,
Franco