NAN source?

[caoxu915683474]

pointer-generator/attention_decoder.py

Line 101 in b29e986

return attn_dist / tf.reshape(masked_sums, [-1, 1]) # re-normalize

[caoxu915683474]

return attn_dist / (tf.reshape(masked_sums, [-1, 1]) + tf.ones_like(tf.reshape(masked_sums, [-1, 1])) * sys.float_info.epsilon)

tf.reshape(masked_sums, [-1, 1]) seems will not be zeros during training, but it is also harmful due to
"division by 0", NAN issue may be caused by this reason.