Evaluation every 10k steps

[adrienohana]

Hello,

I can't figure out how to use the evaluation while training, not sure what the data format is and how to plot the ndcg@k. I've figured out how to do it after training though, by loading the saved models in a loop and predicting on my evaluation data.

The question I have is, does seeing the evaluation only every 10k steps make sense ? How to be sure there aren't some big variations in between ? My training doesn't stop improving after 100k steps and is not as smooth as in your paper.

Any hints would be greatly appreciated.

[shreyasavant]

@nreimers @kwang2049 Would love your help with this. Need this for my implementation as well!

[KuijpersNick0]

@adrienohana Did you find any way to evaluate during training ?