Lunet (train from scatch)

[kailigo]

Nice work. I wonder to know if it is possible to release the implementation of the proposed LuNet? Thanks.

[lucasb-eyer]

Hi, it was initially not planned (too much code to port) but I had to do it anyways for my thesis, so I can confirm that I have a working LuNet reproducing the paper's result in this codebase.

I will make a PR adding it to this repo sometime next week, I have a few other things to do first. Ping me here again if there's no news mid next week.

[kailigo]

Thanks for the reply. I will send you a reminder next week.

Did you try to train from scratch using resnet50? I tried, but failed. Without good initialization, the model tends to stuck in bad local minima, resulting in reasonably low accuracy. I did try to tune the learning rates, but still failed to get reasonable results.

I guessed you must have tried to train from scratch using resnet50. How about your experiments? Do you have some suggestions on improving it. Thanks.

[lucasb-eyer]

Yeah I agree with you re: initialization. And to make matters worse, I discovered problems with TF's ortho init, which we use for the embedding layer. I'll soon make another PR which replaces it by Xavier init, which is also what we originally used in the paper. You could try that already.

But no, as far as I remember, we haven't tried getting best possible results with resnet50 from scratch, and I'm not sure if it's a worthwhile time investment. But if you do try, I'd be happy to hear about your results.

[percarlsen]

Hi, I can't seem to find the LuNet implementation. Do you still plan on uploading the code?
Thanks