Model Breakdown For Larger K-Hops

[RobinRenggli]

I'm interested in reproducing your experiments as related to larger k-hop GNNs, as depicted in Figure 3a in the paper.

However, when training models with k-hops such as 5 or higher, this results in model failure (training starts out well, but then after some number of epochs, the loss explodes).

Could you let me know which hyper-parameters or additional regularization techniques you used to stabilize the model for larger GNNs? Did you also observe model breakdown at larger k-hops?