Train with Skorch +GridSearchCV and test with Pytorch

[AdrienBenamira]

Dear all,

Thank you very much for this very useful library !

I would like to know if it is possible to train a model with Skorch+GridSearchCV

and then

to get the pytorch model trained with the best parameters of the GridSearchCV

regards

Adrien

[BenjaminBossan]

Not exactly, but almost.

GridSearchCV does not store copies of the fitted models (this could be quite a big memory overhead). Therefore, you cannot get back the best models (it's not just 1 best model, because grid search uses cross validation, so it could be 5 best models or so). However, GridSearchCV allows you to refit the net based on the best hyper-parameters. For this, it must be instantiated with the argument refit=True (which is default). Then you can access the skorch net like so:

net = NeuralNet(...)
gs = GridSearchCV(net, ..., refit=True)
gs.fit(X, y)
gs.best_estimator_  # <= the best skorch net
gs.best_estimator_.module_  # the pytorch module of the best net

Note: This is not 100% identical to the best model encountered during the search. The difference is that this model will be fitted on 100% of X and y, whereas the best models encountered during the search would only be fitted on a split of the data, depending on the cv argument.

It would be possible to store copies of each fitted net on disk using the Checkpoint callback, but unfortunately, the latest net would always override the previous on. There is a discussion on #870 on how to solve this, but as is, you would have to write your own solution.

[RoyiAvital]

@BenjaminBossan , It would be good to convert such issues into discussion, no?

[AdrienBenamira]

Thank you very much :D

Regards

Adrien

[BenjaminBossan]

It would be good to convert such issues into discussion, no?

What would be the advantage?

[RoyiAvital]

Less confusion for developers and users?

[BenjaminBossan]

I'll try it out for new issues that would be a better fit for Discussions, but I'll let the old ones as they are.