Revisit CrossValidateReporter.__init__ parameters

[glemaitre]

Right now, CrossValidateReporter.__init__ is taking *args and **kwargs as parameters.

It is pretty hard for user to infer what are the parameters in such case. We should avoid to have these generic signature.

Since we are wrapping the sklearn.model_selection.cross_validate, we can make an editorial choice of the parameters that we want to expose.

Currently, the signature is

sklearn.model_selection.cross_validate(
    estimator, X, y=None, *, groups=None, scoring=None,
    cv=None, n_jobs=None, verbose=0, fit_params=None,
    params=None, pre_dispatch='2*n_jobs', return_train_score=False,
    return_estimator=False, return_indices=False, error_score=nan
)

For instance, there is no need to request the user any of the return_* values since we are going to persist them to make any computation.

We would almost request to not accept groups or fit_params since it would lead to bugs and we should encourage the usage of the metadata routing in this case.

I would also not expose the verbose option because we should come with a nicer way to observe the progress of the computation. This is related with #949.

We should probably also simplify and not expose pre-dispatch or error_score, the first one being a low level component while the second one should be highlighted from the report that we provide.

Therefore, I would think that estimator, X, y, scoring, cv, params, and n_jobs are the sufficient parameters for this class. We should therefore explicitly document these parameters without redirecting to the scikit-learn documentation (when developing in and editor/notebook, I don't really want to be redirected on an external webpage).

[thomass-dev]

In line with a comment on the original PR: #871 (comment)

[augustebaum]

I'm happy to copy-paste cross_validate's arguments to avoid the *args, **kwargs, but changing the argument list further than that might go against the "drop-in replacement" requirement of the cross-validation feature. @MarieS-WiMLDS What are your thoughts on this?

[MarieS-WiMLDS]

It does go against the drop-in replacement indeed. However, it would be more user-friendly to go by this suggestion. I've been convinced that everything that we would remove in this proposed solution is edge-case, or shouldn't be used. Therefore, let's go :)!

[thomass-dev]

@augustebaum i don't understand why you said that changing the argument list is against the "drop-in replacement".
As long as we keep the same positional arguments, we can remove as many kwargs as we want using **kwargs.

For instance:

def skore.cross_validate(estimator, X, y=None, *, error_score=nan, **kwargs)

[MarieS-WiMLDS]

other similar request: #1000. Therefore increasing prio to P1.

[tuscland]

@glemaitre can we take the opportunity of including this issue in the upcoming CrossValidationReport?

[glemaitre]

Yes. I think that it will be part of the revamp.