[FT] Fail faster when passing unsupported metrics to InferenceEndpointModel

[albertvillanova]

Issue encountered

When passing an unsupported metric (e.g. single-token metric) to InferenceEndpointModel, an error is raised from Pipeline.evaluate:

ValueError: Endpoint models can't use single token metrics. Change the metric to the standard version

This happens once the endpoint has been created and the model has been created and loaded.

I wonder if the error could be raised earlier for optimization reasons: the process fails faster and resources are not wasted.

Solution/Feature

Raise the error before the endpoint has been created.

Possible alternatives

Leave it as it is.

[clefourrier]

Hm, good idea for the checks, especially for costly metrics. We would also want the LLM as judge to stop if the open ai key is not provided for example.

[clefourrier]

Also linked to #355