use BaseModelOutput as common interface for all different BaseModelOutputWith*?

[JoanFM]

Hello team,

I have been taking a look at the different output models from your models, and I wonder if it would make sense to inherit all the BaseModelOutputWithPool and all the other flavours of modeling output, instead of using ModelOutput.

transformers/src/transformers/modeling_outputs.py

Line 24 in c301c26

class BaseModelOutput(ModelOutput):

We are trying to build a wrapper around many of the public models hosted on hugging face, and it would be useful to know if we can assume that all the potential outputs of the models will contain hidden_states. Since now they all only inherit from ModelOutput it seems a little confusing.

Am I missing something? Is it not something that can be assumed?