Target tensor check for int in multilabel classification does not agree with torch

[samhavens]

🐛 Bug

I am computing Accuracy and F1Score for a multilabel classification problem. If my target tensors are ints, I get an error from torch:

  File "/path/venv/lib/python3.9/site-packages/torch/nn/functional.py", line 3150, in binary_cross_entropy_with_logits
    return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)
RuntimeError: result type Float can't be cast to the desired output type Long

If my target tensors are floats, I get a torchmetrics error:

File "/path/venv/lib/python3.9/site-packages/torchmetrics/utilities/checks.py", line 47, in _basic_input_validation
    raise ValueError("The `target` has to be an integer tensor.")
ValueError: The `target` has to be an integer tensor.

I think this is a bug, and torchmetrics should not check that targets are ints.

To Reproduce

You can use a transformers sequence classification model set for multilabel classification,

        hf_config  = AutoConfig.from_pretrained(
            model_name,
            num_labels=[fill in],
            problem_type="multi_label_classification"
        )
        model = AutoModelForSequenceClassification.from_pretrained(
            model_name,
            config=hf_config,
        )

roberta-large, for example, uses BCEWithLogits as the loss function, so it will surface this.

Expected behavior

torchmetrics and torch agree on types

Environment

• TorchMetrics version (and how you installed TM, e.g. conda, pip, build from source): 0.9.3, pip installed
• Python & PyTorch Version (e.g., 1.0): torch==1.12.1+cu116, Python 3.9.15
• Any other relevant information such as OS (e.g., Linux): Linux

[github-actions]

Hi! thanks for your contribution!, great first issue!

[Borda]

@stancld mind having a look at it? 🦦

[SkafteNicki]

Hi @samhavens, thanks for reporting this issue.
In this case I would not really call it a bug but more an design decision. The core difference between torch.binary_cross_entropy_with_logits and a metric like F1Score is that the underlying calculation in torch.binary_cross_entropy_with_logits supports values in the [0,1] domain for target and F1Score needs target to be {0,1}, else the calculation will lead to errors. Essentially, this discussion then boils down to if the statement

target = target.long()

should be something the user does or it is something that we should do internally. I think that there are pros and cons for doing both.

[samhavens]

Totally reasonable! Might be worth calling out in the documentation though?

[SkafteNicki]

It is already in the documentation, example:

but maybe we can make it more explicit. I will add it as an todo for the classification metrics in the ongoing update of the documentation (issue #1365).
Closing this issue then.