Cross entropy loss(es) should document hard assumption of labels summing up to 1.0

[hmeine]

For legacy reasons, we have been using keras.losses.categorical_crossentropy() (with from_logits = False) in our own loss wrapper that would also support sample weights (for 3D medical images). Through some version upgrade (some time between TF 2.9 and 2.14), we found that this no longer worked, and it took us quite some time to find that there is nowadays a _get_logits() function that switches to the preferred softmax_cross_entropy_with_logits() function, even though we did not specify that at all! On the one hand, that's great, on the other hand, that loss has stricter requirements, in particular it assumes the target vectors to "be probability distributions" (sum up to 1.0).

I came here to suggest that the relatively weak statement "We expect labels to be provided in a one_hot representation." is re-worded into something stronger such as "This function assumes that the targets are provided in a one_hot representation, and they have to sum up to 1.0."

Maybe the sneaky "we're trying to reveal your logits and might strip your softmax" should also be explicitly mentioned.

[SuryanarayanaY]

Hi @hmeine ,

If we specify from_logits = False to loss function it assumes/expects the values are normalized to summing to the probability of 1 , right ? . If uses from_logits=True will have much more numeric stability. IMO for your case since you are using from_logits = False this should not fail. Could you please provide a minimal code snippet to verify it.

[github-actions]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

[github-actions]

This issue was closed because it has been inactive for 28 days. Please reopen if you'd like to work on this further.