[FIX] ctc_loss_op_test_gpu unit test suite failure

[AleksaArsic]

Motivation

The command bazel test -s --config=rocm //tensorflow/python/kernel_tests/nn_ops:ctc_loss_op_test_gpu will produce failure of CTCLossDeterministicTest.testForwardAndBackward on gfx11xx and gfx12xx. The failure occured due to one element that represents the difference between gradients is greater than the previous absolute tolerance.

Log:

======================================================================
FAIL: testForwardAndBackward2 (True, False) (__main__.CTCLossDeterministicTest) [graph_mode]
CTCLossDeterministicTest.testForwardAndBackward2 (True, False)
testForwardAndBackward(True, False)
----------------------------------------------------------------------
...
Mismatched elements: 1 / 32000 (0.00313%)
Max absolute difference among violations: 7.075071e-05
Max relative difference among violations: 0.00036804
...
======================================================================
FAIL: testForwardAndBackward3 (True, True) (__main__.CTCLossDeterministicTest) [graph_mode]
CTCLossDeterministicTest.testForwardAndBackward3 (True, True)
testForwardAndBackward(True, True)
----------------------------------------------------------------------
...
Mismatched elements: 1 / 32000 (0.00313%)
Max absolute difference among violations: 5.2332878e-05
Max relative difference among violations: 0.00028655
...
----------------------------------------------------------------------

Technical Details

Increase the absolute tolerance between gradients in CTCLossDeterministicTest.testForwardAndBackward due to the floating-point arithmetic.

[ScXfjiang]

Would it be better if we use the same key-value pairs as CUDA, though they return the same thing?

compute capability: desc.rocm_compute_capability().ToString()

[AleksaArsic]

Here we return a specific part of the compute capability; therefore I've used getter specific for returning the gcn_arch_name.
Otherwise, if we inspect device_description.h we can observe that CudaComputeCapability has method ToString() that returns the version of the software in a format "major"."minor". Inspecting RocmComputeCapability we can see that there is no equivalent method in it. Nevertheless, it would not fit our needs. :)

[ScXfjiang]

Does CUDA have the counterpart? What's the behavior if this API is called on the CUDA platform?

Should we add the rocm_ prefix here?

[AleksaArsic]

If this method is called on a system with CUDA support, it will just return empty string, as the CUDA description is not the same as the the ROCm one.

We could be more descriptive with the name and detail design comment.

[i-chaochen]

LGTM