Labels in Python client are out of date

[person142]

With the latest version of the Python sdk and operator, a call like

client = TFJobClient()
# ... do some stuff to launch a job
client.get_logs(<job-name>, namespace=<namespace>, follow=True)

will fail with an error like

Not found Pods of the TFJob <job-name> with the labels {'group-name': 'kubeflow.org', 'job-name': <job-name>, 'job-role': 'master'}

This is because the Python client still looks for the label job-role:

https://github.com/kubeflow/training-operator/blob/master/sdk/python/kubeflow/training/constants/constants.py#L34

whereas in kubeflow/common@6c9fe11 the label was updated to the fully-qualified training.kubeflow.org/job-role.

Some testing reveals that the 1.3 release is the last version to work with the current version of the sdk.

[gaocegege]

Thanks for the issue. Would you help fix it? I saw a commit person142@c1c155e here.

[person142]

Yes indeed - #1588 is the PR. Had to wait a bit to get approval on my end to make it.