[RayCluster] add annotation to enable non-login bash

[fscnick]

Why are these changes needed?

According to #3247, the -l overwrite the PATH resulting in uv not working properly. Introduce an annotation ray.io/non-login-bash-cmd to remove -l from the command.

Previouly, there is a PR #427 to add -l to be convenient for conda init or or set CLASSPATH. Which might have conflict with this current issue. Thus, add an annotation to let user choose which one is suitable for them.

Related issue number

Part of #3247

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • This PR is not tested :(

[fscnick]

screenshot of head-pod:

screenshot of worker-pod:

screenshot of uv run :

[fscnick]

@kevin85421 PTAL

[kevin85421]

The API is too specific; we should make it more general-purpose. We could start by introducing a KubeRay environment variable to define the default container Command for all Pods created by the KubeRay operator.

KUBERAY_DEFAULT_CONTAINER_COMMAND: If the environment variable is not set, the default container Command is []string{"/bin/bash", "-lc", "--"}. If it is set, containers such as the wait-for-gcs init container, the head Pod’s Ray container, the worker Pods’ Ray containers, and the RayJob submitter’s Command will use the value of the environment variable, split by commas ... etc

(Split by commas may not be a good idea. You should figure it out.)

[AvihaiSam]

The API is too specific; we should make it more general-purpose. We could start by introducing a KubeRay environment variable to define the default container Command for all Pods created by the KubeRay operator.

KUBERAY_DEFAULT_CONTAINER_COMMAND: If the environment variable is not set, the default container Command is []string{"/bin/bash", "-lc", "--"}. If it is set, containers such as the wait-for-gcs init container, the head Pod’s Ray container, the worker Pods’ Ray containers, and the RayJob submitter’s Command will use the value of the environment variable, split by commas ... etc

(Split by commas may not be a good idea. You should figure it out.)

@kevin85421 one can use different jobs/clusters/services with different base image with the same KubeRay operator.
I think this should be supported either as the suggested annotation, or extend the CRDs to support it.
either way - it should allow the ability to define the exact container command.

[kevin85421]

one can use different jobs/clusters/services with different base image with the same KubeRay operator.

I understand, but I don’t want to change the CRD at this moment unless we encounter enough actual issues. It’s hard to remove or change behavior once it’s been added to the CRD.

In most cases, the person who deploys KubeRay and the one who builds the Docker images are typically from the same team, while the person who writes the CR YAML may be from a different team.

I think this change should meet most users' needs. I will reconsider updating CRD if the assumptions are incorrect.

either way - it should allow the ability to define the exact container command.

Users can disable the injection of init container.