yaml to python sdk submission example needed

[dandawg]

/kind feature

Describe the solution you'd like
There is a need for an example that submits an Experiment YAML via the Katib SDK. I have written an example, and I plan to submit it as a PR.

My reasoning is this:

The Katib SDK exposes the power of Python programming to Katib users; however, the code structure is significantly more complex than in the case of direct YAML submission using kubectl.

In contrast, direct Katib Experiment submission via YAML offers an easy-to-read (and to maintain) structure, but it is very basic, and it lacks advantages that the SDK route provides.

There is a way to submit a YAML file by converting it to a Python dictionary and then passing it to the Katib SDK, so an example is in order.

Anything else you would like to add:
In the long run, it would be nice to have this functionality in the SDK (that is, load a YAML file, then update chosen parameters, all in an object that can be passed to create_experiment). For now, this example can help users engage easier with the SDK, especially in the case where they already have experiments defined in YAML, but want to submit via the SDK.

---

Love this feature? Give it a 👍 We prioritize the features with the most 👍

[dandawg]

I just submitted PR #2008 to address this issue.

[johnugeorge]

@dandawg Does this https://github.com/kubeflow/katib/blob/master/examples/v1beta1/sdk/tune-train-from-func.ipynb help you? It takes python function as the argument

[dandawg]

That is a great example @johnugeorge ! And I'm super excited about that new tune function. I think it works great for many cases, especially where model training is not too complex, and it can fit in a python function.

I think there is still a need for the yaml route, and the example I submitted shows how to convert a yaml experiment to python, so that it can be submitted via the sdk. I was wondering if this was the right subfolder for the example though.

Longer term, it might be cool to have functions in the sdk like:
load_experiment_from_yaml # creates an experiment object
V1beta1ExperimentSpec.to_yaml(<file_path>)
create_experiment(<yaml-file>,...) # passing yaml file instead of experiment spec python object

This would mimic the create_run_from_pipeline_package function in kfp.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.