How to get pipeline Task via API?

[kiranzo]

So, I made a pipeline, and now I want to clone it and launch it with different config programmatically.

Here is Clearml screenshot:

The code for its creation looked like this:

    pipe = PipelineController(
        project="Protocol Test",
        name="protocol",
        version="0.3",
        add_pipeline_tags=True
    )

Now I need to find its Task via server API and call Task.clone(), right?

However, when I call

clearml.task.Task.get_task(project_name='Protocol Test', task_name='protocol #36')
I get None.

I can get result by either calling clearml.task.Task.get_task(task_name='protocol #36') or clearml.task.Task.get_task(task_id='e389a1c....................') but the first option doen't consider project name (which means I could potentially get wrong task), and the second one isn't user friendly.

When I call clearml.task.Task.get_task(task_name='protocol #36').get_project_name() I get 'Protocol Test/.pipelines/protocol', which isn't how I named pipeline in my project.

The way this pipeline is planned to be used is basically launch it once, get its task and then clone it over and over again and run clones with different configs every time. Why does the project name change? Is there a good way to get pipeline task by project name and task name I created the pipeline under, or should I make sure I use unique name for the pipeline task?

This whole thing is very confusing, I'm not sure researchers will find it convenient to use.

[ainoam]

You're absolutely right @kiranzo!!

We'll make sure to have a PipelineController.clone method available in a near SDK version, so you'lle be able to do something like

my_pipeline = PipelinController.get_pipeline(...)
new_run = PipelineController.clone(my_pipeline)

[kiranzo]

@ainoam so I'll tell researchers to copy pipeline task id from web UI for now :|

[pollfly]

Hey @kiranzo! v1.17.0 is now out supporting cloning pipelines using PipelineController.create() and PipelineController.clone().