Feature: automatically install repo as pip package

[markbarna]

In addition to manually listing python dependencies or using a requirements file, it would be good to have the option to install the repo as a package, if it's structured appropriately. This also would address any potential issues with inconsistent behavior of imports based on the cwd.

The specific request here is that if the repo contains a setup.cfg, setup.py, or pyproject.toml, after cloning the repo and applying any uncommitted changes, the agent will cd into the repo and run pip install ..

I was able to replicate this behavior by having the following in my code:

task = Task.init(....)
task.set_packages(["my-cool-project @ file:///root/.clearml/venvs-builds/task_repository/my-cool-project.git"])

But this seems a bit hacky, especially since I have to hard code the path to where the agent clones my repo.

[jkhenning]

Hi @markbarna,

Notice the root of the repo is always added to the PAYTHONPATH so it almost the equivalent of adding the repo.
You can also add the repo using Task.add_requirements("-e .")

[markbarna]

Hi @jkhenning ,
I could have sworn I had already tried task.set_packages(["-e ."]), however I just tried again and that worked so I guess this does address the need. Can you explain the difference between task.add_requirements(), task.set_packages() and Task. force_requirements_env_freeze()? They all have overlapping functionality so it's not clear when I should use which one. Thank you.