Allow non-package dependencies to be installed when package-mode = false

[RiccardoRomagnoli]

Issue Kind

Change in current behaviour

Description

I need to manage non-package dependencies is a distributed way. I have multiple "root" pyproject.toml which references different shared code, every grouped shared code has its dependencies in another pyproject.toml.

I see that with #1132 #8650 I can set my "roots" pyproject to package-mode = false but then when referring to other local dependencies like this dependencies = [ mydep = { path = "../shared/mydep"} ] I get:

RuntimeError: Building a package is not possible in non-package mode.

But I don't want to build mydep I just want to install his dependencies since I also specify package-mode = false there.

Impact

This brings more flexibility & scalability on non-package projects

Workarounds

Install local non package dependencies as packages. This brings in all the boilerplate of packages when not needed

[radoering]

That's out of scope. Dependencies have to be packages.

[RiccardoRomagnoli]

@radoering Could you elaborate more on the reason why dependencies needs to be packages?

[RiccardoRomagnoli]

@thejohnfreeman what do you think?

[finswimmer]

Could you elaborate more on the reason why dependencies needs to be packages?

This is how it works in the python ecosystem, so this is not a Poetry thing.

[RiccardoRomagnoli]

Poetry already expanded in the package-mode false possibility, I don't see why it can't be expanded in regards of non-package dependencies. Still, I would appreciate if you can elaborate more on the fundamental reasons for which this is not feasible.

Poetry presents it self as "Poetry is a tool for dependency management ... ", I think my feature request is inline with Poetry vision.

[neersighted]

import statements are resolved in the sys.path, which is a set of paths where packages are installed, plus (typically) the current working directory. To import from another project not in sys.path or the CWD, you therefore must have an installed package.

Similarly, Poetry recursively resolves a dependency tree, including transient dependencies. It does this by using the standard package metadata (the core metadata). A valid package is required for this to work (and yes, while maybe Poetry could special-case its own pyproject.toml format, that's an entire level of complexity and additional validations to support a niche use case that is not necessary).

What you ask for is well out of scope: you would have to import code that we define to modify sys.path to include other non-package directories and we would have to gain an entire new dependency resolution logic. In other words, blending more than one directory of non-package Python with packages (because you still have dependencies) is not something we intend to support.

You may be interested in #2270, which may provide something similar to what you ask for, if implemented.

---

This brings in all the boilerplate of packages when not needed

Also, to clarify: packages are not boilerplate. You want to re-use code across projects (where a 'project' is a pyproject.toml): that is a package. The burden of specifying enough metadata to build a valid package is not high.

[RiccardoRomagnoli]

Thanks a lot for expanding the discussion, I now see the whole picture, definitely not an easy addition even though I think It is useful in some project/repo setting.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.