Allow to create empty python package

[ghost]

There is no way to create empty packages with poetry. If we set tool.poetry.packages to an empty array [], pip install raises the following error:

      File "path/to/site-packages/poetry/core/masonry/utils/module.py", line 63, in __init__
        raise ModuleOrPackageNotFound(
    poetry.core.masonry.utils.module.ModuleOrPackageNotFound: No file/folder found for package poetry-dockerize

I understand this is a valid validation error, but there are cases where we need to install the package and dependencies separately, Dockerfiles. To leverage docker layer cache we generally install dependencies first and then install the project code in another command so that changes in project code does not require installing dependencies again.

Right now a simple workaround is to create a dummy python module (my_project.py if tool.poetry.name = my-project) to install dependencies.

FROM python:3.10-slim-buster

WORKDIR /app

# Install dependencies
COPY poetry.lock pyproject.toml ./
RUN touch my_project.py && pip install . && rm my_project.py

# Copy project code
COPY src src

CMD command1

If we could create empty packages like setuptools, then we could just set tool.poetry.packages = [] instead of tricking poetry-core with a dummy python module.

[rrobby86]

I'm also trying to install a project with tool.poetry.packages = [] in development mode using pip install -e . resulting in the error described above.

Curiously, the poetry install command, from which I would expect the same error, seems to work correctly instead. If it is a validation error, I'd expect it to also appear with poetry install.

Still, I would like to be allowed to have an explicitly empty packages list even when installing with pip. If it is desired to keep the validation, maybe allow to specify something like tool.poetry.allow_no_packages = true to still be able to override it.

[onerandomusername]

I'm also trying to install a project with tool.poetry.packages = [] in development mode using pip install -e . resulting in the error described above.

Curiously, the poetry install command, from which I would expect the same error, seems to work correctly instead. If it is a validation error, I'd expect it to also appear with poetry install.

Still, I would like to be allowed to have an explicitly empty packages list even when installing with pip.

But what is pip installing in that case? just the dependencies?

This smells a lot like an a-b problem, what is your end goal, what you are reaching to happen in the end?

[rrobby86]

But what is pip installing in that case? just the dependencies?

Yes

This smells a lot like an a-b problem, what is your end goal, what you are reaching to happen in the end?

In my case I'm trying to setup a mono-repo with multiple subprojects and to create a "meta-project" which just includes them all as dependencies together with some useful dev-dependencies. I'm not sure it is the best possible approach, but in general I think there can be other use cases (like the one described by the OP) where declaring a "dependencies-only" package could be useful.

[HereticSK]

In my case I'm trying to setup a mono-repo with multiple subprojects and to create a "meta-project" which just includes them all as dependencies together with some useful dev-dependencies. I'm not sure it is the best possible approach, but in general I think there can be other use cases (like the one described by the OP) where declaring a "dependencies-only" package could be useful.

I guess a namespace package is suitable for this use case? According to python's official documentation:

Namespace packages can be useful for a large collection of loosely-related packages (such as a large corpus of client libraries for multiple products from a single company).

[lieryan]

+1, having this exact same issue currently.

This smells a lot like an a-b problem, what is your end goal, what you are reaching to happen in the end?

The end goal here is: it should be possible to create a virtualenv (i.e. successfully run pip install -e .) with just pyproject.toml and poetry.lock.

In a Dockerfile, it is considered best practice to install virtualenv/dependencies in a separate image layer than installing the application itself. This is to take best advantage of Docker's layer caching, as it avoids invalidating the dependency layer cache when the dependency itself doesn't actually change, as recreating virtualenv+installing dependencies is very expensive and dependencies usually change at a much slower pace than the application.

In poetry, this is something that should have been doable by copying just pyproject.toml and poetry.lock, as those two files contained sufficient information to build the virtualenv. However, currently when using poetry-core as build-backend, it seems that poetry-core would always unnecessarily require that many parts of the application package itself (as well as other metadata files like readme) to be present even though they are not necessary for the creation of virtualenv.

[michaeloliverx]

I think this is the same issue as #5052.

[lieryan]

Currently, my work around for this is to create another docker layer where I have full poetry just so I can do poetry export to create a requirements.txt. Not ideal but at least workable, though I wish there's a way to do this with just poetry-core and pip.

[neersighted]

Mostly a specialized duplicate of #1132 with regard to the immediate ask of not requiring a valid package to use Poetry.

The Docker-specific stuff is better tracked elsewhere as a combination of best-practices patterns and possible features driven by the process of developing those best practices.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.