Add support for editable pip installs

[jaimergp]

Closes #35

[dholth]

conda/conda-build#5380 is a very rough outline of how I would do it using pypa/build. Especially it gives us the opportunity to install the build system (flit_core, hatch, ...) with our preferred installer instead of necessarily using pip.

[jaimergp]

Isn't pip installing the build system on its own? We have the concept here of "backend", which you can use with grayskull and pip. But yea it's not super perfect. I'm not sure how much I'd like to reimplement from scratch instead of delegating to pip (or uv in the future).

[jaimergp]

Ah, I see what you mean, installing build-system.requires with conda in... a temporary environment? 🤔

[dholth]

In the draft PR I've linked perhaps you can begin to imagine how we could avoid almost all pip invocation, especially if we include our own wheel installer (or https://github.com/zuzukin/whl2conda to convert to .conda and then install)
pypa/build makes the wheel by delegating to each package's build system, setuptools, flit, etc.
Our preferred installer takes care of the environment, dependencies.
Functions from packaging handle the full PEP 508 Python dependency specifications https://packaging.pypa.io/en/stable/markers.html

In "no build isolation" mode, which is what the draft PR halfheartedly implements, you would install the build system "flit_core" or whatever into the current or target conda environment. If it was more like conda-build a temporary conda environment would be used. The build system could also be installed into a temporary directory that is added to PYTHONPATH, or it is allowed that the build system can come from the tree as in when flit builds flit.

Now make it recursive! Imagine an almost seamless experience.

[jaimergp]

Ok, I see, I like that! Added #37 to track these suggestions. Let's consider this PR a barebones prototype for now.

[jezdez]

@jaimergp @dholth Hold on, why would we encourage creating wheel files instead of conda files? There isn't a conda specific platform tag, so I have a hard time seeing a way to differentiate the files from regular Python wheel builds.

[dholth]

I'm talking about the ephemeral "installs a .pth to effect an editable install" wheels from https://peps.python.org/pep-0660/ . We would install and then call into the build system, it spits out that wheel.

We would probably pip install the ephemeral wheel, but it might be interesting to turn it into a conda package and install that way.