uv-managed python installed into .local/bin can't be used with venv module

[bluss]

uv 0.4.30

python3.12 -m venv fails to create a virtual environment.

Steps how I reproduced this problem. Using 3.12.6 to use a version that was not already installed.

rm .local/bin/python3.12
uv python install --preview 3.12.6
# Installed Python 3.12.6 in 2.60s
#  + cpython-3.12.6-linux-x86_64-gnu (python3.12)
which python3.12
# /home/user/.local/bin/python3.12
python3.12 -V
# Python 3.12.6
python3.12 -m venv new_environment
# Error: Command '['/home/user/new_environment/bin/python3.12', '-m', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

Let's try execute the python in the venv:

./new_environment/bin/python3.12

Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = './new_environment/bin/python3.12'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = '/install/lib/python3.12'
  sys._base_executable = '/home/user/.local/share/uv/python/cpython-3.12.6-linux-x86_64-gnu/bin/python3.12'
  sys.base_prefix = '/install'
  sys.base_exec_prefix = '/install'
  sys.platlibdir = 'lib'
  sys.executable = '/home/user/new_environment/bin/python3.12'
  sys.prefix = '/install'
  sys.exec_prefix = '/install'
  sys.path = [
    '/install/lib/python312.zip',
    '/install/lib/python3.12',
    '/install/lib/python3.12/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007136928b6740 (most recent call first):
  <no Python frame>

Is the symlink in the virtual environment resolved? (no)

ls -l ./new_environment/bin/python3.12 
# lrwxrwxrwx 1 user user 33 nov  5 03:08 ./new_environment/bin/python3.12 -> /home/user/.local/bin/python3.12*

[charliermarsh]

This is known and reported upstream: astral-sh/python-build-standalone#380

[zanieb]

See also #8429

[garyo]

I'm running into this; uv-installed python symlinked into $HOME/.local/bin fails to create a working venv. I've read #8429 above and astral-sh/python-build-standalone#380 as well. Is there any workaround at the user level that I can use? My specific use case is in a github workflow; one of the actions calls python -m venv .action-bin and I can't change that, but I can adjust anything else beforehand.
I have uv 0.5.8, installed just now, into a CentOS 7 docker container.

[zanieb]

I think export PYTHONHOME="$(dirname $(dirname $(realpath $(which python))))" should work, though it's pretty hacky.

[garyo]

I think export PYTHONHOME="$(dirname $(dirname $(realpath $(which python))))" should work, though it's pretty hacky.

Sadly, that doesn't work:

[root@4b801b307a52 aswf]# curl -LsSf https://astral.sh/uv/install.sh | sh
System glibc version (`2.17') is too old; checking alternatives
downloading uv 0.5.8 x86_64-unknown-linux-musl-static
no checksums to verify
installing to /root/.local/bin
  uv
  uvx
everything's installed!

To add $HOME/.local/bin to your PATH, either restart your shell or run:

    source $HOME/.local/bin/env (sh, bash, zsh)
    source $HOME/.local/bin/env.fish (fish)
[root@4b801b307a52 aswf]# source ~/.local/bin/env
[root@4b801b307a52 aswf]# uv python install --preview 3.11
Installed Python 3.11.11 in 2.10s
 + cpython-3.11.11-linux-x86_64-gnu (python3.11)
[root@4b801b307a52 aswf]# (cd ~/.local/bin; ln -sf python3.11 python3; ln -sf python3.11 python)
[root@4b801b307a52 aswf]# which python
/root/.local/bin/python
[root@4b801b307a52 aswf]# python --version
Python 3.11.11
[root@4b801b307a52 aswf]# export PYTHONHOME="$(dirname $(dirname $(realpath $(which python))))"
[root@4b801b307a52 aswf]# echo $PYTHONHOME
/root/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu
[root@4b801b307a52 aswf]# python -mvenv foo
Error: Command '['/opt/aswf/foo/bin/python', '-m', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

[zanieb]

Alas, if you use --without-pip it works fine. I'm... not sure why the ensurepip call fails there. Of course there's VIRTUALENV_WITHOUT_PIP but no venv equivalent to my knowledge.

Annoyingly, it works in isolation which makes it hard to inspect:

❯ python -m venv .venv
Error: Command '['/Users/zb/workspace/uv/.venv/bin/python', '-m', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.
 - pip==24.2
❯ .venv/bin/python -m ensurepip --upgrade --default-pip
Looking in links: /var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpgx6oozdb
Processing /var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpgx6oozdb/pip-24.2-py3-none-any.whl
Installing collected packages: pip
Successfully installed pip-24.2

[garyo]

I ended up prepending the actual python bin dir to $PATH. That seems to work for me.

Details at AcademySoftwareFoundation/openfx@8276ac8 if anyone’s interested.