--python python3.11 results in an error

[maxwell-k]

Describe the bug

• The output of --help explains that pipx run supports --python python3.11.
• My testing shows that this results in an error
• I tested on 1.3.3 and the latest version from main
• A workaround is to use --python 3.11
• Edit I tested with the python-launcher package installed on Fedora 38

How to reproduce

Command to reproduce this bug:

pipx run --python python3.11 pycowsay moo

Error message:

Traceback (most recent call last):
  File "/home/maxwell-k/.local/bin/pipx", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/maxwell-k/.local/pipx/venvs/pipx/lib64/python3.11/site-packages/pipx/main.py", line 885, in cli
    return run_pipx_command(parsed_pipx_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/pipx/venvs/pipx/lib64/python3.11/site-packages/pipx/main.py", line 192, in run_pipx_command
    py_launcher_python = find_py_launcher_python(args.python)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/maxwell-k/.local/pipx/venvs/pipx/lib64/python3.11/site-packages/pipx/interpreter.py", line 33, in find_py_launcher_python
    py = subprocess.run(
         ^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/py', '-python3.11', '-c', 'import sys; print(sys.executable)']' returned non-zero exit status 2.

Command to show pipx version:

pipx --version

Output:

1.3.3

Also tested with:

1.3.4.dev8+gb66335c

Expected behavior

Command to show help:

pipx run --help

Excerpt from help:

  --python PYTHON       Python to run with. Possible values can be the executable
                        name (python3.11), the version to pass to py launcher (3.11),
                        or the full path to the executable. Requires Python 3.8 or
                        above.

I therefore expect the two commands below to be equivalent:

pipx run --python python3.11 pycowsay moo
pipx run --python 3.11 pycowsay moo

They should both display a mooing cow:

  ---
< moo >
  ---
   \   ^__^
    \  (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||

[maxwell-k]

I just noticed that the same behaviour - help text and error / traceback - occur with pipx install; for example:

.nox/develop-3-12/bin/python src/pipx install --python python3.11 pycowsay

[maxwell-k]

I see different behaviour if the python launcher is not installed ( https://python-launcher.app/ / https://github.com/brettcannon/python-launcher ).

Testing on Ubuntu 22.04 with/without the Python launcher

Command using executable name:

pipx run --python python3.11 pycowsay moo

Output:

  ---
< moo >
  ---
   \   ^__^
    \  (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||

Command using version number:

pipx run --python 3.11 pycowsay moo

Output:

Traceback (most recent call last):
  File "/usr/local/bin/pipx", line 8, in <module>
    sys.exit(cli())
  File "/opt/pipx/venvs/pipx/lib/python3.10/site-packages/pipx/main.py", line 885, in cli
    return run_pipx_command(parsed_pipx_args)
  File "/opt/pipx/venvs/pipx/lib/python3.10/site-packages/pipx/main.py", line 197, in run_pipx_command
    commands.run(
  File "/opt/pipx/venvs/pipx/lib/python3.10/site-packages/pipx/commands/run.py", line 200, in run
    run_package(
  File "/opt/pipx/venvs/pipx/lib/python3.10/site-packages/pipx/commands/run.py", line 157, in run_package
    _download_and_run(
  File "/opt/pipx/venvs/pipx/lib/python3.10/site-packages/pipx/commands/run.py", line 234, in _download_and_run
    venv.create_venv(venv_args, pip_args, override_shared)
  File "/opt/pipx/venvs/pipx/lib/python3.10/site-packages/pipx/venv.py", line 160, in create_venv
    venv_process = run_subprocess(cmd + venv_args + [str(self.root)])
  File "/opt/pipx/venvs/pipx/lib/python3.10/site-packages/pipx/util.py", line 174, in run_subprocess
    completed_process = subprocess.run(
  File "/usr/lib/python3.10/subprocess.py", line 503, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '3.11'

Commands to install the python launcher:

 curl --location --remote-name https://github.com/brettcannon/python-launcher/releases/download/v1.0.0/python_launcher-1.0.0-x86_64-unknown-linux-gnu.tar.xz
 tar --extract --strip-components 1 --directory /usr/local --file python_launcher-1.0.0-x86_64-unknown-linux-gnu.tar.xz

Now the behaviour matches the initial bug report.

[Gitznik]

@maxwell-k would my linked PR fix this issue? this would essentially make the command with python3.11 and just 3.11 identical when running with pylauncher.

[maxwell-k]

After more testing I think that there are two parts to this issue:

The first part, emphasised in the table below, would be fixed by #1186 .

The second part is the content of the tracebacks. There are two reasons that might produce a trace back here: (1) the launcher is not installed or (2) the requested version of Python is not installed. It is a leap from the displayed tracebacks (examples in #1186 (comment) ) to the underlying cause.

In my opinion, the next steps here depend on if #1186 is accepted.

Certain invocations of pipx like the example below require that the launcher must be installed. For these, should pipx display a warning if the launcher is not installed?

pipx run --python=3.11 pycowsay moo

Today (pipx 1.4.1) including the full path is the only invocation that works both with the launcher installed and absent:

pipx run --python=/usr/bin/python3.11 pycowsay moo

---

Launcher installed	Python 3.11 installed	--python=	Expected	pipx 1.4.1
Yes	Yes	3.11	Success	Success
		python3.11	Success	Success
		/usr/bin/python3.11	Success	Success
	No	3.11	Error	Traceback
		python3.11	Error	Traceback
		/usr/bin/python3.11	Error	Traceback
No	Yes	3.11	Error	Traceback
		python3.11	Success	Traceback
		/usr/bin/python3.11	Success	Success
	No	3.11	Error	Traceback
		python3.11	Error	Traceback
		/usr/bin/python3.11	Error	Traceback

[Gitznik]

Thanks for having a thorough look at this @maxwell-k! I will take some time this weekend to be able to reproduce your examples and see if I can come up with a good solution to the tracebacks as well 👍