[Bug]: Using full path to python executable in webui-user.sh cause problems with venv on macOS and Linux

[viking1304]

Checklist

• The issue exists after disabling all extensions
• The issue exists on a clean installation of webui
• The issue is caused by an extension, but I believe it is caused by a bug in the webui
• The issue exists in the current version of the webui
• The issue has not been reported before recently
• The issue has been reported before but has not been fixed yet

What happened?

If the user, for some reason, sets the full path to the Python executable, venv will be created as usual. Everything will work and look normal. But after closer inspection, you will see that almost all packages are actually installed globally and not in the venv. This basically makes venv useless in this case.

Steps to reproduce the problem

1. Open webui-user.sh in editor
2. Set python_cmd="/opt/homebrew/bin/python3.10" (or any other full path)
3. Run ./webui.sh
4. Wait for the installation to finish
5. Close the web UI
6. Check libraries installed in venv folder and those installed globally

What should have happened?

Since venv was created, I expected that all packages would be installed in venv and not globally.

What browsers do you use to access the UI ?

Brave

Sysinfo

It is not helpful in this case

Console logs

> ./webui.sh

################################################################
Install script for stable-diffusion + Web UI
Tested on Debian 11 (Bullseye), Fedora 34+ and openSUSE Leap 15.4 or newer.
################################################################

################################################################
Running on viking user
################################################################

################################################################
Repo already cloned, using it as install directory
################################################################

################################################################
Create and activate python venv
################################################################

################################################################
Launching launch.py...
################################################################
Python 3.10.14 (main, Mar 19 2024, 21:46:16) [Clang 15.0.0 (clang-1500.3.9.4)]
Version: v1.9.4
Commit hash: feee37d75f1b168768014e4634dcb156ee649c05
Installing torch and torchvision
Collecting torch==2.1.2
  Using cached torch-2.1.2-cp310-none-macosx_11_0_arm64.whl.metadata (25 kB)
Collecting torchvision==0.16.2
  Using cached torchvision-0.16.2-cp310-cp310-macosx_11_0_arm64.whl.metadata (6.6 kB)
Collecting filelock (from torch==2.1.2)
  Using cached filelock-3.15.4-py3-none-any.whl.metadata (2.9 kB)
Collecting typing-extensions (from torch==2.1.2)
  Using cached typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting sympy (from torch==2.1.2)
  Using cached sympy-1.12.1-py3-none-any.whl.metadata (12 kB)
Collecting networkx (from torch==2.1.2)
  Using cached networkx-3.3-py3-none-any.whl.metadata (5.1 kB)
Collecting jinja2 (from torch==2.1.2)
  Using cached jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting fsspec (from torch==2.1.2)
  Using cached fsspec-2024.6.1-py3-none-any.whl.metadata (11 kB)
Collecting numpy (from torchvision==0.16.2)
  Using cached numpy-2.0.0-cp310-cp310-macosx_14_0_arm64.whl.metadata (60 kB)
Collecting requests (from torchvision==0.16.2)
  Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting pillow!=8.3.*,>=5.3.0 (from torchvision==0.16.2)
  Using cached pillow-10.3.0-cp310-cp310-macosx_11_0_arm64.whl.metadata (9.2 kB)
Collecting MarkupSafe>=2.0 (from jinja2->torch==2.1.2)
  Using cached MarkupSafe-2.1.5-cp310-cp310-macosx_10_9_universal2.whl.metadata (3.0 kB)
Collecting charset-normalizer<4,>=2 (from requests->torchvision==0.16.2)
  Using cached charset_normalizer-3.3.2-cp310-cp310-macosx_11_0_arm64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests->torchvision==0.16.2)
  Using cached idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests->torchvision==0.16.2)
  Using cached urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests->torchvision==0.16.2)
  Using cached certifi-2024.6.2-py3-none-any.whl.metadata (2.2 kB)
Collecting mpmath<1.4.0,>=1.1.0 (from sympy->torch==2.1.2)
  Using cached mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Using cached torch-2.1.2-cp310-none-macosx_11_0_arm64.whl (59.6 MB)
Using cached torchvision-0.16.2-cp310-cp310-macosx_11_0_arm64.whl (1.5 MB)
Using cached pillow-10.3.0-cp310-cp310-macosx_11_0_arm64.whl (3.4 MB)
Using cached filelock-3.15.4-py3-none-any.whl (16 kB)
Using cached fsspec-2024.6.1-py3-none-any.whl (177 kB)
Using cached jinja2-3.1.4-py3-none-any.whl (133 kB)
Using cached networkx-3.3-py3-none-any.whl (1.7 MB)
Using cached numpy-2.0.0-cp310-cp310-macosx_14_0_arm64.whl (5.2 MB)
Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Using cached sympy-1.12.1-py3-none-any.whl (5.7 MB)
Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Using cached certifi-2024.6.2-py3-none-any.whl (164 kB)
Using cached charset_normalizer-3.3.2-cp310-cp310-macosx_11_0_arm64.whl (120 kB)
Using cached idna-3.7-py3-none-any.whl (66 kB)
Using cached MarkupSafe-2.1.5-cp310-cp310-macosx_10_9_universal2.whl (18 kB)
Using cached mpmath-1.3.0-py3-none-any.whl (536 kB)
Using cached urllib3-2.2.2-py3-none-any.whl (121 kB)
Installing collected packages: mpmath, urllib3, typing-extensions, sympy, pillow, numpy, networkx, MarkupSafe, idna, fsspec, filelock, charset-normalizer, certifi, requests, jinja2, torch, torchvision
Successfully installed MarkupSafe-2.1.5 certifi-2024.6.2 charset-normalizer-3.3.2 filelock-3.15.4 fsspec-2024.6.1 idna-3.7 jinja2-3.1.4 mpmath-1.3.0 networkx-3.3 numpy-2.0.0 pillow-10.3.0 requests-2.32.3 sympy-1.12.1 torch-2.1.2 torchvision-0.16.2 typing-extensions-4.12.2 urllib3-2.2.2

[notice] A new release of pip is available: 24.0 -> 24.1.1
[notice] To update, run: python3.10 -m pip install --upgrade pip
Installing clip
Installing open_clip
Installing requirements
Launching Web UI with arguments: --skip-torch-cuda-test --opt-sub-quad-attention --upcast-sampling --no-half --lowvram --use-cpu all
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
Warning: caught exception 'Torch not compiled with CUDA enabled', memory monitor disabled
Loading weights [6ce0161689] from /Users/viking/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors
Creating model from config: /Users/viking/stable-diffusion-webui/configs/v1-inference.yaml
/opt/homebrew/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 77.3s (prepare environment: 35.7s, import torch: 5.8s, import gradio: 7.7s, setup paths: 12.3s, initialize shared: 0.2s, other imports: 14.5s, load scripts: 0.2s, initialize extra networks: 0.2s, create ui: 0.3s, gradio launch: 0.3s).
Applying attention optimization: sub-quadratic... done.
Model loaded in 17.2s (load weights from disk: 0.2s, create model: 0.8s, apply weights to model: 16.0s, calculate empty prompt: 0.1s).

Additional information

A1111 works perfectly fine without any issues.

The problem is where Python libraries are installed in this case.

[bhanuj10]

hi actually stable diffusion works in python 3.10.6
3.10.11 may have deprecated some features from the previous version causing the errors
try uninstalling the python 3.10.11 and reinstall 3.10.6

[viking1304]

You completely misunderstood the problem. A1111 works perfectly fine without any issues.

The problem is where Python libraries are installed in this case.

If I clean up globally installed libraries and set python_cmd="python3.10" (without path), all libraries will be installed in venv as expected. The Python version used in this case will be exactly the same since that is the only 3.10 version installed on the system, and it is in the path.

I just wondered if anyone might have any ideas about the reason before I start debugging webui.sh and do some separate tests with venv itself.

[bhanuj10]

do u activate the venv before opening the webui.sh

[viking1304]

do u activate the venv before opening the webui.sh

I found what is the reason and submitted #16116

I will close this issue once the PR is merged or closed.