How to add pip packages via Dockerfile?

[Emporea]

Even tho I searched a lot for this issue online I cant seem to figure it out for code-server.

I had this Dockerfile for a while:

FROM codercom/code-server:4.19.1

# Install Code extensions
RUN code-server --install-extension ms-python.python
RUN code-server --install-extension ms-toolsai.jupyter

# Install apt packages
RUN sudo apt-get update
RUN sudo apt-get install -y python3 python3-pip

# Install pip packages
RUN pip3 install spacy ipykernel matplotlib

RUN python3 -m spacy download en_core_web_lg

Since code-server 4.20.* this throws an error about pip3 installing packages:

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

I've tried lots of different online suggestions, but unfortunately none of them seem to be working with Code Server.
When I try to install something using 'pip install ...', it works perfectly when Code Server is up and running, but I'd really love to be able to install everything I need in the Dockerfile so that once I remove the container, all my packages are there without having to manually reinstall them each time. This was the case until 4.19.1. Also, the spacy download command relies on spacy being installed by pip already, and en_core_web_lg is a rather large package. I don't want to download it each time when deleting the container, so I was wondering if there's a way to install it in the Dockerfile?

So my question: What is the best way to install pip packages inside a dockerfile without breaking any code-server functionality.

[im-coder-lg]

Hmm... So this issue has risen because you are installing it on your base/system-level environment. Why not install Anaconda3/Miniconda3, use Conda to initialise a virtual environment and install the packages there? It might be tedious, time-consuming and could also be hard to code in, but that ought to work since you basically are installing modules into a virtual environment.

P. S. It does say in the error log that you can initialise a Python Virtual ENV for this. I bet that would work!

[im-coder-lg]

And the reason I said Conda is because it is effective in my opinion. But in this situation, I think the python3-venv package from system repos would do the job here.

P.S. referred to this part in the error message before:

If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

[im-coder-lg]

I wrote this Dockerfile up!

Dockerfile code

FROM codercom/code-server:4.19.1

# Install Code extensions
RUN code-server --install-extension ms-python.python
RUN code-server --install-extension ms-toolsai.jupyter

# Install apt packages
RUN sudo apt-get update
RUN sudo apt-get install -y python3 python3-pip python3-venv 
# Added venv module



## UPDATED

RUN python3 -m venv name-of-env

RUN source name-of-env/bin/activate

# Install pip packages
RUN pip3 install spacy ipykernel matplotlib

RUN python3 -m spacy download en_core_web_lg

### NOTE
# ***Please change the name of your venv*** when using this code (name-of-env => `your-name`) :)
# You can add as many modifications as you can
# ***THIS HAS NOT BEEN TESTED IN ANY WAY, SO I HOPE IT WORKS!*** My computer is not powerful enough for this, but I hope that wouldn't be the case in the coming months.
# Again, I hope this works!

Some things I'd like to add(noted in the code itself):

1. This code hasn't been tested or run in any form on my side, I literally typed it in my local code-server instance just a few moments ago. So I don't have any guarantee that this could work, but I hope it does.
2. If this does work, change the name of the virtual ENV to something else, because my name-of-env was just a placeholder for some other name that you could choose
3. Any other Python modules can be added the same way, in the case that the code provided actually does the job properly.

Hope this works!