Connecting to the kernel never succeeds.

[meltingSnowdrift]

Description

What steps will reproduce the problem?

1. Create an environment using miniforge containing Python 3.11 and spyder-kernels.
2. In the Spyder preferences window, set the Python interpreter to that of this environment.
3. Open a new IPython console in Spyder.

The resulting behaviour is that the console continuously shows "Connecting to kernel..." without ever succeeding. This issue does not appear to be affected by the presence of any other packages in the environment.

If I start with the built-in Python interpreter (which works correctly), perform steps 1 and 2, and then use the restart kernel command, horizontal lines repeatedly appear one after the other. This suggests to me, along with some other testing, that the kernel is being repeatedly started but not successfully connected to.

Versions

• Spyder version: 5.5.3 (standalone)
• Python version: 3.8.10 64-bit
• Qt version: 5.15.2
• PyQt5 version: 5.15.10
• Operating System: Windows-10-10.0.22631-SP0

Dependencies

# Mandatory:
atomicwrites >=1.2.0          :  1.4.1 (OK)
chardet >=2.0.0               :  5.2.0 (OK)
cloudpickle >=0.5.0           :  3.0.0 (OK)
cookiecutter >=1.6.0          :  2.6.0 (OK)
diff_match_patch >=20181111   :  20230430 (OK)
intervaltree                  :  None (OK)
IPython >=8.12.2,<8.13.0      :  8.12.3 (OK)
jedi >=0.17.2,<0.20.0         :  0.19.1 (OK)
jellyfish >=0.7               :  1.0.3 (OK)
jsonschema >=3.2.0            :  4.21.1 (OK)
keyring >=17.0.0              :  24.3.1 (OK)
nbconvert >=4.0               :  7.16.2 (OK)
numpydoc >=0.6.0              :  1.6.0 (OK)
paramiko >=2.4.0              :  3.4.0 (OK)
parso >=0.7.0,<0.9.0          :  0.8.3 (OK)
pexpect >=4.4.0               :  4.9.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.9.8 (OK)
pygments >=2.0                :  2.17.2 (OK)
pylint >=2.5.0,<3.1           :  3.0.4 (OK)
pylint_venv >=3.0.2           :  3.0.3 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.10.0,<1.11.0        :  1.10.1 (OK)
pylsp_black >=2.0.0,<3.0.0    :  2.0.0 (OK)
qdarkstyle >=3.2.0,<3.3.0     :  3.2.3 (OK)
qstylizer >=0.2.2             :  0.2.2 (OK)
qtawesome >=1.2.1,<1.3.0      :  1.2.3 (OK)
qtconsole >=5.5.1,<5.6.0      :  5.5.1 (OK)
qtpy >=2.1.0                  :  2.4.1 (OK)
rtree >=0.9.7                 :  1.2.0 (OK)
setuptools >=49.6.0           :  69.2.0 (OK)
sphinx >=0.6.6                :  7.1.2 (OK)
spyder_kernels >=2.5.1,<2.6.0 :  2.5.1 (OK)
textdistance >=4.2.0          :  4.6.1 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog                      :  4.0.0 (OK)
zmq >=24.0.0                  :  25.1.2 (OK)

# Optional:
cython >=0.21                 :  3.0.9 (OK)
matplotlib >=3.0.0            :  3.7.5 (OK)
numpy >=1.7                   :  1.24.4 (OK)
pandas >=1.1.1                :  2.0.3 (OK)
scipy >=0.17.0                :  1.10.1 (OK)
sympy >=0.7.3                 :  1.12 (OK)

Environment

Environment

# packages in environment at C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme:
#
# Name                    Version                   Build  Channel
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
bzip2                     1.0.8                hcfcfb64_5    conda-forge
ca-certificates           2024.2.2             h56e8100_0    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.2.2              pyhd8ed1ab_0    conda-forge
debugpy                   1.8.1           py311h12c1d0e_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
importlib-metadata        7.1.0              pyha770c72_0    conda-forge
importlib_metadata        7.1.0                hd8ed1ab_0    conda-forge
ipykernel                 6.29.3             pyha63f2e9_0    conda-forge
ipython                   8.22.2             pyh7428d3b_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.1              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2           py311h1ea47a8_0    conda-forge
libexpat                  2.6.2                h63175ca_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libsqlite                 3.45.2               hcfcfb64_0    conda-forge
libzlib                   1.2.13               hcfcfb64_5    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
openssl                   3.2.1                hcfcfb64_1    conda-forge
packaging                 24.0               pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
platformdirs              4.2.0              pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.42             pyha770c72_0    conda-forge
psutil                    5.9.8           py311ha68e1ae_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pygments                  2.17.2             pyhd8ed1ab_0    conda-forge
python                    3.11.8          h2628c8c_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pywin32                   306             py311h12c1d0e_2    conda-forge
pyzmq                     25.1.2          py311h9250fbb_0    conda-forge
setuptools                69.2.0             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
spyder-kernels            2.5.1           win_pyh7428d3b_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
tk                        8.6.13               h5226925_1    conda-forge
tornado                   6.4             py311ha68e1ae_0    conda-forge
traitlets                 5.14.2             pyhd8ed1ab_0    conda-forge
typing_extensions         4.10.0             pyha770c72_0    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
vc                        14.3                hcf57466_18    conda-forge
vc14_runtime              14.38.33130         h82b7239_18    conda-forge
vs2015_runtime            14.38.33130         hcb4865c_18    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
wheel                     0.43.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
zeromq                    4.3.5                h63175ca_1    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge

[meltingSnowdrift]

This is what I mean by "horizontal lines" above.

[ccordoba12]

Hey @meltingSnowdrift, thanks for reporting. Please go to this entry in the File menu

and select Verbose:

After Spyder restarts, go to the Tools > Debug logs menu

open the first file and upload it here.

Note: The numbers at the end of server_python and transport_python will be different in your case. Don't worry about it.

[meltingSnowdrift]

Here is the requested file.
spyder-debug.log

[meltingSnowdrift]

Suspecting that the problem may be related to the space which occurs in the file path of the kernel, I installed Anaconda at a path which does not contain spaces and tried to use that as a kernel. This did not fix the issue. I therefore consider it unlikely that the cause of the issue is a file path containing spaces. Because Anaconda was used this time instead of Miniforge, I also consider it unlikely that the cause is specific to Miniforge.

[ccordoba12]

Suspecting that the problem may be related to the space which occurs in the file path of the kernel, I installed Anaconda at a path which does not contain spaces and tried to use that as a kernel.

@meltingSnowdrift, good thinking! Actually, that could very well be the case because you installed Spyder in C:\Program Files. Since your username on Windows also has spaces, it wouldn't work to install Spyder for your own user instead of globally.

But don't worry, we're working right now to solve that problem. A fix will be available in our next version (5.5.4), to be released at the end of the week or beginning of the next one.

[meltingSnowdrift]

I have installed Spyder 5.5.4. The problem is not fixed.

Collecting evidence this time was somewhat frustrating. Upon first launching Spyder with the interpreter set to the one I want to use, the problem occurs as described in this thread. However, upon closing and launching Spyder again (whether manually or as part of restarting in debug mode), I get a different error message in the console area saying that some network port is already in use:

Traceback (most recent call last):
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\spyder_kernels\console\__main__.py", line 24, in
start.main()
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\spyder_kernels\console\start.py", line 319, in main
kernel.initialize()
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\traitlets\config\application.py", line 118, in inner
return method(app, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\ipykernel\kernelapp.py", line 692, in initialize
self.init_sockets()
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\ipykernel\kernelapp.py", line 331, in init_sockets
self.shell_port = self._bind_socket(self.shell_socket, self.shell_port)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\ipykernel\kernelapp.py", line 253, in _bind_socket
return self._try_bind_socket(s, port)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\ipykernel\kernelapp.py", line 229, in _try_bind_socket
s.bind("tcp://%s:%i" % (self.ip, port))
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\zmq\sugar\socket.py", line 302, in bind
super().bind(addr)
File "zmq/backend/cython/socket.pyx", line 564, in zmq.backend.cython.socket.Socket.bind
File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Address in use (addr='tcp://127.0.0.1:51573')
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\spyder_kernels\console\__main__.py", line 24, in
start.main()
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\spyder_kernels\console\start.py", line 319, in main
kernel.initialize()
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\traitlets\config\application.py", line 118, in inner
return method(app, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\ipykernel\kernelapp.py", line 692, in initialize
self.init_sockets()
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\ipykernel\kernelapp.py", line 331, in init_sockets
self.shell_port = self._bind_socket(self.shell_socket, self.shell_port)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\ipykernel\kernelapp.py", line 253, in _bind_socket
return self._try_bind_socket(s, port)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\ipykernel\kernelapp.py", line 229, in _try_bind_socket
s.bind("tcp://%s:%i" % (self.ip, port))
File "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\Lib\site‑packages\zmq\sugar\socket.py", line 302, in bind
super().bind(addr)
File "zmq/backend/cython/socket.pyx", line 564, in zmq.backend.cython.socket.Socket.bind
File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Address in use (addr='tcp://127.0.0.1:51573')

I am unsure whether this error message reveals anything about the problem or even whether it is related to the same problem at all.

This error message happens whenever I start Spyder after having previously started and closed Spyder with that interpreter selected. This state appears to be reset by restarting the computer, but seems to persist even when there are no "python" executables" running. In other words: start Spyder -> waiting forever symptom -> restart Spyder -> error message -> restart computer -> start Spyder -> waiting forever symptom

Such an error message would very occasionally appear during previous efforts to reproduce the issue, but now appears to be appearing consistently.

I therefore present two log files: one in which this happens and another in which, after this happens, I close that console, causing a new one to open, which reproduces the original behaviour I was complaining about.
spyder-debug_address_in_use.log
spyder-debug_close_console.log

Thank you for your patience.

[meltingSnowdrift]

In a further test today, I tried installing both Spyder and Miniforge at paths with no spaces. This did not fix the problem and did not appear to change anything. If the problem is caused by spaces in file paths at all, the only potentially relevant one remaining would seem to be the one in my username.

I have just conducted another test on my personal computer at home. On this computer, my username does not have spaces, and Spyder appears to work correctly there. This is suggestive but not indicative because there are plenty of other differences between this personal computer and the institutional computer on which the problem occurs.

Please do not trust this interpretation too much, as there are many things I am not sure about and many ways in which I may be wrong.

[ccordoba12]

Hey @meltingSnowdrift, sorry for the late reply. You said:

Collecting evidence this time was somewhat frustrating. Upon first launching Spyder with the interpreter set to the one I want to use, the problem occurs as described in this thread.

That's not good, of course.

However, upon closing and launching Spyder again (whether manually or as part of restarting in debug mode), I get a different error message in the console area saying that some network port is already in use

But I think this shows that at least spaces in directories are not the culprit anymore.

I have just conducted another test on my personal computer at home. On this computer, my username does not have spaces, and Spyder appears to work correctly there. This is suggestive but not indicative because there are plenty of other differences between this personal computer and the institutional computer on which the problem occurs.

I agree with your analysis. Do you have a VPN or firewall enabled at your institution? That could be causing this issue.

Also, please open a system terminal (i.e. cmd.exe) and run there

C:\Program Files\Spyder\pkgs\spyder\plugins\ipythonconsole\scripts\conda-activate.bat C:/Program Files/Spyder/pkgs/spyder/bin/micromamba.exe C:/Users/Kiki Lu/.conda/envs/quacktest_basic_extreme C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\python.exe C:\Users\Kiki Lu\AppData\Roaming\jupyter\runtime\kernel-dec4dde9240d.json

and post the output here. That could help us to better understand your problem.

[meltingSnowdrift]

Results of running your specified command

Running that line unmodified in cmd.exe (not Powershell) produced the following error message.
'C:\Program' is not recognized as an internal or external command, operable program or batch file.
This is presumably because of the space in "Program Files".

I then modified the line to surround every file path in it with quotation marks, such that it now reads:

"C:\Program Files\Spyder\pkgs\spyder\plugins\ipythonconsole\scripts\conda-activate.bat" "C:/Program Files/Spyder/pkgs/spyder/bin/micromamba.exe" "C:/Users/Kiki Lu/.conda/envs/quacktest_basic_extreme" "C:\Users\Kiki Lu\.conda\envs\quacktest_basic_extreme\python.exe" "C:\Users\Kiki Lu\AppData\Roaming\jupyter\runtime\kernel-dec4dde9240d.json"

This produced the following output:

0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
NOTE: When using the `ipython kernel` entry point, Ctrl-C will not work.

To exit, you will have to explicitly quit this process, by either sending
"quit" from a client, or using Ctrl-\ in UNIX-like environments.

To read more about this, see https://github.com/ipython/ipython/issues/2049


To connect another client to this kernel, use:
    --existing C:\Users\Kiki

The program then apparently froze, neither producing more output nor terminating. As I write this, it has remained in this state for more than half an hour.

An unexpected clue

A potentially significant piece of evidence recently delivered itself to me in a surprising way. On April 19, when I started up the computer, there appeared the dialog Windows uses to ask the user to select a program to open a file without a file extension. This was not caused by any user action: it occurred immediately when I logged into my account. In this dialog, I selected a text editor, which opened a text file with the following contents:

{
  "shell_port": 51573,
  "iopub_port": 51579,
  "stdin_port": 51574,
  "control_port": 51575,
  "hb_port": 51583,
  "ip": "127.0.0.1",
  "key": "1048e061-244bbbe2358818da9c86cc0c",
  "transport": "tcp",
  "signature_scheme": "hmac-sha256",
  "kernel_name": ""
}

This file exists at the path C:\Users\Kiki. This is not a path to a folder; it is the path of a file named Kiki with no file extension, located in the "Users" folder alongside the normal Windows user folders. This behaviour in which this file is automatically opened upon startup has since occurred every time the computer is started.
To clarify:

• The path C:\Users\Kiki Lu points to a folder which is my user folder.
• The path C:\Users\Kiki, with no file extension, points to a text file containing the content above.

Though I immediately suspected some connection between this file and Spyder problems, I was also concerned that it could be something else with potential security implications, so I asked my organization's technical support department. They assessed with very high confidence that this file is related to Spyder.

I suspect that it is not accidental that the file path of this file is the substring of the path of my user folder which ends at the first space in that path. This leads me to again suspect that spaces in file paths have at least some role in the problem. This suspicion is supported by the presence of something called kernel_name in the file content. It is also suggestive that the path of this file resembles the last line output by your diagnostic command, as described in the previous section, before it froze.

I admittedly have no idea why Windows would try to open this file upon startup. However, this behaviour may prove quite fortunate, as it has made me aware of the existence and contents of this file.

[ccordoba12]

Thanks for the updates @meltingSnowdrift. About them:

I then modified the line to surround every file path in it with quotation marks, such that it now reads

Thanks for doing that. I wasn't sure if my initial command was going to work without the quotes (I don't use Windows, so I couldn't test).

The program then apparently froze, neither producing more output nor terminating. As I write this, it has remained in this state for more than half an hour.

That's fine. The command I asked you to run starts a Spyder kernel, which is like a server waiting for code to be run on it (that's the reason why it stays open for ever).

On April 19, when I started up the computer, there appeared the dialog Windows uses to ask the user to select a program to open a file without a file extension. This was not caused by any user action: it occurred immediately when I logged into my account.

Did you close the console where you started the command I posted? That could be the cause of this.

This file exists at the path C:\Users\Kiki. This is not a path to a folder; it is the path of a file named Kiki with no file extension, located in the "Users" folder alongside the normal Windows user folders.

I think this means we still have an issue with spaces in paths that we need to fix. To give you a bit more context about my guess: every Spyder kernel needs to create a file with its connection info so Spyder can read that file and communicate with it. In this case it seems the command I posted created that file precisely in C:\Users\Kiki, instead of doing it in the appropriate folder inside your user directory.

[ccordoba12]

@dalthviz, could you help me with this one? For this you'd need to create a Windows account with spaces on its name and use our app to start an external kernel on it. If everything goes well, you should be able to reproduce @meltingSnowdrift's issue.

If that doesn't work, you could try to run our activation command directly, i.e. something like this:

"C:\Program Files\Spyder\pkgs\spyder\plugins\ipythonconsole\scripts\conda-activate.bat" "C:/Program Files/Spyder/pkgs/spyder/bin/micromamba.exe" "C:/Users/<user with spaces>/Miniconda3/envs/myenv" "C:\Users\<user with spaces>\Miniconda3\envs\myenv\python.exe" "C:\Users\<user with spaces>\AppData\Roaming\jupyter\runtime\kernel-<available-id>.json"

My guess is that the kernel connection file is not quoted in our activation script and that's what's causing this problem.

[dalthviz]

you could try to run our activation command directly, i.e. something like this:

Checking the command I can reproduce the behavior described at #21937 (comment) I see that a kernel starts but instead of the full path to the kernel connection file, the message shows a truncated one.

My guess is that the kernel connection file is not quoted in our activation script and that's what's causing this problem.

I think that's correct. @meltingSnowdrift could you check if changing C:\Program Files\Spyder\pkgs\spyder\plugins\ipythonconsole\scripts\conda-activate.bat line 27

spyder/spyder/plugins/ipythonconsole/scripts/conda-activate.bat

Line 27 in 1d17d91

"%CONDA_ENV_PYTHON%" -m spyder_kernels.console -f %SPYDER_KERNEL_SPEC%

in your Spyder installation for something like

"%CONDA_ENV_PYTHON%" -m spyder_kernels.console -f "%SPYDER_KERNEL_SPEC%"

helps?

Let us know!

[dalthviz]

Just in case, created #22028 . To test if the fix helps, you can download the Windows installer generated by that PR (note that running it will uninstall the Spyder version you are using!): https://github.com/spyder-ide/spyder/actions/runs/8853573213?pr=22028

[meltingSnowdrift]

Just in case, created #22028 . To test if the fix helps, you can download the Windows installer generated by that PR (note that running it will uninstall the Spyder version you are using!): https://github.com/spyder-ide/spyder/actions/runs/8853573213?pr=22028

I am happy to report that this version works. For the first time in many weeks, Spyder runs in a usable form on the institutional computer.

Did you close the console where you started the command I posted? That could be the cause of this.

The symptom involving the file being opened automatically at startup occurred before I ever tried your command in any form.

Thank you to both of you for your patience and effort.

[ccordoba12]

That's great news @meltingSnowdrift! Thanks for your patience with this problem, your help and feedback were critical to solve it.