Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unusual memory and cpu usage leading to crashes #10567

Open
Kai-Denzel-Jane opened this issue Dec 15, 2024 · 10 comments
Open

Unusual memory and cpu usage leading to crashes #10567

Kai-Denzel-Jane opened this issue Dec 15, 2024 · 10 comments
Assignees
Labels
info-needed Issue requires more information from poster ssh Issue in vscode-remote SSH

Comments

@Kai-Denzel-Jane
Copy link

  • VSCode Version: 1.96.0

  • Local OS Version: Windows 11 24H2

  • Remote OS Version: Ubuntu 24.Server

  • Remote Extension/Connection Type: SSH Server

  • Logs:

23:13:14.343] Enhanced session logs are disabled. [23:13:14.407] [server] Checking /home/kai/.vscode-server/cli/servers/Stable-138f619c86f1199955d53b4166bef66ef252935c/log.txt and /home/kai/.vscode-server/cli/servers/Stable-138f619c86f1199955d53b4166bef66ef252935c/pid.txt for a running server... [23:13:14.407] [server] Installing and setting up Visual Studio Code Server... [23:13:14.407] [server] Server setup complete [23:13:14.408] [server] Starting server... [23:13:14.408] [server] Starting server with command... Command { std: "/home/kai/.vscode-server/cli/servers/Stable-138f619c86f1199955d53b4166bef66ef252935c/server/bin/code-server" "--connection-token=remotessh" "--accept-server-license-terms" "--start-server" "--enable-remote-auto-shutdown" "--socket-path=/tmp/code-ec60f31c-aca4-4ec7-a962-d0a0ae3cf6ce", kill_on_drop: false } [23:13:14.498] [server] * [23:13:14.498] [server] * Visual Studio Code Server [23:13:14.498] [server] * [23:13:14.498] [server] * By using the software, you agree to [23:13:14.498] [server] * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and [23:13:14.498] [server] * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement). [23:13:14.500] [server] Server bound to /tmp/code-ec60f31c-aca4-4ec7-a962-d0a0ae3cf6ce [23:13:14.500] [server] Extension host agent listening on /tmp/code-ec60f31c-aca4-4ec7-a962-d0a0ae3cf6ce [23:13:14.500] [server] parsed location: "/tmp/code-ec60f31c-aca4-4ec7-a962-d0a0ae3cf6ce" [23:13:14.500] [server] [23:13:14.500] [server] Server started [23:13:14.556] [server] [02:13:06] [23:13:14.556] [server] [23:13:14.556] [server] [23:13:14.556] [server] [23:13:14.621] [server] [02:13:06] Extension host agent started. [23:13:14.760] [server] [02:13:06] Deleted uninstalled extension from disk github.copilot /home/kai/.vscode-server/extensions/github.copilot-1.250.0 [23:13:14.760] [server] [02:13:06] Deleted uninstalled extension from disk github.copilot-chat /home/kai/.vscode-server/extensions/github.copilot-chat-0.23.0 [23:13:14.812] [server] [02:13:06] [<unknown>][62388403][ExtensionHostConnection] New connection established. [23:13:14.812] [server] [02:13:06] [<unknown>][796041b3][ManagementConnection] New connection established. [23:13:15.020] [server] [02:13:06] [<unknown>][62388403][ExtensionHostConnection] <464310> Launched Extension Host Process. [23:13:16.822] Opening exec server for ssh-remote+158.179.19.136 [23:13:16.879] Opening exec server for ssh-remote+158.179.19.136 [23:13:16.938] Verified and reusing cached exec server for ssh-remote+158.179.19.136 [23:13:16.939] Enhanced session logs are disabled. [23:13:16.981] Verified and reusing cached exec server for ssh-remote+158.179.19.136 [23:13:16.982] Enhanced session logs are disabled. [23:13:17.028] Opening exec server for ssh-remote+158.179.19.136 [23:13:17.132] Verified and reusing cached exec server for ssh-remote+158.179.19.136 [23:13:17.132] Enhanced session logs are disabled. [23:13:30.897] [server] [02:13:22] Getting Manifest... github.copilot [23:13:31.224] [server] [02:13:22] Installing extension: github.copilot { [23:13:31.225] [server] productVersion: { version: '1.96.0', date: '2024-12-11T02:29:09.626Z' }, [23:13:31.225] [server] pinned: false, [23:13:31.225] [server] operation: 3, [23:13:31.225] [server] scheme: 'file', [23:13:32.744] [server] [02:13:24] Extension signature verification result for github.copilot: Success. Executed: true. Duration: 1015ms. [23:13:33.163] [server] [02:13:24] Extracted extension to file:///home/kai/.vscode-server/extensions/github.copilot-1.252.0: github.copilot [23:13:33.176] [server] [02:13:24] Renamed to /home/kai/.vscode-server/extensions/github.copilot-1.252.0 [23:13:33.192] [server] [02:13:24] Marked extension as uninstalled github.copilot-1.251.0 [23:13:33.214] [server] [02:13:24] Extension installed successfully: github.copilot file:///home/kai/.vscode-server/extensions/extensions.json

Steps to Reproduce:

1.Open a remote session
2.Open something like htop or bashtop and watch memory increase until it crashes

Does this issue occur when you try this locally?: No
Does this issue occur when you try this locally and all extensions are disabled?: No

Image

This is bashtop after opening a small little remote session which is just some python code and it keeps increasing in CPU and Memory usage after this and eventually the system becomes unstable and you can barely use it before it crashes

@Kai-Denzel-Jane
Copy link
Author

If I need to provide anything else point me towards it and I would be happy to provide whatever is needed to resolve this issue

@vs-code-engineering vs-code-engineering bot added the ssh Issue in vscode-remote SSH label Dec 15, 2024
@joshspicer
Copy link
Member

Have you tried disabling your extensions and seeing if it still repros? Also, please attach your log file, and if you could also include the "extension host" output logs, please include those too.

To dig in further, you could capture a CPU profile. Some steps I have saved are:

  1. set the env variable NODE_OPTIONS="--inspect=9229" in your .bashrc or similar so that the env gets injected into the remote server processes when they startup
  2. Restart the server so that the env is injected, you can also verify in the command line args of the process
  3. Port forward 9229 so that you can debug the process from your local machine
  4. Open chrome://inspect/#devices or edge://inspect/#devices
    You should see the server processes show up under Remote Target, if you would like to use a different port than 9229 then click on Open Dedicated Devtools for Node in the page and under the Connection tab in devtools configure the host:port pair.
    This will now allow you to capture the cpu profile

@joshspicer joshspicer added the info-needed Issue requires more information from poster label Dec 16, 2024
@Kai-Denzel-Jane
Copy link
Author

Kai-Denzel-Jane commented Dec 16, 2024

Have you tried disabling your extensions and seeing if it still repros? Also, please attach your log file, and if you could also include the "extension host" output logs, please include those too.

To dig in further, you could capture a CPU profile. Some steps I have saved are:

  1. set the env variable NODE_OPTIONS="--inspect=9229" in your .bashrc or similar so that the env gets injected into the remote server processes when they startup
  2. Restart the server so that the env is injected, you can also verify in the command line args of the process
  3. Port forward 9229 so that you can debug the process from your local machine
  4. Open chrome://inspect/#devices or edge://inspect/#devices
    You should see the server processes show up under Remote Target, if you would like to use a different port than 9229 then click on Open Dedicated Devtools for Node in the page and under the Connection tab in devtools configure the host:port pair.
    This will now allow you to capture the cpu profile

When I am at my computer next I'll try with extensions disabled, should I try both local and remote extensions and I'll look into the CPU Profiler thing when possible thanks.

@itsopi
Copy link

itsopi commented Dec 19, 2024

I have the same problem. It's very strange. Once an hour the memory starts to grow, until the machine dies. Right now I'm fixing it by killing the process. The next hour the same thing happens again. This problem is critical in my eyes. Please give it more serious attention. It may not be specifically related to remote ssh, but to vscode. I don't know.

@Kai-Denzel-Jane
Copy link
Author

I've done a little testing enabling and disabling remote extensions and seem to think something is related to the official python extensions it almost instantly increases CPU and memory usage and doesn't decrease at all

@tomgreenfield
Copy link

My team are also experiencing this and the only way we've found to circumvent it is to to disable the Copilot extension, so this may be related.

@joshspicer
Copy link
Member

Any extension could be related since this process runs the extension host on the remote. There is an "extension bisect" command in VS Code that can help narrow down problematic extensions, as well as the advice above to capture CPU profiles

@itsopi
Copy link

itsopi commented Dec 20, 2024

The only extension I have installed is Remote - SSH. If it helps, it all started when I updated VSCode to v1.96. As far as I remember, there was no problem on v1.95.

@Kai-Denzel-Jane
Copy link
Author

I've tried Bisect Command it didn't seem to help but also I can barely do anything else on my server when this happening so I'm unsure if I'm going to be able to profile the CPU during this situation because the whole server becomes slow and unresponsive very quickly

@Kai-Denzel-Jane
Copy link
Author

Kai-Denzel-Jane commented Dec 20, 2024

Is this related this process keeps spiking to things like this very often while VSCode process is running
Image

Although hat might be bashtop related

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
info-needed Issue requires more information from poster ssh Issue in vscode-remote SSH
Projects
None yet
Development

No branches or pull requests

5 participants