[Linux] (Exit code 133) Error when loading large LLM models #285

yushijinhun · 2025-01-06T17:34:59Z

When loading large LLMs (for example, Meta-Llama-3.1-70B-Instruct-IQ2_S with context window 32768), I would encounter the error (Exit code: 133). Please check settings and try loading the model again.. My machine has 64G RAM and 16G vRAM, and I can load the model with same configuration with llama.cpp. Therefore, the problem should not be caused by insufficient RAM or vRAM.

Further investigation into the coredump shows that, the application crashes in function _ZN15partition_alloc8internal32PartitionExcessiveAllocationSizeEm, which means the application is trying to allocate an excessive amount of memory using PartitionAlloc in a call to posix_memalign. This explains why the problem occurs only on LM Studio but not on llama.cpp, as Electron uses PartitionAlloc by default. This is tracked in issue electron/electron#44291.

Apart from waiting for upstream to fix this issue, a potential workaround is to use a customized Electron build to disable PartitionAlloc.

The text was updated successfully, but these errors were encountered:

yagil · 2025-01-06T17:36:06Z

@yushijinhun which operating system are you on? Assuming Linux?

yushijinhun · 2025-01-06T17:36:35Z

@yushijinhun which operating system are you on? Assuming Linux?

Yes. I'm on Linux.

yagil · 2025-01-06T17:37:24Z

@yushijinhun which operating system are you on? Assuming Linux?

Yes. I'm on Linux.

Thanks. @neilmehta24 from our team is investigating this with priority as we speak.

yushijinhun · 2025-01-07T04:50:57Z

I built Electron v33.0.2 with use_allocator_shim = false and enable_backup_ref_ptr_support = false. After replacing LM Studio's electron with the one I built, the problem is solved.

krones9000 · 2025-01-07T09:20:42Z

Could you give a little more specific detail about exactly how you resolved this? I'm in a similar situation, on linux, 16gbVRAM 128GB system RAM, could load models in Oobabooga but getting the 133 error in LM studio. Not sure where to start or how I would "build" Electron or what Electron is and how it relates to the Linux app image of LM Studio that I load.

laushunyu · 2025-01-07T09:28:30Z

I run LM Studio v0.3.6 on Ubuntu 24.04, with 7840HS CPU, 96GB RAM and 16GB GPU VRAM, encountered the same issue: small models can be loaded normally but large models cannot be loaded.

I would appreciate it if @yushijinhun share your compiled Electron version.

yushijinhun · 2025-01-07T11:07:36Z

Not sure where to start or how I would "build" Electron or what Electron is and how it relates to the Linux app image of LM Studio that I load.

Electron is a UI framework built on Chromium used by LM Studio.

I would appreciate it if @yushijinhun share your compiled Electron version.

Here is my Electron v33.0.2 Linux x86_64 build (without allocator shim): Google Drive

You can also follow Electron's Build Instructions to build it yourself.

To replace LM Studio' stock Electron, first unpack the AppImage:

/path/to/LM-Studio-0.3.6-8-x64.AppImage --appimage-extract

Then you would see a squashfs-root directory. Go into it, extract the Electron distribution zip, and overwrite all existent files. Run the electron executable (you may need to add --no-sandbox flag), and you will see LM Studio starting.

daxime · 2025-01-07T13:36:15Z

@yushijinhun Thank you for that :)

CHesketh76 · 2025-01-10T21:29:53Z

Why is this not set to default? LMstudio exclusively uses GGUF files which implies most users are limited on Vram.

yagil · 2025-01-10T21:32:08Z

This is a new bug in 0.3.6 because we updated our Electron version.
Our recommendation: consider staying on 0.3.5 until we fix it properly in a new release (it's in the works)

CHesketh76 · 2025-01-11T21:43:42Z

Same issue when downgrading.

aamir-gmail · 2025-01-13T00:56:36Z

I am using linux version of LM studio 0 3.6 , with Ubuntu 22.04 LTS , I have two 3090 card and 256 GB RAM and AMD 24 Core CPU , for some reason the Qwen2 VL model not load 7B or 70B I get the following message (Exit code: 133). Please check settings and try loading the model again. I was able to load 7B version of the same model on the windows machine (0.3.6) with 64 GB RAM and 12 GB 4070 card , here neither 72B or 4B does not load my NVidia driver is 550, I thought it was a memory issues , them I tried LLAMA 70B , which loaded just fine , the GPU utilisation was around 15 GB each . Are there any logs collected by LM studion which I can share to help with this problem.

yagil · 2025-01-13T01:37:29Z

Thanks @aamir-gmail! We are aware of this issue and we are working on a fix. The recommendation is to stay on 0.3.5 until it's out. Get 0.3.5 from https://lmstudio.ai/download#beta-and-experimental-releases

aamir-gmail · 2025-01-13T05:25:22Z

FYI, I was able to load LLAMA 70B with 0.3.6 without a problem. I will keep you posted on how go with 0.3.5

…

On Mon, Jan 13, 2025 at 12:37 PM Yagil Burowski ***@***.***> wrote: Thanks @aamir-gmail <https://github.com/aamir-gmail>! We are aware of this issue and we are working on a fix. The recommendation is to stay on 0.3.5 until it's out. Get 0.3.5 from https://lmstudio.ai/download#beta-and-experimental-releases — Reply to this email directly, view it on GitHub <#285 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJA2ECIRZUWL2M6CV27FQXT2KMKG7AVCNFSM6AAAAABUWBROFOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOBWGAZTAMZSG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Kind Regards Aamir Mirza

aamir-gmail · 2025-01-13T06:01:25Z

Tried your link for Qwen2 VL 7B and 70B , on the download link you provided still got the same error message. Are there any logs I can send you. Let me know where to find them if you require them. (Exit code: 133). Please check the settings and try loading the model again.

…

On Mon, Jan 13, 2025 at 4:24 PM Aamir mirza ***@***.***> wrote: FYI, I was able to load LLAMA 70B with 0.3.6 without a problem. I will keep you posted on how go with 0.3.5 On Mon, Jan 13, 2025 at 12:37 PM Yagil Burowski ***@***.***> wrote: > Thanks @aamir-gmail <https://github.com/aamir-gmail>! We are aware of > this issue and we are working on a fix. The recommendation is to stay on > 0.3.5 until it's out. Get 0.3.5 from > https://lmstudio.ai/download#beta-and-experimental-releases > > — > Reply to this email directly, view it on GitHub > <#285 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AJA2ECIRZUWL2M6CV27FQXT2KMKG7AVCNFSM6AAAAABUWBROFOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOBWGAZTAMZSG4> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- Kind Regards Aamir Mirza

-- Kind Regards Aamir Mirza

alphacentrino · 2025-01-22T07:35:43Z

I'm facing with the same issue. I have no GPU, but I have 32GB ram and intel core vPRO i7. I'm using latest cersion of LMS0.3.8-4

codevski · 2025-01-24T01:33:00Z

Same issue here even if I try offload the whole thing to RAM (mave more then enough) errors out. currently just dropping down a model size for the time being. Happy to share any logs that you need.

pseudomo · 2025-01-24T14:04:40Z

Same issue on Ubuntu 24.04 with 64Gb RAM. The model I am trying to load is 34.8 Gb bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF/DeepSeek-R1-Distill-Qwen-32B-Q8_0.gguf
The version of LM Studio is 0.3.8-4

alphacentrino · 2025-01-24T15:55:57Z

I resolved this issue by doing the following, In my case I need to first install 'vulkan' tools

sudo apt install vulkan-tools
Then start LMStudio and under settings--> Runtime(second tab on left) --> Configure RunTime(on Right): set this to 'CPU llama.cpp(limux)' from drop down.
You should now see two runtimes under 'MyRuntImes'(Right Bottom), one with 'CPU llama.cpp(linux)' and another one ' Vulkan llama.cpp(linux)'

Jachyme · 2025-01-24T16:58:47Z

Same problem. 64GB RAM, GeForce RTX 3060, Linux Mint 22.1 Cinnamon, Model 'DeepSeek R1 Distill Llama 70B Q4_K_L': Error loading model. (Exit code: 133)

But with Windows 11 on the same machine, with the same model copied from the Linux folder to the Windows folder, it loads and works without problems.

Both LM Studio 0.3.8 (Build 4)

alphacentrino: your solution doesn't work on my computer.

yagil · 2025-01-24T17:00:04Z

We are working on a fix, sorry for the wait. In the meantime, 0.3.5 should work: https://lmstudio.ai/download#beta-and-experimental-releases

CHesketh76 · 2025-01-24T17:37:51Z

@yagil O.3.5 has the same issue for me. I have two desktops and same issue occurs on both machines with 0.3.5 and the recent release.

alphacentrino · 2025-01-24T23:19:47Z

Hi Jachyme,
mine is Ubuntu 20.04. After starting your LM studio, try loading the model, then as soon as you get error in LM studio, go back to command line and see the logs and you should be able to find what it's looking for. u can share your logs here, may be someone can help.

dynamiccreator · 2025-01-25T22:21:21Z

I have the same issue with ubuntu 22.04 (128GB RAM) (4GB VRAM).

However it works in 0.3.5 (Build 2) if I use:

CPU runtime from the settings
set offload layers explicitly (bin symbol appears) to 0
(GPU runtime might also work with this setting)
reduce batch evaluation size to 128 or lower

The trick with the reduce batch evaluation size shouldn't even be necessary as OP pointed out it all works with vanilla llama.cpp.

BUT:

However going to newer version (0.3.7 and 0.3.8) to use the R1 Distill models for example , the "trick" with the reduced batch eval size does not work anymore and it tells me:

🥲 Failed to load the model

Error loading model.

(Exit code: 133). Please check settings and try loading the model again.

It loads the model and then at 10GB RAM usage or so the error occurs.

EDIT:

BUG LOGS:
With error (when context is higher, like even more than 5000 ) :

[ModelLoadingProvider] Requested to load model bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF/DeepSeek-R1-Distill-Qwen-14B-Q4_K_M.gguf with opts {
identifier: { desired: 'deepseek-r1-distill-qwen-14b', conflictBehavior: 'bump' },
instanceLoadTimeConfig: { fields: [ [Object], [Object], [Object], [Object] ] }
}
[ModelLoadingProvider] Started loading model bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF/DeepSeek-R1-Distill-Qwen-14B-Q4_K_M.gguf
[ModelProxyObject(id=deepseek-r1-distill-qwen-14b)] Forking LLMWorker with custom envVars: {}
22:59:31.212 › [LMSInternal][Client=LM Studio][Endpoint=loadModel] Error in channel handler: Error: Error loading model.
at _0x5bd2a1._0xf4108d (/tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:287003)
at _0x5bd2a1.emit (node:events:519:28)
at _0x5bd2a1.onChildExit (/tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:243719)
at ForkUtilityProcess. (/tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:243035)
at ForkUtilityProcess.emit (node:events:519:28)
at ForkUtilityProcess.a.emit (node:electron/js2c/browser_init:2:71438)
[LMSInternal][Client=LM Studio][Endpoint=loadModel] Error in loadModel channel _0x548ef3 [Error]: Error loading model.
at _0x5bd2a1._0xf4108d (/tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:287003)
at _0x5bd2a1.emit (node:events:519:28)
at _0x5bd2a1.onChildExit (/tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:243719)
at ForkUtilityProcess. (/tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:243035)
at ForkUtilityProcess.emit (node:events:519:28)
at ForkUtilityProcess.a.emit (node:electron/js2c/browser_init:2:71438) {
cause: '(Exit code: 133). Please check settings and try loading the model again. ',
suggestion: '',
errorData: undefined,
data: undefined,
displayData: undefined,
title: 'Error loading model.'
}
22:59:31.212 › [LMSInternal][Client=LM Studio][Endpoint=loadModel] No instance reference assigned before error
22:59:32.214 › [LMSInternal][Client=LM Studio][Endpoint=countTokens] Error in RPC handler: Error: Model is unloaded.
at /tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:283735
22:59:32.214 › [LMSInternal][Client=LM Studio][Endpoint=countTokens] Error in RPC handler: Error: Model is unloaded.
at /tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:283735
22:59:32.214 › [LMSInternal][Client=LM Studio][Endpoint=countTokens] Error in RPC handler: Error: Model is unloaded.
at /tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:283735
22:59:32.214 › Unhandled Rejection at: {} reason: Error: Model is unloaded.
at /tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:283735
22:59:32.214 › Unhandled Rejection at: {} reason: Error: Model is unloaded.
at /tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:283735
22:59:32.214 › Unhandled Rejection at: {} reason: Error: Model is unloaded.
at /tmp/.mount_LM-StuIorlnq/resources/app/.webpack/main/index.js:24:283735

With low context no problem:

[ModelLoadingProvider] Requested to load model bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF/DeepSeek-R1-Distill-Qwen-14B-Q4_K_M.gguf with opts {
identifier: { desired: 'deepseek-r1-distill-qwen-14b', conflictBehavior: 'bump' },
instanceLoadTimeConfig: { fields: [ [Object], [Object], [Object], [Object] ] }
}
[ModelLoadingProvider] Started loading model bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF/DeepSeek-R1-Distill-Qwen-14B-Q4_K_M.gguf
[ModelProxyObject(id=deepseek-r1-distill-qwen-14b)] Forking LLMWorker with custom envVars: {}

aamir-gmail · 2025-01-26T00:24:26Z

Fyi the model Qwen2-VL-72B-Instruct 7B or 72B does not work any LM studio release 3.5 to 3.8, to recap my system is 2x 3090, 256 GB RAm ubuntu 22.04 . As as suggestion please enable detailed logging so we can understand where and what exactly is failing and offer fix.

…

On Sun, Jan 26, 2025 at 9:21 AM dynamiccreator ***@***.***> wrote: I have the same issue with ubuntu 22.04 (128GB RAM) (4GB VRAM). However it works in 0.3.5 (Build 2) if I use: - CPU runtime from the settings - set offload layers explicitly (bin symbol appears) to 0 (GPU runtime might also work with this setting) - reduce batch evaluation size to 128 or lower The trick with the reduce batch evaluation size shouldn't even be necessary as OP pointed out it all works with vanilla llama.cpp. However going to newer version (0.3.7 and 0.3.8) to use the R1 Distill models for example , the "trick" with the reduced batch eval size does not work anymore and it tells me: 🥲 Failed to load the model Error loading model. (Exit code: 133). Please check settings and try loading the model again. It loads the model and then at 10GB RAM usage or so the error occurs. — Reply to this email directly, view it on GitHub <#285 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJA2ECL33Q55RJ3PCQJI4JD2MQE7RAVCNFSM6AAAAABUWBROFOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMJUGEYTQMBZGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Kind Regards Aamir Mirza

dynamiccreator · 2025-01-26T22:08:21Z

I have edited my comment and added logs to my description. I hope these are the log messages you need.

nukopal · 2025-01-28T06:48:43Z

I have met same situation/errors, that says "(Exit code: 133). Please check settings and try loading the model again. ".
I will attach error logs, I hope help you to solve this problem.

error logs

in case of error occurs

[CachedFileDataProvider] Watching file at /home/nukopal-local/.lmstudio/conversations/1738044409708.conversation.json
[LMSInternal][Client=LM Studio][Endpoint=unloadModel] Unloading model KnNQkjVXd68ouJoIz5i1KQDU
Unloading model: KnNQkjVXd68ouJoIz5i1KQDU
15:46:42.668 › [LMSInternal][Client=LM Studio][Endpoint=countTokens] Error in RPC handler: Error: Cannot find model of instance reference. KnNQkjVXd68ouJoIz5i1KQDU
    at _0x6f0a0f.<computed>.getInstanceBySpecifierOrThrow (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:63:12348)
    at _0x6f0a0f.<computed>.getLLMModelBySpecifierOrThrow (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:63:14931)
    at Object.handler (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:68:19977)
    at _0x546cd8.<computed>.receivedRpcCall (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:365:147)
    at _0x41a188.receivedMessage (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:338:13495)
    at MessagePortMain.<anonymous> (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:338:11926)
    at MessagePortMain.emit (node:events:519:28)
    at MessagePortMain._internalPort.emit (node:electron/js2c/browser_init:2:105846)
[CachedFileDataProvider] Watching file at /home/nukopal-local/.lmstudio/.internal/user-concrete-model-default-config/.json
[CachedFileDataProvider] Watching file at /home/nukopal-local/.lmstudio/.internal/user-concrete-model-default-config/lmstudio-community/DeepSeek-R1-Distill-Llama-70B-GGUF/DeepSeek-R1-Distill-Llama-70B-Q4_K_M.gguf.json
[ModelLoadingProvider] Requested to load model lmstudio-community/DeepSeek-R1-Distill-Llama-70B-GGUF/DeepSeek-R1-Distill-Llama-70B-Q4_K_M.gguf with opts {
  identifier: {
    desired: 'deepseek-r1-distill-llama-70b',
    conflictBehavior: 'bump'
  },
  instanceLoadTimeConfig: { fields: [] }
}
[CachedFileDataProvider] Watching file at /home/nukopal-local/.lmstudio/.internal/user-concrete-model-default-config/lmstudio-community/DeepSeek-R1-Distill-Llama-70B-GGUF/DeepSeek-R1-Distill-Llama-70B-Q4_K_M.gguf.json
[ModelLoadingProvider] Started loading model lmstudio-community/DeepSeek-R1-Distill-Llama-70B-GGUF/DeepSeek-R1-Distill-Llama-70B-Q4_K_M.gguf
[ModelProxyObject(id=deepseek-r1-distill-llama-70b)] Forking LLMWorker with custom envVars: {}
15:47:03.426 › [LMSInternal][Client=LM Studio][Endpoint=loadModel] Error in channel handler: Error: Error loading model.
    at _0x5bd2a1._0xf4108d (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:287003)
    at _0x5bd2a1.emit (node:events:519:28)
    at _0x5bd2a1.onChildExit (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:243719)
    at ForkUtilityProcess.<anonymous> (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:243035)
    at ForkUtilityProcess.emit (node:events:519:28)
    at ForkUtilityProcess.a.emit (node:electron/js2c/browser_init:2:71438)
[LMSInternal][Client=LM Studio][Endpoint=loadModel] Error in loadModel channel _0x548ef3 [Error]: Error loading model.
    at _0x5bd2a1._0xf4108d (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:287003)
    at _0x5bd2a1.emit (node:events:519:28)
    at _0x5bd2a1.onChildExit (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:243719)
    at ForkUtilityProcess.<anonymous> (/home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:243035)
    at ForkUtilityProcess.emit (node:events:519:28)
    at ForkUtilityProcess.a.emit (node:electron/js2c/browser_init:2:71438) {
  cause: '(Exit code: 133). Please check settings and try loading the model again. ',
  suggestion: '',
  errorData: undefined,
  data: undefined,
  displayData: undefined,
  title: 'Error loading model.'
}
15:47:03.430 › [LMSInternal][Client=LM Studio][Endpoint=loadModel] No instance reference assigned before error
15:47:04.426 › [LMSInternal][Client=LM Studio][Endpoint=countTokens] Error in RPC handler: Error: Model is unloaded.
    at /home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:283735
15:47:04.427 › [LMSInternal][Client=LM Studio][Endpoint=countTokens] Error in RPC handler: Error: Model is unloaded.
    at /home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:283735
15:47:04.428 › [LMSInternal][Client=LM Studio][Endpoint=countTokens] Error in RPC handler: Error: Model is unloaded.
    at /home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:283735
15:47:04.428 › Unhandled Rejection at: {} reason: Error: Model is unloaded.
    at /home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:283735
15:47:04.429 › Unhandled Rejection at: {} reason: Error: Model is unloaded.
    at /home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:283735
15:47:04.429 › Unhandled Rejection at: {} reason: Error: Model is unloaded.
    at /home/nukopal-local/.cache/LMStudio/.mount_LM-Stu3gPhtc/resources/app/.webpack/main/index.js:24:283735

in case of successful pattern

[ModelLoadingProvider] Requested to load model lmstudio-community/DeepSeek-R1-Distill-Llama-8B-GGUF/DeepSeek-R1-Distill-Llama-8B-Q8_0.gguf with opts {
  identifier: { desired: 'deepseek-r1-distill-llama-8b', conflictBehavior: 'bump' },
  instanceLoadTimeConfig: { fields: [] },
  instanceOperationTimeConfig: { fields: [] }
}
[CachedFileDataProvider] Watching file at /home/nukopal-local/.lmstudio/.internal/user-concrete-model-default-config/lmstudio-community/DeepSeek-R1-Distill-Llama-8B-GGUF/DeepSeek-R1-Distill-Llama-8B-Q8_0.gguf.json
[ModelLoadingProvider] Started loading model lmstudio-community/DeepSeek-R1-Distill-Llama-8B-GGUF/DeepSeek-R1-Distill-Llama-8B-Q8_0.gguf
[ModelProxyObject(id=deepseek-r1-distill-llama-8b)] Forking LLMWorker with custom envVars: {}

environment

Software

OS: Ubuntu 24.04.1 LTS (Deskjtop)
Kernel version : 6.8.0-51-generic #52-Ubuntu SMP PREEMPT_DYNAMIC Thu Dec 5 13:09:44 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
LM Studio version: 0.3.8 (Build 4)
Runtime: CPU llama.cpp (Linux) v1.10.0
(Vulkan llama.cpp v1.10.0 installed, but I don't use because machine only have poor GPU )

Hardware

CPU: Intel Xeon 5218 (single)
RAM: 256GB (Pmem Memory mode, normal DRAM 64GB used as Cache)
chassis: ThinkSystem P920

dynamiccreator · 2025-01-30T10:57:55Z

I want to add that it works with no problem if all layers + cache fits into VRAM. But with 0 layers to GPU it stops loading and that is not bound to the model size but only to the context size. So even a 70B model can be loaded if I set the context to 1000 or so.

illtellyoulater · 2025-01-30T15:11:34Z

We are working on a fix, sorry for the wait. In the meantime, 0.3.5 should work: https://lmstudio.ai/download#beta-and-experimental-releases

@yagil any news on this?

By the way, latest 0.3.9 builds - both stable or beta - don't fix this and users are reporting neither the 0.3.5 you linked does.

Are there any cons to the proposed change in the Electron compilation flag discussed above, which seems to fix the issue?
Or would switching the flag interfere with other LM Studio features?

Insurgent65 · 2025-01-31T10:28:26Z

Does not seem to be solved in the new version 0.3.9

bijavix · 2025-01-31T23:19:56Z

Version 0.3.9 still has the issue. By lowering the Context Length, the model was able to load.

Msakhibullin24 · 2025-02-01T08:37:38Z

Version 0.3.9 still has the issue. By lowering the Context Length, the model was able to load.

how did you solve this problem?

I'm trying to run deepseekr1 gguf which at 20gpu would run

Msakhibullin24 · 2025-02-01T08:37:48Z

how did you solve this problem?

I'm trying to run deepseekr1 gguf which at 20gpu would run

krones9000 · 2025-02-01T08:39:27Z

Could you give a little more specific detail about exactly how you resolved this? I'm in a similar situation, on linux, 16gbVRAM 128GB system RAM, could load models in Oobabooga but getting the 133 error in LM studio. Not sure where to start or how I would "build" Electron or what Electron is and how it relates to the Linux app image of LM Studio that I load.

For what it's worth, I updated to the latest beta and that solved it for me.

Insurgent65 · 2025-02-01T16:15:54Z

I have not been able to get it to work with any version or any of the tricks mentioned.

The user interface of the application is the best of all I've tried, it makes me very angry that I can't use it.

mxfeinberg · 2025-02-02T00:15:20Z

Electron's Build Instructions

I was running into this same issue over the last couple of days when attempting to use DeepSeek-R1 GGUF. This approach worked for me. I am now able to load the 300GB+ versions of the model.

piotrkosecki · 2025-02-04T09:23:40Z

0.3.9 build 6, issue still exists...

fpatrick · 2025-02-04T17:45:48Z

I have same the problem on 0.3.9

ellavs · 2025-02-05T12:46:34Z

the same problem on 0.3.9, model qwen2.5-coder-7b-instruct. Ubuntu 20.04. Error:

Failed to load the model. Exit code 133. Please check settings and try loading the model again

kP700c-github · 2025-02-05T18:15:20Z

Update about models loaded successfully and unsuccessfully with LM Studio 0.3.9 build 6 (Linux Mint 22, AMD Ryzen 7 5700G, 64 GB RAM):

unsuccessful:
- DeepSeek-R1-Distill-Qwen-32B-Q6_K.gguf
- DeepSeek-R1-Distill-Qwen-32B-Q3_K_L.gguf
- DeepSeek-Coder-V2-Lite-Instruct-Q8_0.gguf
- FuseO1-DeekSeekR1-QwQ-SkyT1-32B-Preview-Q4_K_M.gguf
- gemma-2-27b-it-Q4_K_M.gguf
- gemma-2-9b-it-Q8_0.gguf
- QwQ-32B-Preview-IQ3_M.gguf
- qwen2.5-coder-32b-instruct-q4_0.gguf
- qwen2.5-coder-14b-instruct-q4_0.gguf
- Qwen2-VL-7B-Instruct-Q8_0.gguf+mmproj-Qwen2-VL-7B-Instruct-f16.gguf
- Llama-3.3-70B-Instruct-Q3_K_L.gguf
successful:
- DeepSeek-R1-Distill-Qwen-7B-Q8_0.gguf
- DeepSeek-R1-Distill-Qwen-14B-Q8_0.gguf
- Qwen2.5-7B-Instruct-Q8_0.gguf
- Qwen2.5-7B-Instruct-1M-Q8_0.gguf
- Qwen2.5-14B-Instruct-Q8_0.gguf
- Qwen2.5-14B-Instruct-1M-Q8_0.gguf
- Qwen2.5-Math-1.5B-Instruct-Q8_0.gguf
- Dolphin3.0-Llama3.1-8B-Q8_0.gguf
- phi-4-Q8_0.gguf
- Yi-Coder-9B-Chat-Q8_0.gguf
- Llama-3.2-4X3B-MOE-Ultra-Instruct-10B-D_AU-Q8_0.gguf
- MiniCPM-V-2_6-Q8_0.gguf
- MiniCPM-o-2_6-Q8_0.gguf
- Mistral-Nemo-Instruct-2407-Q5_K_S.gguf
- granite-3.1-8b-instruct-Q4_K_M.gguf
- Codestral-22B-v0.1-Q4_K_M.gguf
- gemma-2-2b-it-Q5_K_M.gguf
- llama-3.2-3b-instruct-q8_0.gguf

yagil assigned neilmehta24 Jan 6, 2025

yagil added the bug Something isn't working label Jan 6, 2025

yagil changed the title ~~Error when loading large LLM models~~ (Exit code 133) Error when loading large LLM models Jan 6, 2025

yagil marked this as a duplicate of lmstudio-ai/lms#136 Jan 17, 2025

yagil mentioned this issue Jan 17, 2025

Failed to load the model lmstudio-ai/lms#136

Closed

yagil marked this as a duplicate of #322 Jan 18, 2025

yagil mentioned this issue Jan 18, 2025

Qwen2-VL not load in AMD Linux #322

Closed

yagil changed the title ~~(Exit code 133) Error when loading large LLM models~~ [Linux] (Exit code 133) Error when loading large LLM models Jan 24, 2025

yagil pinned this issue Jan 27, 2025

yagil unpinned this issue Jan 27, 2025

yagil pinned this issue Jan 27, 2025

dynamiccreator mentioned this issue Jan 28, 2025

does not load a model if context size is "too big" lmstudio-ai/lms#111

Open

[Linux] (Exit code 133) Error when loading large LLM models #285

[Linux] (Exit code 133) Error when loading large LLM models #285

Comments

yushijinhun commented Jan 6, 2025

yagil commented Jan 6, 2025

yushijinhun commented Jan 6, 2025

yagil commented Jan 6, 2025

yushijinhun commented Jan 7, 2025

krones9000 commented Jan 7, 2025

laushunyu commented Jan 7, 2025

yushijinhun commented Jan 7, 2025

daxime commented Jan 7, 2025

CHesketh76 commented Jan 10, 2025 • edited Loading

yagil commented Jan 10, 2025

CHesketh76 commented Jan 11, 2025

aamir-gmail commented Jan 13, 2025

yagil commented Jan 13, 2025

aamir-gmail commented Jan 13, 2025 via email

aamir-gmail commented Jan 13, 2025 via email

alphacentrino commented Jan 22, 2025

codevski commented Jan 24, 2025

pseudomo commented Jan 24, 2025

alphacentrino commented Jan 24, 2025 • edited Loading

Jachyme commented Jan 24, 2025

yagil commented Jan 24, 2025

CHesketh76 commented Jan 24, 2025

alphacentrino commented Jan 24, 2025

dynamiccreator commented Jan 25, 2025 • edited Loading

aamir-gmail commented Jan 26, 2025 via email

dynamiccreator commented Jan 26, 2025 • edited Loading

nukopal commented Jan 28, 2025

error logs

in case of error occurs

in case of successful pattern

environment

Software

Hardware

dynamiccreator commented Jan 30, 2025

illtellyoulater commented Jan 30, 2025 • edited Loading

Insurgent65 commented Jan 31, 2025

bijavix commented Jan 31, 2025

Msakhibullin24 commented Feb 1, 2025

Msakhibullin24 commented Feb 1, 2025

krones9000 commented Feb 1, 2025

Insurgent65 commented Feb 1, 2025

mxfeinberg commented Feb 2, 2025

piotrkosecki commented Feb 4, 2025

fpatrick commented Feb 4, 2025

ellavs commented Feb 5, 2025

kP700c-github commented Feb 5, 2025

CHesketh76 commented Jan 10, 2025 •

edited

Loading

alphacentrino commented Jan 24, 2025 •

edited

Loading

dynamiccreator commented Jan 25, 2025 •

edited

Loading

dynamiccreator commented Jan 26, 2025 •

edited

Loading

illtellyoulater commented Jan 30, 2025 •

edited

Loading