-
-
Notifications
You must be signed in to change notification settings - Fork 192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FATAL ERROR]: JavaScript heap out of memory #365
Comments
Thanks for putting this together. Unfortunately there's not much I can do from this information alone. I need a reproduction. Things you could try:
Something I want to try: One thing Knip does is some sort of "workspace merging", based on |
I've added an undocumented |
I tried with
It works ✅
Obviously I can't share the company projects but I am at your disposal for testing 😉
Same error ❌
|
Bummer... but thanks a lot for trying!
It should be reliable, you can read about it here: https://knip.dev/features/monorepos-and-workspaces/#lint-a-single-workspace |
I faced such problem too. Somehow process of knip does not receive new value of NODE_OPTIONS. You can check it in your script with But this works:
|
Same error
|
Thanks for chiming in, @onskulit! I was wondering if it's possible there's a potential circular dependency causing infinite recursion or otherwise recursive calls exhausting memory. I have done some work and try to prevent that from happening, but could be buggy or incomplete. E.g. #359 was opened because of a "RangeError: Maximum call stack size exceeded" Would be great if anyone could look into that, to rule out that the high memory usage is caused by infinite recursion. I mean, if raising the memory for the process does not help, there might be recursive calls going on (probably in the context of multiple workspaces) that leads to the fatal error. I'm not sure how to try and reproduce it myself. |
@ild0tt0re Any chance you could try this for me?
Would there be parts of the output you could share (redacted)? I'm especially interested to see how many "programs" Knip is using (the debug lines with |
Same error:
|
Thanks a lot, how many workspaces do you have in your monorepo? Is it actually 100 or more? |
There are 106 projects:
|
@webpro if can be helpful below you can find the Nodejs memory profiling: Sampling profile:-> Heap-20231208T173058.heapprofile.zip How to load the file:
Preview |
Thanks for all the help, @ild0tt0re! It was really helpful. This issue was one of the reasons I was motivated even more to work on lowering memory usage and improving performance. I wrote about it (at https://knip.dev/blog/slim-down-to-speed-up/) and you can install There's a lot more plans in this area, so even if it doesn't work out today it might be soon. E.g. caching and recovery should be helpful features as well. Also, the size of projects like yours might give false positives, but at least, hopefully, there will be some useful output. (By the way, I couldn't find anything related to Knip in the memory heap profile, there's no matches when I grep for e.g. "knip" or even "typescript". It all seems related to npm cli. I might be misunderstanding the purpose, though.) |
Hey @webpro you did it! 🏆 👏 👏 👏 👏 👏
I cant't wait! A reliable and high-performance tool like this is exactly the missing piece in the JS/TS community
Thanks for the check! |
I'm so glad to hear this! Thanks for all the feedback, let's rock on 🚀 |
Description
I tried Knip in different company projects (Nx integrated-monorepo based). In the small one it works correctly and we were able to leverage the potential of the tool 🚀 but where it would be most useful like in large monorepos (with
34_875
files or more) It fails forJavaScript heap out of memory
Command
Tried with both, knip latest
2.43.0
$ pnpm exec knip
and the canary version
3.0.0-canary.4
the result is the same:
Output
Software Overview
2.43.0
or3.0.0-canary.4
17.0.3
20.9.0
14.1.1
Hardware Overview:
MacBook Pro
Mac14,7
Apple M2
8
(4 performance and 4 efficiency)16 GB
The text was updated successfully, but these errors were encountered: