Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

npm install fails as node-gyp is unable to build lmdb #8152

Closed
rincewindcz opened this issue May 27, 2022 · 35 comments
Closed

npm install fails as node-gyp is unable to build lmdb #8152

rincewindcz opened this issue May 27, 2022 · 35 comments

Comments

@rincewindcz
Copy link

🐛 bug report

When trying to install parcel into npm project, npm install --save-dev parcel fails with exit code 7, complaining about node-gyp failing to build lmdb package.

🎛 Log

$ npm install --save-dev parcel
npm ERR! code 7
npm ERR! path /tmp/parcel-bug/node_modules/lmdb
npm ERR! command failed
npm ERR! command sh -c node-gyp-build-optional-packages
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | linux | x64
npm ERR! gyp info find Python using Python version 3.10.4 found at "/usr/bin/python3"
npm ERR! gyp ERR! UNCAUGHT EXCEPTION 
npm ERR! gyp ERR! stack Error: Cannot find module 'request'
npm ERR! gyp ERR! stack Require stack:
npm ERR! gyp ERR! stack - /usr/lib/node_modules/npm/node_modules/node-gyp/lib/install.js
npm ERR! gyp ERR! stack - /usr/lib/node_modules/npm/node_modules/node-gyp/lib/node-gyp.js
npm ERR! gyp ERR! stack - /usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js
npm ERR! gyp ERR! stack     at Module._resolveFilename (node:internal/modules/cjs/loader:939:15)
npm ERR! gyp ERR! stack     at Module._load (node:internal/modules/cjs/loader:780:27)
npm ERR! gyp ERR! stack     at Module.require (node:internal/modules/cjs/loader:1005:19)
npm ERR! gyp ERR! stack     at require (node:internal/modules/cjs/helpers:102:18)
npm ERR! gyp ERR! stack     at Object.<anonymous> (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/install.js:10:17)
npm ERR! gyp ERR! stack     at Module._compile (node:internal/modules/cjs/loader:1105:14)
npm ERR! gyp ERR! stack     at Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
npm ERR! gyp ERR! stack     at Module.load (node:internal/modules/cjs/loader:981:32)
npm ERR! gyp ERR! stack     at Module._load (node:internal/modules/cjs/loader:827:12)
npm ERR! gyp ERR! stack     at Module.require (node:internal/modules/cjs/loader:1005:19)
npm ERR! gyp ERR! System Linux 5.17.9-arch1-1
npm ERR! gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd /tmp/parcel-bug/node_modules/lmdb
npm ERR! gyp ERR! node -v v18.2.0
npm ERR! gyp ERR! node-gyp -v v7.1.2
npm ERR! gyp ERR! Node-gyp failed to build your package.
npm ERR! gyp ERR! Try to update npm and/or node-gyp and if it does not help file an issue with the package author.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/myawesomeusername/.npm/_logs/2022-05-27T10_20_36_304Z-debug-0.log

🌍 Your Environment

Software Version(s)
Node v18.2.0
npm 8.5.5
Operating System Arch Linux / x64
@alaksandarjesus
Copy link

+1

@mixaildudin
Copy link

mixaildudin commented May 28, 2022

Cannot install the new parcel on two machines with Win10, tried with Node 14.16.0, 14.16.1 and 16.15.0.

On one machine npm says it cannot find Python. On the other one it says it cannot find MSVS, and all because of node-gyp and lmdb.
Meanwhile, [email protected] works ok.

I hope 2.5.0 is not the last version to work on both of my computers. Cause it would be really uncomfortable to install Python, install MSVS and all the necessary build tools for Parcel which used to work like a charm just several days ago, and the main advantage of Parcel is that it's supposed to work just out of the box!

@mixaildudin
Copy link

Here's what I have:

npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node-gyp-build-optional-packages
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | win32 | x64
npm ERR! gyp info find Python using Python version 3.7.2 found at "C:\Users\asdf\AppData\Local\Programs\Python\Python37\python.exe"
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.0/node-v16.15.0-headers.tar.gz
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.0/node-v16.15.0-headers.tar.gz
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.0/SHASUMS256.txt
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.0/win-x64/node.lib
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.0/win-arm64/node.lib
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.0/win-x86/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.0/SHASUMS256.txt
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.0/win-x64/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.0/win-x86/node.lib
npm ERR! gyp http 404 https://nodejs.org/download/release/v16.15.0/win-arm64/node.lib
npm ERR! gyp ERR! find VS
npm ERR! gyp ERR! find VS msvs_version not set from command line or npm config
npm ERR! gyp ERR! find VS VCINSTALLDIR not set, not running in VS Command Prompt
npm ERR! gyp ERR! find VS unknown version "undefined" found at "C:\Program Files\Microsoft Visual Studio\2022\Community"npm ERR! gyp ERR! find VS could not find a version of Visual Studio 2017 or newer to use
npm ERR! gyp ERR! find VS looking for Visual Studio 2015
npm ERR! gyp ERR! find VS - not found
npm ERR! gyp ERR! find VS not looking for VS2013 as it is only supported up to Node.js 8
npm ERR! gyp ERR! find VS
npm ERR! gyp ERR! find VS **************************************************************
npm ERR! gyp ERR! find VS You need to install the latest version of Visual Studio
npm ERR! gyp ERR! find VS including the "Desktop development with C++" workload.
npm ERR! gyp ERR! find VS For more information consult the documentation at:
npm ERR! gyp ERR! find VS https://github.com/nodejs/node-gyp#on-windows
npm ERR! gyp ERR! find VS **************************************************************
npm ERR! gyp ERR! find VS
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: Could not find any Visual Studio installation to use
npm ERR! gyp ERR! stack     at VisualStudioFinder.fail (C:\Users\asdf\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:121:47)
npm ERR! gyp ERR! stack     at C:\Users\asdf\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:74:16
npm ERR! gyp ERR! stack     at VisualStudioFinder.findVisualStudio2013 (C:\Users\asdf\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:351:14)
npm ERR! gyp ERR! stack     at C:\Users\asdf\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:70:14
npm ERR! gyp ERR! stack     at C:\Users\asdf\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\find-visualstudio.js:372:16
npm ERR! gyp ERR! stack     at C:\Users\asdf\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\util.js:54:7
npm ERR! gyp ERR! stack     at C:\Users\asdf\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\util.js:33:16
npm ERR! gyp ERR! stack     at ChildProcess.exithandler (node:child_process:406:5)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:527:28)
npm ERR! gyp ERR! stack     at maybeClose (node:internal/child_process:1092:16)
npm ERR! gyp ERR! System Windows_NT 10.0.19043
npm ERR! gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\asdf\\AppData\\Roaming\\npm\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd C:\Users\asdf\Documents\react-app\node_modules\lmdb
npm ERR! gyp ERR! node -v v16.15.0
npm ERR! gyp ERR! node-gyp -v v7.1.2
npm ERR! gyp ERR! not ok

npm ERR! A complete log of this run can be found in:

@mixaildudin
Copy link

mixaildudin commented May 28, 2022

Okay, I resolved my issue.
I already had VS 2022 Community installed, so I ran the VS installer, checked Node.js tools (and also checked C++ tools for Node.js or something like this), it downloaded 3Gb of data from the internet, now [email protected] installs smoothly.

But this is still lame.
Looks like someone forgot to include their pre-compiled binaries into the package or something.

@devongovett
Copy link
Member

cc @kriszyp

@kriszyp
Copy link
Contributor

kriszyp commented May 28, 2022

The lmdb package definitely has precompiled binaries for these OS/architectures and so you should not be needing to locally compiling binaries with python, MSVS, gcc, or anything else. If these are not present, it seems like it may be an issue with the installation not working right, an incompatible package override, or the binaries being deleted or somehow corrupted.

I would check to make sure your binaries have been properly installed by NPM first. @rincewindcz, your binary should be at /tmp/parcel-bug/node_modules/lmdb-linux-x64/node.abi108.node (from lmdb-linux-x64). @mixaildudin , you binary should be at C:\Users\asdf\Documents\react-app\node_modules\lmdb-win32-x64\node.abi93.node (from lmdb-win32-x64). (If anyone else wants to know the specific location to check, let me know).

If the binary is there, the next thing I would try is to force the npm install to not using any locally compilation by using --ignore-scripts and/or making sure that node_modules/lmdb/build folder is deleted/gone and then try loading lmdb with `node -e "console.log(require('lmdb'))". If there is some binary corruption or CPU incompatibility issue at the OS level, this should report it.

One other thing to check is to make sure that there was no weird package version override and that the correct versions are with npm ls lmdb node-gyp-build-optional-packages, which should have [email protected] and [email protected]. If either of those are wrong, naturally that would cause problems.

Hopefully that helps isolate the issue, certainly let me know if you have any questions.

@yume-chan
Copy link

yume-chan commented May 28, 2022

NPM will rollback failed installation, so --ignore-scripts is required to see package content in the first place.

node-gyp-build-optional-packages script invokes node-gyp-build-test, but node-gyp-build package (required by @parcel/[email protected]) also provides a node-gyp-build-test script, so the prebuilt binary was not detected.

node_modules/.bin/node-gyp-build-test:

#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")

case `uname` in
    *CYGWIN*|*MINGW*|*MSYS*) basedir=`cygpath -w "$basedir"`;;
esac

if [ -x "$basedir/node" ]; then
  exec "$basedir/node"  "$basedir/../node-gyp-build/build-test.js" "$@"
else 
  exec node  "$basedir/../node-gyp-build/build-test.js" "$@"
fi
$ npm ls node-gyp-build
[email protected] <redacted>/test
└─┬ [email protected]
  └─┬ @parcel/[email protected]
    └─┬ @parcel/[email protected]
      └── [email protected]

run npm install --verbose parcel:

npm ERR! code 1
npm ERR! path <redacted>\node_modules\lmdb
npm ERR! command failed
npm ERR! command C:\WINDOWS\system32\cmd.exe /d /s /c node-gyp-build-optional-packages
npm ERR! <redacted>\node_modules\node-gyp-build\index.js:60
npm ERR!   throw new Error('No native build was found for ' + target + '\n    loaded from: ' + dir + '\n')
npm ERR!   ^
npm ERR!
npm ERR! Error: No native build was found for platform=win32 arch=x64 runtime=node abi=93 uv=1 libc=glibc node=16.13.1
npm ERR!     loaded from: <redacted>\node_modules\lmdb
npm ERR!
npm ERR!     at Function.load.path (<redacted>\node_modules\node-gyp-build\index.js:60:9)
npm ERR!     at load (<redacted>\node_modules\node-gyp-build\index.js:22:30)
npm ERR!     at Object.<anonymous> (<redacted>\node_modules\node-gyp-build\build-test.js:19:19)
npm ERR!     at Module._compile (node:internal/modules/cjs/loader:1101:14)
npm ERR!     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
npm ERR!     at Module.load (node:internal/modules/cjs/loader:981:32)
npm ERR!     at Function.Module._load (node:internal/modules/cjs/loader:822:12)
npm ERR!     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
npm ERR!     at node:internal/main/run_main_module:17:47

Notice the stacktrace is fromnode-gyp-build package.

PS. so funny that it only recognizes npm install --verbose, but not npm install -ddd

@kriszyp
Copy link
Contributor

kriszyp commented May 28, 2022

Hmm, this is an interesting and helpful observation @yume-chan . And indeed, when I run node_modules./.bin/node-gyp-build-optional-packages from the lmdb directory, I get this same stack trace. However, if I run npm run install from the lmdb directory, I do not see this error. I believe npm passes in an environmental variables (there are a lot, so not sure which), that allows it to resolve node-gyp-build-test "locally" to the (correct) one in node_modules/lmdb/node_modules/node-gyp-build-optional-packages/.bin/. And this is the script/env that should actually execute during the install process. And indeed that is what I see on all the computers I have tested with (no errors or compilation).

So IIUC, the possibility that you are suggesting is that on some computer/configuration these paths are not being passed correctly by npm, and it is causing the wrong node-gyp-build-test to be executed, and therefore no binary to be detected, and this triggers the build/compilation process (and I would assume this is an unusual configuration, since I presume that hundreds or thousands of users have installed parcel without issue). And if this is the case, that means that the prebuilt binary should actually be present, and as long as the compilation is avoided (--ignore-scripts), the prebuilt binaries should be properly loaded and work fine (node-gyp-build-optional-packages itself should work fine).

And if that is the case (and the users above do actually have the correct native binaries), the issue is about fixing the check for the prebuilt binaries to ensure no mixups in the test script. And I believe I could update node-gyp-build-optional-packages to use its own, unambiguous node-gyp-build-optional-packages-test script to protect against npm lookup issues.

@yume-chan
Copy link

yume-chan commented May 28, 2022

I believe npm passes in an environmental variables

I think it just resolves to local node_modules first.

to the (correct) one in node_modules/lmdb/node_modules/node-gyp-build-optional-packages/.bin/

My node-gyp-build-optional-packages is hoisted to the root node_modules folder. If I add node-gyp-build-optional-packages@5 to my package.json, thus no hosting, the install script of lmdb does work. However, with NPM@7 and NPM@8 they both hoisted it when parcel is the only dependency in package.json.

that means that the prebuilt binary should actually be present

Yes, with --ignore-scripts, I can require lmdb in Node REPL without problem.

kriszyp added a commit to kriszyp/node-gyp-build that referenced this issue May 28, 2022
@kriszyp
Copy link
Contributor

kriszyp commented May 28, 2022

Ok, I published an updated node-gyp-build-optional-packages (patch version, should be semver compatible and automatically used). Hopefully this should fix this issue. I can't reproduce this issue, so I can't verify it, but at least nothing seems broken from my testing.

@asmockler
Copy link

asmockler commented May 29, 2022

Thanks to everyone who has put work in on this already, especially Kris for publishing that update. Unfortunately, I'm still getting this error.

yarn add parcel always seems to fetch [email protected] for me. I added

"resolutions": {
  "node-gyp-build-optional-packages": "4.3.4"
}

to my package.json, which got it to pull 4.3.4, but I still wound up with an error. Here is the message from yarn add parcel --verbose:

[4/4] 🔨  Building fresh packages...
warning Error running install script for optional dependency: "<project>/node_modules/msgpackr-extract: Command failed.
Exit code: 127
Command: node-gyp-build-optional-packages
Arguments: 
Directory: <project>/node_modules/msgpackr-extract
Output:
env: node\\r: No such file or directory"
info This module is OPTIONAL, you can safely ignore this error
verbose 18.347376732 Error: <project>/node_modules/lmdb: Command failed.
Exit code: 127
Command: node-gyp-build-optional-packages
Arguments: 
Directory: <project>/node_modules/lmdb
Output:
env: node\r: No such file or directory
    at ProcessTermError.ExtendableBuiltin (~/.nvm/versions/node/v16.15.0/lib/node_modules/yarn/lib/cli.js:721:66)
    at ProcessTermError.MessageError (~/.nvm/versions/node/v16.15.0/lib/node_modules/yarn/lib/cli.js:750:123)
    at new ProcessTermError (~/.nvm/versions/node/v16.15.0/lib/node_modules/yarn/lib/cli.js:790:113)
    at ChildProcess.<anonymous> (~/.nvm/versions/node/v16.15.0/lib/node_modules/yarn/lib/cli.js:25783:17)
    at ChildProcess.emit (node:events:527:28)
    at maybeClose (node:internal/child_process:1092:16)
    at Socket.<anonymous> (node:internal/child_process:451:11)
    at Socket.emit (node:events:527:28)
    at Pipe.<anonymous> (node:net:709:12)
error <project>/node_modules/lmdb: Command failed.
Exit code: 127
Command: node-gyp-build-optional-packages
Arguments: 
Directory: <project>/node_modules/lmdb
Output:

Edit: I should note that yarn add [email protected] succeeds

@kriszyp
Copy link
Contributor

kriszyp commented May 29, 2022

Ok, trying again with 4.3.5, hopefully the carriage returns are gone now.

@yume-chan
Copy link

4.3.5 is working for me.

@mixaildudin
Copy link

@kriszyp The good news is, your update of node-gyp-build-optional-packages seems to have fixed the issue for me, [email protected] installs successfully on both of my computers, even the one with no python and no MSVS.

Thanks a lot!

@kriszyp
Copy link
Contributor

kriszyp commented May 29, 2022

Great to hear, thank you @yume-chan for actually figuring out the issue!

kriszyp added a commit to kriszyp/node-gyp-build that referenced this issue May 29, 2022
@asmockler
Copy link

Just chiming in to say that 4.3.5 fixed it for me, too. Thanks so much!

@rincewindcz
Copy link
Author

Yes, I can confirm that now it's working too. Thanks to everybody involved!

@CarlosAzpurua
Copy link

Yes, I can confirm that it works now too. Thanks, very helpful

@srishilesh
Copy link

Hey @kriszyp ,

4.3.5 doesn't seem to work for me.

On running this command:

npm i [email protected]

Facing the below error:

> [email protected] install /home/srishilesh/GitHub/Gatsby/node_modules/msgpackr-extract
> node-gyp-build-optional-packages

make: Entering directory '/home/srishilesh/GitHub/Gatsby/node_modules/msgpackr-extract/build'
  CXX(target) Release/obj.target/extract/src/extract.o
../src/extract.cpp: In member function ‘napi_value__* Extractor::extractStrings(napi_env, int, int, uint8_t*)’:
../src/extract.cpp:165:121: error: no matching function for call to ‘v8::Array::New(v8::Isolate*, v8::Local<v8::Value>*, int&)’
  165 |   v8::Local<v8::Array> v8Array = v8::Array::New(v8::Isolate::GetCurrent(), (v8::Local<v8::Value>*) target, writePosition);
      |                                                                                                                         ^
In file included from ../src/extract.cpp:10:
/usr/include/nodejs/deps/v8/include/v8.h:3674:23: note: candidate: ‘static v8::Local<v8::Array> v8::Array::New(v8::Isolate*, int)’
 3674 |   static Local<Array> New(Isolate* isolate, int length = 0);
      |                       ^~~
/usr/include/nodejs/deps/v8/include/v8.h:3674:23: note:   candidate expects 2 arguments, 3 provided
../src/extract.cpp:166:3: error: ‘memcpy’ was not declared in this scope
  166 |   memcpy(&array, &v8Array, sizeof(array));
      |   ^~~~~~
../src/extract.cpp:11:1: note: ‘memcpy’ is defined in header ‘<cstring>; did you forget to ‘#include <cstring>’?
   10 | #include <v8.h>
  +++ |+#include <cstring>
   11 | #endif
make: *** [extract.target.mk:119: Release/obj.target/extract/src/extract.o] Error 1
make: Leaving directory '/home/srishilesh/GitHub/Gatsby/node_modules/msgpackr-extract/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/share/nodejs/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Linux 5.13.0-52-generic
gyp ERR! command "/usr/bin/node" "/usr/bin/node-gyp" "rebuild"
gyp ERR! cwd /home/srishilesh/GitHub/Gatsby/node_modules/msgpackr-extract
gyp ERR! node -v v10.19.0
gyp ERR! node-gyp -v v6.1.0
gyp ERR! not ok 

Could you please help me figure out the issue?

Thanks.

@kriszyp
Copy link
Contributor

kriszyp commented Jul 11, 2022

@srishilesh msgpackr-extract is not a (required) dependency of gatsby. Or parcel. Or lmdb-js. Or msgpackr. So if you have not explicitly added msgpackr-extract as a dependency in your package.json, this should just be logging noise, not a real failure. Also, this is because node.js v10 is not a supported version of node (according to node.js themselves, and generally unsafe to use due to a lack of security patches).

@JavaScriptBach
Copy link

JavaScriptBach commented Jul 11, 2022

I think this is broken again on the latest version of Parcel because 2.6.1 upgraded lmdb which in turn requires node-gyp-build-optional-packages 5.0.3. After updating Parcel to 2.6.2, I'm getting an error running npm ci on all my Docker files:

#12 146.0 npm ERR! code 1
#12 146.0 npm ERR! path /common/node_modules/lmdb
#12 146.0 npm ERR! command failed
#12 146.0 npm ERR! command sh -c node-gyp-build-optional-packages
#12 146.0 npm ERR! gyp info it worked if it ends with ok
#12 146.0 npm ERR! gyp info using [email protected]
#12 146.0 npm ERR! gyp info using [email protected] | linux | x64
#12 146.0 npm ERR! gyp ERR! find Python 
#12 146.0 npm ERR! gyp ERR! find Python Python is not set from command line or npm configuration

@srishilesh
Copy link

@srishilesh msgpackr-extract is not a (required) dependency of gatsby. Or parcel. Or lmdb-js. Or msgpackr. So if you have not explicitly added msgpackr-extract as a dependency in your package.json, this should just be logging noise, not a real failure. Also, this is because node.js v10 is not a supported version of node (according to node.js themselves, and generally unsafe to use due to a lack of security patches).

Thanks for the help. Noted.

@kriszyp
Copy link
Contributor

kriszyp commented Jul 12, 2022

@JavaScriptBach Is this alpine or standard linux? I haven't been able to reproduce this. Do you have a package.json (or lock) that might reproduce this?

@JavaScriptBach
Copy link

@JavaScriptBach Is this alpine or standard linux? I haven't been able to reproduce this. Do you have a package.json (or lock) that might reproduce this?

Standard linux; I've created a repo with instructions here: https://github.com/JavaScriptBach/test-parcel-gyp

The issue seems to only appear when we run npm ci --no-optional. Perhaps some dependencies are incorrectly flagged as optional?

@kriszyp
Copy link
Contributor

kriszyp commented Jul 12, 2022

The issue seems to only appear when we run npm ci --no-optional. Perhaps some dependencies are incorrectly flagged as optional?

@JavaScriptBach Ah, I see. Yes, lmdb-js (as well as parcel-css and probably others) include the prebuilt binaries as optional packages, as this allows npm (or yarn, etc.) to exclude prebuilt binary packages that don't match the current architecture, without error. And the prebuilt binaries are technically "optional"; they aren't a hard requirement since you can build the binaries locally from source, as long as you have build tools/python installed. However, since you don't seem to have python installed, I assume you do want the prebuilt binaries, and therefore need to remove the --no-optional flag.

@tiramisu-zl
Copy link

I resolved my issue by npm install @lmdb/lmdb-win32-x64

1318 verbose Windows_NT 10.0.19044
1319 verbose node v16.18.0
1320 verbose npm  v8.19.2
1321 error code 7
1322 error path D:\todo-js\node_modules\lmdb
1323 error command failed
1324 error command C:\WINDOWS\system32\cmd.exe /d /s /c node-gyp-build-optional-packages
1325 error D:\todo-js\node_modules\node-gyp-build-optional-packages\index.js:64
1325 error   throw new Error('No native build was found for ' + target + '\n    loaded from: ' + dir + ' and package: ' + platformPackage + '\n')
1325 error   ^
1325 error
1325 error Error: No native build was found for platform=win32 arch=x64 runtime=node abi=93 uv=1 libc=glibc node=16.18.0
1325 error     loaded from: D:\todo-js\node_modules\lmdb and package: @lmdb/lmdb-win32-x64
1325 error
1325 error     at Function.load.path (D:\todo-js\node_modules\node-gyp-build-optional-packages\index.js:64:9)
1325 error     at load (D:\todo-js\node_modules\node-gyp-build-optional-packages\index.js:20:30)
1325 error     at Object.<anonymous> (D:\todo-js\node_modules\node-gyp-build-optional-packages\build-test.js:19:19)
1325 error     at Module._compile (node:internal/modules/cjs/loader:1155:14)
1325 error     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1209:10)
1325 error     at Module.load (node:internal/modules/cjs/loader:1033:32)
1325 error     at Function.Module._load (node:internal/modules/cjs/loader:868:12)
1325 error     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
1325 error     at node:internal/main/run_main_module:22:47

@Toxiapo
Copy link

Toxiapo commented Nov 9, 2022

I resolved my issue by npm install @lmdb/lmdb-win32-x64

1318 verbose Windows_NT 10.0.19044
1319 verbose node v16.18.0
1320 verbose npm  v8.19.2
1321 error code 7
1322 error path D:\todo-js\node_modules\lmdb
1323 error command failed
1324 error command C:\WINDOWS\system32\cmd.exe /d /s /c node-gyp-build-optional-packages
1325 error D:\todo-js\node_modules\node-gyp-build-optional-packages\index.js:64
1325 error   throw new Error('No native build was found for ' + target + '\n    loaded from: ' + dir + ' and package: ' + platformPackage + '\n')
1325 error   ^
1325 error
1325 error Error: No native build was found for platform=win32 arch=x64 runtime=node abi=93 uv=1 libc=glibc node=16.18.0
1325 error     loaded from: D:\todo-js\node_modules\lmdb and package: @lmdb/lmdb-win32-x64
1325 error
1325 error     at Function.load.path (D:\todo-js\node_modules\node-gyp-build-optional-packages\index.js:64:9)
1325 error     at load (D:\todo-js\node_modules\node-gyp-build-optional-packages\index.js:20:30)
1325 error     at Object.<anonymous> (D:\todo-js\node_modules\node-gyp-build-optional-packages\build-test.js:19:19)
1325 error     at Module._compile (node:internal/modules/cjs/loader:1155:14)
1325 error     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1209:10)
1325 error     at Module.load (node:internal/modules/cjs/loader:1033:32)
1325 error     at Function.Module._load (node:internal/modules/cjs/loader:868:12)
1325 error     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
1325 error     at node:internal/main/run_main_module:22:47

I tried this and got the error Unsupported platform for @lmdb/[email protected]: wanted {"os":"win32","arch":"x64"} (current: {"os":"darwin","arch":"x64"}) on Mac. I wonder if anyone know a solution for all platforms

@kriszyp
Copy link
Contributor

kriszyp commented Nov 10, 2022

@lmdb/lmdb-darwin-x64 is the corresponding binary file for Mac with Intel.

@decanTyme
Copy link

decanTyme commented Dec 7, 2022

For me this whole gyp ERR! started appearing on just about every other repo I work with when I updated node to v18 (as far as I can remember) and tried to install deps (not just parcel too). Sure enough, downgrading to v16 resolved the entire problem. I am on Windows 10. Writing this possible "fix" here for anyone not yet wanting to upgrade deps for some reason.

@LanguageXange
Copy link

LanguageXange commented Jan 11, 2023

got the same error after upgrading Node to v18
( everything is working properly if I switch node version back to v16 )

@lazerwalker
Copy link

Yet another +1 that downgrading to node v16 (from v19 for me) was a successful (temporary?) fix for me.

@tillsanders
Copy link

I had this issue after copying all my repositories from an Intel MacBook to a M2 MacBook. This is what helped me instantly.

If the binary is there, the next thing I would try is to force the npm install to not using any locally compilation by using --ignore-scripts

@rasi5050
Copy link

rasi5050 commented Dec 1, 2023

I had this issue on M1 MacBook Pro. npm install -D parcel --ignore-scripts resolved the issue.

@imnasnainaec
Copy link

Our GitHub action workflow (https://github.com/sillsdev/TheCombine/blob/parcel-babel-jest/.github/workflows/frontend.yml) has started hitting this gyp error.

In one job, the error is fixed by adding --ignore-scripts:

  test_coverage:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [20]
    steps:
      # See https://docs.stepsecurity.io/harden-runner/getting-started/ for instructions on
      # configuring harden-runner and identifying allowed endpoints.
      - name: Harden Runner
        uses: step-security/harden-runner@17d0e2bd7d51742c71671bd19fa12bdc9d40a3d6 # v2.8.1
        with:
          disable-sudo: true
          egress-policy: block
          allowed-endpoints: >
            api.github.com:443
            github.com:443
            objects.githubusercontent.com:443
            registry.npmjs.org:443
      - name: Checkout repository
        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm ci --ignore-scripts
      - run: npm run test-frontend:coverage
        env:
          CI: true
      - name: Upload coverage artifact
        uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
        with:
          if-no-files-found: error
          name: coverage
          path: coverage/cobertura-coverage.xml
          retention-days: 7

Error before changing - run: npm ci to - run: npm ci --ignore-scripts (from https://github.com/sillsdev/TheCombine/actions/runs/11014185256/job/30584228260#step:6:35):

npm error code 1
npm error path /home/runner/work/TheCombine/TheCombine/node_modules/lmdb
npm error command failed
npm error command sh -c node-gyp-build-optional-packages
npm error /home/runner/work/TheCombine/TheCombine/node_modules/node-gyp-build-optional-packages/index.js:77
npm error   throw new Error('No native build was found for ' + target + '\n    attempted loading from: ' + dir + ' and package:' +
npm error   ^
npm error
npm error Error: No native build was found for platform=linux arch=x64 runtime=node abi=115 uv=1 libc=glibc node=20.17.0
npm error     attempted loading from: /home/runner/work/TheCombine/TheCombine/node_modules/lmdb and package: @lmdb/lmdb-linux-x64
npm error
npm error     at load.path (/home/runner/work/TheCombine/TheCombine/node_modules/node-gyp-build-optional-packages/index.js:77:9)
npm error     at load (/home/runner/work/TheCombine/TheCombine/node_modules/node-gyp-build-optional-packages/index.js:29:25)
npm error     at Object.<anonymous> (/home/runner/work/TheCombine/TheCombine/node_modules/node-gyp-build-optional-packages/build-test.js:19:19)
npm error     at Module._compile (node:internal/modules/cjs/loader:1469:14)
npm error     at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
npm error     at Module.load (node:internal/modules/cjs/loader:1288:32)
npm error     at Module._load (node:internal/modules/cjs/loader:1104:12)
npm error     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:174:12)
npm error     at node:internal/main/run_main_module:28:49
npm error
npm error Node.js v20.17.0
npm error
npm error The failure above indicates the primary issue with the native builds which are included for all major platforms. Will now attempt to build the package locally in case this can be resolved by re-compiling.
npm error gyp info it worked if it ends with ok
npm error gyp info using [email protected]
npm error gyp info using [email protected] | linux | x64
npm error gyp info find Python using Python version 3.10.12 found at "/usr/bin/python3"
npm error gyp http GET https://nodejs.org/download/release/v20.17.0/node-v20.17.0-headers.tar.gz
npm error gyp http fetch GET https://nodejs.org/download/release/v20.17.0/node-v20.17.0-headers.tar.gz attempt 1 failed with ECONNREFUSED
npm error gyp WARN install got an error, rolling back install
npm error gyp ERR! configure error 
npm error gyp ERR! stack FetchError: request to https://nodejs.org/download/release/v20.17.0/node-v20.17.0-headers.tar.gz failed, reason: connect ECONNREFUSED 54.185.253.63:443
npm error gyp ERR! stack at ClientRequest.<anonymous> (/opt/hostedtoolcache/node/20.17.0/x64/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js:130:14)
npm error gyp ERR! stack at ClientRequest.emit (node:events:519:28)
npm error gyp ERR! stack at emitErrorEvent (node:_http_client:108:11)
npm error gyp ERR! stack at _destroy (node:_http_client:891:9)
npm error gyp ERR! stack at onSocketNT (node:_http_client:911:5)
npm error gyp ERR! stack at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
npm error gyp ERR! System Linux 6.8.0-1014-azure
npm error gyp ERR! command "/opt/hostedtoolcache/node/20.17.0/x64/bin/node" "/opt/hostedtoolcache/node/20.17.0/x64/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm error gyp ERR! cwd /home/runner/work/TheCombine/TheCombine/node_modules/lmdb
npm error gyp ERR! node -v v20.17.0
npm error gyp ERR! node-gyp -v v10.1.0
npm error gyp ERR! not ok
npm error A complete log of this run can be found in: /home/runner/.npm/_logs/2024-09-24T12_54_40_1[39](https://github.com/sillsdev/TheCombine/actions/runs/11014185256/job/30584228260#step:6:40)Z-debug-0.log
Error: Process completed with exit code 1.

Adding --ignore-scripts to another job just results in a different error.

  lint_build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [20]
    steps:
      # See https://docs.stepsecurity.io/harden-runner/getting-started/ for instructions on
      # configuring harden-runner and identifying allowed endpoints.
      - name: Harden Runner
        uses: step-security/harden-runner@17d0e2bd7d51742c71671bd19fa12bdc9d40a3d6 # v2.8.1
        with:
          disable-sudo: true
          egress-policy: block
          allowed-endpoints: >
            api.github.com:443
            github.com:443
            objects.githubusercontent.com:443
            registry.npmjs.org:443
      - name: Checkout repository
        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm ci
      - run: npm run fmt-frontend-check
      - run: npm run lint
      - run: npm run build

Error with - run: npm ci (from https://github.com/sillsdev/TheCombine/actions/runs/11014185256/job/30584227659#step:6:35), same as above error.

Error with - run: npm ci --ignore-scripts (from https://github.com/sillsdev/TheCombine/actions/runs/11014556950/job/30585467526#step:9:9):

Error: No prebuild or local build of @parcel/watcher found. Tried @parcel/watcher-linux-x64-glibc. Please ensure it is installed (don't use --no-optional when installing with npm). Otherwise it is possible we don't support your platform yet. If this is the case, please report an issue to https://github.com/parcel-bundler/watcher.
    at Object.<anonymous> (/home/runner/work/TheCombine/TheCombine/node_modules/@parcel/watcher/index.js:26:13)
    at Module._compile (node:internal/modules/cjs/loader:146[9](https://github.com/sillsdev/TheCombine/actions/runs/11014556950/job/30585467526#step:9:10):14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
    at Module.load (node:internal/modules/cjs/loader:1288:32)
    at Module._load (node:internal/modules/cjs/loader:1[10](https://github.com/sillsdev/TheCombine/actions/runs/11014556950/job/30585467526#step:9:11)4:12)
    at Module.require (node:internal/modules/cjs/loader:13[11](https://github.com/sillsdev/TheCombine/actions/runs/11014556950/job/30585467526#step:9:12):19)
    at require (node:internal/modules/helpers:179:18)
    at Object.<anonymous> (/home/runner/work/TheCombine/TheCombine/node_modules/@parcel/fs/lib/index.js:7:28)
    at Module._compile (node:internal/modules/cjs/loader:[14](https://github.com/sillsdev/TheCombine/actions/runs/11014556950/job/30585467526#step:9:15)69:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:[15](https://github.com/sillsdev/TheCombine/actions/runs/11014556950/job/30585467526#step:9:16)48:10)
Error: Process completed with exit code 1.

Here's the package.json: https://github.com/sillsdev/TheCombine/blob/parcel-babel-jest/package.json

@imnasnainaec
Copy link

As suggested here in another issue, I resolved this issue with npm dedupe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests