Issue when building on armv7 system

[jakew009]

BullMQ depends on this project (well it depends on msgpackr, which depends on this).

When msgpackr-extract is installed on Raspbian (32 bit arm), and further down our pipeline we use Yocto for building a firmware image, during the process of copying the files into our firmware image we are getting errors like this:

Subprocess output:arm-poky-linux-gnueabi-objcopy: /builds/rugged-networks/onvp/ifu/yocto/plc-ionopi/.build-ionopi-rpi4/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/ifu-node/1.0-r0/package/usr/lib/node/ifu-node/node_modules/msgpackr-extract/prebuilds/linux-arm64/node.abi93.node: file format not recognized

When we check some of these files:

file node.abi93.node
node.abi93.node: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=90558ecf012514f047835415c93b37ab38b2b37f, not stripped

This would suggest these are 64 bit executable, but our system is only 32 bit support.

Is there any way we can work around this? Is 32 bit OS supported?

Thanks for any help in advance.

[jakew009]

As a bit of an update to this

We found that the corect 32 bit version of msgpackr-extract is in fact being built by make:

jake@thinkpad-t490-jake:~/Downloads$ file msgpackr-extract.node 
msgpackr-extract.node: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=6e71769a8d1eda73ad9bbf3aa47d400004d72b53, not stripped

The problem is that the prebuilt binaries are 'left behind' and still sat in the prebuilds directory.

When Yocto goes to copy these files into our OS, it blows up because it refuses to copy 64bit executables it knows cannot run.

I was able to work around this by simply rm -rf node_modules/msgpackr-extract/prebuilds

Is there any way this could be done as part of the build script to remove these useless files?

Thanks again for the effort you put into this.

[jakew009]

I found this thread which maybe answers my questions: prebuild/prebuildify#54 (comment)

Maybe I should post over there

Edit: created an issue: prebuild/prebuildify#60

[kriszyp]

Glad to hear you found some workarounds. As mentioned in the prebuildify issue, deleting prebuilds after install could be complicated and even counter-productive in some situations. I know that for our application, we run our builds on a different computer than where it is deployed, sometimes the node version differs, so having all the prebuilds available is actually really helpful for us.

Another possibility to note: msgpackr-extract is an optional dependency of msgpackr. msgpackr will run without it, and have decreased performance (typically about 50% reduction in speed with processing strings from larger messagepack documents). If performance isn't a big concern, you could do your npm install with --no-optional to avoid using it altogether.

If you have any other suggestions for how to improve this process, be glad to hear.

[jumo-sballing]

Hello. Same problem here. Yocto doesn't like those foreign architecture binarys.

The native node modules I know so far are using https://github.com/prebuild/prebuild. This keeps the prebuilds on Github and downloads them only when needed.

[kriszyp]

Yes, I used to use prebuild/prebuild for builds, but has prebuildify has seemed to work a lot better for more users, since there is no build step required in most cases, and the package will work (on machines with prebuilds) even with --ignore-scripts.

Another entirely different approach is create a set of platform-specific packages that are all marked optional, but dictate a target OS and architecture. I have been considering this, but it is a pretty complicated set up from a CI/publishing perspective, so I haven't dug into setting it up yet. parcel-css uses this approach, if you are looking for an example, and you can see the build process here: https://github.com/parcel-bundler/parcel-css/blob/master/scripts/build-npm.js

However, this is still a bit of a hack, and I believe there are still warnings using this approach. A more "approved" mechanism for OS/arch specific builds has been proposed as well:
npm/rfcs#120

Anyway, again, any feedback/suggestions you have here are welcome.

[kriszyp]

I believe this should now be fixed in v1.1 (v1.1.3 is latest), by using optional platform-specific packages, which should mean only the binaries for the current platform should be installed.

[jumo-sballing]

That is looking good. Thank you very much.

[jakew009]

@kriszyp this is awesome. I will test it shortly and report back.

[jakew009]

Will you release a new version of msgpackr which includes this upate?

[kriszyp]

Published [email protected] which depends on [email protected].