shakapacker fails to run webpack if modules dir is absolute path

[cryptomilk]

We use /usr/src/node_modules as the node path and yarn module path to install modules in a container. However since shakapacker removed SHAKAPACKER_NODE_MODULES_BIN_PATH this doesn't work anymore.

It looks like the current working directory and NODE_PATH are concatenated. It should actually check if NODE_PATH is an absolute path. Maybe this is happening in package_json now.

/usr/local/bundle/gems/shakapacker-8.1.0/lib/shakapacker/webpack_runner.rb:47:in `exec': No such file or directory - /usr/src/app/usr/src/node_modules/.bin/webpack (Errno::ENOENT)
        from /usr/local/bundle/gems/shakapacker-8.1.0/lib/shakapacker/webpack_runner.rb:47:in `block in run'
        from /usr/local/bundle/gems/shakapacker-8.1.0/lib/shakapacker/webpack_runner.rb:46:in `chdir'
        from /usr/local/bundle/gems/shakapacker-8.1.0/lib/shakapacker/webpack_runner.rb:46:in `run'
        from /usr/local/bundle/gems/shakapacker-8.1.0/lib/shakapacker/runner.rb:11:in `run'
        from /usr/src/app/bin/shakapacker:12:in `block in <main>'
        from /usr/src/app/bin/shakapacker:11:in `chdir'
        from /usr/src/app/bin/shakapacker:11:in `<main>'

Expected behavior:

/usr/src/node_modules/.bin/webpack should be called

Actual behavior:

/usr/src/app/usr/src/node_modules/.bin/webpack is called

Small, reproducible repo:

https://gitlab.com/routenbuch/routenbuch/-/merge_requests/41

podman-compose -p routenbuch up

Setup environment:

• Ruby version: 3.3
• Rails version: 7.1.5
• Shakapacker version: 8.1.0

[justin808]

@cryptomilk PR's welcome!

[cryptomilk]

Thanks Andreas for taking the time to report a bug. The referenced code in https://github.com/shakacode/shakapacker/blob/main/lib/shakapacker/webpack_runner.rb#L47 doesn't contruct the path, but you can find it <insert code location> here. Could you open a PR for it and propose a fix? I'm happy to review and help then.

https://www.debuggingteams.com/

[cryptomilk]

It is unclear where this path is constructed. Looking at the package_json code, and I'm not a ruby developer, kt looks like the native_command should be yarn run webpack. However instead it tries to exec /usr/src/app/usr/src/node_modules/.bin/webpack. The package.json file has "packageManager": "[email protected]". This looks suspicious.

[cryptomilk]

Looks like the culprit is yarn classic:

yarn --modules-folder=/usr/src/node_modules bin
/home/asn/workspace/prj/oss/routenbuch/asn-shakapacker/usr/src/node_modules/.bin

[cryptomilk]

yarnpkg/yarn#7519

[cryptomilk]

Well, using pnpm also isn't a solution: pnpm/pnpm#9113

I guess you have check if the path you want to exec exists, if not try env PACKAGE_JSON_NODE_MODULES_BIN_PATH as a fallback?

[cryptomilk]

I've opened shakacode/package_json#21

[G-Rath]

@cryptomilk as I said on shakacode/package_json#21 (comment) while I'm not sure if this is actually something package_json should be responsible for, I think either way this would be better to start life in shakapacker as we could land an "experimental" solution here first, then pull it out into package_json if we later decide it does actually belong there without it being a breaking change.

In addition to exploring the solution you proposed (which I think would effectively be looking to restore the SHAKAPACKER_NODE_MODULES_BIN_PATH behaviour, though potentially with some differences), I think another one could be to introduce a new config property for setting the "path to binary", which would allow you to completely override it.

Also couple of side questions:

• why are you installing node_modules in a different place than the rest of the app?
• could you use a symlink, either for node_modules entirely or node_modules/.bin? (or whatever the package manager needs/uses - I think it might just be npm that uses .bin, but the principle is the same)
• could you use a multi-stage build instead? i.e. build the assets in a different temp stage where node_modules can be not-absolute, then copy the build assets into the final build

[cryptomilk]

Because the git source directory is mounted to the container for development, but I guess the developers didn't want to pollute the source do with node modules.