[rush] rush deploy fails if it encounters a symbolic link in node_modules

[salieri]

Summary

Some packages, such as Puppeteer, automatically create symbolic links inside node_modules. rush deploy seems to fail when it encounters such a symlink.

Repro steps

Example at:
https://github.com/salieri/rush-issue-with-symlink-and-deploy

This may be MacOS only issue, as I didn't notice it on a Linux VM.

1. Clone the repo
2. rush update
3. rush deploy --overwrite -p test

Expected result:
Deploy command should complete successfully

Actual result:

% rush --debug deploy --overwrite -p test

Rush Multi-Project Build Tool 5.50.0 - https://rushjs.io
Node.js version is 14.17.0 (LTS)

Starting "rush deploy"

Loading deployment scenario: /path/to/common/config/rush/deploy.json
Deploying to target folder:  /path/to/common/deploy
Main project for deployment: test

Deleting target folder contents because "--overwrite" was specified...

Analyzing project: test

Copying folders...

ERROR: Internal Error: The path ends prematurely at: /path/to/common/temp/node_modules/.pnpm/[email protected]/node_modules/puppeteer/.local-chromium/mac-884014/chrome-mac/Chromium.app/Contents/Frameworks/Chromium
Framework.framework/Versions/Current/Chromium Framework
You have encountered a software defect. Please consider reporting the issue to the maintainers of this application.

Error: Internal Error: The path ends prematurely at: /path/to/common/temp/node_modules/.pnpm/[email protected]/node_modules/puppeteer/.local-chromium/mac-884014/chrome-mac/Chromium.app/Contents/Frameworks/Chromium Framework.framework/Versions/Current/Chromium Framework

You have encountered a software defect. Please consider reporting the issue to the maintainers of this application.
    at SymlinkAnalyzer.analyzePath (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/@microsoft/rush-lib/lib/logic/deploy/SymlinkAnalyzer.js:105:23)
    at SymlinkAnalyzer.analyzePath (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/@microsoft/rush-lib/lib/logic/deploy/SymlinkAnalyzer.js:77:45)
    at Object.filter (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/@microsoft/rush-lib/lib/logic/deploy/DeployManager.js:299:53)
    at startCopy (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:35:28)
    at copyDirItem (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:118:10)
    at /Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:111:39
    at Array.forEach (<anonymous>)
    at copyDir (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:111:23)
    at mkDirAndCopy (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:106:3)
    at onDir (/Users/<user>/.rush/node-v14.17.0/rush-5.50.0/node_modules/fs-extra/lib/copy-sync/copy-sync.js:97:37)

Details

The issue is caused by SymlinkAnalyzer.ts expecting nothing but 'real' (non-symlink) directories in the path.

Standard questions

Please answer these questions to help us investigate your issue more quickly:

Question	Answer
@microsoft/rush globally installed version?	5.47.0
rushVersion from rush.json?	5.50.0
useWorkspaces from rush.json?	true
Operating system?	MacOS
Would you consider contributing a PR?	Maybe
Node.js version (node -v)?	14.17.0

[salieri]

Workaround if you don't care about having chromium with Puppeteer when you prepare your deployment package:
PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 rush update --full --purge --recheck

[jeanlescure]

Being affected by this, nothing to do with puppeteer, our project just underwent a very successful refactor (all tests passing in multiple packages/apps) that requires the usage of symlinks. But now our CI fails when deploying, in the Copying folders... stage.

Is there no way to ignore or bypass this check?

[bryanmig]

I have recently started seeing this same behavior in my Rush repo as well. I'm not sure what changed in our repo or environment but I've traced this back to a change in node-gyp (see nodejs/node-gyp#2713) which intentionally creates a symlink to python on the host machine.

It looks like there is a fix that has not yet landed (nodejs/node-gyp#2721) but my solution to this particular problem is simple. I've added a postRushInstall hook that finds and removes these dangling symlinks:

    "postRushInstall": [
      // This is needed to remove node_gyp_bins which contains symlinks outside the root
      // @see https://github.com/nodejs/node-gyp/issues/2713
      // @see https://github.com/nodejs/node-gyp/pull/2721
      "find ./common -name node_gyp_bins -type d -exec rm -r '{}' \\;"
    ],

[microsoftly]

I just encountered this too. My guess is that there's a bug with the folder copy where it doesn't properly handle paths with spaces in them