build fails when using npm workspaces and node_modules is required during build and launch

[pbusko]

Expected Behavior

build should succeed, when using project with npm workspaces and node_modules are required at both build and launch.

Current Behavior

build fails during the Executing launch environment install process phase:

            [builder]   Executing build environment install process
            [builder]     Running 'npm install --unsafe-perm --cache /layers/paketo-buildpacks_npm-install/npm-cache'
            [builder]       
            [builder]       added 167 packages, and audited 169 packages in 12s
            [builder]       
            [builder]       56 packages are looking for funding
            [builder]         run `npm fund` for details
            [builder]       
            [builder]       found 0 vulnerabilities
            [builder]       npm notice 
            [builder]       npm notice New minor version of npm available! 9.5.1 -> 9.6.5
            [builder]       npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.6.5>
            [builder]       npm notice Run `npm install -g [email protected]` to update!
            [builder]       npm notice 
            [builder]       Completed in 12.345s
            [builder] 
            [builder]   Configuring build environment
            [builder]     NODE_ENV -> "development"
            [builder]     PATH     -> "$PATH:/layers/paketo-buildpacks_npm-install/build-modules/node_modules/.bin"
            [builder] 
            [builder]   Generating SBOM for /layers/paketo-buildpacks_npm-install/build-modules
            [builder]       Completed in 344ms
            [builder] 
            [builder]   Executing launch environment install process
            [builder]     Running 'npm prune'
            [builder]       
            [builder]       changed 1 package, and audited 4 packages in 827ms
            [builder]       
            [builder]       found 0 vulnerabilities
            [builder] failed to copy linked module directory to layer path: failed to copy: destination exists: remove /layers/paketo-buildpacks_npm-install/build-modules/src/packages/hello-world: directory not empty
            [builder] ERROR: failed to build: exit status 1

From what we can see, there are currently several issues with the workspace handling:

1. When required at both build and launch, the call to the symlinkerResolver.Resolve inside the if launch block is done with the same set of parameters, as within the if build block.
   After the first call, the source folder no longer exists (it gets deleted by linker.Link() function). Also, the target folder is already present in the build-modules layer

npm-install/build.go

Lines 142 to 145 in 56c8312

	err = symlinkResolver.Resolve(filepath.Join(projectPath, "package-lock.json"), layer.Path)
	if err != nil {
	return packit.BuildResult{}, err
	}

npm-install/build.go

Line 242 in 56c8312

targetLayerPath = buildLayerPath

npm-install/build.go

Lines 252 to 255 in 56c8312

	err = symlinkResolver.Resolve(filepath.Join(projectPath, "package-lock.json"), targetLayerPath)
	if err != nil {
	return packit.BuildResult{}, err
	}

2. workspace modules are not copied from the build layer to the launch layer
3. in the produced image, workspace modules in the projectPath are pointing to nonexistent directory in the /tmp folder, create-symlinks helper manages only symlink to the node_modules folder.

Steps to Reproduce

1. Add build = true here:

   npm-install/integration/testdata/workspaces/commonjs/plan.toml

   Lines 10 to 11 in 56c8312

   	[requires.metadata]
   	launch = true

2. Run integration tests

Motivations

[jansu76]

Not sure if this failure is related, or caused by issues with how workspaces are handled in npm-install paketo-buildpacks/node-run-script#230

[jansu76]

Also this one was related to workspaces, fixed / changed in 1.1.0: #492

[c0d1ngm0nk3y]

Also this one was related to workspaces, fixed / changed in 1.1.0: #492

Are you sure? I can still reproduce this with 1.1.1 and the steps provided above. We didn't have the problem with 1.0.6. I think that was introduced with workspaces.

[tuopppi]

I'm also experiencing the same issue with version 1.1.1.

[thitch97]

This has not yet been resolved. I'm currently finishing up a PR to resolve it.

[jansu76]

Are you sure? I can still reproduce this with 1.1.1 and the steps provided above. We didn't have the problem with 1.0.6. I think that was introduced with workspaces.

No I was just being unclear. I simply meant that the fix #492 did change something in relation to workspace handling, and might be worth looking into.