Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug?]: Parallel execution of yarn workspace $name version apply is unstable #6615

Open
1 task
GeorgeTaveras1231 opened this issue Nov 26, 2024 · 0 comments
Open
1 task
Labels
bug Something isn't working

Comments

@GeorgeTaveras1231
Copy link

GeorgeTaveras1231 commented Nov 26, 2024

Self-service

  • I'd be willing to implement a fix

Describe the bug

Running multiple instances of yarn workspace $name version apply causes unpredictable behavior.

Bug model

At its core, I believe this is a race-condition bug.

Multiple processes attempting to read and modify the same files will inevitably encounter race conditions. Since there is no cross-process orchestration, multiple processes might read the stale version of a given file before performing the assumed update.

sequenceDiagram
  box rgba(33,66,99,0.5) Parallel processes
    participant Process1
    participant Process2
  end
  participant OS
  par yarn  workspace $name version apply (1)
    Process1 ->> OS: Read ./.yarn/versions/1.yml
    OS -->> Process1: Stale or fresh state of ./.yarn/versions/1.yml
    Process1 ->> OS: Write ./.yarn/versions/1.yml (Err - update is based on possibly stale state)
  and yarn  workspace $name version apply (2)
    Process2 ->> OS: Read ./.yarn/versions/1.yml
    OS -->> Process2: Stale or fresh state of ./.yarn/versions/1.yml
    Process2 ->> OS: Write ./.yarn/versions/1.yml (Err - update is based on possibly stale state)
  end

Loading

To reproduce

  1. Create a monorepo with multiple workspaces
  2. Create version bump files (in .yarn/versions/*) - choose a version bump for each workspace. Since there are many OS dependent factors that will impact the result of running yarn version apply, consider creating multiple version files, to increase the chances that the problem will surface.
  3. Run yarn workspace $name version apply for each workspace in parallel.

Environment

System:
    OS: macOS 14.6
    CPU: (12) arm64 Apple M2 Pro
  Binaries:
    Node: 20.11.0 - /private/var/folders/5y/s8blq1h50fn7_qbwt5cndstw0000gq/T/xfs-b7f99751/node
    Yarn: 4.1.1 - /private/var/folders/5y/s8blq1h50fn7_qbwt5cndstw0000gq/T/xfs-b7f99751/yarn
    npm: 10.2.4 - ~/.asdf/plugins/nodejs/shims/npm

Additional context

This bug causes issues in teams wanting more control over which workspaces are bumped or how to apply the version bump (i.e. bump as a prerelease) in a given release.

The problem would be solved if yarn provided a way to apply the version bump on multiple workspaces at once, which will allow yarn to make sure all updates to .yarn/versions files are atomic. At the moment, yarn version apply only allows --all - to apply the version bump on every workspace.

There isn't enough control to apply a prerelease bump to a subset of the workspaces, or to apply a typical bump to a subset of workspaces.

Workarounds

At the moment, I'm choosing to run each yarn workspace $name version apply sequential order. At the moment of writing this, running this in a monorepo with 150 workspaces takes ~15 minutes.

@GeorgeTaveras1231 GeorgeTaveras1231 added the bug Something isn't working label Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant