Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: store package.json and deno.json JSR and npm deps in lockfile for tracking removable dependencies #13

Merged

Conversation

dsherret
Copy link
Member

@dsherret dsherret commented Jan 20, 2024

Updates the lockfile to keep track of JSR and npm dependencies stored in a deno.json or package.json. When a package is removed from a config file, then the dependency is also removed from the lockfile. This supports workspaces, so is nested under the "workspace" key:

{
  // etc...
  "workspace": {
    // these are the deps in the deno.json. They may be jsr or npm deps
    "dependencies": [
      "jsr:@scope/package_a"
    ],
    // stored separately from dependendencies so that if someone runs a one-off script
    // with --no-npm in their repo, then it won't remove this
    "packageJson": [
      "npm:ts-morph@11"
    ]
  }
}

A workspace with multiple members looks like this:

{
  // etc...
  "workspace": {
    "dependencies": [
      "jsr:@scope/some_package"
    ]
    "members": {
      "@dsherret/package_a": {
        "dependencies": [
          "jsr:@scope/package_a"
        ]
      },
      "@dsherret/package_b": {
        "dependencies": [
          "jsr:@scope/package_b"
        ]
      }
    }
  }
}

Additionally, JSR packages with their version constraints are now stored under "packages" in addition to "npm":

{
  "version": "3",
  "packages": {
    "specifiers": {
      "jsr:@scope/package_a@1": "jsr:@scope/[email protected]",
      "jsr:@scope/package_b@1": "jsr:@scope/[email protected]",
      "jsr:@scope/package_c@^2.1": "jsr:@scope/[email protected]"
    },
    "jsr": {
      "@scope/[email protected]": {
        "dependencies": [
          "jsr:@scope/package_b@1",
          "jsr:@scope/package_c@^2.1"
        ]
      },
      "@scope/[email protected]": {
        "dependencies": [
          "jsr:@scope/package_a@1"
        ]
      }
    }
  },
  "remote": {
    "https://jsr.io/@scope/package_a/1.0.1/mod.ts": "09154a97e18c4d6a1692e3b3c8a3b1ec2934f00b7c1caf7491d762d963ada045",
    "https://jsr.io/@scope/package_b/1.0.1/mod.ts": "09154a97e18c4d6a1692e3b3c8a3b1ec2934f00b7c1caf7491d762d963ada046",
    "https://jsr.io/@scope/package_c/2.1.2/mod.ts": "09154a97e18c4d6a1692e3b3c8a3b1ec2934f00b7c1caf7491d762d963ada047"
  },
  "workspace": {
    "dependencies": [
      "jsr:@scope/package_a@1"
    ]
  }
}

When "jsr:@scope/package_a@1" is removed from "workspace" -> "dependencies", then it knows it can remove @scope/[email protected] and therefore @scope/[email protected] (even though it's circular in this case) and @scope/[email protected]. Additionally, it strips out the urls under "remote". This specific example can be seen in https://github.com/denoland/deno_lockfile/pull/13/files#diff-e13153f86e029e465847ec0daa572a7d0e96acaea1ff47c232136f826208fb86

@dsherret dsherret changed the title feat: store package json and deno.json JSR and npm deps in lockfile for tracking removable dependencies feat: store package.json and deno.json JSR and npm deps in lockfile for tracking removable dependencies Jan 20, 2024
Copy link
Member

@bartlomieju bartlomieju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits and questions, but not a blocker to land it


/// Graph used to analyze a lockfile to determine which packages
/// and remotes can be removed based on config file changes.
pub struct LockfilePackageGraph<FNvToJsrUrl: Fn(&str) -> Option<String>> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, I didn't know you could do that (or never seen that in the wild)

src/graphs.rs Outdated Show resolved Hide resolved
Comment on lines +139 to +140
while let Some(id) = pending.pop_back() {
if let Some(package) = packages.get_mut(&id) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if pacakges.get_mut() returns None? Is it assume it will be found by another root?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might occur if someone has been modifying the lockfile themselves. It's not a big deal in that case because then the package is already gone.

src/graphs.rs Outdated Show resolved Hide resolved
Comment on lines +92 to +104
#[error("Integrity check failed for npm package: \"{package_display_id}\". Unable to verify that the package
is the same as when the lockfile was generated.

Actual: {actual}
Expected: {expected}

This could be caused by:
* the lock file may be corrupt
* the source itself may be corrupt

Use \"--lock-write\" flag to regenerate the lockfile at \"{filename}\".",
)]
pub struct IntegrityCheckFailedError {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: will we surface these errors one-by-one or aggregate them and display them all at once?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One-by-one as it currently does. It might be good to improve this in the future.

@dsherret dsherret merged commit c263e01 into denoland:main Jan 21, 2024
1 check passed
@dsherret dsherret deleted the feat_store_package_json_and_import_map_lockfile branch January 21, 2024 22:02
bartlomieju pushed a commit to denoland/deno that referenced this pull request Jan 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants