Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move rules_pkg_lib to //pkg #898

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

jacky8hyf
Copy link
Contributor

... next to the .bzl files. This is because the
//doc_build package is not loadable when rules_pkg itself
is not the root Bazel module, because stardoc is a
dev_dependency. So, //doc_build:rules_pkg_lib is not
usable by clients of rules_pkg.

Move the target to //pkg, next to the .bzl files.

Also add @rules_python//python:defs_bzl used by pkg_install.

Link: #897

HONG Yifan added 2 commits October 1, 2024 12:33
It is incorrect to put a struct in a depset of files.
However, we never use this case, so just delete the
branch.

Signed-off-by: HONG Yifan <[email protected]>
... next to the .bzl files. This is because the
//doc_build package is not loadable when rules_pkg itself
is not the root Bazel module, because stardoc is a
dev_dependency. So, //doc_build:rules_pkg_lib is not
usable by clients of rules_pkg.

Move the target to //pkg, next to the .bzl files.

Also add @rules_python//python:defs_bzl used by pkg_install.

Link: bazelbuild#897
Signed-off-by: HONG Yifan <[email protected]>
@cgrindel
Copy link
Collaborator

cgrindel commented Oct 4, 2024

@jacky8hyf Do you want to address the failing tests before we review?

@jacky8hyf
Copy link
Contributor Author

@jacky8hyf Do you want to address the failing tests before we review?

I don't think the test error sare related to my change. The errors are:

 ERROR: no such package '@@[unknown repo 'mappings_test_external_repo' requested from @@]//pkg': The repository '@@[unknown repo 'mappings_test_external_repo' requested from @@]' could not be resolved: No repository visible as '@mappings_test_external_repo' from main repository. Was the repository introduced in WORKSPACE? The WORKSPACE file is disabled by default in Bazel 8 (late 2024) and will be removed in Bazel 9 (late 2025), please migrate to Bzlmod. See https://bazel.build/external/migration.

mappings_test_external_repo is defined in WORKSPACE file. Perhaps --noenable_workspace is set because it is using Bazel 8: (8.0.0-pre.20240911.1 from the test).

I can't reproduce the same error offline.

@cgrindel
Copy link
Collaborator

cgrindel commented Oct 4, 2024

@aiuto Any thoughts on the test failures?

@jacky8hyf
Copy link
Contributor Author

gentle ping

@aiuto
Copy link
Collaborator

aiuto commented Oct 8, 2024

I do think the failing tests are unrelated, but I am dubious about the change.
We are trying to limit taking on new dependencies. Skylib depends on rules_pkg, so the mutual embrace makes compatibility_level updates really hard.

@jacky8hyf
Copy link
Contributor Author

AFAIK there are no new dependencies. See this line:

load("@rules_pkg//pkg/private:make_starlark_library.bzl", "starlark_library")

This means for anyone that relies on the @rules_pkg//pkg package, the starlark_library.bzl is loaded, which relies on skylib because of

load("@bazel_skylib//:bzl_library.bzl", "StarlarkLibraryInfo")

In addition, skylib is already a non-dev dependency:

bazel_dep(name = "bazel_skylib", version = "1.4.2")

(Skylib does depend on rules_pkg as a dev-dependency, on the other hand:
https://github.com/bazelbuild/bazel-skylib/blob/e8e9d218ff563f93e3a7f7547ea532e66b210620/MODULE.bazel#L20
)

I am not changing MODULE.bazel for rules_pkg so there are no dependency changes. Am I missing something?

@@ -9,13 +9,10 @@ def _make_starlark_library(ctx):
direct = []
transitive = []
for src in ctx.attr.srcs:
if StarlarkLibraryInfo in src:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I am trying to remember why this was important. Is removing that strictly needed?

Copy link
Contributor Author

@jacky8hyf jacky8hyf Oct 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No; in fact I can drop this commit from this particular pull request.

I just find that it is incorrect here because

transitive.append(src[StarlarkLibraryInfo])

This is appending the StarlarkLibraryInfo, not the inner list of files, to the transitive list, causing the depset creation below to fail when there is a bzl_library/starlark_library in srcs because of unmatching types. I found this bug when I was trying to add rules_python/skylib to //pkg:bzl_srcs directly. But later I didn't actually add rules_python/skylib to //pkg:bzl_srcs.

Do you want me to drop this commit from this pull request and create a separate pull request?

Copy link
Contributor Author

@jacky8hyf jacky8hyf Oct 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other words, if I were to use starlark_library instead of bzl_library for the newly added //pkg:rules_pkg_lib target, then I would need this patch (or a variation of it that makes sure the StarlarkLibraryInfo doesn't go into the depset directly).

That being said, I think @bazel_skylib//lib:paths and @rules_python//python:defs_bzl are both bzl_librarys, and due to this:

https://github.com/bazelbuild/bazel-skylib/blob/56b235e700ddd6a15b7d9fa1803fa7a84048471e/rules/private/bzl_library.bzl#L37

It is actually okay to just use the default files from these libraries without poking into StarlarkLibraryInfo, so this contional branch could be deleted, as I am doing in this patch.

Though, a pedantic person may also argue that, for this to be 100% correct, we should use the fields in StarlarkLibraryInfo from dependencies to construct StarlarkLibraryInfo of this starlark_library...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on bazelbuild/bazel#18391 (comment)

it seems we can drop the use of starlarklibrary and bzl_library.

It seem to work for me by changing pkg/BUILD:bzl_srcs to a filegroup(name ="rules_pkg_lib"
and referring to that in doc_build/BUILD, deleting the existing rules_pkg_lib there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants