Update rules_jvm_external to use the Starlark version of aar_import after the native version was removed from Bazel #1149

ahumesky · 2024-05-24T00:16:07Z

The native version of aar_import has been removed from bazel at head, so use the Starlark version instead.

This PR:

Updates rules_jvm_external to use the Starlark version of aar_import automatically if the version of Bazel used does not contain the native version of aar_import.
Updates the version of Bazel in .bazelversion to 7.2.1 because the latest Starlark Android rules are only compatible with 7.2.1+
Updates the version of rules_android used to the latest commit, because version 0.1.1 is basically just wrappers around the native versions of the rules. However, version 0.1.1 of rules_android is still used in the WORKSPACE file so that test that run with Bazel 5 and Bazel 6 use the native rules, and Bazel 7+ uses the Starlark rules via bzlmod / MODULE.bazel, because the latest version of the Starlark rules are only compatible with Bazel 7.2.1+.
Updates the default label to import aar_import from, from @build_bazel_rules_android//android:rules.bzl to @rules_android//rules:rules.bzl, both to match the bzlmod style name of the repo ("rules_android") and to use the latest label inside the repo (//android:rules.bzl exists, but only for backwards compatibility).
Fixes some dependency issues with aar_import_that_consumes_the_downloaded_file_directly and sceneform_rendering aar_imports, because the Starlark version of aar_import performs additional checks ("ImportDepsChecker") to ensure that the dependencies of every class in the imported aar are in the dependencies. Notably, the class com.google.ar.sceneform.assets.Loader is used somewhere in the aar or its dependencies, but I couldn't find this class anywhere in any aars on maven.google.com, so I ended up having to stub it out with an empty implementation to satisfy the check.
Disables --experimental_sibling_repository_layout added in b6631f9 because the latest rules_android uses protos in go, and there is a problem compiling protos in go with --experimental_sibling_repository_layout: rules_go + protobufs + experimental_sibling_repository_layout fails to build rules_go#3947
Adds --experimental_google_legacy_api to .bazelrc until Remove dependencies on --experimental_google_legacy_api and --experimental_enable_android_migration_apis bazelbuild/rules_android#219 is fixed

ahumesky · 2024-05-24T00:30:19Z

Oh, I just noticed the tests run on bazel as far back as bazel 5.4.0:
https://github.com/bazelbuild/rules_jvm_external/blob/71e08c82709b24d12e2d1040d86caa05bf6a8399/.bazelci/presubmit.yml#L40

The starlark Android rules will only work with 7.2.0 (releasing soon). Might need to segment the tests so that the tests with the starlark version of aar_import are only run with 7.2.0

ahumesky · 2024-05-24T03:10:51Z

The windows tests will also be a problem because windows support for starlark rules_android is low priority

shs96c · 2024-05-25T12:32:58Z

We still need to support Windows users, so those tests will need to pass properly before we can land this PR.

ahumesky · 2024-05-29T01:35:22Z

bazelbuild/rules_android#235 should fix up the windows problems

jin · 2024-05-30T06:10:21Z

Error looks legit:


(19:43:56) ERROR: C:/b/rxn2cr7v/external/rules_android~/tools/android/BUILD:273:6: in alias rule @@rules_android~//tools/android:dexsharder: target '@@bazel_tools//src/tools/android/java/com/google/devtools/build/android/dexer:DexFileSplitter' is not visible from target '@@rules_android~//tools/android:dexsharder'. Check the visibility declaration of the former target if you think the dependency is legitimate
--
  | (19:43:56) ERROR: C:/b/rxn2cr7v/external/rules_android~/tools/android/BUILD:273:6: Analysis of target '@@rules_android~//tools/android:dexsharder' failed
  | (19:43:57) ERROR: Analysis of target '//tests/unit/aar_import:starlark_aar_import_test' failed; build aborted: Analysis failed

from https://buildkite.com/bazel/rules-jvm-external/builds/4289#018fc5e0-bc05-4701-80b8-55fdf1f6ddb0 (and others)

ahumesky · 2024-05-30T15:43:54Z

That's fixed in 7.2.0, specifically bazelbuild/bazel#22416, but there are other problems revealed after bazelbuild/rules_android#235

I'll make this a draft until CI is passing

…utomatically if the version of Bazel used does not contain the native version of aar_import. This also updates the version of rules_android to the latest commit, because version 0.1.1 is basically just wrappers around the native versions of the rules. This also disbles `--experimental_sibling_repository_layout` added in bazel-contrib@b6631f9 because the latest rules_android uses protos in go, and there is a problem compiling protos in go with `--experimental_sibling_repository_layout`: bazel-contrib/rules_go#3947 Fixes bazel-contrib#1139.

ahumesky · 2024-07-09T23:02:35Z

The tests now work with the latest version of bazel:
https://buildkite.com/bazel/rules-jvm-external/builds/4383#01909944-3f4b-4b31-9bc5-128f223a586e

and the others are failing due to an older version of bazel being used and incompatible visibility on some targets in @bazel_tools. It might be tricky to work around these but I'll see what workarounds can be put into rules_android.

shs96c · 2024-07-10T10:16:47Z

We do need the CI tests to pass before we can land this. We're fine asking for the latest versions of Bazel 5 and 6, but our version 7 support is more flexible.

… of rules_android that rules_kotlin uses

ahumesky · 2024-07-10T21:22:06Z

We do need the CI tests to pass before we can land this.

Right I wouldn't want to submit something that breaks CI :)

We're fine asking for the latest versions of Bazel 5 and 6, but our version 7 support is more flexible.

After going through most of the initial issues, it looks like the starlark android rules won't work with bazel 5 because at minimum they use visibility() for bzl visibility, which was introduced in bazel 6:

https://github.com/search?q=repo%3Abazelbuild%2Frules_android+%22visibility%28%22&type=code

I might be able to get the rules to work with bazel 6, but I'm not sure how long we can guarantee that (really we've only been developing for bazel 8 and backporting some things as needed to bazel 7).

Maybe I can arrange this so that the native android rules are used with the tests that use bazel 5 and bazel 6, and the starlark rules with latest bazel (8+). That might just amount to removing the use_starlark_android_rules = True and letting it auto select the starlark rules on latest bazel where native aar_import is deleted. There would still be problems with loading and registering the starlark android rules in the WORKSPACE file (e.g. visibility() above) -- but blzmod is used by default in bazel 7+, so I could perhaps just remove the starlark android rules from the WORKSPACE and leave them in MODULE.bazel.

…rectly" because Starlark aar_import checks that there are no missing class imports in aars

…rt regression_testing_coursier_install.json

… use with bazel 5 and bazel 6 in CI

ahumesky · 2024-07-11T03:21:49Z

This stubbed out action is failing on windows:
https://github.com/bazelbuild/rules_android/blob/main/src/validations/aar_import_checks/BUILD

So that will have to be another update in rules_android

But it looks like everything else is working: native android rules on bazel 5 / 6, starlark android rules on 7+

Ran into this while getting rules_jvm_export and the rules_android working together: bazel-contrib/rules_jvm_external#1149 (comment) PiperOrigin-RevId: 651557752 Change-Id: I30bbe6792863b7d5ed178f0d074a3bf73eb631db

ahumesky · 2024-07-12T21:57:07Z

Hm, well bazelbuild/rules_android@746589b should have fixed the windows tests, but looks like the ValidateAAR action is still failing... I'll have to try something else but out of time today. Really that action does nothing (it's just a stub action that actually does nothing) so I might just try to disable it

ahumesky · 2024-08-01T22:07:34Z

With bazelbuild/rules_android@f4b6edb all the tests are now passing

ahumesky · 2024-08-01T22:29:05Z

I updated the description and marked this ready for review

MODULE.bazel

private/rules/coursier.bzl

private/rules/maven_install.bzl

tests/integration/override_targets/Loader.java

ahumesky · 2024-08-07T15:28:38Z

Thank you!

…import after the native version was removed from Bazel (bazel-contrib#1149)" This reverts commit ba7310c. The default label for AAR imports is `@rules_android//android:rules.bzl`. This imports `@rules_android//rules:providers.bzl`, which calls `@rules_android//rules/reexport_providers.bzl`. This means that any downstream user of `rules_jvm_external` with a sufficiently recent version of Bazel needs to add `build --experimental_google_legacy_api` to their `.bazelrc` That's a significantly breaking change, and one that it's not reasonable to ask our users to do, especially since the vast majority of projects don't use the Android rules.

…import after the native version was removed from Bazel (#1149)" (#1215) This reverts commit ba7310c. The default label for AAR imports is `@rules_android//android:rules.bzl`. This imports `@rules_android//rules:providers.bzl`, which calls `@rules_android//rules/reexport_providers.bzl`. This means that any downstream user of `rules_jvm_external` with a sufficiently recent version of Bazel needs to add `build --experimental_google_legacy_api` to their `.bazelrc` That's a significantly breaking change, and one that it's not reasonable to ask our users to do, especially since the vast majority of projects don't use the Android rules.

ahumesky requested review from jin, Wyverald, shs96c and cheister as code owners May 24, 2024 00:16

ahumesky marked this pull request as draft May 30, 2024 15:44

ahumesky added 5 commits July 9, 2024 16:44

Restore .bazelrc

bbc23af

Don't overwrite bazelrc

a006092

Update rules_android commit to inclue bazelbuild/rules_android@d25741d

cde052d

Update comments

9b496c7

ahumesky force-pushed the master branch from 7476b23 to 9b496c7 Compare July 9, 2024 20:49

Update to latest rules_android commit

2191b11

Load the rules_android rules before rules_kotlin to avoid the version…

743be05

… of rules_android that rules_kotlin uses

ahumesky changed the title ~~Update rules_jvm_external to use the Starlark version of aar_import~~ Update rules_jvm_external to use the Starlark version of aar_import after the native version was removed from Bazel Jul 10, 2024

ahumesky added 5 commits July 10, 2024 17:28

Update .bazelversion from 7.1.0 to 7.2.1

d0d10d3

Add dependencies for "aar_import_that_consumes_the_downloaded_file_di…

779baae

…rectly" because Starlark aar_import checks that there are no missing class imports in aars

Workarounds for aar_import ImportDepsChecker checks

9d5969c

Remove unnecessary com.google.ar.sceneform:assets:1.10.0 dep and reve…

fb2bed6

…rt regression_testing_coursier_install.json

Use the native Android rules in WORKSPACE via rules_android 0.1.1 for…

be36692

… use with bazel 5 and bazel 6 in CI

Update to latest rules_android

7474b3e

Update to latest rules_android

2016989

ahumesky marked this pull request as ready for review August 1, 2024 22:24

ahumesky mentioned this pull request Aug 2, 2024

Remove the Native Android Rules from Bazel bazelbuild/bazel#23199

Closed

10 tasks

jin approved these changes Aug 6, 2024

View reviewed changes

MODULE.bazel Outdated Show resolved Hide resolved

private/rules/coursier.bzl Outdated Show resolved Hide resolved

private/rules/maven_install.bzl Outdated Show resolved Hide resolved

tests/integration/override_targets/Loader.java Outdated Show resolved Hide resolved

Update to rules_android 0.5.1, address review comments

d90dd35

jin approved these changes Aug 7, 2024

View reviewed changes

jin merged commit ba7310c into bazel-contrib:master Aug 7, 2024
9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update rules_jvm_external to use the Starlark version of aar_import after the native version was removed from Bazel #1149

Update rules_jvm_external to use the Starlark version of aar_import after the native version was removed from Bazel #1149

ahumesky commented May 24, 2024 •

edited

Loading

ahumesky commented May 24, 2024

ahumesky commented May 24, 2024

shs96c commented May 25, 2024

ahumesky commented May 29, 2024

jin commented May 30, 2024

ahumesky commented May 30, 2024

ahumesky commented Jul 9, 2024 •

edited

Loading

shs96c commented Jul 10, 2024

ahumesky commented Jul 10, 2024

ahumesky commented Jul 11, 2024 •

edited

Loading

ahumesky commented Jul 12, 2024 •

edited

Loading

ahumesky commented Aug 1, 2024

ahumesky commented Aug 1, 2024

ahumesky commented Aug 7, 2024

Update rules_jvm_external to use the Starlark version of aar_import after the native version was removed from Bazel #1149

Update rules_jvm_external to use the Starlark version of aar_import after the native version was removed from Bazel #1149

Conversation

ahumesky commented May 24, 2024 • edited Loading

ahumesky commented May 24, 2024

ahumesky commented May 24, 2024

shs96c commented May 25, 2024

ahumesky commented May 29, 2024

jin commented May 30, 2024

ahumesky commented May 30, 2024

ahumesky commented Jul 9, 2024 • edited Loading

shs96c commented Jul 10, 2024

ahumesky commented Jul 10, 2024

ahumesky commented Jul 11, 2024 • edited Loading

ahumesky commented Jul 12, 2024 • edited Loading

ahumesky commented Aug 1, 2024

ahumesky commented Aug 1, 2024

ahumesky commented Aug 7, 2024

ahumesky commented May 24, 2024 •

edited

Loading

ahumesky commented Jul 9, 2024 •

edited

Loading

ahumesky commented Jul 11, 2024 •

edited

Loading

ahumesky commented Jul 12, 2024 •

edited

Loading