[native_toolchain_c] Add linking for Android #2347

goderbauer · 2025-06-06T10:02:16Z

Towards #1376

This enables linking for Android on Windows, Linux, and MacOS hosts.

With this PR the linker is no longer invoked directly. Instead, it is invoked via the compiler driver (clang, etc.), which has the advantage that we don't have to hand-translate the arguments to the format the linker expects them to be. The compiler driver is doing that for us and we can mostly reuse the compiler driver invocation.

I’ve reviewed the contributor guide and applied the relevant portions to this PR.

Contribution guidelines:

See our contributor guide for general expectations for PRs.
Larger or significant changes should be discussed in an issue before creating a PR.
Contributions to our repos should follow the Dart style guide and use dart format.
Most changes should add an entry to the changelog and may need to rev the pubspec package version.
Changes to packages require corresponding tests.

Note that many Dart repos have a weekly cadence for reviewing PRs - please allow for some latency before initial review feedback.

github-actions · 2025-06-06T10:04:22Z

PR Health

Changelog Entry ✔️

Package	Changed Files

Changes to files need to be accounted for in their respective changelogs.

API leaks ✔️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package	Leaked API symbols

License Headers ✔️

// Copyright (c) 2025, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

Files
no missing headers

All source files should start with a license header.

Unrelated files missing license headers

Files
pkgs/jni/lib/src/third_party/generated_bindings.dart
pkgs/native_doc_dartifier/example/native_doc_dartifier_example.dart
pkgs/native_doc_dartifier/lib/native_doc_dartifier.dart
pkgs/native_doc_dartifier/lib/src/native_doc_dartifier_base.dart
pkgs/objective_c/lib/src/ns_input_stream.dart

github-actions · 2025-06-06T12:41:36Z

Package publishing

Package	Version	Status	Publish tag (post-merge)
package:code_assets	0.19.3	already published at pub.dev
package:data_assets	0.19.1	already published at pub.dev
package:ffi	2.1.4	already published at pub.dev
package:ffigen	19.1.0-wip	WIP (no publish necessary)
package:hooks	0.19.3	already published at pub.dev
package:hooks_runner	0.21.0	already published at pub.dev
package:jni	0.14.2	already published at pub.dev
package:jnigen	0.15.0-wip	WIP (no publish necessary)
package:native_doc_dartifier	0.0.1-pre	already published at pub.dev
package:native_toolchain_c	0.16.3	ready to publish	`native_toolchain_c-v0.16.3`
package:objective_c	8.1.0-wip	WIP (no publish necessary)
package:swift2objc	0.0.1-wip	WIP (no publish necessary)
package:swiftgen	0.0.1-wip	WIP (no publish necessary)

Documentation at https://github.com/dart-lang/ecosystem/wiki/Publishing-automation.

goderbauer · 2025-06-06T13:37:05Z

Hm, it's not clear to me why the bot wants the version to be bumped to 0.17.0

dcharkes · 2025-06-06T14:28:41Z

Hm, it's not clear to me why the bot wants the version to be bumped to 0.17.0

You should be able to repro it with the dart_apitool yourself with https://github.com/dart-lang/ecosystem/blob/64aac3a9c4606950bcf6c8729f01ed8548b9ed87/pkgs/firehose/lib/src/health/health.dart#L152

pkgs/native_toolchain_c/test/clinker/objects_cross_test.dart

goderbauer · 2025-06-10T13:30:24Z

You should be able to repro it with the dart_apitool yourself

Thank you! I am pretty sure that the tool is wrong here based on what it has identified as a breaking change in a local run:

BREAKING CHANGES
├── Function "environmentFromBatchFile" removed (CE10)
├─┬ Class CBuilder
│ ├─┬ Constructor library
│ │ └── Type of parameter "linkModePreference" changed. InvalidType -> LinkModePreference? (CE08)
│ └─┬ Field linkModePreference
│   └── Type of field changed. InvalidType -> LinkModePreference? (CF04)
└─┬ Class CLinker
  ├─┬ Constructor library
  │ └── Type of parameter "linkModePreference" changed. InvalidType -> LinkModePreference? (CE08)
  └─┬ Field linkModePreference
    └── Type of field changed. InvalidType -> LinkModePreference? (CF04)

The environmentFromBtachFile wasn't touched in this PR, it is still there and properly exported. The tool also seems to be unable to correctly infer the type of the linkModePreference parameter in the old version of this package and now that it can determine the correct type it thinks the type has changed. That's not correct, the type remains unchanged. I am going to overwrite the check for now.

Edit: It looks like the dart_apitool may have trouble with native_toolchain_c being part of a pub workspace. It seems to be checking out the previous version of the package from pub and analyzing that code outside of its workspace, which seems to result in some problems. If I actually check out the whole repository at the old version's revision and have the tool compare that with the current revision, it correctly detects no breaking change: No breaking changes!

coveralls · 2025-06-10T14:23:29Z

coverage: 82.3%. first build
when pulling 1ec9e5a on goderbauer:android-linker
into fd9415b on dart-lang:main.

dcharkes · 2025-06-10T19:37:19Z

Edit: It looks like the dart_apitool may have trouble with native_toolchain_c being part of a pub workspace. It seems to be checking out the previous version of the package from pub and analyzing that code outside of its workspace, which seems to result in some problems. If I actually check out the whole repository at the old version's revision and have the tool compare that with the current revision, it correctly detects no breaking change: No breaking changes!

It does indeed check out the package from pub. See the discussion here: bmw-tech/dart_apitool#216

The packages on pub.dev should work, they are used in the flutter template and flutter integration tests. It's fishy that it fails. (Yeah indeed just apply the label for now.)

dcharkes · 2025-06-10T19:39:23Z

pkgs/native_toolchain_c/lib/src/cbuilder/clinker.dart

@@ -53,10 +53,12 @@ class CLinker extends CTool implements Linker {
    required LinkOutputBuilder output,
    required Logger? logger,
  }) async {
-    if (OS.current != OS.linux || input.config.code.targetOS != OS.linux) {
+    const supportedTargetOSs = [OS.linux, OS.android];
+    if (!supportedTargetOSs.contains(input.config.code.targetOS)) {


This no longer checks if the hostOS is Linux if the targetOS is Linux.

I was assuming that we only check for combinations that should work, but currently don't because they are still WIP - similarly to how cbuilder.dart is also not checking that you can only target linux from linux.

pkgs/native_toolchain_c/test/clinker/objects_helper.dart

pkgs/native_toolchain_c/test/clinker/treeshake_cross_android_test.dart

mosuem

Does this mean that users need to have clang or similar installed? Linux will have ld preinstalled, but not clang.

pkgs/native_toolchain_c/lib/src/cbuilder/linker_options.dart

pkgs/native_toolchain_c/test/helpers.dart

pkgs/native_toolchain_c/lib/src/cbuilder/run_cbuilder.dart

goderbauer · 2025-06-11T13:45:21Z

Does this mean that users need to have clang or similar installed? Linux will have ld preinstalled, but not clang.

In theory, it does not require clang on linux and I double checked that in practice everything works if the compiler driver to be used resolves to gcc, which should be installed by default on Linux. However, the native_toolchain_c package currently hard-codes the use of clang on linux systems if host and target OS and architecture match:

native/pkgs/native_toolchain_c/lib/src/cbuilder/compiler_resolver.dart

Lines 65 to 70 in f127f35

    
           // TODO(dacoharkes): Support falling back on other tools. 
        
           if (targetArch == hostArchitecture && 
        
               targetOS == hostOS && 
        
               hostOS == OS.linux) { 
        
             return clang; 
        
           }

Not sure why this is done. When I remove this and let it instead fall through to select the approriate gcc on linux (see below) it just works as expected:

native/pkgs/native_toolchain_c/lib/src/cbuilder/compiler_resolver.dart

Lines 73 to 86 in f127f35

    
           if (hostOS == OS.linux) { 
        
             switch (targetArch) { 
        
               case Architecture.arm: 
        
                 return armLinuxGnueabihfGcc; 
        
               case Architecture.arm64: 
        
                 return aarch64LinuxGnuGcc; 
        
               case Architecture.ia32: 
        
                 return i686LinuxGnuGcc; 
        
               case Architecture.x64: 
        
                 return x86_64LinuxGnuGcc; 
        
               case Architecture.riscv64: 
        
                 return riscv64LinuxGnuGcc; 
        
             } 
        
           }

@dcharkes Since you added this selection code originally, do you have any insights into this? Is it on purpose that we force clang on linux if you don't cross-compile? If yes, do we consider it a requirement that clang is installed on linux if one wants to use the native_toolchain_c package?

dcharkes · 2025-06-11T16:26:03Z

Is it on purpose that we force clang on linux if you don't cross-compile?

No. But we haven't implemented fallback tools yet:

// TODO(dacoharkes): Support falling back on other tools.

The tool recognizer works with both clang and gcc if passed in in the CCompilerConfig, but the tool discovery doesn't use fallbacks.

Related issue: [native_toolchain_c] Tool variants #20

We do have a requirement that we use the compiler/linker/archiver that belong together. We shouldn't be using a linker from a different compiler toolchain. So the fallback logic can't be separate for the compiler/linker. So we might need to introduce something like a ToolGroup and do fallbacks at that level. (And if we make the tool resolver logic more fancy, we should probably also address #19.)

goderbauer · 2025-06-12T07:54:15Z

We shouldn't be using a linker from a different compiler toolchain.

This PR actually simplifies this then since it removes calling the linker directly and instead goes through clang or gcc or whatever compiler driver is available.

I am submitting this as is then since #20 is a separate open issue regarding clang vs. gcc on Linux.

github-actions bot added the package:native_toolchain_c label Jun 6, 2025

goderbauer force-pushed the android-linker branch 3 times, most recently from 84b1b48 to ca6c274 Compare June 6, 2025 13:22

goderbauer mentioned this pull request Jun 6, 2025

[native_toolchain_c] Implement CLinker for different platforms #1376

Open

6 tasks

goderbauer requested review from dcharkes and mosuem June 6, 2025 13:36

dcharkes reviewed Jun 6, 2025

View reviewed changes

pkgs/native_toolchain_c/test/clinker/objects_cross_test.dart Show resolved Hide resolved

goderbauer added the skip-breaking-check label Jun 10, 2025

goderbauer requested a review from dcharkes June 10, 2025 13:30

goderbauer added 5 commits June 10, 2025 15:46

[native_toolchain_c] Add linking for Android

f4846fa

fix -u syntax

c0987f5

speculative windows fix

7a18291

test on more android API levels

c4c8cc4

fix naming

330b8db

goderbauer force-pushed the android-linker branch from ca6c274 to 330b8db Compare June 10, 2025 13:46

goderbauer added skip-breaking-check and removed skip-breaking-check labels Jun 10, 2025

dcharkes approved these changes Jun 10, 2025

View reviewed changes

mosuem reviewed Jun 11, 2025

View reviewed changes

reviews

1ec9e5a

goderbauer merged commit 08c98ed into dart-lang:main Jun 12, 2025
18 checks passed

goderbauer deleted the android-linker branch June 12, 2025 07:54

dcharkes mentioned this pull request Jun 13, 2025

[infra] The breaking-change health bot is broken with workspaces #2361

Open

[native_toolchain_c] Add linking for Android #2347

[native_toolchain_c] Add linking for Android #2347

Uh oh!

Conversation

goderbauer commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Health

Uh oh!

github-actions bot commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Package publishing

Uh oh!

goderbauer commented Jun 6, 2025

Uh oh!

dcharkes commented Jun 6, 2025

Uh oh!

Uh oh!

goderbauer commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dcharkes commented Jun 10, 2025

Uh oh!

dcharkes Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

goderbauer Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mosuem left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

goderbauer commented Jun 11, 2025

Uh oh!

dcharkes commented Jun 11, 2025

Uh oh!

goderbauer commented Jun 12, 2025

Uh oh!

Uh oh!

Uh oh!

goderbauer commented Jun 6, 2025 •

edited

Loading

github-actions bot commented Jun 6, 2025 •

edited

Loading

github-actions bot commented Jun 6, 2025 •

edited

Loading

goderbauer commented Jun 10, 2025 •

edited

Loading

coveralls commented Jun 10, 2025 •

edited

Loading