Proposal: Babylon React Native Build Architecture

[okwasniewski]

The Problem

The current build architecture of Babylon React Native is not scalable for multiple (Apple) platforms.

Currently, in the library podspec, we ship a bunch of .a files compiled for iOS.

If we want to support other platforms like macOS we need to compile the static libraries for macOS as well, same thing goes for visionOS.

Here is the podspec of react-native-babylon:

require "json"

package = JSON.parse(File.read(File.join(__dir__, "package.json")))

Pod::Spec.new do |s|
  s.name         = "react-native-babylon"
  s.version      = package["version"]
  s.summary      = package["description"]
  s.homepage     = package["homepage"]
  s.license      = package["license"]
  s.authors      = package["author"]

  s.platforms    = { :ios => "12.0" }
  s.source       = { :git => package["repository"]["url"], :tag => s.version }

  s.source_files = "ios/**/*.{h,m,mm}"
  s.requires_arc = true
  s.xcconfig     = { 'USER_HEADER_SEARCH_PATHS' => '$(inherited) ${PODS_TARGET_SRCROOT}/shared ${PODS_TARGET_SRCROOT}/../react-native/shared' }

  s.vendored_libraries = 'ios/libs/*.a'

  s.frameworks = "MetalKit", "ARKit"

  s.dependency "React"
end

The Solution

The solution from the Apple ecosystem is to use xcframeworks which are a bundle of .frameworks for multiple platforms. We could create Babylon.xcframework that would contain .frameworks for iOS, macOS and visionOS. On top of that, we could also create a version of the framework without XR support (BabylonBaseKit.xcframework).

Then depending of the build type we could link the correct framework to the project.

This would also fix the issue with linking in the Babylon React Native RNTA example app. Currently, for the example app we generate a new .xcodeproj and add it to the workspace. This solution will work for iOS but not for other platforms, we would need to generate a new .xcodeproj for each platform which is not scalable.

Currently, on post-install we run iosCmake which generates the .xcodeproj:

function iosCMake() {
  console.log(chalk.black.bgCyan("Running CMake for iOS..."));
  exec("cmake -B ../../Build/iOS -G Xcode", {
    cwd: "node_modules/@babylonjs/react-native-iosandroid/ios",
  });
}

But for additional platforms, we would need to generate a new .xcodeproj for ex for visionOS:

function visionOSCMake() {
  console.log(chalk.black.bgCyan("Running CMake for visionOS..."));
  exec("cmake -B ../../Build/visionOS -G Xcode VISIONOS=ON", {
    cwd: "node_modules/@babylonjs/react-native-iosandroid/ios",
  });
}

Then we would get duplicated linked projects in the workspace for each platform.

The question is whether linking of the xcodeproj is necessary for your workflow? Do you need to modify the source code of BabylonNative from Babylon React Native? If not we can just link the .xcframework to the project and not the source code.

This is a common practice for libraries that depend on the source code of other libraries.

Another option would be to integrate Babylon Native as a cocoapod which would allow for source code modifications but this would require even bigger changes to the build system.

I think adjusting the CMake scripts of BabylonNative to generate .xcframeworks would be the best solution for now. That would also allow people to use BabylonNative in their projects without the need to link the source code.

Im looking forward to your feedback on this proposal.

[ryantrem]

Yes, we've talked about switching to xcframeworks many times in the past. One of the common reasons is to be able to include arm64 simulator libs in the package, which we cannot do with multi-architecture static libs. Some thoughts and questions:

This solution will work for iOS but not for other platforms, we would need to generate a new .xcodeproj for each platform which is not scalable.

Is it really not scalable? It's still only a small handful of platforms. Could we not have a generated xcodeproj for each, and if needed a xcworkspace for each?

Do you need to modify the source code of BabylonNative from Babylon React Native?

I would think there probabably needs to be some way of doing this to make it practicaly to debug Babylon Native code in the context of Babylon React Native, especially for cases where JSI behavior is different from N-API behavior.

Another option would be to integrate Babylon Native as a cocoapod which would allow for source code modifications but this would require even bigger changes to the build system.

I assume you mean cocoapod with source code (since yous aid "allow for source code modifications"). I feel like that would be quite hard though since we rely on CMake to generate the xcodeproj. CMake can't generate a podspec, right? We have discussed the idea in the past of having BN binary packages that can be used standalone, and would also be used by BRN, but I think there are a lot of complexities with this (again, JSI). But this was also more in the context of the end user, not the BRN dev workflow.

I think adjusting the CMake scripts of BabylonNative to generate .xcframeworks would be the best solution for now. That would also allow people to use BabylonNative in their projects without the need to link the source code.

So this is having BN CMake generate .xcframeworks, and then we would just include a single .xcframework supporting multiple platforms/architectures in the BRN xcworkspace, yes?

[okwasniewski]

Yes, we've talked about switching to xcframeworks many times in the past. One of the common reasons is to be able to include arm64 simulator libs in the package, which we cannot do with multi-architecture static libs.

Great! This is going to enhance the user's experience for sure!

In my proposal I didn't properly separate internal use case from external (publishing).

So the internal use case of getting React Native Test App to work with current architecture it's possible. As you mentioned we can generate and link a project per platform and if (maintainer) wants to change the platform, they re-generate the project using CMake.

Is it really not scalable? It's still only a small handful of platforms. Could we not have a generated xcodeproj for each, and if needed a xcworkspace for each?

I've meant it's not scalable in terms of publishing the package. We can work around this with as mentioned above using Test App.

I feel like that would be quite hard though since we rely on CMake to generate the xcodeproj. CMake can't generate a podspec, right?

It can't and I agree this would be hard and time-consuming.

So this is having BN CMake generate .xcframeworks, and then we would just include a single .xcframework supporting multiple platforms/architectures in the BRN xcworkspace, yes?

Yes, exactly! This is how Hermes xcframework looks like (one framework for every platform):

I'm going to continue working on the BRNPlayground and hopefully find a fix for real devices (now it works properly only on simulators) and then we can continue the discussion to support xcframework publishing

[ryantrem]

Ok great, and just to be clear, beyond how this impacts local development of BRN, it would also mean distributing an xcframework with .so files in the package instead of .a files, correct? I think this might be the topic of discussion you refer to above as "continue the discussion to support xcframework publishing," yes?

[CedricGuillemet]

is prebuilt .so only containing BN code? If yes, can it be prebuilt as a post merge PR step, packaged and used in this new build architecture? I'd like to reduce build times. Overall, is there any step that can be extracted from current builds?

[okwasniewski]

Local development will stay the same (just needs to be fixed) and added support for multiple platforms. We could also use xcframework there but as you mentioned you want to be able to debug Babylon Native code while working on BRN.

Regarding xcframework, typically they contain .framework files (which contain executables) for each platform.

can it be prebuilt as a post merge PR step, packaged and used in this new build architecture? I'd like to reduce build times.

@CedricGuillemet Yeah, for sure we can build it on the CI. By "reduce build times" you mean reduce it in the internal testing app or for users of Babylon?

[CedricGuillemet]

Internal testing/building mainly. Like PR or local builds. If BN was not built each time, it would reduce a lot build times. Build time for users is OK, I think.

[okwasniewski]

Yeah for users its already prebuilt 😅

In theory we could leverage xcframeworks for local builds and optionally allow to build babylon native from source. What do you think?

[CedricGuillemet]

yes, more freedom to build or not BN depending on use. I've added this installation steps recently in BN that run after a PR is merged: https://github.com/BabylonJS/BabylonNative/blob/master/.github/jobs/test_install_ios.yml
Is it possible to bundle that and (optionaly) use them in BRN build?

[okwasniewski]

Yeah, I think we can modify it to upload an artifact that can be later on downloaded locally

[okwasniewski]

I've successfully integrated the generated .xcframework into the RNTA TestApp.

The builds are now super fast and it works on the device!

Are you okay with keeping the "build from source" option available only for CMake < 3.24? This can be fixed later while I can iterate on this xcframework build script and integration. I've been trying to fix this CMake issue for quite a while now and Im running out of ideas 😅 By using xcframework I can faster integrate visionOS support and work on new architecture support.