Adding Mac Catalyst target to the MacOS build

[torfluor]

When I'm trying to use the MacOS NuGet package with a .NET MAUI Catalyst project the dll is not found. I get the following error:

Exception has occurred: CLR/System.Reflection.TargetInvocationException
An unhandled exception of type 'System.Reflection.TargetInvocationException' occurred in Microsoft.MacCatalyst.dll: 'Exception has been thrown by the target of an invocation.'
 Inner exceptions found, see $exception in variables window for more details.
 Innermost exception 	 System.DllNotFoundException : pdfium

I also tried to use the iOS NuGet package, but I get the following compile error:

/usr/local/share/dotnet/packs/Microsoft.MacCatalyst.Sdk/17.2.8053/targets/Xamarin.Shared.Sdk.targets(1560,3): error : ld: building for Mac Catalyst, but linking in dylib built for iOS, file 'obj/Debug/net8.0-maccatalyst/maccatalyst-arm64/nativelibraries/Contents/MonoBundle/libpdfium.dylib' [/Users/tor/dev/MauiPdfiumTest/MauiPdfiumTest/MauiPdfiumTest.csproj::TargetFramework=net8.0-maccatalyst]
/usr/local/share/dotnet/packs/Microsoft.MacCatalyst.Sdk/17.2.8053/targets/Xamarin.Shared.Sdk.targets(1560,3): error : clang: error: linker command failed with exit code 1 (use -v to see invocation) [/Users/tor/dev/MauiPdfiumTest/MauiPdfiumTest/MauiPdfiumTest.csproj::TargetFramework=net8.0-maccatalyst]

[torfluor]

By manually adding the "maccatalyst" target in the libpdfium.dylib with vtool and then copying the libpdfium.dylib file into my .app package I'm able to use Pdfium in my MAUI application on Mac.

Here is the vtool command I used:
vtool -set-build-version "maccatalyst" 11.0.0 14.0.0 -set-build-tool "maccatalyst" 4 19.0.0 -output catalyst.dylib libpdfium.dylib

Would be great if the maccatalyst target could be added in your build process :)

[bblanchon]

@sungaila, I think this one is for you.

[sungaila]

Hi @torfluor, if you can help to create iOS and Catalyst binaries for pdfium, I'll add the missing NuGet packages needed for MAUI. I don't have a Mac myself to test any of this.

Please take a look at the build scripts and modify them to create the required binaries. It would be very helpful if someone who is familiar with Mac development could help here.

[torfluor]

Hi @sungaila, I would be happy to contribute! I don't have much experience with build scrips and cross platform builds, but looking at the script files I think it would be easies to do separate builds for maccatalyst instead of trying to add multiple targets to the Mac builds. As the library works on Catalyst by just adding the "maccatalyst" target, I would try with the following changes:

build.sh
Add maccatalyst to the OS_NAMES

05-configure.sh
copy the mac case into a new maccatalyst case and set mac_deployment_target = "10.15.0" which is the minimum for Catalyst

07-stage.sh
Add maccatalyst to the mac|ios case

08-test.sh
Add maccatalyst to the Mac case

I have a M1 MacBook Air and an old Intel MacBook Pro that I can test the NuGet packages on. I can also set up a cloud build server to test the suggested changes, but then I would need instructions on how to set it up.

[sungaila]

I've tried for a few hours to get this working in my fork but I couldn't do it. The pdfium build files don't have catalyst as a target and I'm not clever enough to write git patches to get these working (see https://github.com/sungaila/pdfium-binaries/actions/runs/9596534182/job/26463597042).

Maybe @bblanchon can help out with that. I'd still help with the NuGet packages once the workflows, bash scripts and git patches are in place.

[bblanchon]

The OS names are defined by Google; we cannot add them like this (okay, we did for wasm but that's different).
@torfluor managed to make it work from the original dylib, so I think this is more of a packaging issue, and that's why I pinged @sungaila.

I don't know anything about Mac Catalyst or its interaction with .NET and NuGet.
@torfluor can you explain?

[sungaila]

AFAIK Mac Catalyst is its own build target for dylibs consumed by both macOS and iOS (plus derivatives like iPadOS).

.NET has the maccatalyst target to seperate it from osx and ios. I'd have to create a separate NuGet package for it.

[torfluor]

@bblanchon: What I did with vtool is more of a hack to see if it would run on Catalyst. It's recommended to rebuild the library instead of modifying the dylibs as described by Quinn from Apple.

Maybe this blog entry might be helpful: How to build WebRTC for Mac-Catalyst or Ninja intro for Apple-developers.
It mentions that the target for Catalyst is x86_64-apple-ios13.0-macabi

[sungaila]

So maybe the pdfium build should be run with mac as target_os but with slight compiler argument changes? Like setting the Mac Catalyst ABI and a higher SDK version?

[bblanchon]

An additional build would add another maintenance burden to this project.
If there is no measurable benefit, I prefer going with the vtool approach.
We'll still have the option to do it differently in the future.

[torfluor]

But is vtool available in your build environment? vtool ships as part of Xcode.

[sungaila]

The hosted runners by GitHub have Xcode preinstalled. But I personally would prefer a clean build over a hack (which is not yet confirmed to run on iOS).

[bblanchon]

The macOS and iOS builds have a cumulated build time of 4h20 every week.
It's even worse on my old MacBook: I need a full day to complete a V8 build!
That's why I highly prefer a solution that doesn't require doubling the number of builds.

@torfluor, please provide the full list of commands we must run to produce the catalyst library from the existing builds.

[sungaila]

Maybe the mac_deployment_target could be set higher as a compromise. I mean besides the vtool hacking.

[torfluor]

@bblanchon Only the vtool command is needed.

If you run vtool -show libpdfium.dylib on the Mac lib you will get the following output:

libpdfium.dylib:
Load command 9
      cmd LC_BUILD_VERSION
  cmdsize 32
 platform MACOS
    minos 11.0
      sdk 14.0
   ntools 1
     tool 4
  version 19.0

Now run vtool -set-build-version "maccatalyst" 11.0.0 14.0.0 -set-build-tool "maccatalyst" 4 19.0.0 -output libpdfium.dylib libpdfium.dylib to add the Catalyst target.

Run vtool -show libpdfium.dylib to verify that the output lib has catalyst target. The output should be:

libpdfium.dylib:
Load command 9
     cmd LC_BUILD_VERSION
 cmdsize 32
platform MACOS
   minos 11.0
     sdk 14.0
  ntools 1
    tool 4
 version 19.0
Load command 19
     cmd LC_BUILD_VERSION
 cmdsize 32
platform MACCATALYST
   minos 11.0
     sdk 14.0
  ntools 1
    tool 4
 version 19.0

[sungaila]

I've drafted a pull request for the mentioned changes: #169

@torfluor Please verify if the following NuGet packages are working (macOS, iOS and so on):
bblanchon.PDFium.MacCatalyst.0.1.0.zip
bblanchon.PDFiumV8.MacCatalyst.0.1.0.zip

[torfluor]

PDFiumV8 works great on my M1 MacBook Air, but using the other crashes my test app with the following error message:

dyld[57063]: Library not loaded: ./libpdfium.dylib
  Referenced from: <FCB5AC57-014F-3D1B-88C5-7CC02BD17DA4> /Users/tor/dev/MauiPdfiumTest/MauiPdfiumTest/bin/Debug/net8.0-maccatalyst/maccatalyst-arm64/MauiPdfiumTest.app/Contents/MacOS/MauiPdfiumTest
  Reason: tried: '/System/iOSSupport./libpdfium.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/System/iOSSupport./libpdfium.dylib' (no such file), '/System/iOSSupport./libpdfium.dylib' (no such file), './libpdfium.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS./libpdfium.dylib' (no such file), './libpdfium.dylib' (no such file), '/System/iOSSupport/Users/tor/dev/MauiPdfiumTest/MauiPdfiumTest/libpdfium.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/System/iOSSupport/Users/tor/dev/MauiPdfiumTest/MauiPdfiumTest/libpdfium.dylib' (no such file), '/System/iOSSupport/Users/tor/dev/MauiPdfiumTest/MauiPdfiumTest/libpdfium.dylib' (no such file), '/Users/tor/dev/MauiPdfiumTest/MauiPdfiumTest/libpdfium.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tor/dev/MauiPdfiumTest/MauiPdfiumTest/libpdfium.dylib' (no such file), '/Users/tor/dev/MauiPdfiumTest/MauiPdfiumTest/libpdfium.dylib' (no such file)
zsh: abort      

[sungaila]

@torfluor Are you sure that the dylib was correctly copied and loaded? The logs says 'libpdfium.dylib' (no such file), over and over.

And since the V8 build worked, it is very odd that the "normal" stripped down build isn't.

I would test it myself but I don't want to invest into a Mac Mini just for debugging Catalyst. 😅

[torfluor]

I think there must be some difference in the Nuget package. Here is how I tested:

1. Copied the nuget packages to my local nuget repository directory
2. Added "" to my .csproj file
3. Ran the project and got the error.
4. Changed to "" in the .csproj file
5. Ran the project again and now to works fine.

I'm using the PDFiumCore NuGet Package for the bindings, and this works fine with the Windows, iOS, and V8 MacCatalyst packages.

[sungaila]

The difference between those NuGet packages are the contained dylib and some unrelated metadata.

I've had the chance to create a blank MAUI app in Visual Studio for Mac. Both NuGet packages will fail the build with the error message

/usr/local/share/dotnet/packs/Microsoft.MacCatalyst.Sdk/16.4.7141/targets/Xamarin.Shared.Sdk.targets(3,3): Error: install_name_tool exited with code 1:
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: changing install names or rpaths can't be redone for: obj/Debug/net7.0-maccatalyst/maccatalyst-x64/nativelibraries/Contents/MonoBundle/libpdfium.dylib.tmp (for architecture x86_64) because larger updated load commands do not fit (the program must be relinked, and you may need to use -headerpad or -headerpad_max_install_names) (Test1)

No idea what that means but the part the program must be relinked, and you may need to use -headerpad or -headerpad_max_install_names sounds like there are some compiler/linker arguments missing in the bash scripts.

[torfluor]

I sometimes got the same build error when I switched between the two NuGet packages. Running dotnet restore and then building again solved the issue for me.

[torfluor]

Any update on this issue? Did the dotnet restore solve your build error?

[sungaila]

@torfluor No, the issues were unrelated to NuGet and restore, I couldn't get the hacked binaries to run.

Someone with knowledge in C++ development and Catalyst libraries is needed here. Best I can do is to help producing the NuGet packages.

[rmarinho]

So I m not an expert, but I do have some friends like I can try ping to see if it helps.
I also have 2 Macs here to help debug this . Right now I just have M1 and M3 so arm64 .
I think opening this up for full .net MAUI support would be great.

I want to start by trying to build locally and generate the nupkg. Is that possible ? do we have build scripts locally ?

[rmarinho]

Is there any chat on discord or something we can chat?

Ok first try

➜  pdfium-binaries git:(master) ✗ ./build.sh mac arm64

~/Projects/pdfium-binaries/build ~/Projects/pdfium-binaries
++ cmake -D CMAKE_OSX_ARCHITECTURES=arm64 /Users/ruimarinho/Projects/pdfium-binaries/example
steps/08-test.sh: line 145: cmake: command not found

[sungaila]

@rmarinho This project has no group chat, you have to ask your questions in the issues.

I would suggest you create a fork of this project and run the GitHub workflows (e.g. build-one.yml) to test the build scripts. Then change the build scripts and re-run the workflows as needed. This is easier than setting up the build toolchain locally.

Also I think there is a proper Mac Catalyst build needed here. The project maintainer does not wish for an other build target so the binaries should be distributed in a repo fork (I have one for Blazor WebAssembly).

I can help you with creating the NuGet packages as soon as you have successfully created Mac Catalyst builds.

[rmarinho]

ok @sungaila I m going to try locally

got stuck here but need to look more ..

++ pushd /Users/ruimarinho/Projects/pdfium-binaries/pdfium
~/Projects/pdfium-binaries/pdfium ~/Projects/pdfium-binaries
++ gn gen /Users/ruimarinho/Projects/pdfium-binaries/pdfium/out
ERROR at //build/config/BUILDCONFIG.gn:278:3: Assertion failed.
  assert(false, "Unsupported target_os: $target_os")
  ^-----
Unsupported target_os: maccatalyst

[bblanchon]

The project maintainer does not wish for an other build target so the binaries should be distributed in a repo fork

@sungaila, that's not what I said.
I said that I strongly prefer generating the maccatalyst from the artifacts of the other Mac builds, as we do for the mac universal binaries.
This greatly simplifies the maintenance and shortens the build time.
If there is a proven value (and not just theoretical) to add four more hours to the build, that's different.

Also, any new build configuration increases the risk of a failed build.
You might have noticed that we missed many builds in recent weeks.
The failures were for configurations that very few people use in practice (V8, Wasm, Musl).
Meanwhile, the regular builds that most people use are blocked.

@rmarinho, I don't think anyone in this thread knows how to build for maccatalyst, but if you know how to do it, we'll be happy to help.

[rmarinho]

Ok, I don't know but I work with people that I almost sure know how to do it , so I will ask around , ( I work in .net maui at Microsoft). And I have a worker that does skiasharp nugetpackages so I think I might ask him for help.
Yeah not sure, but if needed I can setup the build on my fork for net8/net9 packages just for .NET MAUI for example..

[rmarinho]

https://github.com/rmarinho/pdfium-binaries/actions/runs/10634188063/job/29480963251

[sungaila]

https://github.com/rmarinho/pdfium-binaries/actions/runs/10634188063/job/29480963251

It isn't enough to set target_os to maccatalyst and starting the build. This project uses the pdfium build scripts and patches them to add unsupported platforms.

Here is how it is done for iOS: https://github.com/bblanchon/pdfium-binaries/blob/master/patches/ios/pdfium.patch

Here is the original BUILD.gn: https://pdfium.googlesource.com/pdfium/+/refs/heads/main/core/fxge/BUILD.gn

You will need to check the original pdfium build stuff and create git patches for maccatalyst.

[rmarinho]

yeah we want target_environment , I reading it .. and testing locally .. if we see the gn files for iOS they already have support for catalyst in there

[rmarinho]

So now we test this ?

https://github.com/rmarinho/pdfium-binaries/actions/runs/10634833862

[sungaila]

@rmarinho I took your lib and packaged it into a nupkg:
bblanchon.PDFium.MacCatalyst.0.0.0.zip

Just a few suggestions:

• Please use maccatalyst instead of catalyst for the output directory
• We need a x64 build as well (you built arm64 only)
• We need a V8 build as well (both arm64 and x64)

I don't have the time test it right now but I'll create a .NET MAUI app, add this nuget package and run it on macOS later on. Maybe you (@rmarinho) can test this on macOS and iOS/iPadOS as well.

[rmarinho]

Ok I did a build all that added xc64 .. didn t do v8 yet, lets test this works.. what's the variable for the output dir ?

https://github.com/rmarinho/pdfium-binaries/actions/runs/10635311877

[sungaila]

@rmarinho The directory is named like the target_os. I'd suggest you rename catalyst into maccatalyst there.

I have some nuspec files and icons prepared for the nuget packages. If it is ok with you, I'll open a PR to your fork so you can create the nuget packages yourself (with the workflow).

[rmarinho]

YEah sure I m using this branch https://github.com/rmarinho/pdfium-binaries/tree/add-maccatalyst can pr agains it

[rmarinho]

hum.. it worked now it s broken.. need to figure what am I missing now

[rmarinho]

OK not sure why I building with v8 now .. https://github.com/rmarinho/pdfium-binaries/actions/runs/10637396968

[sungaila]

@rmarinho I've created nuget packages out of your artifacts:
bblanchon.PDFium.MacCatalyst.1.0.0.zip
bblanchon.PDFiumV8.MacCatalyst.1.0.0.zip

Also please consider to merge my PR (rmarinho#1) into your fork. Then you should be able to create nuget packages yourself.

[rmarinho]

bblanchon.PDFium.MacCatalyst.130.0.6679.zip

https://github.com/rmarinho/pdfium-binaries/actions/runs/10639243588#artifacts

buhh I did had a release complete, but tested the catalyst artifact but still fails for me

[rmarinho]

Ok we need some tweaks I think ..

➜  net9.0-maccatalyst git:(move-net9) ✗ vtool -show maccatalyst-arm64/libpdfium.dylib 
maccatalyst-arm64/libpdfium.dylib:
Load command 12
      cmd LC_BUILD_VERSION
  cmdsize 32
 platform MACCATALYST
    minos 17.4
      sdk 17.0
   ntools 1
     tool 4
  version 20.0

but hey.. progress...

[sungaila]

I can't get it to run either. Building a .NET MAUI app with the libpdfium.dylib included results in this error:

Tool xcrun execution finished (exit code = 1).
          
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: changing install names or rpaths can't be redone for: obj/Debug/net8.0-maccatalyst/maccatalyst-arm64/nativelibraries/Contents/MonoBundle/libpdfium.dylib.tmp (for architecture arm64) because larger updated load commands do not fit (the program must be relinked, and you may need to use -headerpad or -headerpad_max_install_names)

I noticed that SkiaSharp.NativeAssets.MacCatalyst distributes a framework instead of a dylib. Maybe this is something that has to be done for pdfium as well?

But since I have no idea about native Mac development, my tests end here.

[rmarinho]

Yeah something is missing, I was reading the code for the ninja scripts trying to see what makes sense. but it might that is just configured to be build only as framework. I will continue to investigate.