Android port

[yan12125]

I created a branch android that enables building Darling for Android. Currently it builds but doesn't run. darling shell fails at pivot_root().

Overlayfs requires Linux kernel 3.18+, and stock Android emulators ships with 3.10, so building a custom kernel is necessary. I put steps to build and run the custom kernel at https://gist.github.com/yan12125/78a9004acb1bed5faf2ffd442163e2ef

And then built Darling and install files onto the virutal device. Android NDK is necessary in this step. Here I use NDK r15 beta1.

cd /path/to/darling
git checkout -b android
mkdir build
cmake .. -DCMAKE_TOOLCHAIN_FILE=../Toolchain.cmake -DBUILD_ANDROID=ON -DANDROID_PLATFORM=android-21 -DANDROID_ABI=x86_64 -DCMAKE_INSTALL_PREFIX=/data/local/tmp
make -j8
make install DESTDIR=/tmp
cd /tmp
tar cvf android-darling.tar data
adb -s emulator-5554 push android-darling.tar /sdcard/

Go into adb shell and extract it:

su
tar xf /sdcard/android-darling.tar

Finally it's possible to run the Darling binary:

cd /data/local/tmp
export HOME=$(pwd)
./bin/darling shell

The current status is:

Setting up a new Darling prefix at /data/local/tmp/.darling
Cannot pivot_root: Invalid argument
Cannot open mount namespace file: No such file or directory

The reason is that a sanity check in SYS_pivot_root fails: https://android.googlesource.com/kernel/goldfish/+/android-goldfish-3.18/fs/namespace.c#2985. Seems pivot_root allows the global mount namespace only, while each App has its own mount namespace in Android. I guess there's a way to run a program without creating a new namespace - which is still under investigation.

By the way, anyone know how to setup KGDB on Android?

[bugaevc]

Totally awesome!

Seems pivot_root allows the global mount namespace only, while each App has its own mount namespace in Android.

No, pivot_root works in other mount namespaces too -- and in fact, Darling creates a new mount namespace for the container before calling pivot_root: https://github.com/darlinghq/darling/blob/master/src/startup/darling.c#L469-L507.

That being said, if you really need to run a program in the original namespace, it's fairly easy to join PID1's namespace.

[yan12125]

and in fact, Darling creates a new mount namespace for the container before calling pivot_root

Thanks. That's right. So the reason is still unknown :/

[ylluminate]

Might want to look at Remix OS' methodology and / or implementation for this. Heck, honestly they might throw some financial and dev support behind this to add iOS to Remix. If I recall correctly, they're using: http://www.android-x86.org

[LubosD]

@ylluminate Not really. Android is based on platform-independent Java/Dalvik bytecode + JNI libs (which are usually ARM, but x86 is also widespread). That enables you to run Android apps on desktop Linux "easily" as long as you bootstrap all Android services, get Binder running, hack SurfaceFlinger (or something else?) to render into an X11 window and forward input events.

iOS is different. Everything is native arm/arm64, just like everything is native i386/x86_64 on macOS. Running iOS apps on Linux would require either of these two:

• Using qemu-user while having a fully working arm/arm64 chroot on the hosting Linux desktop. This is because Darling (even if compiled for ARM) needs to talk to native libs eventually to get something on the screen. Could get troublesome with OpenGL.
• Implement arm->x86 translation directly in Darling's mldr.

The bottom line is iOS emulation is a completely different matter from running Android apps on desktop Linux, although it may not seem so from user's perspective.

[bugaevc]

You could still run iOS apps on ARM non-Android Linux desktops, e.g. people like to put Linux onto Raspberry Pi

[LubosD]

Just did some research on how Rosetta worked - it only translated the application itself, the rest of the system was always x86. So it was sitting between dyld and the PPC app, the system libraries & frameworks just used a special process hint (that could be checked with IF_ROSETTA()) to reverse byte order (which wouldn't be needed for ARM).

[bugaevc]

How did it work? There are multiple nuances in app <-> library interaction: how to call native/untranslated code in the first place, what happens to the pointers passed both ways (are emulated addresses equal to real ones? they certainly can't be for code sections -- how does a library call back into the app at all?), how are ABI (e.g. structure layout) differences handled (you can't "just" convert them one level deep), etc

[fervi]

To run Android on Linux you can use Anbox
To Linux on Android - Linux Deploy.

On Android you can use XServer XSDL if it is necessary

[yan12125]

Thanks - an X Server on Android can be useful. That can reduce efforts on porting.

@bugaevc AFAIK qemu translates ARM and writes equivalent x86 machine codes to a writable page. After that it marks the page as executable and jumps to it. As a result, running an ARM binary is not different than running an x86 one, and thus there are no ABI mismatches. I guess Rosetta is similar.

For memory adresses: qemu emulates the page table and TLB by inserting additional instructions.

---

Implement arm->x86 translation directly in Darling's mldr.

Personally I feel it not the best option as implementing virtual machines requires quite a few efforts and there are way many edge cases. Integrating qemu or other VM sounds better.

[bugaevc]

there are no ABI mismatches. I guess Rosetta is similar.

Qemu requires you to install the whole set of necessary dynamic libraries for the emulated architecture — those libraries are then loaded as usual and get translated to native instructions in the same way as the app is.

The only way the emulated code interacts with the outside world is by system calls, and Qemu notices and translates those.

But according to @LubosD's research, Rosetta only translated the app, whereas all the libraries it used were native (x86) — so my question is about how native and emulated code do interact without breaking ABI.

[LubosD]

Integrating QEMU means you have to be emulating the whole ecosystem. What made Rosetta so successful was the fact that only the app itself had to be emulated. I know that there were some attempts to translate one architecture's assembly into LLVM IR and have LLVM emit another architecture's assembly, but I can't find the link right now.

[stek29]

@LubosD Probably https://zneak.github.io/fcd/2016/02/16/lifting-x86-code.html

[fervi]

This is my hypothesis, maybe you will succeed.

The best emulation method is to use QEMU User Emulation. It works much faster than full emulation.

There is also a proot project, something like chroot, but it does not require root, it can simulate the /dev directory and others and use the qemu user emulator.

So you can like Exagear emulator on Android emulate linux on which Darling is running.

Perhaps rewriting this ready-made Linux (without root) and modifying the proot will enable darling (x86) on Android:
https://github.com/pelya/debian-noroot

[M1cha]

If we ever wanna let iOS apps run on ARM Android devices we don't even need qemu ;)

[stek29]

@M1cha afaik Darling relies on x86 features heavily, and @yan12125 also used x86_64 build of Android.

[bugaevc]

Darling relies on x86 features heavily

Not really. Many architecture-specific parts are only implemented for x86/x86-64 for now, but ARM support is planned.

Emulation aside, two main goals are to run macOS apps on x64 desktop Linux and iOS apps on ARM Android devices. We can additionally consider macOS apps on x64 Android and iOS apps on ARM desktop Linux.

[stephengroat]

I'd look at termux's package build environment. it does a really nice job of wrapping the ndk and gets some additional critical packages into the system

[TheBrokenRail]

Any updates on this with #642?