Use autolinking on Apple platforms to prevent the need to link with sqlite3 explicitly

[abdulowork]

On Apple platforms an explicit linking configuration is usually not required, because clang and swiftc compilers embed linker commands into .o files they produce. These linker commands can be printed out via otool -l /path/to/object | grep -A 4 LC_LINKER_OPTION.

This MR uses this technique to remove the requirement to explicitly link with -lsqlite3 on Apple platforms by embedding LC_LINKER_OPTION command into object files produced by K/N. This may be most useful for projects that link against K/N static libraries in Xcode, but should also remove the need for consumer side linker configuration.

As an integration test, I disabled explicit linking in sqldelight and ran the tests I could find against this MR: 1, 2

[kpgalligan]

That's an interesting option. We'd need to figure out how to use things like sqlcipher in this context, which is a "drop-in" replacement for the system sqlite library. Will have to look at this in more detail, though.

[abdulowork]

Disabling default search paths via -Z and reordering them manually might be one way to link against a custom sqlite3. So for example given:

// main.c
#include "sqlite3.h"
int main(void) {
    sqlite3_sourceid();
}

and a custom implementation:

// my_sqlite3.c
const char *sqlite3_sourceid(void) { return "foo"; }

one could:

# Compile main
clang -Xclang --linker-option=-lsqlite3 -c main.c -o main.o

# Default option: link against system's sqlite3
clang main.o -o mainSystem && otool -L mainSystem | grep sqlite

# Compile and link custom sqlite3 dylib
mkdir dylib
clang my_sqlite3.c -dynamiclib -o "$(realpath dylib)"/libsqlite3.dylib

# Link against custom sqlite3
clang -Z -Ldylib -L/usr/lib main.o -o mainCustom && otool -L mainCustom | grep sqlite

[abdulowork]

Hi Kevin. Were you able to look into this PR? Perhaps implementing autolinking and linking against a custom sqlite3 version could be achieved in a simpler way than what I originally suggested. I think it is sufficient to add linkerOpts in sqlite3.def for all Apple targets:

linkerOpts.${apple_target} = -lsqlite3

When producing a framework:

• with a static archive, K/N will embed these linkage flags as LC_LINKER_OPTION
• with a dynamic library, K/N will pass these linkage flags to the linker

I believe in case you need to link against a custom libsqlite3.dylib or libsqlite3.a, if you pass the -L search path it will have precedence over platform libraries. For example you could pass it as follows:

kotlin
    macosArm64 {
        binaries.executable {
            linkerOpts("-L", "/path/to/libs")
        }
    }
}

[kpgalligan]

Were you able to look into this PR?

I was not. It is an interesting option, but the overall todo list is rather long (outside of SQLiter). I'll bump it back up, but there are follow-on considerations, and other conversations happening concurrently.

• Linking against other custom builds would need to be definitively verified. As an example, linking against a custom sqlcipher build vs the system sqlite, on apple targets, is a critical thing, and a silent fail. So much so that on a recent client we added a (dev build) runtime check to crash the app if the created database wasn't actually encrypted. It is that critical, and also quite easy to undo by config changes.
• SqlDelight, the primary, probably only, consumer of SQLiter has a flag which presumably toggles adding the flag, which would at minimum need a deprecation. How to resolve the situation of users who expect there to be no flag added by explicitly setting that value would also need some kind of resolution. The simplest version would probably be removing it rather than deprecating it, as a failed build would be preferred to a runtime surprise. That change, and it's release, and updated docs, well, there are conversations involved.
• There is ongoing discussion with various parties about the Room native driver and perhaps some kind of collab, which might make the PR, and potentially SQLiter, irrelevant.

So, in summary, it's on the radar, but it's complicated :)