Load DuckDB native library by name first

[hubertp]

When loading a native library, attempt to first load the library, duckdb_java, by name. Only when that fails, loading will revert to the old behaviour of searching via classpath/local file path.

The main motivation for this work was to allow creating a correct Native Image that will not have issues during initialization. The change does not provide the support out-of-the-box.
But it works in Native Image if someone (like us) extracts the native library (wrt OS/Arch), renames it to follow the specification and places it on JNI search path.

For System.loadLibrary to work, the native library has to have the correct prefix (lib for Mac/Linux, no lib for Windows) and suffix (.so, .dll or .dylib for Linux/Windows/Mac, respectively). No architecture suffix is allowed.
As it stands the current packaging logic does not create native library files with the right names but that seems like a deeper change in the build setup. Also, I have zero knowledge about how the artifacts are being used internally, in dev work, so I don't feel comfortable with changing it without guidance.
One could look at sqlite-jdbc for some inspiration on how to improve it (which is roughly what we had to do when repackaging DuckDB's Java client jar).

Also, if maintainers are fine with this change we can also include Native Image configuration in the final duckdb_jdbc.jar which is a requirement for GraalVM's Native Image. More and more Java libraries provide similar Native Image configuration.
That would mostly enable #180.

[staticlibs]

Hi, thanks for the PR! It looks good to me.

About the Graal metadata - yes, I think it is a good idea to include it.

To fix the formatting errors (to pass the CI checks) you can run the following:

python -m pip install --user clang_format==11.0.1
make format
make format-check

Or just apply the following patch:

diff --git a/src/main/java/org/duckdb/DuckDBNative.java b/src/main/java/org/duckdb/DuckDBNative.java
index 599ba33d..78354799 100644
--- a/src/main/java/org/duckdb/DuckDBNative.java
+++ b/src/main/java/org/duckdb/DuckDBNative.java
@@ -24,7 +24,7 @@ final class DuckDBNative {
     }
 
     private static void loadLibraryFromFile(String libname) {
-        String full_libname  = "lib" + libname;
+        String full_libname = "lib" + libname;
         String lib_suffix = ".so";
         String os_name = "";
         String os_arch;
@@ -53,8 +53,7 @@ final class DuckDBNative {
         } else if (os_name_detect.startsWith("linux")) {
             os_name = "linux";
         }
-        String lib_res_name = "/" + full_libname + lib_suffix
-                              + "_" + os_name + "_" + os_arch;
+        String lib_res_name = "/" + full_libname + lib_suffix + "_" + os_name + "_" + os_arch;
         try {
             Path lib_file = Files.createTempFile(full_libname, lib_suffix);
             URL lib_res = DuckDBNative.class.getResource(lib_res_name);

Just note that this area with unpacking/loading the shared lib is a subject to change. There are no exact plans right now, but in general the following changes are intended:

1. pack less architectures into the "main" JAR file - additional arches (like linux_aarch64_musl for Alpine Linux) are now going into separate JAR files with Maven classifiers. It is intended to move MacOS amd64 out of the universal lib into a separate JAR.

2. logic for shared lib into a temp file should be more robust

3. in general the intention is to give users more control over distributing and loading the native lib. For example, to have a separate jar in Maven without any bundled native libs that requires user to make sure that the desired shared lib is on the java.library.path. I think this PR covers a part of it (some version checks may be still needed though).

4. currently DuckDB engine is packed into the same shared lib along with the JNI code. It is intended to split them into 2 shared lib to be able to run with the libduckdb.so built from the mainline duckdb/duckdb repo

5. there is an intention to support FFM API (along with the JNI one) to be able to run using only libduckdb.so on newer versions of JDK (JNI with JDK 8 remains supported).

[staticlibs]

@hubertp

Thanks for updating the PR! Just FYI, this change is intended to be included into 1.4.2 update planned for Nov 10.

[hubertp]

@hubertp

Thanks for updating the PR! Just FYI, this change is intended to be included into 1.4.2 update planned for Nov 10.

Thanks, I will create a separate PR with Native Image properties.

[staticlibs]

@hubertp

I took the idea from this PR a little bit further and filed #450, I wonder if you can take a look at it whether it (along with #447) covers your requirements?