Skip to content

DuckDBNative.java should call System.loadLibrary("duckdb_java"), starting faster if native lib is in -Djava.library.path #444

@giftick

Description

@giftick

Hi,
first thank you for this wonderful database :)

The java driver duckdb_jdbc-1.4.1.0.jar contains huge (>60 MB each) zipped native libs (each is the full DuckDB engine!) for multiple platforms, and on loading unzips/copies the right one into a temp file, then loads it.
See: https://github.com/duckdb/duckdb-java/blob/main/src/main/java/org/duckdb/DuckDBNative.java#L48

This may work out-of-the-box and be convenient for most users.
But it creates I/O and lag on each start, and fails if the system's security forbids loading binaries from /tmp.
Therefore I always extract native libs and store them uncompressed on disk in the java.library.path.

But the duckdb driver does not even try to load from java.library.path,
even if I delete the native libs from the jar file and the driver has no working option left.
Any other java lib with native lib(s) that I know does this correctly, only duckdb does not.
So I had to patch DuckDBNative.java myself:
Now it calls System.loadLibrary("duckdb_java") and works perfectly fine.

I kindly ask to fix this upstream.
Ideally, DuckDBNative.java should call System.loadLibrary("duckdb_java") first.
Only when this has failed shoud it upzip/copy the >60 MB from jar to /tmp.
This might even be more convenient for you duckdb developers than the current hack
System.load(Paths.get("build/debug", lib_res_name) ...
in the production duckdb_jdbc-1.4.1.0.jar, which works only if the native lib is not found as a resource (e.g. in the jar).

Additional suggestion:
Offer an alternative java driver jar that does NOT contain all the huge native libs for all the platforms,
and instead include the JNI binding library (which would be very small) in the DuckDB binary packages.
Then I (and others) could download that much smaller java jar, and build the DuckDB binary from source,
which allows "gcc -march=native" to optimize for the local CPU,
and feels safer than using a precompiled binary downloaded from the internet.

Thanks a lot and best regards,
Giftick

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions