"Linking CXX shared library libnunchuk.so" error while building on ubuntu

[MrUnfunny]

After building bitcoin and sqlcipher according to build instructions, I am getting this linking error on trying to create a shared library. Error is:

/usr/bin/ld: ../../libnunchuk/contrib/bitcoin/src/libbitcoin_util.a(libbitcoin_util_a-threadnames.o): relocation R_X86_64_TPOFF32 against `__tls_guard' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_filesystem.a(operations.o): relocation R_X86_64_PC32 against symbol `_ZN5boost6system6detail10cat_holderIvE24system_category_instanceE' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make[2]: *** [libnunchuk/CMakeFiles/nunchuk.dir/build.make:652: libnunchuk/libnunchuk.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:247: libnunchuk/CMakeFiles/nunchuk.dir/all] Error 2
make: *** [Makefile:106: all] Error 2

CMakeLIsts.txt

cmake_minimum_required(VERSION 3.7 FATAL_ERROR)

project(btcwallet VERSION 0.1.0)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_INCLUDE_DIRECTORIES_BEFORE ON)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
set(Boost_USE_STATIC_LIBS ON)
set(OPENSSL_USE_STATIC_LIBS ON)
set(UR__DISABLE_TESTS ON)
set(CMAKE_POLICY_DEFAULT_CMP0077 NEW)

add_library(btcwallet SHARED main.cpp)

add_subdirectory(libnunchuk)
target_link_libraries(btcwallet PUBLIC nunchuk) 



set_target_properties(btcwallet PROPERTIES
    PUBLIC_HEADER btcwallet.h
    VERSION ${PROJECT_VERSION}
    SOVERSION 1
    OUTPUT_NAME "btcwallet"
    XCODE_ATTRIBUTE_CODE_SIGN_IDENTITY "Hex_Identity_ID_Goes_Here"
)

[giahuy98]

try adding add_compile_options(-fPIC) to CMakeLists.txt

[MrUnfunny]

@giahuy98 thanks for the reply. Adding it also doesn't seem to work. Got following error

/usr/bin/ld: ../../../contrib/bitcoin/src/libbitcoin_util.a(libbitcoin_util_a-threadnames.o): relocation R_X86_64_TPOFF32 against `__tls_guard' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status
gmake[2]: *** [lib/CMakeFiles/nunchuk.dir/build.make:701: lib/libnunchuk.so] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:262: lib/CMakeFiles/nunchuk.dir/all] Error 2
gmake: *** [Makefile:111: all] Error 2

[giahuy98]

@MrUnfunny you need to compile bitcoin with -fPIC too, see #15 (comment)

[MrUnfunny]

@giahuy98 Okay. I'll get back to you after trying this out.

[MrUnfunny]

@giahuy98 after building bitcoin with -fPIC I'm getting this error:

/usr/bin/ld: libur-2.a(ur.cpp.o): warning: relocation against `_ZTIN2ur2UR12invalid_typeE' in read-only section `.text'
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_filesystem.a(directory.o): relocation R_X86_64_PC32 against symbol `_ZN5boost6system6detail10cat_holderIvE24system_category_instanceE' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
gmake[2]: *** [CMakeFiles/nunchuk.dir/build.make:701: libnunchuk.so] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:214: CMakeFiles/nunchuk.dir/all] Error 2
gmake: *** [Makefile:111: all] Error 2

[giahuy98]

Did you run the whole scrip @MrUnfunny, libboost_filesystem need to link against objects in libnunchuk/contrib/bitcoin/depends/<android target>/lib not in /usr/lib/x86_64-linux-gnu/ (for Linux not android). The script should be run in the parent directory of libnunchuk.

[MrUnfunny]

Hey @giahuy98, After running the script in parent directory of libnunchuk, I'm now getting this error on trying to run cmake .. in build directory.

-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.74.0/BoostConfig.cmake (found suitable version "1.74.0", minimum required is "1.47.0") found components: filesystem 
CMake Error at libnunchuk/embedded/CMakeLists.txt:159 (message):
  bitcoin_crypto_shani not found

[giahuy98]

@MrUnfunny you don't have to run cmake .. in build directory. libnunchuk will be built with your cmake project project(btcwallet VERSION 0.1.0) when using gradle

[MrUnfunny]

Okay. I'll try that tomorrow and will get back to you. Thanks for bearing with me.

[MrUnfunny]

Hey @giahuy98. On running it in android, it's unable to locate Boost libraries. It throws this error:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:configureCMakeDebug[armeabi-v7a]'.
> [CXX1405] error when building with cmake using /home/mohit/StudioProjects/ffitest/testlibnun/CMakeLists.txt: Build command failed.
  Error while executing process /home/mohit/Android/Sdk/cmake/3.18.1/bin/cmake with arguments {-H/home/mohit/StudioProjects/ffitest/testlibnun -DCMAKE_SYSTEM_NAME=Android -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_SYSTEM_VERSION=16 -DANDROID_PLATFORM=android-16 -DANDROID_ABI=armeabi-v7a -DCMAKE_ANDROID_ARCH_ABI=armeabi-v7a -DANDROID_NDK=/home/mohit/Android/Sdk/ndk/21.1.6352462 -DCMAKE_ANDROID_NDK=/home/mohit/Android/Sdk/ndk/21.1.6352462 -DCMAKE_TOOLCHAIN_FILE=/home/mohit/Android/Sdk/ndk/21.1.6352462/build/cmake/android.toolchain.cmake -DCMAKE_MAKE_PROGRAM=/home/mohit/Android/Sdk/cmake/3.18.1/bin/ninja -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/mohit/StudioProjects/ffitest/build/app/intermediates/cxx/Debug/4n6s2v1s/obj/armeabi-v7a -DCMAKE_RUNTIME_OUTPUT_DIRECTORY=/home/mohit/StudioProjects/ffitest/build/app/intermediates/cxx/Debug/4n6s2v1s/obj/armeabi-v7a -DCMAKE_BUILD_TYPE=Debug -B/home/mohit/StudioProjects/ffitest/android/app/.cxx/Debug/4n6s2v1s/armeabi-v7a -GNinja}
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /home/mohit/Android/Sdk/ndk/21.1.6352462/toolchains/llvm/prebuilt/linux-x86_64/bin/clang - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /home/mohit/Android/Sdk/ndk/21.1.6352462/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Looking for pthread.h
  -- Looking for pthread.h - found
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - not found
  -- Check if compiler accepts -pthread
  -- Check if compiler accepts -pthread - yes
  -- Found Threads: TRUE  
  -- Configuring incomplete, errors occurred!
  See also "/home/mohit/StudioProjects/ffitest/android/app/.cxx/Debug/4n6s2v1s/armeabi-v7a/CMakeFiles/CMakeOutput.log".
  See also "/home/mohit/StudioProjects/ffitest/android/app/.cxx/Debug/4n6s2v1s/armeabi-v7a/CMakeFiles/CMakeError.log".

  CMake Error at /home/mohit/Android/Sdk/cmake/3.18.1/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
    Could NOT find Boost (missing: Boost_INCLUDE_DIR filesystem) (Required is
    at least version "1.47.0")
  Call Stack (most recent call first):
    /home/mohit/Android/Sdk/cmake/3.18.1/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
    /home/mohit/Android/Sdk/cmake/3.18.1/share/cmake-3.18/Modules/FindBoost.cmake:2177 (find_package_handle_standard_args)
    libnunchuk/CMakeLists.txt:41 (find_package)

[giahuy98]

If you gonna build for armeabi-v7a you need to run the script like this ./script.sh armeabi-v7a

[MrUnfunny]

Hey @giahuy98
on running ./script.sh armeabi-v7a, I got an error sayinh "incompatible platform".

However if we change ANDROID_TOOLCHAIN_BIN in line 85 in installBitcoinDeps in #15 from
ANDROID_TOOLCHAIN_BIN=$TOOLCHAIN to ANDROID_TOOLCHAIN_BIN=$TOOLCHAIN/bin, the script succeeds but make in gradle fails with following error:

CMake Error in Gradle

  env_posix.cc:(.text._ZN7leveldb11PosixLogger4LogvEPKcSt9__va_list[_ZN7leveldb11PosixLogger4LogvEPKcSt9__va_list]+0x49c): undefined reference to `__fwrite_chk'
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/embedded/../contrib/bitcoin/depends/aarch64-linux-android/lib/libevent.a(evutil.o): In function `evutil_check_ifaddrs':
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/aarch64-linux-android/libevent/2.1.12-stable-6a0759a478e/evutil.c:683: undefined reference to `getifaddrs'
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/aarch64-linux-android/libevent/2.1.12-stable-6a0759a478e/evutil.c:694: undefined reference to `freeifaddrs'
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/embedded/../contrib/bitcoin/depends/aarch64-linux-android/lib/libevent.a(log.o): In function `event_log':
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/aarch64-linux-android/libevent/2.1.12-stable-6a0759a478e/log.c:242: undefined reference to `stderr'
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/aarch64-linux-android/libevent/2.1.12-stable-6a0759a478e/log.c:242: undefined reference to `stderr'
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/aarch64-linux-android/libevent/2.1.12-stable-6a0759a478e/log.c:242: undefined reference to `stderr'
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/aarch64-linux-android/libevent/2.1.12-stable-6a0759a478e/log.c:242: undefined reference to `stderr'
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/aarch64-linux-android/libevent/2.1.12-stable-6a0759a478e/log.c:242: undefined reference to `stderr'
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/embedded/../contrib/bitcoin/depends/aarch64-linux-android/lib/libevent.a(log.o):/home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/aarch64-linux-android/libevent/2.1.12-stable-6a0759a478e/log.c:242: more undefined references to `stderr' follow
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/embedded/../contrib/bitcoin/src/libbitcoin_util.a(libbitcoin_util_a-sock.o): In function `Sock::Send(void const*, unsigned long, int) const':
  sock.cpp:(.text+0x5d4): undefined reference to `__sendto_chk'
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/embedded/../contrib/bitcoin/src/libbitcoin_util.a(libbitcoin_util_a-sock.o): In function `Sock::Wait(std::__ndk1::chrono::duration<long long, std::__ndk1::ratio<1l, 1000l> >, unsigned char, unsigned char*) const':
  sock.cpp:(.text+0x92c): undefined reference to `__poll_chk'
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/embedded/../contrib/bitcoin/src/libbitcoin_util.a(libbitcoin_util_a-sock.o): In function `NetworkErrorString(int)':
  sock.cpp:(.text+0xf3c): undefined reference to `__gnu_strerror_r'
  /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/embedded/../contrib/bitcoin/src/libbitcoin_util.a(libbitcoin_util_a-tokenpipe.o): In function `TokenPipeEnd::TokenWrite(unsigned char)':
  tokenpipe.cpp:(.text+0x2cc): undefined reference to `__write_chk'
  clang++: error: linker command failed with exit code 1 (use -v to see invocation)
  ninja: build stopped: subcommand failed.

CMakeError.log

CMakeOutput.log

Another thing to mention is that Bitcoin docs say that minimum supported NDK version is r23 but running the script with r23 or 25 throws the following error.

Error with ndk >=23

/bin/sh: 2: /home/mohit/Android/Sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android-ar: not found
/bin/sh: 3: /home/mohit/Android/Sdk/ndk/25.1.8937393/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android-ranlib: not found
...failed clang-linux.archive bin.v2/libs/test/build/clang-linux-14.0.6/release/link-static/threading-multi/visibility-hidden/libboost_test_exec_monitor-mt-s-a64.a...
...skipped <p/home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/aarch64-linux-android/boost/1_71_0-d7e7ce9beb9/stage/lib>libboost_test_exec_monitor-mt-s-a64.a for lack of <pbin.v2/libs/test/build/clang-linux-14.0.6/release/link-static/threading-multi/visibility-hidden>libboost_test_exec_monitor-mt-s-a64.a...
...failed updating 10 targets...
...skipped 10 targets...
...updated 64 targets...
make: *** [funcs.mk:281: /home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/aarch64-linux-android/boost/1_71_0-d7e7ce9beb9/./.stamp_built] Error 1

However the script succeeds with version 21.1.6352462 but throws the cmake error mentioned before while building with gradle. Also I'm on Ubuntu 22.04.1 LTS with AMD ryzen 5 CPU. Can this be related to any of the above issues?

[giahuy98]

look like something wrong if you use ./script.sh armeabi-v7a
It should search in directory
/home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/armv7a-linux-androideabi
Instead of
/home/mohit/StudioProjects/ffitest/testlibnun/libnunchuk/contrib/bitcoin/depends/work/build/aarch64-linux-android

Can you try run ./script.sh arm64-v8a then set build.grade similar to

android {
    compileSdkVersion androidConfig.compileSdkVersion
    defaultConfig {
        minSdkVersion androidConfig.minSdkVersion
        targetSdkVersion androidConfig.targetSdkVersion
        externalNativeBuild {
            cmake {
                cppFlags "-std=c++17 -stdlib=libc++"
                ndk {
                    'armeabi-v7a'
                }
            }
        }
    }

    flavorDimensions 'cpuArch'
    productFlavors {
        arm8 {
            dimension 'cpuArch'
            ndk {
                abiFilters 'arm64-v8a', 'armeabi-v7a'
            }
        }
        arm64_v8a {
            dimension 'cpuArch'
            ndk {
                abiFilters 'arm64-v8a'
            }
        }
        armeabi_v7a {
            dimension 'cpuArch'
            ndk {
                abiFilters 'armeabi-v7a'
            }
        }
        x86_64 {
            dimension 'cpuArch'
            ndk {
                abiFilters 'x86_64'
            }
        }
        universal {
            dimension 'cpuArch'
            // include all default ABIs. with NDK-r16,  it is:
            //   armeabi-v7a, arm64-v8a, x86, x86_64
        }
    }
...