[question] "cpp_info.system_libs" and static linking of some system libs

[amachado-pie]

What is your question?

Whe are porting some custom makefiles with Conan v1 to Cmake + Conan 2.

Some recipes, such as libpng, use this approach:

def package_info(self):
    if self.settings.os in ["Linux", "Android", "FreeBSD", "SunOS", "AIX"]:
        self.cpp_info.system_libs.append("m")

This raises an issue when you want to mix static linking of just some system libraries, such as the case of libm using -Wl,-Bstatic -lstdc++ -lm -Wl,-Bdynamic

The issue is that system libraries of the conan recipes that added it as a dependency are repeated in the linking command:

[100%] Linking CXX executable
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/project.dir/link.txt --verbose=1
/usr/bin/g++  -m32
...
-lpthread
-static-libstdc++
-static-libgcc
-Bstatic -lstdc++ -lm -Wl,-Bdynamic
/root/.conan2/p/b/freetafaa4833c1df1/p/lib/libfreetype.a
/root/.conan2/p/b/libpnb01f5f21fd40f/p/lib/libpng.a
...
/root/.conan2/p/b/opens5b1ba6b148939/p/lib/libcrypto.a
-lrt
/root/.conan2/p/b/xmlsead5ac171be143/p/lib/libxmlsec1.a
/root/.conan2/p/b/libxmab7a8911f883b/p/lib/libxml2.a
...
/root/.conan2/p/b/zlib8d00874752b7e/p/lib/libz.a
/root/.conan2/p/b/libic52d0797df088e/p/lib/libiconv.a
/root/.conan2/p/b/libic52d0797df088e/p/lib/libcharset.a
-ldl
-lpthread
-lm            <---
/root/.conan2/p/b/libjp57a4193473e2a/p/lib/libjpeg.a

So the executable is linked dynamically to libm:

ldd build/Release/project 
        linux-gate.so.1 (0xed5bd000)
        libdl.so.2 => /lib32/libdl.so.2 (0xec1dc000)
        librt.so.1 => /lib32/librt.so.1 (0xec1d3000)
        libpthread.so.0 => /lib32/libpthread.so.0 (0xec1b6000)
        libm.so.6 => /lib32/libm.so.6 (0xec161000)  <---
        libc.so.6 => /lib32/libc.so.6 (0xebfa9000)
        /lib/ld-linux.so.2 (0xed5bf000)

We need to still statically link some system libraries like libm because we need to support old distros that have outdated libs.

I tried to hack the dependencies in CMake to remove -lm like this but without success:

function(remove_transitive_dependencies target)
  if(TARGET ${target})
    message(STATUS "Target: ${target}")
    get_target_property(libs ${target} INTERFACE_LINK_LIBRARIES)
    if(libs)
      foreach(lib ${libs})
      message(STATUS "  lib ${lib}")
        if(lib STREQUAL "-lm")
          message(STATUS "    Removing transitive dependency '-lm' from target '${target}'")
          list(REMOVE_ITEM libs ${lib})
        endif()
      endforeach()
      set_target_properties(${target} PROPERTIES INTERFACE_LINK_LIBRARIES "${libs}")
    endif()
  endif()
endfunction()

# Remove transitive dependencies from common libraries
foreach(lib ${project_libraries})
  remove_transitive_dependencies(${lib})
endforeach()

Any idea about how can we deal with this issue?
Thanks

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @amachado-pie

Thanks for your question

Whe are porting some custom makefiles with Conan v1 to Cmake + Conan 2.

From what I understand reading the issue, this is not something new due to the migration. I think that system_libs treatment as always been very similar, and it was not possible to differentiate between static/dynamic in Conan 1 either.

This raises an issue when you want to mix static linking of just some system libraries, such as the case of libm using -Wl,-Bstatic -lstdc++ -lm -Wl,-Bdynamic

Quick question, you are using recipes and packages from ConanCenter, or are you using your own recipes? Or maybe at least building the binaries from a conan-center-index Github source repo fork? I am asking in case you could change the recipes to customize it.

Regarding your attempt to remove the libraries from the consumer side CMake code, a couple of quick questions:

• foreach(lib ${project_libraries}) how do you get project_libraries? The targets that actually get the flags are inner targets, the package::package targets are typically just interface targets that link other internal targets that define the properties, flags, etc. So it would be needed to iterate deeper in the targets graph.
• Are you sure that if(lib STREQUAL "-lm") works? I think the property is a list of libraries, not a list of flags. It would be very useful to print the values to check.

I'd try to add more logs to the cmake code, printing the targets, the target properties, whether the if() matches or not, etc.

[amachado-pie]

Hi @memsharded thanks for your reply.

Sorry if I wasn't clear enough. We were using Makefiles with our own recipes for 3rd party libs in Conan v1.
Now we are migrating to CMake and using the recipes for 3rd party libs with Conan v2 obtained directly from conan-center to simplify our dependency update process.

I believe there were no changes in the way Conan handles dependencies defined in cpp_info.system_libs.
The difference is that we didn't define them in our own recipes, we defined the system libs directly in the makefile as linker flags once.
Now that we were porting to CMake and we don't know much about it, we were asking if there was any way to deal with this kind of need when some of the libs defined as system libs in the recipes have to be statically linked.

A suggestion they gave us on a cmake forum is to put together a sysroot with the libraries we need and setup a cmake toolchain file to set CMAKE_SYSROOT variable. I don't now if that will make difference to Conan translates the "m" dependency properly to use the static version in its generated files and appends it to the interface link libraries of the targets it provides.

I also tried a suggestion to use CMAKE_LINK_LIBRARY_USING_ to apply a pattern to redefine the way "-libm" is injected into the command line in the linker, but I don't know if it's due to an error in the way I tried to use it, I also didn't get the desired effect: https://cmake.org/cmake/help/latest/variable/CMAKE_LINK_LIBRARY_USING_FEATURE.html

Probably the solution of creating a repo, cloning the recipes for the dependencies we need and modifying them to remove the system lib definitions will be the alternative...

[memsharded]

Sorry if I wasn't clear enough. We were using Makefiles with our own recipes for 3rd party libs in Conan v1.
Now we are migrating to CMake and using the recipes for 3rd party libs with Conan v2 obtained directly from conan-center to simplify our dependency update process.

Thanks for the clarification, now I understand, using Makefiles it is more clear how to manage flags and other stuff, things are more direct, with CMake not that much.

Now that we were porting to CMake and we don't know much about it, we were asking if there was any way to deal with this kind of need when some of the libs defined as system libs in the recipes have to be statically linked.
A suggestion they gave us on a cmake forum is to put together a sysroot with the libraries we need and setup a cmake toolchain file to set CMAKE_SYSROOT variable. I don't now if that will make difference to Conan translates the "m" dependency properly to use the static version in its generated files and appends it to the interface link libraries of the targets it provides.

I think your original approach might still be doable, and it could have some advantages, as not having to fork and customize recipes, and also not having to manage a sysroot. It would need some expertise in CMake, and checking the internals of Conan generated .cmake files, but I have been successful with this small proof of concept:

• Started with a default conan new cmake_lib -d name=mypkg template
• Added a self.cpp_info.system_libs = ["invented"] in the package_info() method of the root conanfile.py.
• The consumer test_package/conanfile.py, which is similar to a regular consumer project, will fail under a normal conan create . because invented is not found, as expected. Note the package creates correctly in the cache, but the test_package consumption fails
• The consumer test_package/CMakeLists.txt`` is modified to:

  find_package(mypkg CONFIG REQUIRED)
  
  function(remove_transitive_dependencies pkg)
    set(target "${pkg}_DEPS_TARGET")
    if(TARGET ${target})
      get_target_property(libs ${target} INTERFACE_LINK_LIBRARIES)
      if(libs)
        foreach(lib ${libs})
          if(lib STREQUAL "$<$<CONFIG:Release>:invented>")
              list(REMOVE_ITEM libs ${lib})
          endif()
        endforeach()
        set_target_properties(${target} PROPERTIES INTERFACE_LINK_LIBRARIES "${libs}")
      endif()
    endif()
  endfunction()
  
  # Remove transitive dependencies from common libraries
  foreach(pkg "mypkg")
    remove_transitive_dependencies(${pkg})
  endforeach()
  
  
  add_executable(example src/example.cpp)
  target_link_libraries(example mypkg::mypkg)

• Now a conan create will work.

The key is using for each package the <pkgname>_DEPS_TARGET which is the one that accumulates the system_libs, and also taking into account that the value is a CMake generator expression (you might need to do the same for Debug or other build configurations)

Please let me know if this helps.

[amachado-pie]

@memsharded sorry for the delay in response.

We had to hack the CXX Linker via CMAKE_CXX_LINK_EXECUTABLE to forces the use of "gcc" instead of "g++" as linker because the C++ runtime libstdc++ requires libm, so if you compile a C++ program with g++, you will automatically get libm linked in dinamically and we want to avoid that to force be statically linked because we need to support:

set(CMAKE_CXX_LINK_EXECUTABLE 
  "<CMAKE_C_COMPILER> <FLAGS> <CMAKE_CXX_LINK_FLAGS> <LINK_FLAGS> 
   <OBJECTS> -o <TARGET> 
   <LINK_LIBRARIES> -Wl,-Bstatic -lstdc++ -lm -Wl,-Bdynamic")

In addition, to handle multiple -lm injected in the linker command because transitive dependencies of 3rd party libs to libm , I tried the approach you suggested but I still couldn't remove the '-lm' from the dependencies in the linker command line, I don't know if it's because of the recursion between them.

We end doing this to prevent that:

if(LINUX)
  # force transitive dependencies of libs to libm be statically linked
  add_library(m INTERFACE)
  target_link_options(m INTERFACE "SHELL:-Wl,-Bstatic -lm -Wl,-Bdynamic")
endif()

This generates the linking step as:

gcc -m32 -march=pentium-mmx -m32 -Wl,-Bstatic -lm -Wl,-Bdynamic 
...(*.o) 
-o target 
...(*.a) 
-Wl,-Bstatic -lstdc++ -lm -Wl,-Bdynamic

I don't have in-depth knowledge of CMake if there was a more orthodox approach to what was intended, but for now I've managed to overcome it this way and move on.

Thank you very much for your support!