Building with local hdf5 created with FetchContent

[jackStalnaker]

I am having a lot of trouble building with a local hdf5 I build using FetchContent. First and foremost, the CMake recommended way does not work (unless you have a nice environment where you can have a really recent CMake that supports the new OVERRIDE_FIND_PACKAGE argument to FetchContent_declare:

FetchContent_Declare(hdf5
    GIT_REPOSITORY https://github.com/HDFGroup/hdf5
    GIT_TAG hdf5-1_12_0
)

FetchContent_Declare(HighFive
    GIT_REPOSITORY https://github.com/BlueBrain/HighFive
    GIT_TAG v2.7.1
)

FetchContent_MakeAvailable(hdf5 HighFive)

fails because FetchContent has not set the variables find_package(HDF5) does. If you set those variables manually:

set(HDF5_FOUND TRUE)
set(HDF5_INCLUDE_DIRS "${hdf5_SOURCE_DIR}/src/;${hdf5_SOURCE_DIR}/c++/src;${hdf5_BINARY_DIR}")
set(HDF5_LIBRARIES ....)

you get a bunch of errors, because HighFive's dependencies are added as part of an interface library. This assumes that the local HDF5 is bad:

CMake Error in deps/HighFive/CMakeLists.txt:
Target "libdeps" INTERFACE_INCLUDE_DIRECTORIES property contains path:
     "/my/dir/build/_deps/hdf5-build"
which is prefixed in the build directory.

which also happens even if I just use a git submodule for HighFive, and forego FetchContent. It doesn't matter. It's the local hdf5 it doesn't like.

I can try to fix this by using a generator expression:

set(HDF5_INCLUDE_DIRS "$<BUILD_INTERFACE:${hdf5_SOURCE_DIR}/src/;${hdf5_SOURCE_DIR}/c++/src;${hdf5_BINARY_DIR}>")

but once again that fails because it's unquoted in CMake/HighFiveTargetDeps.cmake:

target_include_directories(libdeps SYSTEM INTERFACE ${HDF_INCLUDE_DIRS})

I tried escaping the quotes several different ways, and nothing worked. Finally, I can fix it by adding quotes to the line above:

target_include_directories(libdeps SYSTEM INTERFACE "${HDF_INCLUDE_DIRS}")

Except now I've negated the entire reason for using FetchContent or a git submodule, because I have to modify the source.

I can't think I'm the only one who has run into this issue. Anyone attempting to avoid "dependency hell" and build a local, static build environment that links to a specific HDF5 will run into this. Is there an obvious workaround I'm missing?

[jackStalnaker]

I mean, I guess there's the obvious route of just adding the git submodule directory to the target include directories directly, but that feels a little hacky.

[1uc]

This is what we do at the BlueBrainProject. Thanks for reporting the missing quotes. I've put it into an issue #742.

[jackStalnaker]

To be clear, I don't mean as in the README.md. If you use:

add_subdirectory(highfive_folder)
add_executable(bar bar.cpp)
target_link_libraries(bar HighFive)

the same issue happens, because the dependency cmake still doesn't like the local hdf5. I mean just using:

target_include_directories(highfive_folder/include)

Never adding the subdirectory. Is that the same thing you do?

[1uc]

HDF5 itself is usually already installed, either by the system package manager, in a module or via Spack. If needed we pass the location of the exact version we want to use via -DCMAKE_PREFIX_PATH="${HDF5_DIR}" and then find_package picks it up and sets all the flags correctly.

As for HighFive in MorphIO we do it like this:
https://github.com/BlueBrain/MorphIO/blob/01700d4405e7a6ae5dd0c67fa1e942836c814bfa/3rdparty/CMakeLists.txt#L3

and in libsonata we chose to include it sorta along the lines you describe:
https://github.com/BlueBrain/libsonata/blob/f26db3226ce40a02373a9794956f23f0013de8b3/extlib/CMakeLists.txt#L2

[1uc]

Unfortunately, I'm unable to get it to run even as far as you. For me it errors out on

-- Cannot execute TEST flushrefresh - perl not found

What I would have wanted to check is what happens if we replace the use of ${HDF5_*} variables with the CMake target HDF5::HDF5. However, I then reduce the test to only use HDF5 via FetchContent, no HighFive involved and it failed with the above message; and didn't generate my executable. This happens if one adds a find_package(hdf5) before the add_executable, which would simulate that HighFive will do. If one removes that, then the configure fails complaining that HDF5::HDF5 isn't a target. My limited understanding of FetchContent is that it'll grant us access to all the target of the fetch dependency.

Do you have a complete CMakeLists.txt which shows how to use FetchContent for HDF5? This would provide us with a much better starting point to decide if there are issues inside of HighFive's CMake code that affect you, and then how one could fix those.

Note, I've also tried #745 which only includes HighFive via FetchContent, but (presumably) assumes that HDF5 is preinstalled. That part, i.e. the part of using FetchContent to get HighFive and use is via its target HighFive seems to work, at least locally.

[jackStalnaker]

This is an excerpt from the CMakeLists.txt I was using for my project:

cmake_minimum_required(VERSION 3.10)

project(
    myproject
    VERSION 1.0
    DESCRIPTION "my project"
    LANGUAGES CXX
)

# Includes
include(FetchContent)
include(CTest)

# Global settings
set(CMAKE_CXX_STANDARD 17)
set(FETCHCONTENT_QUIET FALSE)
set_property(GLOBAL PROPERTY USE_FOLDERS ON)

# ----- HIGHFIVE ------
# HighFive is a high level layer that sits on top of HDF5 and
# just makes life a bit easier.
set(HIGHFIVE_EXAMPLES OFF)
set(HIGHFIVE_USE_BOOST OFF)
set(HIGHFIVE_USE_EIGEN OFF)
set(HIGHFIVE_USE_XTENSOR OFF)
set(HIGHFIVE_USE_HALF_FLOAT OFF)
set(HIGHFIVE_USE_OPENCV OFF)
FetchContent_Declare(HighFive
    GIT_REPOSITORY https://github.com/BlueBrain/HighFive
    GIT_TAG v2.7.1
    GIT_PROGRESS   TRUE
)

# ----- HDF5 ----------
find_package(HDF5 COMPONENTS CXX REQUIRED)
if (NOT HDF5_FOUND)
    FetchContent_Declare(hdf
        GIT_REPOSITORY https://github.com/HDFGroup/hdf5
        GIT_TAG hdf5-1_12_1
        GIT_PROGRESS TRUE
    )

    # The find_package(hdf5) call in HighFive's CMakeLists expects the following to be set:
    set(HDF5_FOUND TRUE)
    set(HDF5_INCLUDE_DIRS "$<BUILD_INTERFACE:${hdf5_SOURCE_DIR}/src/;${hdf5_SOURCE_DIR}/c++/src;${hdf5_BINARY_DIR}>")
    set(HDF5_LIBRARIES "${PROJECT_BUILD_DIR}/_deps/hdf5-build}")

    FetchContent_MakeAvailable(hdf5 HighFive)
else()
    FetchContent_MakeAvailable(HighFive)
endif ()

# ----- MYPROJECT ------
add_subdirectory(src)

# ----- TESTS -----------
add_subdirectory(tests)

I am recounting the library directory from memory, so it may need tweaking.