Amalgamated Sources

[ddevienne]

Have you considered an SQLite-style amalgamated sources release? For further ease of deployment?

[alkino]

Not yet, but after discussions we are not sure about the pros/cons.
How should we name it?

[ddevienne]

The main PRO is ease of integration in an existing project. I can just add a couple files, to my SCM, and be done.
I say couple, because often there's a header and source file, but in your case, maybe a single file.
As to the name, I'd simply call it highfive.hpp.

I'm considering HighFive to replace hdf5_cpp and hdf_hl, and having a single header would definitely be a plus.

The CON is of course more work for the project, to write a script that amalgamates the sources, in the right order,
possibly getting rid of internal #include's.

Another PRO is that you'd be listed in https://github.com/nothings/single_file_libs :)

Just my $0.02. I'd completely understand if you don't bite on that request :).

[1uc]

My concern is that it'll create two variants of the library. The current version where code such as:

#include <highfive/H5File.hpp>
#include <highfive/H5Group.hpp> // Redundant.

compiles. The second line is redundant, but I've seen it "in the wild".

The amalgamated version would not allow the above code instead it would require something like:

#include <highfive/highfive.hpp>

The part about this I really dislike is that an instruction such as:

In order to use HighFive please include

#include <highfive/H5File.hpp>

is incorrect and should read:

If you're using HighFive version 2.x.y or below you need to include <highfive/H5File.hpp>. As of version 2.x.y. you can continue to use <highfive/H5File.hpp> if you're building against the non-amalgamated version of HighFive. However, if you're using the amalgamated version you must include <highfive/highfive.hpp>. You may wish to include <highfive/highfive.hpp> also with the non-amalgamated version, if you don't require compatibility with versions pre 2.x.y.

While HighFive is header only, it's not without dependencies or build-time options, e.g. -DHIGHFIVE_USE_EIGEN=ON. Which implies some subset of the instructions starting at:
https://github.com/BlueBrain/HighFive/blob/master/CMake/HighFiveTargetDeps.cmake#L42

At a minimum one would need to find Eigen, perform target_link_libraries, and set the correct target_compile_definition; all semi trivial, but definitely more involved than it would be for a single-file header only library without any real dependencies.

The good news is:

• Spack is a source-code package manager that as far as I can tell works nicely for the C++ ecosystem. The refreshing thing about Spack is that it a) seems to be quite well suited for the types of build problems I see in the HPC context; b) you don't need to ask anyone for permission to (temporaily) locally add a new version of a library; and c) can be used non-intrusively to install dependencies:

  spack install highfive
  cmake -DCMAKE_PREFIX_PATH=$(spack location --install-dir highfive) -B build .
  

• Modern CMake enables using add_subdirectory(deps/highfive) to integrate HighFive into a project that wants to use HighFive. The cannonical way of getting deps/highfive would be to use Git submodules (which are better than their reputation, for certain usecases, such as this one). We test this type of integration during CI, see tests/test_project_integration.sh.

In summary: Personally, I think the advantage this approach usually offers doesn't fully materialize for HighFive. While, the threat of multiple incompatible variants of the same version of HighFive seems real, as does the maintenance effort to keep 3 systems running "everywhere". Additionally, it takes away the ability to move away from header-only, or perform other restructuring of the code that would allow improving compile times (neither is currently being considered).

[ddevienne]

The goal here though, is to use HighFive in existing legacy large code bases, with ad-hoc build-systems and all dependencies vendored.
Thus IMHO Spack or Modern CMake is not an option. And that's where dropping a header-only file in that legacy code-base shines.

Furthermore, in my use-case, I only want plain std interop, no internal libraries (like your mentioned Eigen, etc...).
Maybe I'm in the minority, not sure. In any case, I fully understand my request might not align well with the project goals.
I do believe a single-file-amalgamated release would lower barrier to entry for HighFive, but that's just my opinion of course.

[1uc]

Would a script to download the contents of include form a particular version of HighFive to a path of your choosing be enough?

[ddevienne]

I don't see what that gains me, to be honest. Adding and updating a single file is easy in our (non-Git) SCM. While an update with a file-hierarchy might involve additions/removals of files, in addition to requiring checkout of all vendored files, for possible update. Not insurmontable. Just not as plain and simple as dealing with a single file. Anyways, doesn't sound like you consider single-file-lib valuable. That's OK. Either I just won't use HighFive (easy status quo on my side), or I'll deal with the added complexity of multiple files and folders. As I wrote before, the lower the barrier to entry, the better in my book. My $0.02. --DD

[alkino]

To be honest, I was trying to produce a clean single_header in a mechanical way but did not succeed, if you want to give a try, feel free.

[ddevienne]

Fair enough. Thank you for trying.