Add Apache Avro wrap

[stephanlachnit]

Adds a wrap for the C++ bindings of Apache Avro. Since it's only possible to build it without Boost using main, we need to wait until a new release.

TODO:

• Get release tarball from https://dlcdn.apache.org/avro/ and adjust folder structure
• Tests (optional)

/cc @WillAyd

[stephanlachnit]

Can we add program_names to a wrap for a program that might be disabled to be built or is that an issue?

[WillAyd]

Cool - thanks for doing this!

[WillAyd]

FYI on other Apache projects we have been using feature options instead of boolean. I don't think its a huge deal, but probably nice to stick with that pattern for consistency in the ecosystem

[stephanlachnit]

@bgilbert any idea why the CI fails with this cryptic message:

Run ./tools/fake_tty.py ./tools/sanity_checks.py
  ./tools/fake_tty.py ./tools/sanity_checks.py
  shell: /usr/bin/bash -e {0}
  env:
    TEST_ANNOTATE_CONTEXT: yes
    TEST_SKIP_BUILD: yes
Traceback (most recent call last):
  File "/home/runner/work/wrapdb/wrapdb/./tools/fake_tty.py", line 6, in <module>
    sys.exit(os.waitstatus_to_exitcode(pty.spawn(sys.argv[1:])))
                                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/pty.py", line 195, in spawn
    os.execlp(argv[0], *argv)
  File "<frozen os>", line 563, in execlp
  File "<frozen os>", line 580, in execvp
  File "<frozen os>", line 603, in _execvpe
PermissionError: [Errno 13] Permission denied
Error: Process completed with exit code 1.

[bgilbert]

Can we add program_names to a wrap for a program that might be disabled

Yes, add a skip_program_check in ci_config.json.

any idea why the CI fails with this cryptic message

The patch to sanity_checks.py removes its executable bit for some reason.

[WillAyd]

I think if you just include the hash here instead of the version you can work around the sanity check issue. So 82a2bc8b034de34626e2ab8bf091234122474d50-1

[stephanlachnit]

I'm not quite sure what exactly will do the trick, since differences between the json and the meson build file will lead to some error at some point. Do you happen to know when there will be a new avro release? That might be the easiest...

[WillAyd]

Not sure. That's also probably a question for @wgtmac

Although as far as the wrapdb is concerned, I think we will wait for the release before merging this. Any commit-hash inclusion here is just a workaround for now to trick CI

[wgtmac]

I'm not a maintainer of Apache Avro so I cannot make any promise here. Perhaps @Fokko @martin-g @thiru-mg can help with this?

[wgtmac]

BTW, iceberg-cpp is also depending on a hash tag of avro-cpp: https://github.com/apache/iceberg-cpp/blob/main/cmake_modules/IcebergThirdpartyToolchain.cmake#L186

[stephanlachnit]

@WillAyd how should the dependency be named? For avro-c the pkg-config file is named avro-c, but avro-cpp does not have a pkg-config file upstream. The CMake target is called Avro - should we go with that, or rather go with a more descriptive avro-cpp? Probably makes sense to add pkg-config support upstream then as well.

[WillAyd]

Hmm following the c pattern it sounds like avro-cpp is best. @eli-schwartz may have a suggestion here as well

[eli-schwartz]

We definitely want to align with upstream. I agree that given the name "avro-c" it makes a decent amount of sense to use "avro-cpp" as well.

Avro upstream has only added CMake package config files for avro-cpp, not for avro-c, and it did that in January which hasn't been incorporated in a release yet. So it can be changed.

https://issues.apache.org/jira/browse/AVRO-3088 asked for "this way one could just use find_package(avrocpp) but it looks like unfortunately a decision was made to provide find_package(Avro) instead, which would mean it will be hard to have consistent naming.

[WillAyd]

FYI @wgtmac - you may be interested in the discussion around the package name for Avro

[wgtmac]

Its CMake project name is project (Avro-cpp). So perhaps its package name and namespace should be AvroCpp? It is not too late to fix that before the next Avro release.

Proposed change: apache/avro#3477

[stephanlachnit]

In principle it wouldn't be a problem to have a CMake target and pkg-config target with different names - this is also the case for zstd (pkg-config: libzstd). However then we should ensure that upstream actually generates a pkg-config file for the C++ version to be consistent. I opened a PR which will include one named avro-cpp (upper-case names are rather unusual for pkg-config files): apache/avro#3479

[eli-schwartz]

In principle it wouldn't be a problem to have a CMake target and pkg-config target with different names - this is also the case for zstd (pkg-config: libzstd).

Let's stop right there.

Official meson policy is that it's a big problem, breaks assumptions, and requires what is officially termed as "a workaround" in projects that do dependency('pkgconfig-name') and consequently don't see any CMake package configs by other names. That means idiomatic projects cannot find the dependency at all, even if the CMake dependency finder is operational, if pkg-config isn't installed (and this scenario happens not infrequently on Windows).

I will add to this by pointing out that zstandard canonically, officially, doesn't provide CMake package config files at all. The sole supported build system is GNU Makefiles, all other build systems are unofficial. The unofficial meson support carefully matches the Makefiles, while the unofficial CMake and visual studio solution files do not. One of the ways the CMake unofficial files differ from the Makefiles is that they (alone) do provide unvetted CMake package config files that the zstandard developers have made no real effort to consider the correctness of.

You can't assume that those zstd CMake package config files exist at all -- e.g. there are some Linux distros that do provide them, while others do not provide them.

[stephanlachnit]

Right, I have not considered the case where pkg-config is not installed, so point taken. I guess then if the CMake dependency is called AvroCpp, so should the pkg-config file. I will rename it in the PR then once a name is settled on in apache/avro#3477.

[WillAyd]

Thanks for all of the upstream work on this @wgtmac and @stephanlachnit !