Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packages and build systems confusion: Separate or combined libraries? #968

Open
drhpc opened this issue Dec 30, 2023 · 0 comments
Open

Packages and build systems confusion: Separate or combined libraries? #968

drhpc opened this issue Dec 30, 2023 · 0 comments

Comments

@drhpc
Copy link

drhpc commented Dec 30, 2023

TL;DR: Support combined library builds in the reference to capture an (emerging) standard?

I am discussion packaging of LAPACK libraries (the four of BLAS, CBLAS, LAPACK, LAPACKE for now) with NumPy and Meson folks. See

mesonbuild/meson#2835
mesonbuild/meson#10921

One point of contention is my insistence on supporting independent explicit choices for the dependencies blas, cblas, lapack, lapacke, which are separate libraries and pkg-config modules in pkgsrc. I switched to the CMake build some time ago to support shared and static libs without patching the build. As I see it, this build always produces

libblas.so
libcblas.so
liblapack.so
liblapacke.so

where you could also replace libblas.so (and liblapack.so) with some external optimized library to only get the C interface or LAPACK on top. Now, this use has been falling out of favour for quite some time now and folks are used to libraries like MKL or OpenBLAS to just combine all APIs in one library.

This has gotten so far that Debian

https://lists.debian.org/debian-devel/2019/10/msg00273.html

has chosen to combine BLAS and CBLAS into its netlib libblas.so. They do that via custom build rules, as the Netlib build system(s) do not even know that option. Or am I missing it?

I would like some standard to follow here. How should the APIs be packaged? When I discuss them as separate libraries, I get told that in practice they are combined, at least CBLAS being included in the library delivering BLAS. This is the case for all optimized libraries with the notable (and funny, isnt' it?) exception of ATLAS, which still provides libcblas.so that is unrelated to netlib's libcblas.so in Debian-derived distros. The assumption in numpy/scipy build that a libblas.so is accompanied by a matching libcblas.so is dead wrong here. It may work in other contexts.

This is a nice mess. What should I do as a packager here, and as someone trying to bugger build system providers to consider my 'special' use case? Is my idea that you should be able to combine any (API-conforming...) BLAS library with netlib's CBLAS wrapper wrong? I'm advocating for support of free-form choice in build systems for each of the four component libraries via differing pkg-config module names. But that is deemed as unrealistic flexibility. Build system implementors want to detect various implementations they know in the field. Should they support hypothetical combinations of wrapper libraries?

Should the netlib standard practice adapt to the precedent given by Debian and optimized libs and put all into one library file? Should I hack this as Debian does? I do not like the idea that there upstream, but nobody installs it like it is prepared and released, lots of build system hacks around. Netlib is the reference, but it is followed in a limited extend. It is the special case now regarding packaging, because other (modern) implementations don't split libraries. I'd like some unification there. Ideally, build systems should not care at all about which BLAS they use, but they do a lot and this causes lost of redundant writing of detection code. I'm not even sure how well LAPACKE is handled compared to (included) CBLAS. Debian does build separate liblapacke, which feels inconsistent. What standard could be set here? I guess this doesn't cause as much confusion as LAPACKE usage is relatively rare.

The situation is really blurry … for the most part, I guess CMake and others also have trouble thinking about the CBLAS library being a separate wrapper library or just an aspect of one combined library.

This text is too long, sorry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant