[boost] add support for multiple boost_python versions #11998

markferry · 2022-08-01T17:27:29Z

Fixes #12038.

Specify library name and version: boost >=1.70

Boost (>=1.67) supports building multiple versions of boost_pythonXY for various python versions.

This adds support for building these libraries.

Python versions are specified using boost:pythonXY=True and boost:pythonXY_executable.
The options are provided for python 2.7 and 3.5+.

Example:

conan create ... -o boost:without_python=False -o boost:python27=True -o boost:python27_executable=`which python2.7`  -o boost:python38=True -o boost:python38_executable=`which python3.8` recipes/boost/all 1.76.0@

As part of these changes python-related methods were refactored to class PythonTool.

Example Usage

Implicit: `boost:without_python=False`

Enable current python (as used for conan invocation) and set correct pythonXY.

e.g.

docker python:X.Y conan create ... -o boost:without_python=False

Explicit: `boost:pythonXY_version=True`

Build one or more specific versions. Disables detection of current python.
Requires a matching boost:pythonXY_executable option.
Does not imply boost:without_python=False which must also be specified explicitly.

e.g.

conan create -o boost:without_python=False -o boost:python27=True -o boost:python27_executable=python2.7 -o boost:python310=True -o boost:python310=python ...

Workflow

This does not work:

conan install --build missing -r conancenter . -pr=default
conan install -o boost:without_atomic=True -o boost:without_chrono=True -o boost:without_container=True -o boost:without_context=True -o boost:without_contract=True -o boost:without_coroutine=True -o boost:without_date_time=True -o boost:without_exception=True -o boost:without_fiber=True -o boost:without_filesystem=True -o boost:without_graph=True -o boost:without_graph_parallel=True -o boost:without_iostreams=True -o boost:without_json=True -o boost:without_locale=True -o boost:without_log=True -o boost:without_math=True -o boost:without_mpi=True -o boost:without_nowide=True -o boost:without_program_options=True -o boost:without_python=False -o boost:without_random=True -o boost:without_regex=True -o boost:without_serialization=True -o boost:without_stacktrace=True -o boost:without_system=True -o boost:without_test=True -o boost:without_thread=True -o boost:without_timer=True -o boost:without_type_erasure=True -o boost:without_wave=True -o boost:python310=True -o boost:python310_executable=`which python3` -o boost:python39=True -o boost:python39_executable=$HOME/src/conan-center-index/conan-venv39/bin/python .
conan build .
conan package .
conan export-pkg  -o boost:without_atomic=True -o boost:without_chrono=True -o boost:without_container=True -o boost:without_context=True -o boost:without_contract=True -o boost:without_coroutine=True -o boost:without_date_time=True -o boost:without_exception=True -o boost:without_fiber=True -o boost:without_filesystem=True -o boost:without_graph=True -o boost:without_graph_parallel=True -o boost:without_iostreams=True -o boost:without_json=True -o boost:without_locale=True -o boost:without_log=True -o boost:without_math=True -o boost:without_mpi=True -o boost:without_nowide=True -o boost:without_program_options=True -o boost:without_python=False -o boost:without_random=True -o boost:without_regex=True -o boost:without_serialization=True -o boost:without_stacktrace=True -o boost:without_system=True -o boost:without_test=True -o boost:without_thread=True -o boost:without_timer=True -o boost:without_type_erasure=True -o boost:without_wave=True -o boost:python310=True -o boost:python310_executable=`which python3` -o boost:python39=True -o boost:python39_executable=$HOME/src/conan-center-index/conan-venv39/bin/python . boost/1.76.0@gnome/test

conan test test_package boost/1.76.0@gnome/test  # fails with missing libs!

Alternatively:

conan create -k -o boost:without_atomic=True -o boost:without_chrono=True -o boost:without_container=True -o boost:without_context=True -o boost:without_contract=True -o boost:without_coroutine=True -o boost:without_date_time=True -o boost:without_exception=True -o boost:without_fiber=True -o boost:without_filesystem=True -o boost:without_graph=True -o boost:without_graph_parallel=True -o boost:without_iostreams=True -o boost:without_json=True -o boost:without_locale=True -o boost:without_log=True -o boost:without_math=True -o boost:without_mpi=True -o boost:without_nowide=True -o boost:without_program_options=True -o boost:without_python=False -o boost:without_random=True -o boost:without_regex=True -o boost:without_serialization=True -o boost:without_stacktrace=True -o boost:without_system=True -o boost:without_test=True -o boost:without_thread=True -o boost:without_timer=True -o boost:without_type_erasure=True -o boost:without_wave=True -o boost:python310=True -o boost:python310_executable=`which python3` -o boost:python39=True -o boost:python39_executable=$HOME/src/conan-center-index/conan-venv39/bin/python . 1.76.0@gnome/test

# then trial install with
conan install --build=never ...

I've read the guidelines for contributing.
I've followed the PEP8 style guides for Python code in the recipes.
I've used the latest Conan client version.
I've tried at least one configuration locally with the conan-center hook activated.

CLAassistant · 2022-08-01T17:27:33Z

All committers have signed the CLA.

ghost · 2022-08-01T17:31:15Z

I detected other pull requests that are modifying boost/all recipe:

Added patch that disables Boost deprecated code which fixes compilation on Clang-15 #13053

This message is automatically generated by https://github.com/ericLemanissier/conan-center-conflicting-prs so don't hesitate to report issues/improvements there.

markferry · 2022-08-03T23:28:11Z

New version of the patch adds new options boost:pythonXY_executable for most relevant versions of python (2.7, 3.5+).

markferry · 2022-08-04T01:12:03Z

FIXME: doesn't yet package the extra libs.

Boost b2 supports building Boost.Python for multiple python versions. (https://www.boost.org/doc/libs/1_79_0/libs/python/doc/html/building/configuring_boost_build.html) Pass option `boost:pythonXY_executable` to enable building boost_python for X.Y of python. For example when in a python 3.10 environment, adding these options: -o boost:python27_executable=`which python2.7` \ -o boost:python36_executable=`which python3.6` \ ... Will build libboost_python310 (the default), libboost_python27 and libboost_python36.

FIXME: raise an error when explicitly specifying `pythonXY_version` without the matching `pythonXY_executable` option. FIXME: extra python libs are missing python_buildid and buildid

Tests only one detected python.

self._pythons is only populated in the configure method so use the explicit options instead. FIXME: does this work for implicit (detected) python version?

markferry · 2022-08-24T15:00:38Z

FIXME: fix ABI compatibility between python versions

Given:

conan create -o boost:python310=True -o boost:python27=True ...

This should work:

conan install -o boost:python310=True --build=never

And should install the pre-built package.

markferry · 2022-08-24T22:12:20Z

recipes/boost/all/test_package/conanfile.py

+                (v, _) = self._all_pythons()[0]
+                cmake.definitions["Python_ADDITIONAL_VERSIONS"] = "{}.{}".format(v.major, v.minor)
+                cmake.definitions["PYTHON_COMPONENT_SUFFIX"] = "{}{}".format(v.major, v.minor)


This (somewhat arbitrary) selection of python lib to link against means all enabled pythons must have their equivalent pythonXY-devel package installed.

Otherwise no /usr/lib/libpythonXY.so and cmake configure fails with errors of the form:

CMake Error at CMakeLists.txt:113 (target_link_libraries): Target "hello_ext" links to: Boost::python39 but the target was not found. Possible reasons include: * There is a typo in the target name. * A find_package call is missing for an IMPORTED target. * An ALIAS target is missing.

markferry · 2022-08-24T22:59:36Z

find_package(Boost COMPONENTS python REQUIRED) breaks test_package if more than one libpythonXY.so is detected.

/usr/bin/ld: $CONAN_USER_HOME/.conan/data/boost/1.76.0/gnome/test/package/e68cab19bbb84f8e32a323e0a11b6364bcd37c17/lib/libboost_python310.a(module.o):(.bss+0x0): multiple definition of `boost::python::detail::current_scope'; $CONAN_USER_HOME/.conan/data/boost/1.76.0/gnome/test/package/e68cab19bbb84f8e32a323e0a11b6364bcd37c17/lib/libboost_python27.a(module.o):(.bss+0x0): first defined here

Should be compatible with *any* extra versions (max 2^len(PYTHON_VERSIONS) packages) but limiting it to just any one extra. e.g. `conan install -o boost:python310=True` could fall back to `-o boost:python310=True -o boost:python27=True`

markferry · 2022-08-25T16:49:20Z

find_package(Boost COMPONENTS python REQUIRED) breaks test_package if more than one libpythonXY.so is detected.

boost_python27 and boost_python310 are mutually exclusive but both are
included in the lib deps when linking test_package resulting in multiply-defined
symbols.

I could remove all python libs from package_info and rely instead on magic_autolink but is there any other way?

Any ideas to resolve this last stumbling block @SSE4, @uilianries ?

conan-center-bot · 2022-08-25T19:36:52Z

All green in build 28 (aa5b781c82233f0e14e0558013236138daeeb16d):

boost/1.71.0@:
All packages built successfully! (All logs)