Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-44950: [C++] Bump minimum CMake version to 3.25 #44989

Draft
wants to merge 23 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
f6ca263
GH-44950: [C++] Bump minimum CMake version to 3.25
raulcd Dec 10, 2024
4e58214
Try installing CMake 3.25 manually on Ubuntu
raulcd Dec 10, 2024
5465906
Install cmake in linux-apt-lint.dockerfile and update install_cmake.s…
raulcd Dec 10, 2024
6be9597
Install CMake 3.25.0 on several Ubuntu jobs and Windows jobs that req…
raulcd Dec 10, 2024
ecd437e
Several fixes to install_cmake
raulcd Dec 10, 2024
00f66f1
Some more fixes and upgrade minimal examples to use Ubuntu 24.04
raulcd Dec 10, 2024
8b482b5
Some more fixes for R
raulcd Dec 10, 2024
c62ecd0
Update from CMAKE to CMAKE_VERSION
raulcd Dec 13, 2024
e5d5211
Ensure that versions returned are strings, bump cmake download
jonkeane Dec 14, 2024
b73073c
Try forcing MINGW package to pull a newer CMake
raulcd Dec 17, 2024
5bbb6a9
Revert trying to pull newer CMake
raulcd Dec 17, 2024
1097be0
Try to completely remove mingw CMake from CRAN
raulcd Dec 19, 2024
a9a5611
/c/cmake/bin/cmake.exe should be present and in path, try with .exe
raulcd Dec 19, 2024
11597b0
Try with full path for CMake
raulcd Dec 19, 2024
5201d8b
Try removing manual installation of CMake and installing CMake via pa…
raulcd Dec 19, 2024
ecf01cc
Remove manual Installation of CMake from pacman not present
raulcd Dec 19, 2024
034393b
More tries
raulcd Dec 19, 2024
08fe858
Temporarily try using msys2 GH action to install cmake
raulcd Dec 19, 2024
2550502
Remove CMake from PKGBUILD
raulcd Dec 19, 2024
39e8572
CMake should be in the path
raulcd Dec 19, 2024
d9e0c2c
Debug new CMake location
raulcd Dec 19, 2024
e2b0c6e
Try with C:\Program Files\CMake\bin\cmake.exe
raulcd Dec 19, 2024
17c0d57
Just testing at this point
raulcd Dec 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ UBUNTU=22.04

# Default versions for various dependencies
CLANG_TOOLS=14
CMAKE_VERSION=3.25.0
CUDA=11.2.2
DASK=latest
DOTNET=8.0
Expand Down
10 changes: 10 additions & 0 deletions .github/workflows/cpp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,11 @@ jobs:
with:
fetch-depth: 0
submodules: recursive
- name: Install CMake 3.25.0
shell: bash
run: |
ci/scripts/install_cmake.sh 3.25.0 /tmp/local/
echo "/tmp/local/bin" >> $GITHUB_PATH
- name: Check CMake presets
run: |
cd cpp
Expand Down Expand Up @@ -352,6 +357,11 @@ jobs:
with:
fetch-depth: 0
submodules: recursive
- name: Install CMake 3.25.0
shell: bash
run: |
ci/scripts/install_cmake.sh 3.25.0 /c/cmake
echo "c:\\cmake\\bin" >> $GITHUB_PATH
- name: Download Timezone Database
shell: bash
run: ci/scripts/download_tz_database.sh
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,11 @@ jobs:
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
python-version: 3.12
- name: Install CMake 3.25.0
shell: bash
run: |
ci/scripts/install_cmake.sh 3.25.0 /tmp/local/
echo "/tmp/local/bin" >> $GITHUB_PATH
- name: Install pre-commit
run: |
python -m pip install pre-commit
Expand Down
17 changes: 17 additions & 0 deletions .github/workflows/r.yml
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,23 @@ jobs:
restore-keys: |
r-${{ matrix.config.rtools }}-ccache-mingw-${{ matrix.config.arch }}-${{ hashFiles('cpp/src/**/*.cc','cpp/src/**/*.h)') }}-
r-${{ matrix.config.rtools }}-ccache-mingw-${{ matrix.config.arch }}-
- name: Install CMake via MSYS2
uses: msys2/setup-msys2@v2
with:
msystem: UCRT64
update: true
install: mingw-w64-ucrt-x86_64-cmake mingw-w64-ucrt-x86_64-aws-sdk-cpp
- name: Debug CI paths
shell: bash
run: |
echo "## /c/msys64"
ls /c/msys64
echo "## /c/msys64/usr/lib"
ls /c/msys64/usr/lib
echo "## /c/msys64/ucrt64"
ls /c/msys64/ucrt64
echo "## /c/msys64/ucrt64/lib"
ls /c/msys64/ucrt64/lib
- uses: r-lib/actions/setup-r@v2
with:
# Note: RTools must be 40 here because RTools40 + ucrt is how we build the Arrow C++
Expand Down
4 changes: 4 additions & 0 deletions ci/docker/linux-r.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,10 @@ RUN /arrow/ci/scripts/r_docker_configure.sh
COPY ci/scripts/install_sccache.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin

ARG cmake
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/

COPY ci/scripts/r_deps.sh /arrow/ci/scripts/
COPY r/DESCRIPTION /arrow/r/
RUN /arrow/ci/scripts/r_deps.sh /arrow
2 changes: 1 addition & 1 deletion ci/docker/python-wheel-manylinux.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ ENV PATH=/opt/python/${CPYTHON_VERSION}-${CPYTHON_VERSION}/bin:${PATH}
# Install CMake
ARG cmake=3.29.2
COPY ci/scripts/install_cmake.sh arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${arch} linux ${cmake} /usr/local
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local

# Install Ninja
ARG ninja=1.10.2
Expand Down
5 changes: 4 additions & 1 deletion ci/docker/ubuntu-20.04-cpp-minimal.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ RUN apt-get update -y -q && \
apt-get install -y -q \
build-essential \
ccache \
cmake \
curl \
gdb \
git \
Expand Down Expand Up @@ -68,6 +67,10 @@ RUN latest_system_llvm=10 && \
apt-get clean && \
rm -rf /var/lib/apt/lists*

ARG cmake
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/

COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local

Expand Down
5 changes: 4 additions & 1 deletion ci/docker/ubuntu-20.04-cpp.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@ RUN apt-get update -y -q && \
autoconf \
ca-certificates \
ccache \
cmake \
curl \
g++ \
gcc \
Expand Down Expand Up @@ -120,6 +119,10 @@ RUN apt-get update -y -q && \
apt-get clean && \
rm -rf /var/lib/apt/lists*

ARG cmake
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/

COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local

Expand Down
4 changes: 4 additions & 0 deletions ci/docker/ubuntu-22.04-cpp-minimal.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,10 @@ RUN latest_system_llvm=14 && \
apt-get clean && \
rm -rf /var/lib/apt/lists*

ARG cmake
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/

COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local

Expand Down
5 changes: 4 additions & 1 deletion ci/docker/ubuntu-22.04-cpp.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@ RUN apt-get update -y -q && \
ceph \
ceph-fuse \
ceph-mds \
cmake \
curl \
gdb \
git \
Expand Down Expand Up @@ -172,6 +171,10 @@ RUN if [ "${gcc_version}" = "" ]; then \
# make sure zlib is cached in the EMSDK folder
RUN source ~/emsdk/emsdk_env.sh && embuilder --pic build zlib

ARG cmake
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/

COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local

Expand Down
6 changes: 2 additions & 4 deletions ci/scripts/PKGBUILD
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,7 @@ pkgdesc="Apache Arrow is a cross-language development platform for in-memory dat
arch=("any")
url="https://arrow.apache.org/"
license=("Apache-2.0")
depends=("${MINGW_PACKAGE_PREFIX}-aws-sdk-cpp"
"${MINGW_PACKAGE_PREFIX}-bzip2"
depends=("${MINGW_PACKAGE_PREFIX}-bzip2"
"${MINGW_PACKAGE_PREFIX}-curl" # for google-cloud-cpp bundled build
"${MINGW_PACKAGE_PREFIX}-libutf8proc"
"${MINGW_PACKAGE_PREFIX}-re2"
Expand All @@ -36,7 +35,6 @@ depends=("${MINGW_PACKAGE_PREFIX}-aws-sdk-cpp"
"${MINGW_PACKAGE_PREFIX}-zstd"
"${MINGW_PACKAGE_PREFIX}-brotli")
makedepends=("${MINGW_PACKAGE_PREFIX}-ccache"
"${MINGW_PACKAGE_PREFIX}-cmake"
"${MINGW_PACKAGE_PREFIX}-gcc")
options=("staticlibs" "strip" "!buildflags")

Expand Down Expand Up @@ -83,7 +81,7 @@ build() {
# segfaults in tests

MSYS2_ARG_CONV_EXCL="-DCMAKE_INSTALL_PREFIX=" \
${MINGW_PREFIX}/bin/cmake.exe \
"${PROGRAMFILES}\CMake\bin\cmake.exe" \
${ARROW_CPP_DIR} \
-G "MSYS Makefiles" \
-DARROW_ACERO=ON \
Expand Down
46 changes: 33 additions & 13 deletions ci/scripts/install_cmake.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,29 +17,49 @@
# specific language governing permissions and limitations
# under the License.

set -e
set -ex

if [ "$#" -ne 2 ]; then
echo "Usage: $0 <version> <prefix>"
exit 1
fi

declare -A archs
archs=([amd64]=x86_64
[arch64]=aarch64
archs=([x86_64]=x86_64
[arm64]=aarch64
[arm64v8]=aarch64
[x86_64]=x86_64)
[aarch64]=aarch64)

arch=$(uname -m)
if [ -z ${archs[$arch]} ]; then
echo "Unsupported architecture: ${arch}"
exit 0
fi
arch=${archs[$arch]}

declare -A platforms
platforms=([linux]=linux
[macos]=macos
[windows]=windows)

if [ "$#" -ne 4 ]; then
echo "Usage: $0 <architecture> <platform> <version> <prefix>"
exit 1
fi
version=$1
prefix=$2

arch=${archs[$1]}
platform=${platforms[$2]}
version=$3
prefix=$4
platform=$(uname)
case ${platform} in
Linux)
platform=linux
;;
Darwin)
platform=macos
;;
MSYS_NT*|MINGW64_NT*)
platform=windows
;;
*)
echo "Unsupported platform: ${platform}"
exit 0
;;
esac

mkdir -p ${prefix}
url="https://github.com/Kitware/CMake/releases/download/v${version}/cmake-${version}-${platform}-"
Expand Down
10 changes: 10 additions & 0 deletions ci/scripts/r_windows_build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,21 @@ export ARROW_HOME="$(cd "${ARROW_HOME}" && pwd)"

# Uncomment L38-41 if you're testing a new rtools dependency that hasn't yet sync'd to CRAN
# curl https://raw.githubusercontent.com/r-windows/rtools-packages/master/pacman.conf > /etc/pacman.conf
# cp /etc/pacman.conf /etc/pacman.conf.bak
# curl https://raw.githubusercontent.com/r-windows/rtools-packages/master/pacman.conf > /etc/pacman.conf
# cat /etc/pacman.conf
# curl -OSsl "http://repo.msys2.org/msys/x86_64/msys2-keyring-r21.b39fb11-1-any.pkg.tar.xz"
# pacman -U --noconfirm msys2-keyring-r21.b39fb11-1-any.pkg.tar.xz && rm msys2-keyring-r21.b39fb11-1-any.pkg.tar.xz
# pacman --noconfirm -Scc

# pacman --noconfirm -Syy
# pacman --noconfirm -S ${MINGW_PACKAGE_PREFIX}-cmake

#Try reverting to the original pacman.conf
# cp /etc/pacman.conf.bak /etc/pacman.conf
# cat /etc/pacman.conf
pacman --noconfirm -Syy

RWINLIB_LIB_DIR="lib"
: ${MINGW_ARCH:="mingw32 mingw64 ucrt64"}

Expand Down
2 changes: 1 addition & 1 deletion cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.

cmake_minimum_required(VERSION 3.16)
cmake_minimum_required(VERSION 3.25)
message(STATUS "Building using CMake version: ${CMAKE_VERSION}")

# https://www.cmake.org/cmake/help/latest/policy/CMP0025.html
Expand Down
2 changes: 1 addition & 1 deletion cpp/examples/minimal_build/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.

cmake_minimum_required(VERSION 3.16)
cmake_minimum_required(VERSION 3.25)

project(ArrowMinimalExample)

Expand Down
2 changes: 1 addition & 1 deletion cpp/examples/minimal_build/minimal.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.

FROM ubuntu:focal
FROM ubuntu:24.04

ENV DEBIAN_FRONTEND=noninteractive

Expand Down
2 changes: 1 addition & 1 deletion cpp/examples/minimal_build/system_dependency.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.

FROM ubuntu:focal
FROM ubuntu:24.04

ENV DEBIAN_FRONTEND=noninteractive

Expand Down
2 changes: 1 addition & 1 deletion cpp/examples/parquet/parquet_arrow/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
# under the License.

# Require cmake that supports BYPRODUCTS in add_custom_command, ExternalProject_Add [1].
cmake_minimum_required(VERSION 3.16)
cmake_minimum_required(VERSION 3.25)

project(parquet_arrow_example)

Expand Down
2 changes: 1 addition & 1 deletion cpp/examples/tutorial_examples/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.

cmake_minimum_required(VERSION 3.16)
cmake_minimum_required(VERSION 3.25)

project(ArrowTutorialExamples)

Expand Down
2 changes: 1 addition & 1 deletion dev/tasks/python-wheels/github.osx.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
- name: Install CMake 3.29.0
shell: bash
run: |
arrow/ci/scripts/install_cmake.sh $(arch) macos 3.29.0 ${PWD}/local
arrow/ci/scripts/install_cmake.sh 3.29.0 ${PWD}/local
echo "${PWD}/local/bin" >> $GITHUB_PATH

- name: Retrieve VCPKG version from arrow/.env
Expand Down
2 changes: 1 addition & 1 deletion dev/tasks/vcpkg-tests/github.windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
- name: Install CMake 3.29.0
shell: bash
run: |
arrow/ci/scripts/install_cmake.sh amd64 windows 3.29.0 /c/cmake
arrow/ci/scripts/install_cmake.sh 3.29.0 /c/cmake
echo "c:\\cmake\\bin" >> $GITHUB_PATH
- name: Download Timezone Database
shell: bash
Expand Down
3 changes: 3 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -389,6 +389,7 @@ services:
arch: ${ARCH}
base: "${ARCH}/ubuntu:${UBUNTU}"
clang_tools: ${CLANG_TOOLS}
cmake: ${CMAKE_VERSION}
llvm: ${LLVM}
gcc_version: ${GCC_VERSION}
shm_size: *shm-size
Expand Down Expand Up @@ -620,6 +621,7 @@ services:
args:
arch: ${ARCH}
clang_tools: ${CLANG_TOOLS}
cmake: ${CMAKE_VERSION}
llvm: ${LLVM}
shm_size: *shm-size
volumes: *ubuntu-volumes
Expand Down Expand Up @@ -1569,6 +1571,7 @@ services:
tz: ${TZ}
r_prune_deps: ${R_PRUNE_DEPS}
r_custom_ccache: ${R_CUSTOM_CCACHE}
cmake: ${CMAKE_VERSION}
shm_size: *shm-size
environment:
<<: [*common, *sccache]
Expand Down
2 changes: 1 addition & 1 deletion docs/source/cpp/build_system.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ file into an executable linked with the Arrow C++ shared library:

.. code-block:: cmake

cmake_minimum_required(VERSION 3.16)
cmake_minimum_required(VERSION 3.25)

project(MyExample)

Expand Down
2 changes: 1 addition & 1 deletion matlab/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.

cmake_minimum_required(VERSION 3.20)
cmake_minimum_required(VERSION 3.25)

# Build the Arrow C++ libraries using ExternalProject_Add.
function(build_arrow)
Expand Down
2 changes: 1 addition & 1 deletion python/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
# Includes code assembled from BSD/MIT/Apache-licensed code from some 3rd-party
# projects, including Kudu, Impala, and libdynd. See python/LICENSE.txt

cmake_minimum_required(VERSION 3.16)
cmake_minimum_required(VERSION 3.25)
project(pyarrow)

# This is needed for 3.13 free-threading. CMake used to add Python
Expand Down
Loading
Loading