Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-44950: [C++] Bump minimum CMake version to 3.25 #44989

Draft
wants to merge 33 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
f6ca263
GH-44950: [C++] Bump minimum CMake version to 3.25
raulcd Dec 10, 2024
4e58214
Try installing CMake 3.25 manually on Ubuntu
raulcd Dec 10, 2024
5465906
Install cmake in linux-apt-lint.dockerfile and update install_cmake.s…
raulcd Dec 10, 2024
6be9597
Install CMake 3.25.0 on several Ubuntu jobs and Windows jobs that req…
raulcd Dec 10, 2024
ecd437e
Several fixes to install_cmake
raulcd Dec 10, 2024
00f66f1
Some more fixes and upgrade minimal examples to use Ubuntu 24.04
raulcd Dec 10, 2024
8b482b5
Some more fixes for R
raulcd Dec 10, 2024
c62ecd0
Update from CMAKE to CMAKE_VERSION
raulcd Dec 13, 2024
e5d5211
Ensure that versions returned are strings, bump cmake download
jonkeane Dec 14, 2024
b73073c
Try forcing MINGW package to pull a newer CMake
raulcd Dec 17, 2024
5bbb6a9
Revert trying to pull newer CMake
raulcd Dec 17, 2024
1097be0
Try to completely remove mingw CMake from CRAN
raulcd Dec 19, 2024
a9a5611
/c/cmake/bin/cmake.exe should be present and in path, try with .exe
raulcd Dec 19, 2024
11597b0
Try with full path for CMake
raulcd Dec 19, 2024
5201d8b
Try removing manual installation of CMake and installing CMake via pa…
raulcd Dec 19, 2024
ecf01cc
Remove manual Installation of CMake from pacman not present
raulcd Dec 19, 2024
034393b
More tries
raulcd Dec 19, 2024
08fe858
Temporarily try using msys2 GH action to install cmake
raulcd Dec 19, 2024
2550502
Remove CMake from PKGBUILD
raulcd Dec 19, 2024
39e8572
CMake should be in the path
raulcd Dec 19, 2024
d9e0c2c
Debug new CMake location
raulcd Dec 19, 2024
e2b0c6e
Try with C:\Program Files\CMake\bin\cmake.exe
raulcd Dec 19, 2024
17c0d57
Just testing at this point
raulcd Dec 19, 2024
371c06f
Bundle AWSSDK under RTools (#90)
amoeba Dec 23, 2024
63e8847
Check for mingw
raulcd Dec 23, 2024
820b285
Check Windows compiler
raulcd Dec 23, 2024
ceeab3d
Try modifying patch due to redefinition
raulcd Dec 23, 2024
84ab6ed
AWS is bundled on Windows R now, do not use MSYS libraries
raulcd Dec 23, 2024
b875632
Fix license header on patch + add CMake condition for AWS patch
raulcd Dec 23, 2024
6bb2000
Update configure.win to remove specific AWS CPP libs
raulcd Dec 23, 2024
80501c7
Update r/configure.win
raulcd Dec 23, 2024
1da91c0
Update r/configure.win
amoeba Dec 23, 2024
cbc1c4e
Test turning ccache back on
amoeba Dec 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ UBUNTU=22.04

# Default versions for various dependencies
CLANG_TOOLS=14
CMAKE_VERSION=3.25.0
CUDA=11.2.2
DASK=latest
DOTNET=8.0
Expand Down
10 changes: 10 additions & 0 deletions .github/workflows/cpp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,11 @@ jobs:
with:
fetch-depth: 0
submodules: recursive
- name: Install CMake 3.25.0
shell: bash
run: |
ci/scripts/install_cmake.sh 3.25.0 /tmp/local/
echo "/tmp/local/bin" >> $GITHUB_PATH
- name: Check CMake presets
run: |
cd cpp
Expand Down Expand Up @@ -352,6 +357,11 @@ jobs:
with:
fetch-depth: 0
submodules: recursive
- name: Install CMake 3.25.0
shell: bash
run: |
ci/scripts/install_cmake.sh 3.25.0 /c/cmake
echo "c:\\cmake\\bin" >> $GITHUB_PATH
- name: Download Timezone Database
shell: bash
run: ci/scripts/download_tz_database.sh
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,11 @@ jobs:
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
python-version: 3.12
- name: Install CMake 3.25.0
shell: bash
run: |
ci/scripts/install_cmake.sh 3.25.0 /tmp/local/
echo "/tmp/local/bin" >> $GITHUB_PATH
- name: Install pre-commit
run: |
python -m pip install pre-commit
Expand Down
17 changes: 17 additions & 0 deletions .github/workflows/r.yml
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,23 @@ jobs:
restore-keys: |
r-${{ matrix.config.rtools }}-ccache-mingw-${{ matrix.config.arch }}-${{ hashFiles('cpp/src/**/*.cc','cpp/src/**/*.h)') }}-
r-${{ matrix.config.rtools }}-ccache-mingw-${{ matrix.config.arch }}-
- name: Install CMake via MSYS2
uses: msys2/setup-msys2@v2
with:
msystem: UCRT64
update: true
install: mingw-w64-ucrt-x86_64-cmake mingw-w64-ucrt-x86_64-aws-sdk-cpp
- name: Debug CI paths
shell: bash
run: |
echo "## /c/msys64"
ls /c/msys64
echo "## /c/msys64/usr/lib"
ls /c/msys64/usr/lib
echo "## /c/msys64/ucrt64"
ls /c/msys64/ucrt64
echo "## /c/msys64/ucrt64/lib"
ls /c/msys64/ucrt64/lib
- uses: r-lib/actions/setup-r@v2
with:
# Note: RTools must be 40 here because RTools40 + ucrt is how we build the Arrow C++
Expand Down
4 changes: 4 additions & 0 deletions ci/docker/linux-r.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,10 @@ RUN /arrow/ci/scripts/r_docker_configure.sh
COPY ci/scripts/install_sccache.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin

ARG cmake
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/

COPY ci/scripts/r_deps.sh /arrow/ci/scripts/
COPY r/DESCRIPTION /arrow/r/
RUN /arrow/ci/scripts/r_deps.sh /arrow
2 changes: 1 addition & 1 deletion ci/docker/python-wheel-manylinux.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ ENV PATH=/opt/python/${CPYTHON_VERSION}-${CPYTHON_VERSION}/bin:${PATH}
# Install CMake
ARG cmake=3.29.2
COPY ci/scripts/install_cmake.sh arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${arch} linux ${cmake} /usr/local
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local

# Install Ninja
ARG ninja=1.10.2
Expand Down
5 changes: 4 additions & 1 deletion ci/docker/ubuntu-20.04-cpp-minimal.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ RUN apt-get update -y -q && \
apt-get install -y -q \
build-essential \
ccache \
cmake \
curl \
gdb \
git \
Expand Down Expand Up @@ -68,6 +67,10 @@ RUN latest_system_llvm=10 && \
apt-get clean && \
rm -rf /var/lib/apt/lists*

ARG cmake
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/

COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local

Expand Down
5 changes: 4 additions & 1 deletion ci/docker/ubuntu-20.04-cpp.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@ RUN apt-get update -y -q && \
autoconf \
ca-certificates \
ccache \
cmake \
curl \
g++ \
gcc \
Expand Down Expand Up @@ -120,6 +119,10 @@ RUN apt-get update -y -q && \
apt-get clean && \
rm -rf /var/lib/apt/lists*

ARG cmake
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/

COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local

Expand Down
4 changes: 4 additions & 0 deletions ci/docker/ubuntu-22.04-cpp-minimal.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,10 @@ RUN latest_system_llvm=14 && \
apt-get clean && \
rm -rf /var/lib/apt/lists*

ARG cmake
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/

COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local

Expand Down
5 changes: 4 additions & 1 deletion ci/docker/ubuntu-22.04-cpp.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@ RUN apt-get update -y -q && \
ceph \
ceph-fuse \
ceph-mds \
cmake \
curl \
gdb \
git \
Expand Down Expand Up @@ -172,6 +171,10 @@ RUN if [ "${gcc_version}" = "" ]; then \
# make sure zlib is cached in the EMSDK folder
RUN source ~/emsdk/emsdk_env.sh && embuilder --pic build zlib

ARG cmake
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/

COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local

Expand Down
66 changes: 66 additions & 0 deletions ci/rtools/BuildAwsCCommon.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

diff --git a/cmake/AwsCFlags.cmake b/cmake/AwsCFlags.cmake
index b717bca..5aa8ac9 100644
--- a/cmake/AwsCFlags.cmake
+++ b/cmake/AwsCFlags.cmake
@@ -120,6 +120,10 @@ function(aws_set_common_properties target)
list(APPEND AWS_C_FLAGS -Wno-strict-aliasing)
endif()

+ if(CMAKE_C_IMPLICIT_LINK_LIBRARIES MATCHES "mingw32")
+ list(APPEND AWS_C_FLAGS -D__USE_MINGW_ANSI_STDIO=1 -Wno-unused-local-typedefs)
+ endif()
+
# -moutline-atomics generates code for both older load/store exclusive atomics and also
# Arm's Large System Extensions (LSE) which scale substantially better on large core count systems.
#
diff --git a/include/aws/common/byte_order.inl b/include/aws/common/byte_order.inl
index 1204be0..0abd9cb 100644
--- a/include/aws/common/byte_order.inl
+++ b/include/aws/common/byte_order.inl
@@ -13,7 +13,7 @@
# include <stdlib.h>
#else
# include <netinet/in.h>
-#endif /* _MSC_VER */
+#endif /* _WIN32 */

AWS_EXTERN_C_BEGIN

@@ -39,7 +39,7 @@ AWS_STATIC_IMPL uint64_t aws_hton64(uint64_t x) {
uint64_t v;
__asm__("bswap %q0" : "=r"(v) : "0"(x));
return v;
-#elif defined(_MSC_VER)
+#elif defined(_WIN32)
return _byteswap_uint64(x);
#else
uint32_t low = x & UINT32_MAX;
diff --git a/source/windows/thread.c b/source/windows/thread.c
index 447fcd2..01e643f 100644
--- a/source/windows/thread.c
+++ b/source/windows/thread.c
@@ -10,6 +10,7 @@
#include <aws/common/private/thread_shared.h>
#include <aws/common/string.h>

+#define _WIN32_WINNT 0x0601
#include <Windows.h>

#include <inttypes.h>
79 changes: 79 additions & 0 deletions ci/rtools/aws_c_io_ep.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

diff --git a/source/windows/host_resolver.c b/source/windows/host_resolver.c
index 59fbb85..ad4a99e 100644
--- a/source/windows/host_resolver.c
+++ b/source/windows/host_resolver.c
@@ -4,6 +4,7 @@
*/

/* don't move this below the Windows.h include!!!!*/
+#define _WIN32_WINNT 0x0601
#include <winsock2.h>
#include <ws2tcpip.h>

diff --git a/source/windows/iocp/iocp_event_loop.c b/source/windows/iocp/iocp_event_loop.c
index 9ccce30..5cbbef7 100644
--- a/source/windows/iocp/iocp_event_loop.c
+++ b/source/windows/iocp/iocp_event_loop.c
@@ -12,6 +12,7 @@

#include <aws/io/logging.h>

+#define _WIN32_WINNT 0x0601
#include <Windows.h>

/* The next set of struct definitions are taken directly from the
diff --git a/source/windows/secure_channel_tls_handler.c b/source/windows/secure_channel_tls_handler.c
index 50caf02..7f2ab55 100644
--- a/source/windows/secure_channel_tls_handler.c
+++ b/source/windows/secure_channel_tls_handler.c
@@ -2,6 +2,7 @@
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0.
*/
+#define _WIN32_WINNT 0x0601
#define SECURITY_WIN32

#include <aws/io/tls_channel_handler.h>
@@ -35,6 +36,25 @@
# pragma warning(disable : 4306) /* Identifier is type cast to a larger pointer. */
#endif

+#ifndef SP_PROT_TLS1_0_SERVER
+#define SP_PROT_TLS1_0_SERVER SP_PROT_TLS1_SERVER
+#endif
+#ifndef SP_PROT_TLS1_0_CLIENT
+#define SP_PROT_TLS1_0_CLIENT SP_PROT_TLS1_CLIENT
+#endif
+#ifndef SP_PROT_TLS1_1_SERVER
+#define SP_PROT_TLS1_1_SERVER 0x00000100
+#endif
+#ifndef SP_PROT_TLS1_1_CLIENT
+#define SP_PROT_TLS1_1_CLIENT 0x00000200
+#endif
+#ifndef SCH_USE_STRONG_CRYPTO
+#define SCH_USE_STRONG_CRYPTO 0x00400000
+#endif
+#ifndef SECBUFFER_ALERT
+#define SECBUFFER_ALERT 0x11
+#endif
+
#define KB_1 1024
#define READ_OUT_SIZE (16 * KB_1)
#define READ_IN_SIZE READ_OUT_SIZE
Loading
Loading