Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Better ppc64le support and minor improvements #184

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Better ppc64le support and minor improvements #184

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
31e1b22
Better support for ppc64le
mgiessing Aug 25, 2022
1acc43e
Add logic to build gn if no binary
mgiessing Aug 25, 2022
f15ef90
Misspelled arch
mgiessing Aug 25, 2022
e0a2c74
Fix arch issue & add build target
mgiessing Aug 25, 2022
9d40695
Fix arch
mgiessing Aug 25, 2022
199a19d
Fix build issue
mgiessing Aug 25, 2022
3a867ae
Simplified install logic
mgiessing Sep 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 19 additions & 6 deletions build_container/build_container_common.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,21 +11,34 @@ function download_and_check () {

function install_gn(){
# Install gn tools which will be used for building wee8
# amd64 & arm64 install binary, else compile from source
case "$(uname -m)" in
"x86_64")
GN_ARCH=amd64
wget -O gntool.zip "https://chrome-infra-packages.appspot.com/dl/gn/gn/linux-${GN_ARCH}/+/latest"
unzip gntool.zip -d gntool
cp gntool/gn /usr/local/bin/gn
chmod +x /usr/local/bin/gn
rm -rf gntool*
;;

"aarch64")
GN_ARCH=arm64
wget -O gntool.zip "https://chrome-infra-packages.appspot.com/dl/gn/gn/linux-${GN_ARCH}/+/latest"
unzip gntool.zip -d gntool
cp gntool/gn /usr/local/bin/gn
chmod +x /usr/local/bin/gn
rm -rf gntool*
;;

"ppc64le")
git clone https://gn.googlesource.com/gn && cd gn
CC=/opt/llvm/bin/clang CXX=/opt/llvm/bin/clang++ python3 build/gen.py && \
ninja -C out && \
ln -sf out/gn /usr/local/bin/gn
;;
esac

wget -O gntool.zip "https://chrome-infra-packages.appspot.com/dl/gn/gn/linux-${GN_ARCH}/+/latest"
unzip gntool.zip -d gntool
cp gntool/gn /usr/local/bin/gn
chmod +x /usr/local/bin/gn
rm -rf gntool*
esac
}

if [[ "$(uname -m)" == "x86_64" ]]; then
Expand Down
24 changes: 9 additions & 15 deletions build_container/build_container_ubuntu.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,7 @@ curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | \

# docker-ce-cli
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
case $ARCH in
'ppc64le' )
add-apt-repository "deb [arch=ppc64le] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
;;
'x86_64' )
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
;;
'aarch64' )
add-apt-repository "deb [arch=arm64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
;;
esac
add-apt-repository "deb [arch=$(dpkg --print-architecture)] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

# CMake
curl -fsSL https://apt.kitware.com/keys/kitware-archive-latest.asc | apt-key add -
Expand All @@ -53,11 +43,11 @@ PACKAGES=(
gdb
git
gnupg2
google-cloud-sdk
graphviz
jq
libffi-dev
libncurses-dev
libtinfo5
libtool
make
ninja-build
Expand All @@ -78,6 +68,10 @@ PACKAGES=(
xz-utils
zip)

if [[ "${ARCH}" == "x86_64" || "${ARCH}" == "aarch64" ]]; then
PACKAGES+=("google-cloud-sdk")
fi

apt-get install -y --no-install-recommends "${PACKAGES[@]}"

# Set LLVM version for each cpu architecture.
Expand All @@ -94,16 +88,15 @@ case $ARCH in
'aarch64' )
LLVM_DISTRO=aarch64-linux-gnu
LLVM_SHA256SUM=1792badcd44066c79148ffeb1746058422cc9d838462be07e3cb19a4b724a1ee
apt-get install -y --no-install-recommends libtinfo5 # LLVM dependencies on Ubuntu 20.04
;;
esac

# Bazel and related dependencies.
case $ARCH in
'ppc64le' )
BAZEL_LATEST="$(curl https://oplab9.parqtec.unicamp.br/pub/ppc64el/bazel/ubuntu_16.04/latest/ 2>&1 \
BAZEL_LATEST="$(curl https://oplab9.parqtec.unicamp.br/pub/ppc64el/bazel/ubuntu_$(lsb_release -r | awk '{print $2}')/latest/ 2>&1 \
| sed -n 's/.*href="\([^"]*\).*/\1/p' | grep '^bazel' | head -n 1)"
curl -fSL https://oplab9.parqtec.unicamp.br/pub/ppc64el/bazel/ubuntu_16.04/latest/${BAZEL_LATEST} \
curl -fSL https://oplab9.parqtec.unicamp.br/pub/ppc64el/bazel/ubuntu_$(lsb_release -r | awk '{print $2}')/latest/${BAZEL_LATEST} \
-o /usr/local/bin/bazel
chmod +x /usr/local/bin/bazel
;;
Expand Down Expand Up @@ -132,6 +125,7 @@ pip3 install -U pyyaml virtualenv
source ./build_container_common.sh

# Soft link the gcc compiler (required by python env)
ARCH=${ARCH/ppc64le/powerpc64le} # For ppc64le the ARCH variable must be renamed accordingly
update-alternatives --install "/usr/bin/${ARCH}-linux-gnu-gcc" "${ARCH}-linux-gnu-gcc" "/usr/bin/${ARCH}-linux-gnu-gcc-9" 1

apt-get clean
2 changes: 1 addition & 1 deletion build_container/docker_build_linux.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ config_env() {

if [[ -z "${BUILD_TOOLS_PLATFORMS}" ]]; then
if [[ "${OS_DISTRO}" == "ubuntu" ]]; then
export BUILD_TOOLS_PLATFORMS=linux/arm64,linux/amd64
export BUILD_TOOLS_PLATFORMS=linux/arm64,linux/amd64,linux/ppc64le
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's hold off this for now as it drastically increases image build time in CI with emulation + building tools.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure - I can remove this with the next commit I'm just wondering if you don't want to have it built at all or just while this PR is not finally approved.

Thanks!

else
export BUILD_TOOLS_PLATFORMS=linux/amd64
fi
Expand Down