Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alpine docker build fails #480

Open
robot-ranger opened this issue Jul 16, 2024 · 18 comments
Open

Alpine docker build fails #480

robot-ranger opened this issue Jul 16, 2024 · 18 comments
Labels

Comments

@robot-ranger
Copy link
Contributor

robot-ranger commented Jul 16, 2024

Formal issue stemming from discussion here: https://github.com/orgs/mtconnect/discussions/476

Only seems to affect Alpine/arm64.

Alpine w/o buildx works fine.
Ubuntu images build successfully.

Alpine docker build fails. Here is the github action error from my own fork:

#15 146.5 boost/1.85.0: RUN: b2 -q target-os=linux architecture=x86 address-model=64 binary-format=elf abi=sysv --layout=system --user-config=/root/.conan2/p/boost5fdc72773338c/s/src/tools/build/user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=off --disable-icu boost.locale.iconv=on boost.locale.iconv.lib=libc threading=multi visibility=hidden link=static variant=release --with-atomic --with-charconv --with-chrono --with-container --with-context --with-coroutine --with-date_time --with-exception --with-filesystem --with-iostreams --with-json --with-locale --with-log --with-program_options --with-random --with-regex --with-stacktrace --with-system --with-thread --with-timer --with-type_erasure --with-url toolset=gcc cxxstd=17 cxxstd-dialect=gnu define=_GLIBCXX_USE_CXX11_ABI=1 pch=on -sLIBBACKTRACE_PATH=/root/.conan2/p/b/libbab04427202a521/p linkflags="" cxxflags="-fPIC -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=/usr/bin/addr2line" install --prefix=/root/.conan2/p/b/boostdd77d3b1dab0b/p -j2 --abbreviate-paths -d0 --debug-configuration --build-dir="/root/.conan2/p/b/boostdd77d3b1dab0b/b/build-release"
#15 146.5 Assertion failed: destlen >= total (libgcompat/string.c: __strncat_chk: 251)
#15 146.6 
#15 146.6 boost/1.85.0: ERROR: 
#15 146.6 Package '74482cd480d12051a8a314bff4f706866d4488a6' build failed
#15 146.6 boost/1.85.0: WARN: Build folder /root/.conan2/p/b/boostdd77d3b1dab0b/b/build-release
#15 146.6 ERROR: boost/1.85.0: Error in build() method, line 1130
#15 146.6 	self.run(full_command)
#15 146.6 	ConanException: Error -6 while executing
#15 ERROR: process "/bin/sh -c if [ -z \"$WITH_TESTS\" ] || [ \"$WITH_TESTS\" = \"false\" ]; then        WITH_TESTS_ARG=\"--test-folder=\";      else        WITH_TESTS_ARG=\"\";      fi   && . /python/conan/bin/activate   && conan profile detect   && conan create cppagent        --build=missing        -c \"tools.build:jobs=$CONAN_CPU_COUNT\"        -o agent_prefix=mtc        -o cpack=True        -o \"with_ruby=$WITH_RUBY\"        -o cpack_destination=/root/agent        -o cpack_name=dist        -o cpack_generator=TGZ        -pr \"$CONAN_PROFILE\"        \"${WITH_TESTS_ARG}\"   && tar xf dist.tar.gz   && rm dist.tar.gz" did not complete successfully: exit code: 1

#18 [linux/arm64 build 4/4] RUN  if [ -z "false" ] || [ "false" = "false" ]; then        WITH_TESTS_ARG="--test-folder=";      else        WITH_TESTS_ARG="";      fi   && . /python/conan/bin/activate   && conan profile detect   && conan create cppagent        --build=missing        -c "tools.build:jobs=2"        -o agent_prefix=mtc        -o cpack=True        -o "with_ruby=True"        -o cpack_destination=/root/agent        -o cpack_name=dist        -o cpack_generator=TGZ        -pr "/root/agent/cppagent/conan/profiles/docker"        "argument"   && tar xf dist.tar.gz   && rm dist.tar.gz
#18 CANCELED
------
 > [linux/amd64 build 4/4] RUN  if [ -z "false" ] || [ "false" = "false" ]; then        WITH_TESTS_ARG="--test-folder=";      else        WITH_TESTS_ARG="";      fi   && . /python/conan/bin/activate   && conan profile detect   && conan create cppagent        --build=missing        -c "tools.build:jobs=2"        -o agent_prefix=mtc        -o cpack=True        -o "with_ruby=True"        -o cpack_destination=/root/agent        -o cpack_name=dist        -o cpack_generator=TGZ        -pr "/root/agent/cppagent/conan/profiles/docker"        "argument"   && tar xf dist.tar.gz   && rm dist.tar.gz:
146.5 boost/1.85.0: WARN: b2 -q target-os=linux architecture=x86 address-model=64 binary-format=elf abi=sysv --layout=system --user-config=/root/.conan2/p/boost5fdc72773338c/s/src/tools/build/user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=off --disable-icu boost.locale.iconv=on boost.locale.iconv.lib=libc threading=multi visibility=hidden link=static variant=release --with-atomic --with-charconv --with-chrono --with-container --with-context --with-coroutine --with-date_time --with-exception --with-filesystem --with-iostreams --with-json --with-locale --with-log --with-program_options --with-random --with-regex --with-stacktrace --with-system --with-thread --with-timer --with-type_erasure --with-url toolset=gcc cxxstd=17 cxxstd-dialect=gnu define=_GLIBCXX_USE_CXX11_ABI=1 pch=on -sLIBBACKTRACE_PATH=/root/.conan2/p/b/libbab04427202a521/p linkflags="" cxxflags="-fPIC -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=/usr/bin/addr2line" install --prefix=/root/.conan2/p/b/boostdd77d3b1dab0b/p -j2 --abbreviate-paths -d0 --debug-configuration --build-dir="/root/.conan2/p/b/boostdd77d3b1dab0b/b/build-release"
146.5 boost/1.85.0: RUN: b2 -q target-os=linux architecture=x86 address-model=64 binary-format=elf abi=sysv --layout=system --user-config=/root/.conan2/p/boost5fdc72773338c/s/src/tools/build/user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=off --disable-icu boost.locale.iconv=on boost.locale.iconv.lib=libc threading=multi visibility=hidden link=static variant=release --with-atomic --with-charconv --with-chrono --with-container --with-context --with-coroutine --with-date_time --with-exception --with-filesystem --with-iostreams --with-json --with-locale --with-log --with-program_options --with-random --with-regex --with-stacktrace --with-system --with-thread --with-timer --with-type_erasure --with-url toolset=gcc cxxstd=17 cxxstd-dialect=gnu define=_GLIBCXX_USE_CXX11_ABI=1 pch=on -sLIBBACKTRACE_PATH=/root/.conan2/p/b/libbab04427202a521/p linkflags="" cxxflags="-fPIC -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=/usr/bin/addr2line" install --prefix=/root/.conan2/p/b/boostdd77d3b1dab0b/p -j2 --abbreviate-paths -d0 --debug-configuration --build-dir="/root/.conan2/p/b/boostdd77d3b1dab0b/b/build-release"
146.5 Assertion failed: destlen >= total (libgcompat/string.c: __strncat_chk: 251)
146.6 
146.6 boost/1.85.0: ERROR: 
146.6 Package '74482cd480d12051a8a314bff4f706866d4488a6' build failed
146.6 boost/1.85.0: WARN: Build folder /root/.conan2/p/b/boostdd77d3b1dab0b/b/build-release
146.6 ERROR: boost/1.85.0: Error in build() method, line 1130
146.6 	self.run(full_command)
146.6 	ConanException: Error -6 while executing
------

 1 warning found (use --debug to expand):
 - UndefinedVar: Usage of undefined variable '$HOME' (line 47)
Dockerfile:80
--------------------
  79 |     # Build and optionally test. Unpack the dist to reduce overhead in the following release step.
  80 | >>> RUN  if [ -z "$WITH_TESTS" ] || [ "$WITH_TESTS" = "false" ]; then \
  81 | >>>        WITH_TESTS_ARG="--test-folder="; \
  82 | >>>      else \
  83 | >>>        WITH_TESTS_ARG=""; \
  84 | >>>      fi \
  85 | >>>   && . /python/conan/bin/activate \
  86 | >>>   && conan profile detect \
  87 | >>>   && conan create cppagent \
  88 | >>>        --build=missing \
  89 | >>>        -c "tools.build:jobs=$CONAN_CPU_COUNT" \
  90 | >>>        -o agent_prefix=mtc \
  91 | >>>        -o cpack=True \
  92 | >>>        -o "with_ruby=$WITH_RUBY" \
  93 | >>>        -o cpack_destination=/root/agent \
  94 | >>>        -o cpack_name=dist \
  95 | >>>        -o cpack_generator=TGZ \
  96 | >>>        -pr "$CONAN_PROFILE" \
  97 | >>>        "${WITH_TESTS_ARG}" \
  98 | >>>   && tar xf dist.tar.gz \
  99 | >>>   && rm dist.tar.gz
 100 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c if [ -z \"$WITH_TESTS\" ] || [ \"$WITH_TESTS\" = \"false\" ]; then        WITH_TESTS_ARG=\"--test-folder=\";      else        WITH_TESTS_ARG=\"\";      fi   && . /python/conan/bin/activate   && conan profile detect   && conan create cppagent        --build=missing        -c \"tools.build:jobs=$CONAN_CPU_COUNT\"        -o agent_prefix=mtc        -o cpack=True        -o \"with_ruby=$WITH_RUBY\"        -o cpack_destination=/root/agent        -o cpack_name=dist        -o cpack_generator=TGZ        -pr \"$CONAN_PROFILE\"        \"${WITH_TESTS_ARG}\"   && tar xf dist.tar.gz   && rm dist.tar.gz" did not complete successfully: exit code: 1
Error: buildx failed with: ERROR: failed to solve: process "/bin/sh -c if [ -z \"$WITH_TESTS\" ] || [ \"$WITH_TESTS\" = \"false\" ]; then        WITH_TESTS_ARG=\"--test-folder=\";      else        WITH_TESTS_ARG=\"\";      fi   && . /python/conan/bin/activate   && conan profile detect   && conan create cppagent        --build=missing        -c \"tools.build:jobs=$CONAN_CPU_COUNT\"        -o agent_prefix=mtc        -o cpack=True        -o \"with_ruby=$WITH_RUBY\"        -o cpack_destination=/root/agent        -o cpack_name=dist        -o cpack_generator=TGZ        -pr \"$CONAN_PROFILE\"        \"${WITH_TESTS_ARG}\"   && tar xf dist.tar.gz   && rm dist.tar.gz" did not complete successfully: exit code: 1
@robot-ranger
Copy link
Contributor Author

Github Copliot tells me to check if this Boost version is compatible with this Conan version? 🤷

Any other ideas?

@wsobel
Copy link
Member

wsobel commented Jul 16, 2024 via email

@robot-ranger
Copy link
Contributor Author

something clearly changed

@wsobel
Copy link
Member

wsobel commented Jul 17, 2024

Try changing qemu versions or docket buildx. That's where I left off. Went through all the obvious settings like you, no luck. I think it is the cross compile environment that is killing the build. Can't think of anything else.

@robot-ranger
Copy link
Contributor Author

i must be reading my action's logs wrong; im reading that my runs are failing building linux/amd64?

@robot-ranger
Copy link
Contributor Author

robot-ranger commented Jul 17, 2024

yup - its the amd64 build that is failing, not arm64. Which is even weirder. cross-compile works, but native arch doesnt? 🤔 🤷

I removed linux/amd64 in my workflow file and it built successfully:

# build docker image and push to docker hub

# note: this won't work because the cross-platform build takes around 10h,
# but github limits us to 6h.
# so need to build it locally and push to docker hub.
# see Dockerfile for instructions.

# needs github secrets set for -
#   DOCKERHUB_USERNAME - ie mtconnect
#   DOCKERHUB_TOKEN - token for that account

# Set VERSION below before building.
# See CMakeLists.txt for the current version.
# Note: this will tag the image with that version, as well as 'latest'.

name: Build Docker image

env:
  PLATFORMS: linux/arm64

# when to run workflow
on:
  # # run on push or pull request events for the master branch
  push:
    tags:
      - "v*.*.*"

  # allows you to run this workflow manually from the actions tab
  workflow_dispatch:

# a workflow run has one or more jobs that can run sequentially or in parallel
jobs:
  # this workflow contains a single job called "build"
  build:
    # the type of runner that the job will run on
    runs-on: ubuntu-latest
    strategy:
      max-parallel: 1
      matrix:
        include:
          - dockerfile: docker/alpine/Dockerfile
            name: agent
#          - dockerfile: demo/Dockerfile
#            name: demo

    # steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # checks-out your repository under $GITHUB_WORKSPACE
      # see https://github.com/actions/checkout
      - name: Checkout Agent
        uses: actions/checkout@v3

      # the QEMU emulator lets us build for arm processors also
      # see https://github.com/docker/setup-qemu-action
      - name: Set up QEMU emulator
        uses: docker/setup-qemu-action@v3

      # see https://github.com/docker/setup-buildx-action
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to DockerHub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      
      - name: Remove leading v from the version
        run: |
          echo "VERSION=$(echo ${{ github.ref_name }} | sed 's/^v//')" >> $GITHUB_ENV
          echo "MAJOR_VERSION=$(echo ${{ github.ref_name }} | sed -E 's/^v([0-9]+)\.([0-9]+)\..+/\1.\2/')" >> $GITHUB_ENV
          
      # see https://github.com/docker/build-push-action
      - name: Build image and push to DockerHub
        uses: docker/build-push-action@v5
        with:
          context: .
          file: ${{ matrix.dockerfile }}
          platforms: ${{ env.PLATFORMS }}

          # docker hub user/repo:tag
          tags: |
            ${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.name }}:latest
            ${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.name }}:${{ env.VERSION }}
            ${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.name }}:${{ env.MAJOR_VERSION }}

          # push to docker hub
          push: true

image

@wsobel
Copy link
Member

wsobel commented Jul 17, 2024

I have the reverse because I'm building on a Mac M1. Arm builds Amd fails.

@wsobel
Copy link
Member

wsobel commented Jul 17, 2024

Not sure if we can build on GitHub using ARM architecture and then push using build instead of buildx.

@robot-ranger
Copy link
Contributor Author

Not sure if we can build on GitHub using ARM architecture and then push using build instead of buildx.

We looked into that recently, I don't believe github offers anything other than amd64 runners. You can host your own runner, which can be arm. However that all but defeats the point of using github actions cloud

@robot-ranger
Copy link
Contributor Author

Looks like public github-hosted arm runners are coming "this year" :

https://github.com/orgs/community/discussions/73040

@wsobel
Copy link
Member

wsobel commented Jul 17, 2024

I was wondering if the MacOS runner is ARM based.

@wsobel
Copy link
Member

wsobel commented Jul 17, 2024

Looks like they entered private beta in Jan. Maybe the my are now available.

@robot-ranger
Copy link
Contributor Author

we are building both amd and arm, into alpine images, with qemu on github for our other projects, and they build fine. so this is something to do with conan/boost build process in qemu

@wsobel
Copy link
Member

wsobel commented Jul 17, 2024

Or a more complex build.

@wsobel wsobel added the bug label Jul 29, 2024
@robot-ranger
Copy link
Contributor Author

robot-ranger commented Nov 26, 2024

For now, I am building the alpine image for arm64 in my fork since it appears the amd64 is what is failing, arm64 seems to build fine.

if anyone wants/needs alpine image sizes for arm64, you can find it here: https://hub.docker.com/repository/docker/robotranger/agent/general

@wsobel
Copy link
Member

wsobel commented Nov 26, 2024

Cross compiling in docker is broken for the moment in alpine. I have tried to work around it and have had no success. If you find a solution, please post it.

@robot-ranger
Copy link
Contributor Author

The crazy thing is the cross compiling is the part that is working!

ARM64 builds fine.

AMD64 is failing.

Just so I don't duplicate work, have we tried building AMD WITHOUT qemu?

@wsobel
Copy link
Member

wsobel commented Nov 26, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants