Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Docker aliases #493

Merged
merged 1 commit into from
Jan 28, 2025
Merged

Conversation

stanhu
Copy link
Contributor

@stanhu stanhu commented Jan 16, 2025

rake-compiler-dock now makes a clear distinction between GNU and Musl builds:
https://github.com/rake-compiler/rake-compiler-dock?tab=readme-ov-file#linux-gnu-and-musl-important-details

This change makes it possible to build images using the explicit aarch64-linux-gnu and x86_64-linux-gnu targets needed by rake-compiler-dock.

@stanhu stanhu marked this pull request as draft January 16, 2025 18:49
@stanhu stanhu force-pushed the sh-alias-gnu-builds branch from 86a6068 to 7b8dfb3 Compare January 16, 2025 20:50
- ruby_target: x64-mingw32
- ruby_target: x64-mingw-ucrt
# - ruby_target: x86-linux
toolchain: ${{ fromJSON(needs.fetch_ci_data.outputs.toolchains-data) }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will include x86-linux again; do we want to omit it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it builds, i'm ok with it! there was an issue before breaking builds :/

@stanhu stanhu force-pushed the sh-alias-gnu-builds branch 4 times, most recently from 68ba67e to 1fd942d Compare January 16, 2025 22:18
@stanhu stanhu marked this pull request as ready for review January 16, 2025 22:21
@stanhu
Copy link
Contributor Author

stanhu commented Jan 16, 2025

I think this is working, though https://github.com/oxidize-rb/rb-sys/actions/runs/12818599166/job/35744460713?pr=494 is failing because the tag has a leading /. I'm not sure why; that probably is an artifact of my test build. It seems in the working builds, I see***/ instead.

@stanhu stanhu requested a review from ianks January 16, 2025 22:22
@stanhu stanhu marked this pull request as draft January 16, 2025 22:44
@stanhu stanhu force-pushed the sh-alias-gnu-builds branch 2 times, most recently from 51be4be to ddbeb51 Compare January 16, 2025 22:53
@stanhu stanhu marked this pull request as ready for review January 16, 2025 22:54
@stanhu stanhu force-pushed the sh-alias-gnu-builds branch from ddbeb51 to 8b61115 Compare January 16, 2025 22:55
@stanhu
Copy link
Contributor Author

stanhu commented Jan 16, 2025

In https://github.com/oxidize-rb/rb-sys/actions/runs/12819077576/job/35745982383?pr=494, you can see the aarch64-linux aliases:

Run docker/build-push-action@v6
  with:
    builder: builder-1503a5e7-484e-44a4-9b6d-97db6304e621
    context: ./docker
    file: ./docker/Dockerfile.aarch64-linux
    platforms: linux/amd64
    load: true
    tags: /aarch64-linux:pr-494
  /aarch64-linux:sha-7a3717b98d71408bd827408b535d76240a36ac62
  /aarch64-linux-gnu:pr-494
  /aarch64-linux-gnu:sha-7a3717b98d71408bd827408b535d76240a36ac62
  /aarch64-unknown-linux-gnu:pr-494
  /aarch64-unknown-linux-gnu:sha-7a3717b98d71408bd827408b535d76240a36ac62

In https://github.com/oxidize-rb/rb-sys/actions/runs/12819077576/job/35745984135?pr=494, you can see the x86_64-linux aliases:

Run docker/build-push-action@v6
  with:
    builder: builder-50dab0ec-b470-43c1-b718-5e0a2ad651c0
    context: ./docker
    file: ./docker/Dockerfile.x86_64-linux
    platforms: linux/amd64
    load: true
    tags: /x86_64-linux:pr-494
  /x86_64-linux:sha-7a3717b98d71408bd827408b535d76240a36ac62
  /x86_64-linux-gnu:pr-494
  /x86_64-linux-gnu:sha-7a3717b98d71408bd827408b535d76240a36ac62
  /x86_64-unknown-linux-gnu:pr-494
  /x86_64-unknown-linux-gnu:sha-7a3717b98d71408bd827408b535d76240a36ac62

@stanhu
Copy link
Contributor Author

stanhu commented Jan 22, 2025

@ianks Friendly ping. Could you review this?

rake-compiler-dock now makes a clear distinction between GNU and Musl
builds:
https://github.com/rake-compiler/rake-compiler-dock?tab=readme-ov-file#linux-gnu-and-musl-important-details

This change makes it possible to build images using the explicit
aarch64-linux-gnu and x86_64-linux-gnu targets needed by
rake-compiler-dock.
@stanhu stanhu force-pushed the sh-alias-gnu-builds branch from dcebf6e to b7b7d6c Compare January 27, 2025 21:04
@stanhu
Copy link
Contributor Author

stanhu commented Jan 28, 2025

@ianks I think this pull request is needed to unblock a few Rust extensions that use the latest version of rack-compiler-dock. Would you mind reviewing?

@ianks
Copy link
Collaborator

ianks commented Jan 28, 2025

Thank you! I will but a release after docker builds pass

@ianks ianks merged commit b5aa9a6 into oxidize-rb:main Jan 28, 2025
56 checks passed
@stanhu
Copy link
Contributor Author

stanhu commented Jan 28, 2025

I think we have a bit of a chicken and egg issue here. https://github.com/oxidize-rb/rb-sys/actions/runs/13020345559/job/36319402338 is failing because bundle install is installing rb-sys v0.9.108 instead of the latest that contains https://github.com/oxidize-rb/rb-sys/pull/496/files?

UPDATE: No, something else is not right...

@ianks
Copy link
Collaborator

ianks commented Jan 28, 2025

https://github.com/oxidize-rb/rb-sys/actions/runs/13020479341/job/36319817845 is the most recent one, not sure what's wrong :/

@stanhu
Copy link
Contributor Author

stanhu commented Jan 28, 2025

I think the issue is that oxi-test doesn't depend on rack_compiler_dock, so when it tries to run rb-sys-dock it's not there.

I think we can fix this by making rack_compiler_dock a dependency in gem/rb_sys.gemspec? But maybe we can avoid that by fixing the test to run bundle install from the rb-sys dir.

@stanhu
Copy link
Contributor Author

stanhu commented Jan 28, 2025

I submitted #499.

@ianks
Copy link
Collaborator

ianks commented Jan 28, 2025

great minds think alike :)

0891b17

@stanhu
Copy link
Contributor Author

stanhu commented Jan 28, 2025

In https://github.com/oxidize-rb/rb-sys/actions/runs/13020479341/job/36319819895, I see this failure:

  Installing rake-compiler-dock 1.9.1
  Installing rake-compiler 1.2.9
  Installing minitest 5.25.4
  Bundle complete! 5 Gemfile dependencies, 7 gems now installed.
  Bundled gems are installed into `/tmp/rb-sys-dock/bundle`
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-3.4)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-3.3)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-3.2)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-3.1)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-3.0)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-2.7)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-2.6)
  rake aborted!
  Don't know how to build task 'native:x86_64-linux' (See the list of available tasks with `rake --tasks`)
  /tmp/rb-sys-dock/bundle/ruby/3.4.0/gems/rake-13.2.1/exe/rake:27:in '<top (required)>'
  /usr/local/rbenv/versions/3.4.1/bin/bundle:25:in 'Kernel#load'
  /usr/local/rbenv/versions/3.4.1/bin/bundle:25:in '<main>'
  (See full trace by running task with --trace)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-3.4)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-3.3)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-3.2)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-3.1)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-3.0)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-2.7)
  no configuration section for specified version of Ruby (rbconfig-x86_64-linux-2.6)
  rake aborted!
  Don't know how to build task 'native:x86_64-linux' (See the list of available tasks with `rake --tasks`)
  

The main difference between that build and the working one in https://github.com/oxidize-rb/rb-sys/actions/runs/12814269578/job/35730294848 is the inclusion of rake-compiler-dock:

  Installing rake-compiler 1.2.9
  Installing minitest 5.25.4
  Bundle complete! 5 Gemfile dependencies, 6 gems now installed.
  Bundled gems are installed into `/tmp/rb-sys-dock/bundle`
  /usr/local/rbenv/versions/3.1.6/bin/ruby -I. ../../../../ext/extconf.rb
  mkdir -p lib/oxi/test/3.4
  mkdir -p tmp/x86_64-linux/oxi-test/3.4.1
  cd tmp/x86_64-linux/oxi-test/3.4.1
  checking for x86_64-linux-gnu-gcc... yes
  checking for x86_64-linux-gnu-g++... yes
  checking for x86_64-linux-gnu-gcc-ar... yes
  checking for install_name_tool... no
  /usr/bin/make
  cd -
  cd tmp/x86_64-linux/oxi-test/3.4.1

I've seen the error above when the Ruby versions weren't right for the image.

@ianks
Copy link
Collaborator

ianks commented Jan 28, 2025

We may need to lock versions to rake-compiler dock now that i think of it

@stanhu
Copy link
Contributor Author

stanhu commented Jan 28, 2025

It looks like ghcr.io/rake-compiler/rake-compiler-dock-image:1.9.1-mri-x86-linux is also Ubuntu 20.04, so we need to get rid of the CentOS 8-specific commands in the Dockerfile.

@stanhu
Copy link
Contributor Author

stanhu commented Jan 28, 2025

I've reproduced the current issue. I think the issue is the full Ruby patch release version has to be used, but they're not be specified right now for some reason. RUBY_CC_VERSION isn't set?

@stanhu
Copy link
Contributor Author

stanhu commented Jan 28, 2025

@ianks #501 and #502 should help here.

stanhu added a commit to stanhu/rb-sys that referenced this pull request Jan 29, 2025
`matrix.platform.ruby_target` should now be `env.ruby_platform`. These
were missed in oxidize-rb#493.
@stanhu stanhu mentioned this pull request Jan 29, 2025
ianks pushed a commit that referenced this pull request Jan 29, 2025
`matrix.platform.ruby_target` should now be `env.ruby_platform`. These
were missed in #493.
stanhu added a commit to stanhu/rb-sys that referenced this pull request Jan 29, 2025
oxidize-rb#364 upgraded the x86-mingw32
LLVM version to v17.0.6
(https://github.com/mstorsjo/llvm-mingw/releases/tag/20231128) but the
version and include directories were not incremented. Previously the
x86-mingw32 builds never ran, so this issue was not found until
oxidize-rb#493.
@stanhu stanhu mentioned this pull request Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants