ci: use official action to setup rust toolchain #585

roeap · 2024-12-10T15:03:47Z

What changes are proposed in this pull request?

The actions-rs/toolchain action is deprecated in favor of actions-rust-lang/setup-rust-toolchain. This PR updates the usages of the respective actions in the github workflows. THe new action already includes an integration with the rust-cache action, so no need to set that up separately anymore.

This also sets up a dependabot configuration for cargo and github-actions which we may or may not choose to keep.

How was this change tested?

no code changes.

kernel/src/engine/arrow_data.rs

codecov · 2024-12-10T15:09:55Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.48%. Comparing base (702e12f) to head (511a1e2).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #585   +/-   ##
=======================================
  Coverage   83.48%   83.48%           
=======================================
  Files          74       74           
  Lines       16884    16884           
  Branches    16884    16884           
=======================================
  Hits        14096    14096           
  Misses       2131     2131           
  Partials      657      657

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

zachschuermann

nice thanks! changes to existing CI LGTM, just a question on adding dependabot and also overall question: it looks like the two most popular rust toolchains in CI are dtolnay/rust-toolchain and actions-rust-lang/setup-rust-toolchain with the latter saying that it was heavily inspired by dtolnay's? I guess this has caching built-in as the main benefit?

kernel/src/engine/arrow_data.rs

zachschuermann · 2024-12-10T16:44:31Z

.github/dependabot.yaml

oh nice! are we planning on enabling dependabot for the repo?

well - i guess thats up for debate. but since this was literally just copy and paste from somewhere else, it was low cost to spark the discussion by just providing the solution 😆.

From my end, I think the utility may unfortunately be limited as auto-updates for the major dependencies (e.g. arrow) almost never work out of the box - at least in delta-rs. That said, I still landed on "let's add it" to keep the actions fresh and have the occasional rust version bump that works.

yea I think it is helpful insofar as it just automatically opens small PRs to bump versions and we may (probably) have to manually help upgrade but the 'reminder' to upgrade seems useful? curious to hear what others think

the dependabot-quickstart-guide doesn't recommend setting up manually like this, but rather suggests going into settings and letting github create this file. Wondering if we should just do it that way?

I'm okay trying this out, but also a bit skeptical of the utility here. As far as I can tell it only runs cargo update. Generally with how we've set up our Cargo.toml this will only do minor updates (i.e. it'll never update arrow 53->54). Maybe that's good, but we need a process to also run cargo outdated and try and stay on top of more major updates as well, so I'm not sure how much time this saves us.

zachschuermann · 2024-12-10T16:55:34Z

just want to make sure: is the caching working? I see the following errors in test (ubuntu-latest) in the post-run step:

Error: ENOENT: no such file or directory, opendir '/home/runner/work/delta-kernel-rs/delta-kernel-rs/target/tests/target'
Error: Error: ENOENT: no such file or directory, opendir '/home/runner/work/delta-kernel-rs/delta-kernel-rs/target/tests/target'
... Cleaning cargo registry (cache-all-crates: false) ...
... Cleaning cargo/bin ...
... Cleaning cargo git cache ...
... Saving cache ...
/usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/delta-kernel-rs/delta-kernel-rs --files-from manifest.txt --use-compress-program zstdmt
Failed to save: Unable to reserve cache with key v0-rust-test-64e5ee6e-0ce599cb, another job may be creating this cache. More details: Cache already exists. Scope: refs/pull/585/merge, Key: v0-rust-test-64e5ee6e-0ce599cb, Version: 0d[39](https://github.com/delta-io/delta-kernel-rs/actions/runs/12258931755/job/34199879152#step:7:40)aea52349942f5791850f64e514ac9b6831c456410ea5407f320467a8948d

nicklan

looks good for the runner changes. not as sure about dependabot :)

nicklan · 2024-12-11T01:43:18Z

.github/workflows/build.yml

@@ -109,7 +92,7 @@ jobs:
      - name: Setup cmake
        uses: jwlawson/actions-setup-cmake@v2
        with:
-          cmake-version: '3.30.x'
+          cmake-version: "3.30.x"


In yaml double quotes allow escapes and other things, so I tend to prefer single quotes unless I need that, as, fewer surprises. Any reason you changed this?

auto-formatting ... maybe i need to make prettier or whoever it was less agressive :).

nicklan · 2024-12-11T01:44:58Z

.github/dependabot.yaml

@@ -0,0 +1,14 @@
+# Please see the documentation for all configuration options:
+# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates


This link doesn't seem to work. it just goes to a generic support page

nicklan · 2024-12-11T01:47:25Z

.github/dependabot.yaml

the dependabot-quickstart-guide doesn't recommend setting up manually like this, but rather suggests going into settings and letting github create this file. Wondering if we should just do it that way?

nicklan · 2024-12-11T01:52:28Z

.github/dependabot.yaml

I'm okay trying this out, but also a bit skeptical of the utility here. As far as I can tell it only runs cargo update. Generally with how we've set up our Cargo.toml this will only do minor updates (i.e. it'll never update arrow 53->54). Maybe that's good, but we need a process to also run cargo outdated and try and stay on top of more major updates as well, so I'm not sure how much time this saves us.

zachschuermann

LGTM long as all the caching still works :)

zachschuermann · 2024-12-17T21:56:37Z

.github/workflows/build.yml

@@ -10,27 +10,20 @@ jobs:
  format:
    runs-on: ubuntu-latest
    steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
      - name: Install minimal stable with clippy and rustfmt


nit

Suggested change

- name: Install minimal stable with clippy and rustfmt

- name: Install minimal stable with rustfmt

zachschuermann · 2024-12-17T21:57:04Z

.github/workflows/build.yml

@@ -65,18 +55,16 @@ jobs:
          - ubuntu-latest
          - windows-latest
    steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
      - name: Install minimal stable with clippy and rustfmt


nit

Suggested change

- name: Install minimal stable with clippy and rustfmt

- name: Install minimal stable with clippy

zachschuermann · 2024-12-17T21:57:30Z

.github/workflows/build.yml

@@ -109,7 +92,7 @@ jobs:
      - name: Setup cmake
        uses: jwlawson/actions-setup-cmake@v2
        with:
-          cmake-version: '3.30.x'
+          cmake-version: "3.30.x"


Suggested change

cmake-version: "3.30.x"

cmake-version: '3.30.x'

github-actions bot assigned roeap Dec 10, 2024

ci: use official action to setup toolchain

f334d14

roeap force-pushed the ci/toolchain-action branch from b779455 to f334d14 Compare December 10, 2024 15:05

roeap commented Dec 10, 2024

View reviewed changes

kernel/src/engine/arrow_data.rs Outdated Show resolved Hide resolved

roeap requested a review from zachschuermann December 10, 2024 15:09

zachschuermann reviewed Dec 10, 2024

View reviewed changes

zachschuermann requested review from nicklan, scovich and OussamaSaoudi-db December 10, 2024 16:55

chore: revert formatting changes

f9cc2a1

nicklan requested changes Dec 11, 2024

View reviewed changes

roeap and others added 3 commits December 11, 2024 07:49

fix: remove dependabot

08e630c

Merge branch 'main' into ci/toolchain-action

76ebf99

Merge branch 'main' into ci/toolchain-action

0ee5257

zachschuermann requested review from nicklan and zachschuermann December 11, 2024 20:55

Merge branch 'main' into ci/toolchain-action

511a1e2

zachschuermann approved these changes Dec 17, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: use official action to setup rust toolchain #585

ci: use official action to setup rust toolchain #585

roeap commented Dec 10, 2024

codecov bot commented Dec 10, 2024 •

edited

Loading

zachschuermann left a comment

zachschuermann Dec 10, 2024

roeap Dec 10, 2024

zachschuermann Dec 10, 2024

nicklan Dec 11, 2024

nicklan Dec 11, 2024

zachschuermann commented Dec 10, 2024

nicklan left a comment

nicklan Dec 11, 2024

roeap Dec 11, 2024

nicklan Dec 11, 2024

nicklan Dec 11, 2024

nicklan Dec 11, 2024

zachschuermann left a comment

zachschuermann Dec 17, 2024

zachschuermann Dec 17, 2024

zachschuermann Dec 17, 2024

		@@ -0,0 +1,14 @@
		# Please see the documentation for all configuration options:
		# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates

	- name: Install minimal stable with clippy and rustfmt
	- name: Install minimal stable with rustfmt

ci: use official action to setup rust toolchain #585

Are you sure you want to change the base?

ci: use official action to setup rust toolchain #585

Conversation

roeap commented Dec 10, 2024

What changes are proposed in this pull request?

How was this change tested?

codecov bot commented Dec 10, 2024 • edited Loading

Codecov Report

zachschuermann left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zachschuermann commented Dec 10, 2024

nicklan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zachschuermann left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Dec 10, 2024 •

edited

Loading