Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(l1, l2, levm) #1433

Merged
merged 40 commits into from
Dec 6, 2024
Merged

fix(l1, l2, levm) #1433

merged 40 commits into from
Dec 6, 2024

Conversation

lima-limon-inc
Copy link
Contributor

@lima-limon-inc lima-limon-inc commented Dec 6, 2024

Motivation
Fix the loc test done in daily_reports.yaml. ATTOW it does not include line changes.

Description
The loc directive should send a message to the slack channel showing lines added and removed. This implied changes to the CI.

No issue for this PR.

@lima-limon-inc lima-limon-inc changed the title Ci diffs fix(l1, l2, levm) Dec 6, 2024
Bash tries to execute the content in between them; and we are simply
trying to print
@lima-limon-inc lima-limon-inc marked this pull request as ready for review December 6, 2024 20:04
@lima-limon-inc lima-limon-inc requested a review from a team as a code owner December 6, 2024 20:04
@@ -157,7 +158,6 @@ jobs:
- name: Post results to levm slack channel
env:
url: ${{ secrets.LEVM_SLACK_WEBHOOK }}
run: sh .github/scripts/publish_loc.sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we removing this line? isn't it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! That's why I was getting an error. I deleted the line by mistake.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed here: ef7bd71

Comment on lines +127 to +130
- name: Check previous loc report
run: |
cat loc_report.json

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this directive here to read the loc_report.json that was saved in the cache.

@ilitteri ilitteri added this pull request to the merge queue Dec 6, 2024
Merged via the queue into main with commit 07d342a Dec 6, 2024
13 checks passed
@ilitteri ilitteri deleted the ci-diffs branch December 6, 2024 22:16
@lima-limon-inc
Copy link
Contributor Author

lima-limon-inc commented Dec 7, 2024

@ilitteri I think I found the problem. I think the problem is in this line: https://github.com/lambdaclass/ethrex/blob/main/.github/workflows/daily_reports.yaml#L132

We are saving the log_report.json file on a cache entry that has as key loc-report-${{ github.ref_name }}.
github.ref_name is the name of a branch not a commit hash. That name is not "main" but rather the name of the PR it came from. In this case ci-diffs.

What's the problem with that?
If the branch names don't match EXACTLY, the cache fails. HOWEVER, we have a restore-keys as a backup (here: https://github.com/lambdaclass/ethrex/blob/main/.github/workflows/daily_reports.yaml#L124-L125).

restor-keys is a backup that acts like a sort of regex. It tells github-cache "Hey, seen as you failed fetching that EXACT key, give me, as a backup, a value that matches the following prefix". That prefix should match everytime, since every key we generate uses that exact same prefix.

So what's the problem? The problem is that we are only changing the file name IF there is a cache hit aka this if: https://github.com/lambdaclass/ethrex/blob/main/.github/workflows/daily_reports.yaml#L132. So, once again, what's the problem? We are getting a cache hit everytime, we either get it with the exact key or using the restore-keys as a fallback.

And there is the problem, if the value is fetched using restore-keys, it doesn't count as a cache hit.
This behavior is described here: https://github.com/actions/cache/blob/main/README.md?plain=1#L129.

That would also explain why the script worked while I was working on this PR: I was always on the same branch, so it was always a cache hit, it was never using restore-keys. However, when the cron job is executed, it probably fails every time and ends up using the backup.

That is also strange, one would think that it would end up using "main" branch as a key name. My hunch is that it uses commits coming from PR's, so it's maybe using those PR branch names for keys.

I'm gonna create a new PR for it to be merged on Tuesday, so we can see the difference in output and check that this indeed is the issue.

@mpaulucci
Copy link
Collaborator

@lima-limon-inc please in the future, add a more descriptive title.

@lima-limon-inc
Copy link
Contributor Author

@lima-limon-inc please in the future, add a more descriptive title.

Noted. I apologize @mpaulucci for the inconvenience.

github-merge-queue bot pushed a commit that referenced this pull request Dec 12, 2024
…en when there a `restore-keys` is used. (#1438)

Even when there is no cache-hit, the file is fetched. If it came from
the `restore-keys` directive, it doesn't count as a cache-hit even if
the file is fetched.

**Motivation**
Show LOC diffs in the different slack channels.
<!-- Why does this pull request exist? What are its goals? -->

**Description**
This description was taken from this comment:
#1433 (comment)

<!-- A clear and concise general description of the changes this PR
introduces -->
We are saving the log_report.json file on a cache entry that has as key
loc-report-${{ github.ref_name }}.
github.ref_name is the name of a branch not a commit hash. That name is
not "main" but rather the name of the PR it came from. In this case
ci-diffs.

What's the problem with that?
If the branch names don't match EXACTLY, the cache fails. HOWEVER, we
have a restore-keys as a backup (here:
https://github.com/lambdaclass/ethrex/blob/main/.github/workflows/daily_reports.yaml#L124-L125).

restor-keys is a backup that acts like a sort of regex. It tells
github-cache "Hey, seen as you failed fetching that EXACT key, give me,
as a backup, a value that matches the following prefix". That prefix
should match everytime, since every key we generate uses that exact same
prefix.

So what's the problem? The problem is that we are only changing the file
name IF there is a cache hit aka this if:
https://github.com/lambdaclass/ethrex/blob/main/.github/workflows/daily_reports.yaml#L132.
So, once again, what's the problem? We are getting a cache hit
everytime, we either get it with the exact key or using the restore-keys
as a fallback.

And there is the problem, if the value is fetched using restore-keys, it
doesn't count as a cache hit.
This behavior is described here:
https://github.com/actions/cache/blob/main/README.md?plain=1#L129.

That would also explain why the script worked while I was working on
this PR: I was always on the same branch, so it was always a cache hit,
it was never using restore-keys. However, when the cron job is executed,
it probably fails every time and ends up using the backup.

That is also strange, one would think that it would end up using "main"
branch as a key name. My hunch is that it uses commits coming from PR's,
so it's maybe using those PR branch names for keys.

I'm gonna create a new PR for it to be merged on Tuesday, so we can see
the difference in output and check that this indeed is the issue.
<!-- Link to issues: Resolves #111, Resolves #222 -->

No associated issue number
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants