CI: Disable all jobs for macOS and Windows #14377
Conversation
github-actions
bot
added
Area: CI
Size: S
|
[Experimental Bot, please feedback here] Yes, this PR seems to meet the NuttX requirements. Here's a breakdown:
Overall, the PR provides sufficient information for reviewers to understand the change, its impact, and the testing conducted. |
|
@lupyuen can we reduce the board on Linux host to keep macOS/Windows? it's very easy to break these host if without these basic coverage. |
|
@xiaoxiang781216 I suggest that we monitor the GitHub Cost after disabling macOS and Windows Jobs. It's possible that macOS and Windows Jobs are contributing a huge part of the cost. We could re-enable and simplify them after monitoring. @jerpelea I hope you are agreeable to this. |
why don't you disable all tests, including linux? :-) i feel macOS/Windows jobs are more important to have on the CI than linux ones my impression is we simply build too many configs, regardless of platforms. |
When we submit or update a Complex PR that affects All Architectures (Arm, RISC-V, Xtensa, etc): CI Workflow shall run only half the jobs. Previously CI Workflow will run `arm-01` to `arm-14`, now we will run only `arm-01` to `arm-07`. When the Complex PR is Merged: CI Workflow will still run all jobs `arm-01` to `arm-14` Simple PRs with One Single Arch / Board will build the same way as before: `arm-01` to `arm-14` This is explained here: apache#14376 Note that this version of `arch.yml` has diverged from `nuttx-apps`, since we are unable to merge apache#14377
When we submit or update a Complex PR that affects All Architectures (Arm, RISC-V, Xtensa, etc): CI Workflow shall run only half the jobs. Previously CI Workflow will run `arm-01` to `arm-14`, now we will run only `arm-01` to `arm-07`. When the Complex PR is Merged: CI Workflow will still run all jobs `arm-01` to `arm-14` Simple PRs with One Single Arch / Board will build the same way as before: `arm-01` to `arm-14` This is explained here: apache/nuttx#14376 Note that this version of `arch.yml` has diverged from `nuttx` repo, since we are unable to merge apache/nuttx#14377
|
Do we know how much actually Windoze and macOS cost per month? Maybe we can disable macOS for not just to save the CI? macOS is Unix so close to Linux builds but 10x the price. Windoze is another story and true not many people use / test it so would be good to keep the test builds? Someone from Apache would be good to have in all this conversations.. |
|
@cederom The official pricing is here: https://docs.github.com/en/billing/managing-billing-for-your-products/managing-billing-for-github-actions/about-billing-for-github-actions#minute-multipliers macOS is 10x more expensive than Linux. There's actually a high possibility that we are overrunning our GitHub Costs mainly due to macOS. And ASF might have ignored this, because no other ASF Project is using macOS like us. |
|
I have a suggestion: I have a super powerful Mac Mini at home that's underutilised, I would be happy to run NuttX CI once every day. There's a problem though: The macOS CI Scripts were created for the obsolete Intel Macs, they don't run so well on my M2 Pro Mac Mini. The scripts need to be updated. (Also it sounds very odd: Why are we running NuttX CI on an obsolete Intel Mac hosted at GitHub?) |
|
Yes I have also asked Apache Infra do they know how we can introduce external distributed build farms to the testing process. Maybe we don't need to reinvent the wheel just use some existing project that could allow us to test build/run on various boards that we have at hand at home/work/lab :-) |
|
I believe Self-Hosted GitHub Runners is their preferred solution: https://cwiki.apache.org/confluence/display/INFRA/GitHub+self-hosted+runners But like the article says: Server Security will require special expertise to setup and maintain. |
When we submit or update a Complex PR that affects All Architectures (Arm, RISC-V, Xtensa, etc): CI Workflow shall run only half the jobs. Previously CI Workflow will run `arm-01` to `arm-14`, now we will run only `arm-01` to `arm-07`. When the Complex PR is Merged: CI Workflow will still run all jobs `arm-01` to `arm-14` Simple PRs with One Single Arch / Board will build the same way as before: `arm-01` to `arm-14` This is explained here: apache/nuttx#14376 Note that this version of `arch.yml` has diverged from `nuttx` repo, since we are unable to merge apache/nuttx#14377
When we submit or update a Complex PR that affects All Architectures (Arm, RISC-V, Xtensa, etc): CI Workflow shall run only half the jobs. Previously CI Workflow will run `arm-01` to `arm-14`, now we will run only `arm-01` to `arm-07`. When the Complex PR is Merged: CI Workflow will still run all jobs `arm-01` to `arm-14` Simple PRs with One Single Arch / Board will build the same way as before: `arm-01` to `arm-14` This is explained here: #14376 Note that this version of `arch.yml` has diverged from `nuttx-apps`, since we are unable to merge #14377
This PR disables all CI Jobs for macOS and Windows, to reduce GitHub Cost. Details here: apache#14376
lupyuen
force-pushed
the
disable-macos-windows
branch
from
c632e6f
to
6add166
Compare
|
I have just rebased to |
intel mac is not completely obsolete. apple still supports them. (and i'm using them. :-) also, some of tested code are host arch dependent. (eg. low-level logic in sim) |
|
Refurbished Intel Mac Mini costs USD 340, maybe I should buy one and run it at home for CI Tests? I'm now running CI Tests on a refurbished Ubuntu PC, seems OK so far. |
|
The best build HW for macOS is MacStudio, but not the cheapest one, more expensive than MacMini, cheaper than MacBookPro, more powerful than both of them.. 4x faster than my PC but still 5x more expensive :D I was forced to buy MacStudio to get latest macOS and XCode as my old Intel based MacBookPro got no more updates. But Open-Source builds also work on that old MacBookPro with no problem :-) Another solution may be Hackintosh ;-) |
Summary
This PR disables all CI Jobs for macOS and Windows, to reduce GitHub Cost. Details here: #14376
Impact
CI Jobs for macOS and Windows will no longer run, until we find a way to manage their costs.
Testing
We tested the skipping of macOS and Windows jobs: https://github.com/lupyuen5/label-nuttx/actions/runs/11379543159