Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ci runners #169

Merged
merged 63 commits into from
Jun 18, 2024
Merged
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
d81c2b1
Merge in patch branch and re-target develop
brtietz Dec 13, 2023
65866a1
Merge pull request #168 from NREL/patch_to_develop_w_ci
brtietz Dec 13, 2023
932a6ad
add windows build
dguittet May 15, 2024
4fb7abe
windows ci
dguittet May 15, 2024
3093659
windows
dguittet May 15, 2024
dc311ac
windows
dguittet May 15, 2024
93c3c1e
fix paths
dguittet May 15, 2024
391fac8
windows lk
dguittet May 15, 2024
f831e2c
debug
dguittet May 15, 2024
a029e32
check python
dguittet May 17, 2024
d60c696
python version
dguittet May 17, 2024
7e10a3f
try downloading wx
dguittet May 20, 2024
b6a2f01
update linux
dguittet May 20, 2024
861e0a9
linux
dguittet May 20, 2024
052204a
fix syntax
dguittet May 20, 2024
bdc6259
download wx
dguittet May 20, 2024
2c6af7b
add secrets
dguittet May 20, 2024
9a91be9
tokens
dguittet May 20, 2024
e756be2
download_wx
dguittet May 20, 2024
e92072b
download_wx
dguittet May 20, 2024
7013dd2
token
dguittet May 20, 2024
800b490
msg
dguittet May 20, 2024
d0323a9
fix
dguittet May 20, 2024
7327890
cache build off
dguittet May 20, 2024
bbbeebb
windows
dguittet May 20, 2024
e59477d
add mac build
dguittet May 20, 2024
602ff8a
fix dirs
dguittet May 20, 2024
e4dd805
fix mac
dguittet May 20, 2024
6e80b22
pip on mac
dguittet May 20, 2024
c143994
pip
dguittet May 20, 2024
49c9ff4
mac
dguittet May 20, 2024
6030324
fix git ls-remote
dguittet May 20, 2024
b5535f2
fix mac
dguittet May 20, 2024
09e779a
mac
dguittet May 20, 2024
c1e0539
update all
dguittet May 20, 2024
6014fd5
windows secrets
dguittet May 20, 2024
43e22a6
fix paths
dguittet May 21, 2024
d83cf3d
add error message
dguittet May 21, 2024
29a8ab5
build lk always
dguittet May 21, 2024
117b10e
fix paths
dguittet May 21, 2024
a4662ec
fix paths
dguittet May 21, 2024
8f77c5c
fix paths
dguittet May 21, 2024
bbf0b67
windows
dguittet May 21, 2024
5fbda4f
move windows download
dguittet May 21, 2024
14d78f0
clean up and fix artifacts
dguittet May 21, 2024
a03f282
use caches
dguittet May 21, 2024
d8c7cab
windows cache
dguittet May 21, 2024
aeee722
fix caching
dguittet May 21, 2024
3e50ebd
fix paths for artifacts
dguittet May 21, 2024
a0b67aa
fix caching path
dguittet May 21, 2024
632559f
fix caching
dguittet May 21, 2024
cb63a70
get branch name
dguittet May 21, 2024
de98735
cleanup
dguittet May 21, 2024
11d729b
fix ssc cache
dguittet May 21, 2024
37d9a7c
fix windows
dguittet May 21, 2024
4dbb1b5
use caching for wx 3.2.4
dguittet May 22, 2024
01bb928
release mode
dguittet May 22, 2024
1ddbe87
release ver
dguittet May 22, 2024
7c09402
fix windows path
dguittet May 22, 2024
16d26c4
remove download_wx.py
dguittet May 23, 2024
4382f36
add default branch
dguittet May 23, 2024
db71c0a
debug vs rel for MSVC
dguittet May 28, 2024
6502153
select branch based on push or pull_request
dguittet Jun 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
325 changes: 288 additions & 37 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,30 @@ on:
push:
pull_request:

env:
WX_VERSION: '3.2.4'

jobs:
build-on-ubuntu:
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ secrets.TOKEN_GITHUB }}

steps:
- name: Setup cmake
uses: jwlawson/actions-setup-cmake@v1.12
uses: jwlawson/actions-setup-cmake@v2
with:
cmake-version: '3.24.x'
- name: Test cmake version
run: cmake --version
- name: Checkout
uses: actions/checkout@v4
with:
path: wex

- name: Set relative paths
run: |
WXMSW3=$HOME/wx-$WX_VERSION
echo "WXMSW3=$WXMSW3" >> $GITHUB_ENV

- name: Install OS dependencies
run: |
sudo apt-get update --fix-missing
Expand All @@ -27,30 +41,53 @@ jobs:
libgtk2.0-dev \
mesa-common-dev \
unzip
- name: Get GCC version
run: gcc --version
- name: Get libc version
run: ldd --version

- name: Get cached build of wxWidgets
uses: actions/cache@v4
id: cachedwx
with:
path: ${{env.WXMSW3}}/
key: wxWidgets-${{ env.WX_VERSION }}-linux

- name: Install wxWidgets
if: steps.cachedwx.outputs.cache-hit != 'true'
run: |
sudo apt-get install -y libwxgtk*-dev
sudo ln -s $(which wx-config) /usr/local/bin/wx-config-3
wx-config-3 --cflags | grep I
curl -L https://github.com/wxWidgets/wxWidgets/releases/download/v$WX_VERSION/wxWidgets-$WX_VERSION.tar.bz2 -o wxWidgets-$WX_VERSION.tar.bz2
tar jxf wxWidgets-$WX_VERSION.tar.bz2
cd wxWidgets-$WX_VERSION
./configure --prefix=$HOME/wx-$WX_VERSION --enable-shared=no --enable-debug=no --with-gtk=2 --with-libjpeg=builtin --with-libpng=builtin --with-regex=builtin --with-libtiff=builtin --with-zlib=builtin --with-expat=builtin --without-libjbig --without-liblzma --without-gtkprint --with-libnotify=no --with-libmspack=no --with-gnomevfs=no --with-opengl=yes --with-sdl=no --with-cxx=11
make -j4
make install
sudo ln -s $HOME/wx-$WX_VERSION/bin/wx-config /usr/local/bin/wx-config-3
wx-config-3 --cflags
echo $HOME/wx-$WX_VERSION/bin >> $GITHUB_PATH

- name: Save wxWidgets build
if: steps.cachedwx.outputs.cache-hit != 'true'
uses: actions/upload-artifact@v4
with:
name: wxWidgets-${{ env.WX_VERSION }}-linux
path: |
${{ env.WXMSW3 }}

- name: Get branch name
run: |
# Short name for current branch. base ref is set on push builds, head ref is for pull request builds
if [ -z ${GITHUB_HEAD_REF+x} ]; then GIT_BRANCH=${GITHUB_BASE_REF:-${GITHUB_REF#refs/heads/}}; else GIT_BRANCH=${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}; fi
echo "GIT_BRANCH=$GIT_BRANCH" >> $GITHUB_ENV
- name: Get git ref of sibling dependency LK
run: |
ref=$(git ls-remote --exit-code git://github.com/NREL/lk.git refs/heads/patch | awk '{print $1}')
ref=$(git ls-remote --exit-code https://github.com/NREL/lk.git refs/heads/$GIT_BRANCH | awk '{print $1}')
echo "ref_of_lk=$ref" | tee --append $GITHUB_ENV
- name: Get cached build data of sibling dependency LK
uses: actions/cache@v2
uses: actions/cache@v4
id: cachedlk
with:
path: ${{ env.GH_WORKSPACE }}/lk
key: ${{ env.RUNS_ON }}-${{ env.ref_of_lk }}-LK
path: lk
key: linux-${{ env.ref_of_lk }}-LK
- name: Clone sibling dependency LK
if: steps.cachedlk.outputs.cache-hit != 'true'
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
ref: ${{ env.ref_of_lk }}
path: lk
Expand All @@ -59,49 +96,263 @@ jobs:
if: steps.cachedlk.outputs.cache-hit != 'true'
run: |
cd $GITHUB_WORKSPACE/lk
cmake -Bbuild_linux -DCMAKE_BUILD_TYPE=Debug
cmake --build build_linux -- -j
- name: Set LKDIR
cmake -Bbuild_linux -DCMAKE_BUILD_TYPE=Release
cmake --build build_linux -j4
- name: Set env vars
run: |
echo "LKDIR=$GITHUB_WORKSPACE/lk" >>$GITHUB_ENV
- name: Set LKD_LIB
run: |
echo "LKD_LIB=$GITHUB_WORKSPACE/lk/build_linux" >>$GITHUB_ENV
- name: Set RAPIDJSONDIR
run: |
echo "LK_LIB=$GITHUB_WORKSPACE/lk/build_linux" >>$GITHUB_ENV
echo "RAPIDJSONDIR=$GITHUB_WORKSPACE/ssc" >>$GITHUB_ENV
- name: Get git ref of sibling dependency SSC
run: |
ref=$(git ls-remote --exit-code git://github.com/NREL/ssc.git refs/heads/patch | awk '{print $1}')
ref=$(git ls-remote --exit-code https://github.com/NREL/ssc.git refs/heads/develop | awk '{print $1}')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like there will be a branch mismatch between patch and develop. Should this be patch?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this "base" branch handled? Are there different ci.yml for each branch?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For events triggered by pull_request, then you have the pull request merge branch, but for the standalone push, how should the default branch be determined as develop or patch?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently in SAM's patch branch, the base branch is also develop: https://github.com/NREL/SAM/blob/1ed59e9b181cf1f23d7ac1f696b5cb91805c7936/.github/workflows/ci.yml#L70

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not intentional: NREL/SAM#1664 I'm not sure how that got reverted but it should be patch.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For events triggered by pull_request, then you have the pull request merge branch, but for the standalone push, how should the default branch be determined as develop or patch?

Branches from patch should have patch as the default in the CI script, whereas branches from develop should have develop.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For events triggered by pull_request, then you have the pull request merge branch, but for the standalone push, how should the default branch be determined as develop or patch?

Branches from patch should have patch as the default in the CI script, whereas branches from develop should have develop.

Yes, that is clear. My question was mechanistic. I may just have two versions of ci.yml on each branch.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most years, a commit when we switch from patch to develop (and vice versa) has been sufficient, but as we see right now it's error prone. How would two ci.yml instances work?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be a bad idea but I was thinking we'd have a different version of ci.yml on each branch. I think then the first time we merge patch into develop, we have to manually unselect the changes in ci.yml. But then after that, that difference won't be a tracked diff, and shouldn't show up. I'm not 100% sure it'll work but I could try.

There kind of is a way to get the "closest parent branch" but it uses a lot of bash magic (that I don't understand) and might be unreliable.

echo "ref_of_ssc=$ref" | tee --append $GITHUB_ENV
- name: Get cached build data of sibling dependency SSC
uses: actions/cache@v2
uses: actions/cache@v4
id: cachedssc
with:
path: ${{ env.GH_WORKSPACE }}/ssc
key: ${{ env.RUNS_ON }}-${{ env.ref_of_ssc }}-SSC
path: ssc
key: windows-${{ env.ref_of_ssc }}-SSC
- name: Clone sibling dependency SSC
if: steps.cachedssc.outputs.cache-hit != 'true'
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
ref: ${{ env.ref_of_ssc }}
path: ssc
repository: NREL/ssc
- name: Build WEX
run: |
cd $GITHUB_WORKSPACE/wex
cmake -Bbuild_linux -DCMAKE_BUILD_TYPE=Release
cmake --build build_linux
- name: Save static lib, Dview & wexsandbox
uses: actions/upload-artifact@v4
with:
name: WEX-linux-x86_64
path: |
wex/build_linux/tools/DView*
wex/build_linux/tools/wexsandbox*
wex/build_linux/wex*.a

build-on-windows:
runs-on: windows-latest
env:
GITHUB_TOKEN: ${{ secrets.TOKEN_GITHUB }}
steps:
- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v2
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
path: wex
- name: Set relative paths
shell: bash
run: |
WXMSW3=$GITHUB_WORKSPACE/wx-$WX_VERSION
echo "WXMSW3=$WXMSW3" >> $GITHUB_ENV
- name: Setup cmake
uses: jwlawson/actions-setup-cmake@v2
with:
cmake-version: '3.24.x'
- name: Get cached build of wxWidgets
uses: actions/cache@v4
id: cachedwx
with:
path: ${{env.WXMSW3}}/
key: wxWidgets-${{ env.WX_VERSION }}-windows
- name: Download wxWidgets
if: steps.cachedwx.outputs.cache-hit != 'true'
shell: bash
run: |
curl -L https://github.com/wxWidgets/wxWidgets/releases/download/v$WX_VERSION/wxWidgets-$WX_VERSION.tar.bz2 -o wxWidgets-$WX_VERSION.tar.bz2
tar jxf wxWidgets-$WX_VERSION.tar.bz2
- name: Install wxWidgets
if: steps.cachedwx.outputs.cache-hit != 'true'
run: |
cd wxWidgets-$env:WX_VERSION
msbuild build/msw/wx_vc17.sln /t:Build /p:Configuration=Release /p:Platform=x64
mkdir $env:WXMSW3
cp -r include $env:WXMSW3
cp -r lib $env:WXMSW3
- name: Get branch name
shell: bash
run: |
# Short name for current branch. base ref is set on push builds, head ref is for pull request builds
if [ -z ${GITHUB_HEAD_REF+x} ]; then GIT_BRANCH=${GITHUB_BASE_REF:-${GITHUB_REF#refs/heads/}}; else GIT_BRANCH=${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}; fi
echo "GIT_BRANCH=$GIT_BRANCH" >> $GITHUB_ENV
- name: Determine branches for other repos
shell: bash
run: |
echo "GITHUB_REPOSITORY_OWNER=${GITHUB_REPOSITORY_OWNER}" >> $GITHUB_ENV
git ls-remote --heads --exit-code https://github.com/${GITHUB_REPOSITORY_OWNER}/lk.git $GIT_BRANCH
if [[ $? != "0" ]]; then echo "LK_BRANCH=develop" >> $GITHUB_ENV; else echo "LK_BRANCH=$GIT_BRANCH" >> $GITHUB_ENV; fi
dguittet marked this conversation as resolved.
Show resolved Hide resolved
- name: Get git ref of sibling dependency LK
shell: bash
run: |
ref=$(git ls-remote --exit-code https://github.com/NREL/lk.git refs/heads/$GIT_BRANCH | awk '{print $1}')
echo "ref_of_lk=$ref" | tee --append $GITHUB_ENV
- name: Get cached build data of sibling dependency LK
uses: actions/cache@v4
id: cachedlk
with:
path: lk
key: windows-${{ env.ref_of_lk }}-LK
- name: Checkout lk
if: steps.cachedlk.outputs.cache-hit != 'true'
uses: actions/checkout@v4
with:
path: lk
repository: ${{ env.GITHUB_REPOSITORY_OWNER }}/lk
ref: ${{ env.LK_BRANCH }}
- name: Build LK
if: steps.cachedlk.outputs.cache-hit != 'true'
run: |
cd lk
mkdir build
cd build
cmake .. -G "Visual Studio 17 2022" -DCMAKE_CONFIGURATION_TYPES="Release;Debug" -DCMAKE_SYSTEM_VERSION=10 -DSAM_SKIP_TOOLS=1
MSBuild.exe .\lk.sln /t:Build /p:Configuration=Release
MSBuild.exe .\lk.sln /t:Build /p:Configuration=Debug
- name: Set LK paths
shell: bash
run: |
echo "LKDIR=$GITHUB_WORKSPACE/lk" >>$GITHUB_ENV
echo "LKD_LIB=$GITHUB_WORKSPACE/lk/build/Debug" >>$GITHUB_ENV
echo "LK_LIB=$GITHUB_WORKSPACE/lk/build/Release" >>$GITHUB_ENV
echo "RAPIDJSONDIR=$GITHUB_WORKSPACE/ssc" >>$GITHUB_ENV
- name: Get git ref of sibling dependency SSC
shell: bash
run: |
ref=$(git ls-remote --exit-code https://github.com/NREL/ssc.git refs/heads/develop | awk '{print $1}')
dguittet marked this conversation as resolved.
Show resolved Hide resolved
echo "ref_of_ssc=$ref" | tee --append $GITHUB_ENV
- name: Get cached build data of sibling dependency SSC
uses: actions/cache@v4
id: cachedssc
with:
path: ssc
key: windows-${{ env.ref_of_ssc }}-SSC
- name: Checkout ssc
if: steps.cachedssc.outputs.cache-hit != 'true'
uses: actions/checkout@v4
with:
path: ssc
repository: ${{ env.GITHUB_REPOSITORY_OWNER }}/ssc
ref: ${{ env.SSC_BRANCH }}
- name: Build WEX
run: |
cd $GITHUB_WORKSPACE/wex
cmake -Bbuild_linux -DCMAKE_BUILD_TYPE=Debug
cmake --build build_linux
cd wex
mkdir build
cd build
cmake -G "Visual Studio 17 2022" -DCMAKE_CONFIGURATION_TYPES="Release" -DCMAKE_SYSTEM_VERSION=10 ..
MSBuild.exe .\wex.sln /t:Build /p:Configuration=Release
- name: Save static lib, Dview & wexsandbox
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
with:
name: WEX-${{ env.RUNS_ON }}-x86_64
name: WEX-windows-x86_64
path: |
${{ env.GH_WORKSPACE }}/wex/build_linux/tools/DView*
${{ env.GH_WORKSPACE }}/wex/build_linux/tools/wexsandbox*
${{ env.GH_WORKSPACE }}/wex/build_linux/wex*.a
wex/build/tools/Release/tools/DView.exe
wex/build/tools/Release/tools/wexsandbox.exe
wex/build/Release/wex.lib

build-on-mac:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [macos-14-large, macos-latest]
env:
GITHUB_TOKEN: ${{ secrets.TOKEN_GITHUB }}
steps:
- name: Set relative paths
run: |
WXMSW3=$HOME/wx-$WX_VERSION
echo "WXMSW3=$WXMSW3" >> $GITHUB_ENV
- name: Checkout
uses: actions/checkout@v4
with:
path: wex
- name: Get cached build of wxWidgets
uses: actions/cache@v4
id: cachedwx
with:
path: ${{env.WXMSW3}}/
key: wxWidgets-${{ env.WX_VERSION }}-${{ matrix.os }}
- name: Install wxWidgets
if: steps.cachedwx.outputs.cache-hit != 'true'
run: |
curl -L https://github.com/wxWidgets/wxWidgets/releases/download/v$WX_VERSION/wxWidgets-$WX_VERSION.tar.bz2 -o wxWidgets-$WX_VERSION.tar.bz2
tar jxf wxWidgets-$WX_VERSION.tar.bz2
cd wxWidgets-$WX_VERSION
./configure --prefix=$HOME/wx-$WX_VERSION --enable-stl=yes --enable-shared=no --disable-debug_flag --with-cocoa --enable-universal_binary=x86_64,arm64 --enable-unicode --enable-webview --disable-mediactrl --with-cxx=11 --with-macosx-version-min=10.15 --with-libjpeg=builtin --with-libpng=builtin --with-regex=builtin --with-libtiff=builtin --with-zlib=builtin --with-expat=builtin
make -j3
make install
sudo ln -s $HOME/wx-$WX_VERSION/bin/wx-config /usr/local/bin/wx-config-3
wx-config-3 --cflags
echo ${HOME}/wx-$WX_VERSION/bin >> $GITHUB_PATH
- name: Get branch name
run: |
# Short name for current branch. base ref is set on push builds, head ref is for pull request builds
if [ -z ${GITHUB_HEAD_REF+x} ]; then GIT_BRANCH=${GITHUB_BASE_REF:-${GITHUB_REF#refs/heads/}}; else GIT_BRANCH=${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}; fi
echo "GIT_BRANCH=$GIT_BRANCH" >> $GITHUB_ENV
- name: Get git ref of sibling dependency LK
run: |
ref=$(git ls-remote --exit-code https://github.com/NREL/lk.git refs/heads/$GIT_BRANCH | awk '{print $1}')
echo "ref_of_lk=$ref" | tee -a $GITHUB_ENV
- name: Get cached build data of sibling dependency LK
uses: actions/cache@v4
id: cachedlk
with:
path: lk
key: ${{ matrix.os }}-${{ env.ref_of_lk }}-LK
- name: Clone sibling dependency LK
if: steps.cachedlk.outputs.cache-hit != 'true'
uses: actions/checkout@v4
with:
ref: ${{ env.ref_of_lk }}
path: lk
repository: NREL/lk
- name: Build LK
if: steps.cachedlk.outputs.cache-hit != 'true'
run: |
cd $GITHUB_WORKSPACE/lk
cmake -Bbuild_linux -DCMAKE_BUILD_TYPE=Release
cmake --build build_linux -j4
- name: Set env vars
run: |
echo "LKDIR=$GITHUB_WORKSPACE/lk" >>$GITHUB_ENV
echo "LK_LIB=$GITHUB_WORKSPACE/lk/build_linux" >>$GITHUB_ENV
echo "RAPIDJSONDIR=$GITHUB_WORKSPACE/ssc" >>$GITHUB_ENV
- name: Get git ref of sibling dependency SSC
run: |
ref=$(git ls-remote --exit-code https://github.com/NREL/ssc.git refs/heads/develop | awk '{print $1}')
dguittet marked this conversation as resolved.
Show resolved Hide resolved
echo "ref_of_ssc=$ref" | tee -a $GITHUB_ENV
- name: Get cached build data of sibling dependency SSC
uses: actions/cache@v4
id: cachedssc
with:
path: ssc
key: ${{ matrix.os }}-${{ env.ref_of_ssc }}-SSC
- name: Clone sibling dependency SSC
if: steps.cachedssc.outputs.cache-hit != 'true'
uses: actions/checkout@v4
with:
ref: ${{ env.ref_of_ssc }}
path: ssc
repository: NREL/ssc
- name: Setup cmake
uses: jwlawson/actions-setup-cmake@v2
with:
cmake-version: '3.24.x'
- name: Build WEX
run: |
cd wex
cmake -Bbuild_linux -DCMAKE_BUILD_TYPE=Release
cmake --build build_linux
- name: Save static lib, Dview & wexsandbox
uses: actions/upload-artifact@v4
with:
name: WEX-${{ matrix.os }}-x86_64
path: |
wex/build_linux/tools/DView*
wex/build_linux/tools/wexsandbox*
wex/build_linux/wex*.a

Loading