Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nightly build for TileDB-Go #251

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/scripts/nightly/install_tiledb_source_macos.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
set -e -x
git clone https://github.com/TileDB-Inc/TileDB.git -b dev --depth 1
cd TileDB
mkdir build && cd build
cmake -DTILEDB_WEBP=OFF -DTILEDB_VERBOSE=OFF -DTILEDB_S3=ON -DTILEDB_SERIALIZATION=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local ..
make -j$(sysctl -n hw.ncpu)
sudo make -C tiledb install
37 changes: 37 additions & 0 deletions .github/scripts/nightly/open-issue.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#!/bin/bash
set -ex

# Open new Issue (or comment on existing) after build failure

if [[ -z "$GH_TOKEN" ]]
then
echo "The env var GH_TOKEN is missing"
echo "Please define it as a GitHub PAT with write permissions to Issues"
exit 1
fi

theDate="$(date '+%A (%Y-%m-%d)')"
theMessage="Nightly build failed on $theDate in run [$GITHUB_RUN_ID]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID)"

existing=$(gh issue list \
--label "nightly-failure" \
--limit 1 \
--jq '.[].number' \
--json "number" \
--state "open")

if [[ -z "$existing" ]]
then
# open new issue
gh issue create \
--assignee Shelnutt2,ihnorton,anastasop \
--body "$theMessage" \
--label "nightly-failure" \
--title "Nightly build failed on $theDate"
else
# comment on existing issue
gh issue comment "$existing" \
--body "$theMessage"
fi

echo "Success!"
105 changes: 105 additions & 0 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
name: TileDB-Go with nightly libtiledb

on:
push:
paths:
- '.github/workflows/nightly.yml'
- '.github/scripts/nightly/**'
pull_request:
paths:
- '.github/workflows/nightly.yml'
- '.github/scripts/nightly/**'
schedule:
- cron: "0 4 * * *" # Every night at 4 AM UTC (11 PM EST; 12 AM EDT)
workflow_dispatch:

env:
go: "1.18"

jobs:
# Quick build using nightly libtiledb installed as conda binary
# https://anaconda.org/tiledb/tiledb/files?channel=nightlies
# https://github.com/TileDB-Inc/conda-forge-nightly-controller
ubuntu:
runs-on: ubuntu-20.04
defaults:
run:
shell: bash -l {0}
steps:

- uses: actions/checkout@v3

- name: Install Conda environment with go and nightly libtiledb
uses: mamba-org/provision-with-micromamba@main
with:
environment-file: false
environment-name: nightly
extra-specs: |
gcc
go=${{ env.go }}
tiledb
channels: tiledb/label/nightlies, conda-forge

- name: Install dependencies
run: go get -t .

- name: Test TileDB-Go
run: |
export CPATH=$CONDA_PREFIX/include
export LIBRARY_PATH=$CONDA_PREFIX/lib
export LD_LIBRARY_PATH=$CONDA_PREFIX/lib

go test -v ./...

# Longer build because libtiledb is built from source
#
# Couldn't build against the conda binary on macOS. When I set
# `DYLD_LIBRARY_PATH=$CONDA_PREFIX/lib`, then it failed because it found the
# system libarchive instead of the conda-installed one (even when I explicitly
# installed it from conda-forge). Error message below:
#
# dyld: Symbol not found: _iconv
# Referenced from: /usr/lib/libarchive.2.dylib
# Expected in: /Users/runner/micromamba-root/envs/nightly/lib/libiconv.2.dylib
# in /usr/lib/libarchive.2.dylib
macos:
runs-on: macos-11
needs: ubuntu
steps:

- uses: actions/checkout@v3

- name: Install nightly TileDB from source
run: ./.github/scripts/nightly/install_tiledb_source_macos.sh

- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: ${{ env.go }}

- name: Install dependencies
run: go get -t .

- name: Test TileDB-Go
run: go test -v ./...

# Report if either the Ubuntu or macOS builds fail
#
# If there is an existing "nightly-failure" Issue open, then it comments
# instead of creating a new one
#
# Only reports a failure for scheduled builds on the TileDB-Inc repo
issue:
permissions:
issues: write
runs-on: ubuntu-latest
needs: [ubuntu, macos]
if: ( failure() || cancelled() ) && github.repository_owner == 'TileDB-Inc' && github.event_name == 'schedule'
steps:

- uses: actions/checkout@v3
- name: Open Issue
env:
GH_TOKEN: ${{ github.token }}
TZ: "America/New_York"
run: ./.github/scripts/nightly/open-isue.sh