Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TrustEVM Contract CI #293

Merged
merged 58 commits into from
Feb 9, 2023
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
5ef46db
Create CI pipeline to build TrustEVM contract(s)
kj4ezj Jan 18, 2023
a983ca9
Rename "build.tar.gz" to "contract.tar.gz" to all build assets from a…
kj4ezj Jan 18, 2023
9cf20e0
Disable submodule checkout, for testing
kj4ezj Jan 18, 2023
319053e
Fix shell globbing
kj4ezj Jan 18, 2023
9c668bc
Download and install Leap dev package
kj4ezj Jan 18, 2023
e3024b8
Put verb first in job step name
kj4ezj Jan 18, 2023
4a9e78d
Rename script to match job step name
kj4ezj Jan 18, 2023
725692a
Pull script name from BASH instead of hard-coding it
kj4ezj Jan 18, 2023
2704bb3
Build and upload contract unit test(s)
kj4ezj Jan 18, 2023
1f54dde
Use a GitHub app to obtain an ephemeral token with permissions to pri…
kj4ezj Jan 25, 2023
4d0f259
Clone top-level submodules
kj4ezj Jan 25, 2023
542baf0
Skip some steps, for testing leap download
kj4ezj Jan 18, 2023
847aada
Attach the leap-dev*.deb binary as an artifact, for debugging
kj4ezj Jan 25, 2023
6c983ed
Revert "Attach the leap-dev*.deb binary as an artifact, for debugging"
kj4ezj Jan 25, 2023
af86cf4
Point to EOSIO CMake files from leap-dev bin
kj4ezj Jan 25, 2023
fe4393f
Print version of Leap found
kj4ezj Jan 25, 2023
e4653d6
I don't remember why this wasn't already set to "recursive"...
kj4ezj Feb 2, 2023
f33007b
Example building contract tests without building contracts or node
kj4ezj Feb 2, 2023
16f8456
Build with contracts & contract unit test(s)
kj4ezj Feb 2, 2023
666015b
Use GCC 10 to be in parity with the node build
kj4ezj Feb 2, 2023
6cd472e
Document TrustEVM Contracts CI system
kj4ezj Feb 2, 2023
2967aa7
Link to CI system and to CI documentation from root README.md
kj4ezj Feb 2, 2023
553ef03
Try pushing CI documentation to GitHub step summary
kj4ezj Feb 2, 2023
f0d6bbe
Separate document for each pipeline
kj4ezj Feb 2, 2023
2cb3f3f
Reformat pipeline documentation to stand alone
kj4ezj Feb 2, 2023
ac64793
Formatting
kj4ezj Feb 2, 2023
9d58dd0
Fail if the expected artifacts are not present
kj4ezj Feb 2, 2023
8296ef9
Add debugging code
kj4ezj Feb 2, 2023
c711f72
Fix path issue for docs
kj4ezj Feb 2, 2023
1977b19
Append instead of overwriting in case this code gets copied around
kj4ezj Feb 2, 2023
5d06610
Revert "Add debugging code"
kj4ezj Feb 2, 2023
9d5f728
Fix path issue for test artifact
kj4ezj Feb 2, 2023
63f7030
Clarify relationship between "Authenticate" step and checkout step
kj4ezj Feb 2, 2023
dd5c51a
Formatting
kj4ezj Feb 2, 2023
2f4e298
Write about the implementation, benefits, and drawbacks of GitHub app…
kj4ezj Feb 2, 2023
f82e153
Be more specific about the source of the build outputs
kj4ezj Feb 2, 2023
fc5d8c8
Change trigger conditions to meet ticket requirements
kj4ezj Feb 2, 2023
51590e9
Reword output description
kj4ezj Feb 2, 2023
1b7ff4e
Merge branch 'main' into zach-contract-ci
kj4ezj Feb 8, 2023
7d4ae3a
Remove "-Deosio_DIR" cmake flag from contract tests
kj4ezj Feb 8, 2023
87d6b7e
Merge pull request #311 from eosnetworkfoundation/zach-eosio-dir
kj4ezj Feb 8, 2023
2d3ecc9
Specify release build type for contract tests
kj4ezj Feb 8, 2023
4d3f8e9
Pull DWITH_TEST_ACTIONS definition out of script and into action YAML
kj4ezj Feb 8, 2023
b276a7d
Pull DCMAKE_BUILD_TYPE out of scripts into action YAML, as well
kj4ezj Feb 8, 2023
2314df6
Include value of cmake vars from action YAML in cmake command printed…
kj4ezj Feb 8, 2023
bfa6f68
Use build matrix to provide CI with and without test actions enabled
kj4ezj Feb 8, 2023
53a7dd0
Lowercase test action flag
kj4ezj Feb 8, 2023
a06d4ed
De-duplicate contract.tar.gz filename in matrix
kj4ezj Feb 8, 2023
1305944
De-duplicate documentation upload step from matrix
kj4ezj Feb 8, 2023
5752dd3
Lowercase if statement test, as well
kj4ezj Feb 8, 2023
5cbe3b5
Support boolean flags in addition to on/off strings
kj4ezj Feb 8, 2023
7239da9
Docs - sort GH actions inputs
kj4ezj Feb 8, 2023
582dcef
Add DCMAKE_BUILD_TYPE to CI docs
kj4ezj Feb 8, 2023
d89596b
Document DWITH_TEST_ACTIONS
kj4ezj Feb 8, 2023
f782e44
Expand input summary
kj4ezj Feb 8, 2023
08bad50
Update CI steps in documentation based on my changes
kj4ezj Feb 8, 2023
68474bc
Update CI outputs documentation for matrix build
kj4ezj Feb 8, 2023
abf0a46
Format note about GitHub Actions "archive in archive" bug
kj4ezj Feb 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/build-contract-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/bin/bash
set -eo pipefail

# print and run a command
function ee()
{
echo "$ $*"
eval "$@"
}

export Deosio_DIR='/usr/lib/x86_64-linux-gnu/cmake/eosio'
# debug code
ee cmake --version
echo 'Leap version:'
cat "$Deosio_DIR/EosioTester.cmake" | grep 'EOSIO_VERSION' | grep -oP "['\"].*['\"]" | tr -d "'\"" || :

# build
ee mkdir -p contract/tests/build
ee pushd contract/tests
ee pushd build
ee "cmake -Deosio_DIR=$Deosio_DIR .."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend adding -DCMAKE_BUILD_TYPE=Release. It shouldn't matter much, but every little bit helps.

Also, did you need the -Deosio_DIR thing? afaik that should not be required

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually just tested, the difference is more on the order of 25%, so highly recommend doing Release here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, did you need the -Deosio_DIR thing? afaik that should not be required

Apparently not, though my previous testing had indicated otherwise. Removed.

Copy link
Contributor Author

@kj4ezj kj4ezj Feb 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend adding -DCMAKE_BUILD_TYPE=Release. It shouldn't matter much, but every little bit helps.

Done. Thank you!

ee make -j "$(nproc)" unit_test

# pack
ee popd
ee 'tar -czf ../../contract-test.tar.gz build/*'
ee popd

echo "Done! - ${0##*/}"
27 changes: 27 additions & 0 deletions .github/workflows/build-contract.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/bin/bash
set -eo pipefail

# print and run a command
function ee()
{
echo "$ $*"
eval "$@"
}

# debug code
ee cdt-cc --version
ee cmake --version

# build
ee mkdir -p contract/build
ee pushd contract
ee pushd build
ee cmake -DCMAKE_BUILD_TYPE=Release -DWITH_TEST_ACTIONS=On ..
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should build the contract once w/o test actions, and once with test actions. It's okay that we only test the one with test actions, but the non-test action build is the one we'd deploy so it's nice to make sure that it's at least building successfully.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree with @spoonincode. maybe we can add build-contract-release.sh?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

No need to duplicate the BASH script, I abstracted the cmake vars out from BASH into the GitHub Actions YAML. They probably should have been here all along. The BASH script still prints the values of the cmake vars in the CI logs when invoking cmake so contributors do not have to go hunting in the CI code in order to figure out what they are.

ee make -j "$(nproc)"

# pack
ee popd
ee 'tar -czf ../contract.tar.gz build/*'
ee popd

echo "Done! - ${0##*/}"
64 changes: 64 additions & 0 deletions .github/workflows/contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# TrustEVM Contract CI
This GitHub Actions workflow builds the TrustEVM contract and its associated tests.

### Index
1. [Triggers](#triggers)
1. [Inputs](#inputs)
1. [Steps](#steps)
1. [Outputs](#outputs)
1. [GitHub App Integration](#github-app-integration)
1. [See Also](#see-also)

## Triggers
This GitHub action will run under the following circumstances:
1. When code is pushed to the `main` branch.
1. When code is pushed to any branch with a name starting with `release/`.
1. Workflow dispatch event, which is triggered manually using the "Workflow Dispatch" button in the Actions tab of the GitHub repository.

## Inputs
The inputs for this GitHub action are:
1. `TRUSTEVM_CI_APP_ID` - the app ID of the `trustevm-ci-submodule-checkout` GitHub App.
1. `TRUSTEVM_CI_APP_KEY` - the private key to the `trustevm-ci-submodule-checkout` GitHub App.
1. `GITHUB_TOKEN` - a GitHub Actions intrinsic used to access the repository and other public resources.

These inputs are used in various steps of the workflow to perform actions such as authentication, downloading artifacts, and uploading artifacts.

## Steps
This workflow performs the following steps:
1. Authenticate to the `trustevm-ci-submodule-checkout` GitHub app using the [AntelopeIO/github-app-token-action](https://github.com/AntelopeIO/github-app-token-action) action to obtain an ephemeral token.
1. Checkout the repo and submodules using the ephemeral token.
1. Attach an annotation to the build with CI documentation.
1. Download the CDT binary using the [AntelopeIO/asset-artifact-download-action](https://github.com/AntelopeIO/asset-artifact-download-action) action.
1. Install the CDT binary.
1. Build the TrustEVM contract using `make` and `cmake`.
1. Upload the contract build folder to GitHub Actions.
1. Download the `leap-dev` binary using [AntelopeIO/asset-artifact-download-action](https://github.com/AntelopeIO/asset-artifact-download-action) action.
1. Install the `leap-dev` binary.
1. Build the TrustEVM contract tests using `make` and `cmake`.
1. Upload the build folder for the contract test code to GitHub Actions.

## Outputs
This workflow produces the following outputs:
1. Contract Build Artifacts - `contract.tar.gz` containing the built contract from the `contract/build` folder.
1. Contract Test Artifacts - `contract-test.tar.gz` containing the built contract test artifacts from the `contract/tests/build` folder.

Note that, due to actions/upload-artifact [issue 39](https://github.com/actions/upload-artifact/issues/39) which has been open for over _three years_ and counting, the archives attached as artifacts will be zipped by GitHub when you download them such that you get a `*.zip` containing the `*.tar.gz`. There is nothing anyone can do about this except for GitHub.

## GitHub App Integration
This workflow uses the [AntelopeIO/github-app-token-action](https://github.com/AntelopeIO/github-app-token-action) GitHub action to assume the role of a GitHub application installed to the AntelopeIO organization to clone the private submodules. It requests a token from the GitHub app, clones everything using this token under the identity of the app, then the token expires. This is advantageous over a persistent API key from a GitHub service account because this does not consume a paid user seat, the "account" associated with the app cannot be logged into in the GitHub web UI, the app is scoped to exactly the permissions it needs to perform the clones for this repo _and nothing more_, and the API key expires very quickly so a bad actor who exfiltrates this key from the CI system should find it is not useful.

**The downside is that if TrustEVM adds additional private submodules, the GitHub app must be granted permissions to these new submodules.** The CI system will not work until this happens.

## See Also
- [asset-artifact-download-action](https://github.com/AntelopeIO/asset-artifact-download-action) GitHub Action
- [github-app-token-action](https://github.com/AntelopeIO/github-app-token-action) GitHub action
- [TrustEVM Documentation](../../README.md)

For assistance with the CI system, please open an issue in this repo or reach out in the `#help-automation` channel via IM.

***
**_Legal notice_**
This document was generated in collaboration with ChatGPT from OpenAI, a machine learning algorithm or weak artificial intelligence (AI). At the time of this writing, the [OpenAI terms of service agreement](https://openai.com/terms) §3.a states:
> Your Content. You may provide input to the Services (“Input”), and receive output generated and returned by the Services based on the Input (“Output”). Input and Output are collectively “Content.” As between the parties and to the extent permitted by applicable law, you own all Input, and subject to your compliance with these Terms, OpenAI hereby assigns to you all its right, title and interest in and to Output.

This notice is required in some countries.
82 changes: 82 additions & 0 deletions .github/workflows/contract.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
name: TrustEVM Contract CI

on:
push:
branches:
- main
- release/*
pull_request:
workflow_dispatch:

jobs:
build:
name: TrustEVM Contract Build
runs-on: ubuntu-20.04
env:
CC: gcc-10
CXX: g++-10

steps:
- name: Authenticate
id: auth
uses: AntelopeIO/github-app-token-action@v1
with:
app_id: ${{ secrets.TRUSTEVM_CI_APP_ID }}
private_key: ${{ secrets.TRUSTEVM_CI_APP_KEY }}

- name: Checkout Repo
uses: actions/checkout@v3
with:
fetch-depth: 0
submodules: 'recursive'
token: ${{ steps.auth.outputs.token }}

- name: Attach Documentation
run: cat .github/workflows/contract.md >> $GITHUB_STEP_SUMMARY

- name: Download CDT
uses: AntelopeIO/asset-artifact-download-action@v2
with:
owner: AntelopeIO
repo: cdt
target: 'v3.1.0'
prereleases: false
file: 'cdt_.*amd64.deb'
token: ${{ secrets.GITHUB_TOKEN }}

- name: Install CDT
run: sudo apt-get install -y ./cdt*.deb

- name: Build TrustEVM Contract
run: .github/workflows/build-contract.sh

- name: Upload Artifacts
uses: actions/upload-artifact@v3
with:
name: contract.tar.gz
path: contract.tar.gz
if-no-files-found: error

- name: Download Leap - dev binary
uses: AntelopeIO/asset-artifact-download-action@v2
with:
owner: AntelopeIO
repo: leap
target: 'v3.1.3'
prereleases: false
file: 'leap-dev.*(x86_64|amd64).deb'
container-package: experimental-binaries
token: ${{ secrets.GITHUB_TOKEN }}

- name: Install Leap
run: sudo apt-get install -y ./leap*.deb

- name: Build TrustEVM Contract Tests
run: .github/workflows/build-contract-test.sh

- name: Upload Artifacts
uses: actions/upload-artifact@v3
with:
name: contract-test.tar.gz
path: contract-test.tar.gz
if-no-files-found: error
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,3 +115,11 @@ https://github.com/eosnetworkfoundation/TrustEVM/blob/main/docs/local_testnet_de

For public testnet deployment, please refer to
https://github.com/eosnetworkfoundation/TrustEVM/blob/main/docs/public_testnet_deployment_plan.md

## CI
This repo contains the following GitHub Actions workflows for CI:
- TrustEVM Contract CI - build the TrustEVM contract and its associated tests
- [Pipeline](https://github.com/eosnetworkfoundation/TrustEVM/actions/workflows/contract.yml)
- [Documentation](./.github/workflows/contract.md)

See the pipeline documentation for more information.