From 3a36468c9fb0532f497df4b536e99be85c13721c Mon Sep 17 00:00:00 2001 From: Mike MacCana Date: Thu, 17 Oct 2024 13:39:24 -0400 Subject: [PATCH 1/5] Minor markdown spelling fixes --- CONTRIBUTING.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index da858900..307ad168 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -22,13 +22,13 @@ Specifically for code in this repo: 1. Use pnpm as the default package manager for the project. You can [install pnpm by following the instructions](https://pnpm.io/installation). Commit `pnpm-lock.yaml` to the repository. -2. Anchor programs should be in directory `anchor`, programs written for Solana Native should be in directory `native`, TypeScript in `posidon` and Python in `seahorse. +2. Anchor programs should be in directory `anchor`, programs written for Solana Native should be in directory `native`, TypeScript in `posidon` and Python in `seahorse`. 3. Tests for Solana native, Steel and Anchor programs should be written with [ts-mocha](https://github.com/piotrwitek/ts-mocha). -4. Tests for solana native programs and steel framework programs should be written with [solana-bankrun](https://kevinheavey.github.io/solana-bankrun) +4. Tests for Solana native programs and steel framework programs should be written with [solana-bankrun](https://kevinheavey.github.io/solana-bankrun) -5. For Solana native programs and Steel framework programs ensure adding these mandatory pnpm run scripts to your `package.json`. file for successful ci/cd builds: +5. For Solana native programs and Steel framework programs ensure adding these mandatory pnpm run scripts to your `package.json` file for successful CI/CD builds: ```json "scripts": { @@ -39,7 +39,7 @@ Specifically for code in this repo: }, ``` -6. Test command for anchor should execute `pnpm test` instead of `yarn run test` for anchor programs. Replace `yarn` with `pnpm` in `[script]` table inside [Anchor.toml file.](https://www.anchor-lang.com/docs/manifest#scripts-required-for-testing) +6. Test command for Anchor should execute `pnpm test` instead of `yarn run test` for anchor programs. Replace `yarn` with `pnpm` in `[script]` table inside [Anchor.toml file.](https://www.anchor-lang.com/docs/manifest#scripts-required-for-testing) 7. TypeScript, JavaScript and JSON files are formatted and linted using [Biome](https://biomejs.dev/). Execute the following command to format and lint your code at the root of this project before submitting a pull request: From 46edd4bc4b21fafc121365ba8c815cd102a853c2 Mon Sep 17 00:00:00 2001 From: Mike MacCana Date: Mon, 21 Oct 2024 12:40:14 -0400 Subject: [PATCH 2/5] fix typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1c81114f..b4b59a85 100644 --- a/README.md +++ b/README.md @@ -87,7 +87,7 @@ Store and retrieve state in Solana. [anchor](./basics/program-derived-addresses/anchor) [native](./basics/program-derived-addresses/native) -### Handling accounts that expland in size +### Handling accounts that expand in size How to store state that changes size in Solana. From dfaa6d9becf9f352f0b3d548c3f4baafe59c9eee Mon Sep 17 00:00:00 2001 From: The Maker <168169829+adpthegreat@users.noreply.github.com> Date: Mon, 21 Oct 2024 17:42:47 +0100 Subject: [PATCH 3/5] fixed minor grammar errors in CONTRIBUTING.md (#151) * fixed minor grammar errors in CONTRIBUTING.md * fix:added more fixes --- CONTRIBUTING.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 307ad168..b3e966ea 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -22,7 +22,7 @@ Specifically for code in this repo: 1. Use pnpm as the default package manager for the project. You can [install pnpm by following the instructions](https://pnpm.io/installation). Commit `pnpm-lock.yaml` to the repository. -2. Anchor programs should be in directory `anchor`, programs written for Solana Native should be in directory `native`, TypeScript in `posidon` and Python in `seahorse`. +2. Anchor programs should be in directory `anchor`, programs written for Solana Native should be in directory `native`, TypeScript in `poseidon` and Python in `seahorse`. 3. Tests for Solana native, Steel and Anchor programs should be written with [ts-mocha](https://github.com/piotrwitek/ts-mocha). @@ -44,14 +44,15 @@ Specifically for code in this repo: 7. TypeScript, JavaScript and JSON files are formatted and linted using [Biome](https://biomejs.dev/). Execute the following command to format and lint your code at the root of this project before submitting a pull request: -8. Some projects can be ignored from the building and testing process by adding the project name to the `.ghaignore` file. -When removing or updating an example, please ensure that the example is removed from the `.ghaignore` file -and there's a change in that example's directory. - ```bash pnpm check:fix ``` +8. Some projects can be ignored from the building and testing process by adding the project name to the `.gitignore` file. +When removing or updating an example, please ensure that the example is removed from the `.gitignore` file +and there's a change in that example's directory. + + ## Code of Conduct We are committed to providing a friendly, safe, and welcoming environment for all contributors, regardless of their background, experience level, or personal characteristics. As a contributor, you are expected to: From 45e302abb5213363fef5f4db9defbcf51a5c794a Mon Sep 17 00:00:00 2001 From: Jacob Creech <82475023+jacobcreech@users.noreply.github.com> Date: Tue, 22 Oct 2024 22:58:10 -0500 Subject: [PATCH 4/5] Update CONTRIBUTING.md --- CONTRIBUTING.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b3e966ea..ed0528a3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -24,11 +24,9 @@ Specifically for code in this repo: 2. Anchor programs should be in directory `anchor`, programs written for Solana Native should be in directory `native`, TypeScript in `poseidon` and Python in `seahorse`. -3. Tests for Solana native, Steel and Anchor programs should be written with [ts-mocha](https://github.com/piotrwitek/ts-mocha). +3. Tests for Solana native programs, steel framework programs, and Anchor should be written with [solana-bankrun](https://kevinheavey.github.io/solana-bankrun) -4. Tests for Solana native programs and steel framework programs should be written with [solana-bankrun](https://kevinheavey.github.io/solana-bankrun) - -5. For Solana native programs and Steel framework programs ensure adding these mandatory pnpm run scripts to your `package.json` file for successful CI/CD builds: +4. For Solana native programs and Steel framework programs ensure adding these mandatory pnpm run scripts to your `package.json` file for successful CI/CD builds: ```json "scripts": { @@ -39,16 +37,16 @@ Specifically for code in this repo: }, ``` -6. Test command for Anchor should execute `pnpm test` instead of `yarn run test` for anchor programs. Replace `yarn` with `pnpm` in `[script]` table inside [Anchor.toml file.](https://www.anchor-lang.com/docs/manifest#scripts-required-for-testing) +5. Test command for Anchor should execute `pnpm test` instead of `yarn run test` for anchor programs. Replace `yarn` with `pnpm` in `[script]` table inside [Anchor.toml file.](https://www.anchor-lang.com/docs/manifest#scripts-required-for-testing) -7. TypeScript, JavaScript and JSON files are formatted and linted using +6. TypeScript, JavaScript and JSON files are formatted and linted using [Biome](https://biomejs.dev/). Execute the following command to format and lint your code at the root of this project before submitting a pull request: ```bash pnpm check:fix ``` -8. Some projects can be ignored from the building and testing process by adding the project name to the `.gitignore` file. +7. Some projects can be ignored from the building and testing process by adding the project name to the `.gitignore` file. When removing or updating an example, please ensure that the example is removed from the `.gitignore` file and there's a change in that example's directory. From 46f66f3da17966c90374fc325f9560d90f05d45b Mon Sep 17 00:00:00 2001 From: Ayush Date: Thu, 7 Nov 2024 17:12:11 +0530 Subject: [PATCH 5/5] fix: Solana Native and Steel Action should only work on changes. (#325) * run actions only when project is added or changed * test actions * solana version matrix workaround * try action fix * rename job names for setup --- .github/workflows/solana-native.yml | 293 +++++++++++++----- .github/workflows/steel.yml | 293 +++++++++++++----- .husky/pre-commit | 3 - CONTRIBUTING.md | 4 +- basics/hello-solana/native/package.json | 2 +- .../native/tests/{test.ts => index.test.ts} | 0 basics/hello-solana/steel/package.json | 2 +- .../steel/tests/{test.ts => index.test.ts} | 0 8 files changed, 436 insertions(+), 161 deletions(-) rename basics/hello-solana/native/tests/{test.ts => index.test.ts} (100%) rename basics/hello-solana/steel/tests/{test.ts => index.test.ts} (100%) diff --git a/.github/workflows/solana-native.yml b/.github/workflows/solana-native.yml index 5e79ae97..c3462dd5 100644 --- a/.github/workflows/solana-native.yml +++ b/.github/workflows/solana-native.yml @@ -11,107 +11,246 @@ on: branches: - main +env: + MAX_JOBS: 64 + MIN_PROJECTS_PER_JOB: 4 + MIN_PROJECTS_FOR_MATRIX: 4 + jobs: - build: + changes: runs-on: ubuntu-latest - strategy: - matrix: - node-version: [20.x] - solana-version: [stable] + permissions: + pull-requests: read + outputs: + changed_projects: ${{ steps.analyze.outputs.changed_projects }} + total_projects: ${{ steps.analyze.outputs.total_projects }} + matrix: ${{ steps.matrix.outputs.matrix }} steps: - uses: actions/checkout@v4 - - name: Use Node.js ${{ matrix.node-version }} - uses: actions/setup-node@v4 + - uses: dorny/paths-filter@v3 + id: changes + if: github.event_name == 'pull_request' with: - node-version: ${{ matrix.node-version }} - check-latest: true - - uses: heyAyushh/setup-solana@v5.5 - with: - solana-cli-version: ${{ matrix.solana-version }} - - run: solana -V - shell: bash - - name: Install pnpm + list-files: shell + filters: | + native: + - added|modified: '**/native/**' + workflow: + - added|modified: '.github/workflows/solana-native.yml' + - name: Analyze Changes + id: analyze run: | - npm install --global pnpm - - name: Build Native programs + # Generate ignore pattern, excluding comments + ignore_pattern=$(grep -v '^#' .github/.ghaignore | grep -v '^$' | tr '\n' '|' | sed 's/|$//') + echo "Ignore pattern: $ignore_pattern" + + function get_projects() { + find . -type d -name "native" | grep -vE "$ignore_pattern" | sort + } + + # Determine which projects to build and test + if [[ "${{ github.event_name }}" == "push" || "${{ github.event_name }}" == "schedule" || "${{ steps.changes.outputs.workflow }}" == "true" ]]; then + projects=$(get_projects) + elif [[ "${{ steps.changes.outputs.native }}" == "true" ]]; then + changed_files=(${{ steps.changes.outputs.native_files }}) + projects=$(for file in "${changed_files[@]}"; do dirname "${file}" | grep native | sed 's#/native/.*#/native#g'; done | grep -vE "$ignore_pattern" | sort -u) + else + projects="" + fi + + # Output project information + if [[ -n "$projects" ]]; then + echo "Projects to build and test" + echo "$projects" + total_projects=$(echo "$projects" | wc -l) + echo "Total projects: $total_projects" + echo "total_projects=$total_projects" >> $GITHUB_OUTPUT + echo "changed_projects=$(echo "$projects" | jq -R -s -c 'split("\n")[:-1]')" >> $GITHUB_OUTPUT + else + echo "No projects to build and test." + echo "total_projects=0" >> $GITHUB_OUTPUT + echo "changed_projects=[]" >> $GITHUB_OUTPUT + fi + - name: Generate matrix + id: matrix run: | - declare -a ProjectDirs=($(find . -type d -name "native"| grep -v -f <(grep . .github/.ghaignore | grep -v '^$'))) - echo "Projects to Build:" - printf "%s\n" "${ProjectDirs[@]}" - for projectDir in "${ProjectDirs[@]}"; do - echo " - ******** - Building $projectDir - ********" - cd $projectDir - if pnpm build; then - echo "Build succeeded for $projectDir." - else - failed=true - failed_builds+=($projectDir) - echo "Build failed for $projectDir. Continuing with the next program." - fi - cd - > /dev/null - done - if [ "$failed" = true ]; then - echo "Programs that failed building:" - printf "%s\n" "${failed_builds[@]}" - exit 1 + total_projects=${{ steps.analyze.outputs.total_projects }} + max_jobs=${{ env.MAX_JOBS }} + min_projects_per_job=${{ env.MIN_PROJECTS_PER_JOB }} + min_projects_for_matrix=${{ env.MIN_PROJECTS_FOR_MATRIX }} + + if [ "$total_projects" -lt "$min_projects_for_matrix" ]; then + echo "matrix=[0]" >> $GITHUB_OUTPUT else - echo "All programs built successfully." + projects_per_job=$(( (total_projects + max_jobs - 1) / max_jobs )) + projects_per_job=$(( projects_per_job > min_projects_per_job ? projects_per_job : min_projects_per_job )) + num_jobs=$(( (total_projects + projects_per_job - 1) / projects_per_job )) + + indices=$(seq 0 $(( num_jobs - 1 ))) + echo "matrix=[$(echo $indices | tr ' ' ',')]" >> $GITHUB_OUTPUT fi - shell: bash - test: + build-and-test: + needs: changes + if: needs.changes.outputs.total_projects != '0' runs-on: ubuntu-latest strategy: + fail-fast: false matrix: - node-version: [20.x] - solana-version: [1.18.17, stable] + index: ${{ fromJson(needs.changes.outputs.matrix) }} + name: build-and-test-group-${{ matrix.index }} + outputs: + failed_projects: ${{ steps.set-failed.outputs.failed_projects }} steps: - uses: actions/checkout@v4 - - name: Use Node.js ${{ matrix.node-version }} + - name: Use Node.js uses: actions/setup-node@v4 with: - node-version: ${{ matrix.node-version }} + node-version: 20.x check-latest: true - - uses: heyAyushh/setup-solana@v5.4 - with: - solana-cli-version: ${{ matrix.solana-version }} - - run: solana block - shell: bash - - name: Install pnpm + - name: Setup build environment + id: setup run: | + # Create the build and test function + cat << 'EOF' > build_and_test.sh + function build_and_test() { + local project=$1 + local solana_version=$2 + echo "Building and Testing $project with Solana $solana_version" + cd "$project" || return 1 + + # Install dependencies + if ! pnpm install --frozen-lockfile; then + echo "::error::pnpm install failed for $project" + echo "$project: pnpm install failed with $solana_version" >> $GITHUB_WORKSPACE/failed_projects.txt + cd - > /dev/null + return 1 + fi + + # Build + if ! pnpm build; then + echo "::error::build failed for $project" + echo "$project: build failed with $solana_version" >> $GITHUB_WORKSPACE/failed_projects.txt + cd - > /dev/null + return 1 + fi + + # Test + if ! pnpm build-and-test; then + echo "::error::tests failed for $project" + echo "$project: tests failed with $solana_version" >> $GITHUB_WORKSPACE/failed_projects.txt + cd - > /dev/null + return 1 + fi + + echo "Build and tests succeeded for $project with $solana_version version." + cd - > /dev/null + return 0 + } + + function process_projects() { + local solana_version=$1 + + readarray -t all_projects < <(echo '${{ needs.changes.outputs.changed_projects }}' | jq -r '.[]?') + start_index=$(( ${{ matrix.index }} * ${{ env.MIN_PROJECTS_PER_JOB }} )) + end_index=$(( start_index + ${{ env.MIN_PROJECTS_PER_JOB }} )) + end_index=$(( end_index > ${{ needs.changes.outputs.total_projects }} ? ${{ needs.changes.outputs.total_projects }} : end_index )) + + echo "Projects to build and test in this job" + for i in $(seq $start_index $(( end_index - 1 ))); do + echo "${all_projects[$i]}" + done + + failed=false + for i in $(seq $start_index $(( end_index - 1 ))); do + echo "::group::Building and testing ${all_projects[$i]}" + if ! build_and_test "${all_projects[$i]}" "$solana_version"; then + failed=true + fi + echo "::endgroup::" + done + + return $([ "$failed" = true ] && echo 1 || echo 0) + } + EOF + + # Make the script executable + chmod +x build_and_test.sh + + # Install pnpm npm install --global pnpm - - name: Test solana native programs + - name: Setup Solana stable + uses: heyAyushh/setup-solana@v5.5 + with: + solana-cli-version: stable + - name: Build and Test with Stable + run: | + source build_and_test.sh + solana -V + rustc -V + process_projects "stable" + - name: Setup Solana 1.18.17 + uses: heyAyushh/setup-solana@v5.5 + with: + solana-cli-version: 1.18.17 + - name: Build and Test with 1.18.17 run: | + source build_and_test.sh solana -V rustc -V - declare -a ProjectDirs=($(find . -type d -name "native"| grep -v -f <(grep . .github/.ghaignore | grep -v '^$'))) - echo "Projects to Test:" - printf "%s\n" "${ProjectDirs[@]}" - for projectDir in "${ProjectDirs[@]}"; do - echo " - ******** - Testing $projectDir - ********" - cd $projectDir - pnpm install --frozen-lockfile - if pnpm build-and-test; then - echo "Tests succeeded for $projectDir." + process_projects "1.18.17" + + - name: Set failed projects output + id: set-failed + if: failure() + run: | + if [ -f "$GITHUB_WORKSPACE/failed_projects.txt" ]; then + failed_projects=$(cat $GITHUB_WORKSPACE/failed_projects.txt | jq -R -s -c 'split("\n")[:-1]') + echo "failed_projects=$failed_projects" >> $GITHUB_OUTPUT + else + echo "failed_projects=[]" >> $GITHUB_OUTPUT + fi + + summary: + needs: [changes, build-and-test] + if: always() + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Create job summary + run: | + echo "## Native Workflow Summary" >> $GITHUB_STEP_SUMMARY + echo "- Total projects: ${{ needs.changes.outputs.total_projects }}" >> $GITHUB_STEP_SUMMARY + + # List all processed projects + echo "
" >> $GITHUB_STEP_SUMMARY + echo "Projects processed (click to expand)" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo '${{ needs.changes.outputs.changed_projects }}' | jq -r '.[]' | while read project; do + echo "- $project" >> $GITHUB_STEP_SUMMARY + done + echo "" >> $GITHUB_STEP_SUMMARY + echo "
" >> $GITHUB_STEP_SUMMARY + + # Report build and test results + if [[ "${{ needs.build-and-test.result }}" == "failure" ]]; then + echo "## :x: Build or tests failed" >> $GITHUB_STEP_SUMMARY + echo "
" >> $GITHUB_STEP_SUMMARY + echo "Failed projects (click to expand)" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + failed_projects='${{ needs.build-and-test.outputs.failed_projects }}' + if [[ -n "$failed_projects" ]]; then + echo "$failed_projects" | jq -r '.[]' | while IFS=: read -r project failure_reason; do + echo "- **$project**" >> $GITHUB_STEP_SUMMARY + echo " - Failure reason: $failure_reason" >> $GITHUB_STEP_SUMMARY + done else - failed=true - failed_tests+=($projectDir) - echo "Tests failed for $projectDir. Continuing with the next program." + echo "No failed projects reported. This might indicate an unexpected error in the workflow." >> $GITHUB_STEP_SUMMARY fi - cd - > /dev/null - done - if [ "$failed" = true ]; then - echo "*****************************" - echo "Programs that failed testing:" - printf "%s\n" "${failed_tests[@]}" - exit 1 + echo "" >> $GITHUB_STEP_SUMMARY + echo "
" >> $GITHUB_STEP_SUMMARY + elif [[ "${{ needs.build-and-test.result }}" == "success" ]]; then + echo "## :white_check_mark: All builds and tests passed" >> $GITHUB_STEP_SUMMARY else - echo "All tests passed." + echo "## :warning: Build and test job was skipped or canceled" >> $GITHUB_STEP_SUMMARY fi - shell: bash diff --git a/.github/workflows/steel.yml b/.github/workflows/steel.yml index 84222275..3b64292e 100644 --- a/.github/workflows/steel.yml +++ b/.github/workflows/steel.yml @@ -11,107 +11,246 @@ on: branches: - main +env: + MAX_JOBS: 64 + MIN_PROJECTS_PER_JOB: 4 + MIN_PROJECTS_FOR_MATRIX: 4 + jobs: - build: + changes: runs-on: ubuntu-latest - strategy: - matrix: - node-version: [20.x] - solana-version: [stable] + permissions: + pull-requests: read + outputs: + changed_projects: ${{ steps.analyze.outputs.changed_projects }} + total_projects: ${{ steps.analyze.outputs.total_projects }} + matrix: ${{ steps.matrix.outputs.matrix }} steps: - uses: actions/checkout@v4 - - name: Use Node.js ${{ matrix.node-version }} - uses: actions/setup-node@v4 + - uses: dorny/paths-filter@v3 + id: changes + if: github.event_name == 'pull_request' with: - node-version: ${{ matrix.node-version }} - check-latest: true - - uses: heyAyushh/setup-solana@v5.4 - with: - solana-cli-version: ${{ matrix.solana-version }} - - run: solana block - shell: bash - - name: Install pnpm + list-files: shell + filters: | + steel: + - added|modified: '**/steel/**' + workflow: + - added|modified: '.github/workflows/steel.yml' + - name: Analyze Changes + id: analyze run: | - npm install --global pnpm - - name: Build Steel native programs + # Generate ignore pattern, excluding comments + ignore_pattern=$(grep -v '^#' .github/.ghaignore | grep -v '^$' | tr '\n' '|' | sed 's/|$//') + echo "Ignore pattern: $ignore_pattern" + + function get_projects() { + find . -type d -name "steel" | grep -vE "$ignore_pattern" | sort + } + + # Determine which projects to build and test + if [[ "${{ github.event_name }}" == "push" || "${{ github.event_name }}" == "schedule" || "${{ steps.changes.outputs.workflow }}" == "true" ]]; then + projects=$(get_projects) + elif [[ "${{ steps.changes.outputs.steel }}" == "true" ]]; then + changed_files=(${{ steps.changes.outputs.steel_files }}) + projects=$(for file in "${changed_files[@]}"; do dirname "${file}" | grep steel | sed 's#/steel/.*#/steel#g'; done | grep -vE "$ignore_pattern" | sort -u) + else + projects="" + fi + + # Output project information + if [[ -n "$projects" ]]; then + echo "Projects to build and test" + echo "$projects" + total_projects=$(echo "$projects" | wc -l) + echo "Total projects: $total_projects" + echo "total_projects=$total_projects" >> $GITHUB_OUTPUT + echo "changed_projects=$(echo "$projects" | jq -R -s -c 'split("\n")[:-1]')" >> $GITHUB_OUTPUT + else + echo "No projects to build and test." + echo "total_projects=0" >> $GITHUB_OUTPUT + echo "changed_projects=[]" >> $GITHUB_OUTPUT + fi + - name: Generate matrix + id: matrix run: | - declare -a ProjectDirs=($(find . -type d -name "steel"| grep -v -f <(grep . .github/.ghaignore | grep -v '^$'))) - echo "Projects to Build:" - printf "%s\n" "${ProjectDirs[@]}" - for projectDir in "${ProjectDirs[@]}"; do - echo " - ******** - Building $projectDir - ********" - cd $projectDir - if pnpm build; then - echo "Build succeeded for $projectDir." - else - failed=true - failed_builds+=($projectDir) - echo "Build failed for $projectDir. Continuing with the next program." - fi - cd - > /dev/null - done - if [ "$failed" = true ]; then - echo "Programs that failed building:" - printf "%s\n" "${failed_builds[@]}" - exit 1 + total_projects=${{ steps.analyze.outputs.total_projects }} + max_jobs=${{ env.MAX_JOBS }} + min_projects_per_job=${{ env.MIN_PROJECTS_PER_JOB }} + min_projects_for_matrix=${{ env.MIN_PROJECTS_FOR_MATRIX }} + + if [ "$total_projects" -lt "$min_projects_for_matrix" ]; then + echo "matrix=[0]" >> $GITHUB_OUTPUT else - echo "All programs built successfully." + projects_per_job=$(( (total_projects + max_jobs - 1) / max_jobs )) + projects_per_job=$(( projects_per_job > min_projects_per_job ? projects_per_job : min_projects_per_job )) + num_jobs=$(( (total_projects + projects_per_job - 1) / projects_per_job )) + + indices=$(seq 0 $(( num_jobs - 1 ))) + echo "matrix=[$(echo $indices | tr ' ' ',')]" >> $GITHUB_OUTPUT fi - shell: bash - test: + build-and-test: + needs: changes + if: needs.changes.outputs.total_projects != '0' runs-on: ubuntu-latest strategy: + fail-fast: false matrix: - node-version: [20.x] - solana-version: [1.18.17, stable] + index: ${{ fromJson(needs.changes.outputs.matrix) }} + name: build-and-test-group-${{ matrix.index }} + outputs: + failed_projects: ${{ steps.set-failed.outputs.failed_projects }} steps: - uses: actions/checkout@v4 - - name: Use Node.js ${{ matrix.node-version }} + - name: Use Node.js uses: actions/setup-node@v4 with: - node-version: ${{ matrix.node-version }} + node-version: 20.x check-latest: true - - uses: heyAyushh/setup-solana@v5.4 - with: - solana-cli-version: ${{ matrix.solana-version }} - - run: solana block - shell: bash - - name: Install pnpm + - name: Setup build environment + id: setup run: | + # Create the build and test function + cat << 'EOF' > build_and_test.sh + function build_and_test() { + local project=$1 + local solana_version=$2 + echo "Building and Testing $project with Solana $solana_version" + cd "$project" || return 1 + + # Install dependencies + if ! pnpm install --frozen-lockfile; then + echo "::error::pnpm install failed for $project" + echo "$project: pnpm install failed with $solana_version" >> $GITHUB_WORKSPACE/failed_projects.txt + cd - > /dev/null + return 1 + fi + + # Build + if ! pnpm build; then + echo "::error::build failed for $project" + echo "$project: build failed with $solana_version" >> $GITHUB_WORKSPACE/failed_projects.txt + cd - > /dev/null + return 1 + fi + + # Test + if ! pnpm build-and-test; then + echo "::error::tests failed for $project" + echo "$project: tests failed with $solana_version" >> $GITHUB_WORKSPACE/failed_projects.txt + cd - > /dev/null + return 1 + fi + + echo "Build and tests succeeded for $project with $solana_version version." + cd - > /dev/null + return 0 + } + + function process_projects() { + local solana_version=$1 + + readarray -t all_projects < <(echo '${{ needs.changes.outputs.changed_projects }}' | jq -r '.[]?') + start_index=$(( ${{ matrix.index }} * ${{ env.MIN_PROJECTS_PER_JOB }} )) + end_index=$(( start_index + ${{ env.MIN_PROJECTS_PER_JOB }} )) + end_index=$(( end_index > ${{ needs.changes.outputs.total_projects }} ? ${{ needs.changes.outputs.total_projects }} : end_index )) + + echo "Projects to build and test in this job" + for i in $(seq $start_index $(( end_index - 1 ))); do + echo "${all_projects[$i]}" + done + + failed=false + for i in $(seq $start_index $(( end_index - 1 ))); do + echo "::group::Building and testing ${all_projects[$i]}" + if ! build_and_test "${all_projects[$i]}" "$solana_version"; then + failed=true + fi + echo "::endgroup::" + done + + return $([ "$failed" = true ] && echo 1 || echo 0) + } + EOF + + # Make the script executable + chmod +x build_and_test.sh + + # Install pnpm npm install --global pnpm - - name: Test Steel native programs + - name: Setup Solana stable + uses: heyAyushh/setup-solana@v5.4 + with: + solana-cli-version: stable + - name: Build and Test with Stable + run: | + source build_and_test.sh + solana -V + rustc -V + process_projects "stable" + - name: Setup Solana 1.18.17 + uses: heyAyushh/setup-solana@v5.4 + with: + solana-cli-version: 1.18.17 + - name: Build and Test with 1.18.17 run: | + source build_and_test.sh solana -V rustc -V - declare -a ProjectDirs=($(find . -type d -name "steel"| grep -v -f <(grep . .github/.ghaignore | grep -v '^$'))) - echo "Projects to Test:" - printf "%s\n" "${ProjectDirs[@]}" - for projectDir in "${ProjectDirs[@]}"; do - echo " - ******** - Testing $projectDir - ********" - cd $projectDir - pnpm install --frozen-lockfile - if pnpm build-and-test; then - echo "Tests succeeded for $projectDir." + process_projects "1.18.17" + + - name: Set failed projects output + id: set-failed + if: failure() + run: | + if [ -f "$GITHUB_WORKSPACE/failed_projects.txt" ]; then + failed_projects=$(cat $GITHUB_WORKSPACE/failed_projects.txt | jq -R -s -c 'split("\n")[:-1]') + echo "failed_projects=$failed_projects" >> $GITHUB_OUTPUT + else + echo "failed_projects=[]" >> $GITHUB_OUTPUT + fi + + summary: + needs: [changes, build-and-test] + if: always() + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Create job summary + run: | + echo "## Steel Workflow Summary" >> $GITHUB_STEP_SUMMARY + echo "- Total projects: ${{ needs.changes.outputs.total_projects }}" >> $GITHUB_STEP_SUMMARY + + # List all processed projects + echo "
" >> $GITHUB_STEP_SUMMARY + echo "Projects processed (click to expand)" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo '${{ needs.changes.outputs.changed_projects }}' | jq -r '.[]' | while read project; do + echo "- $project" >> $GITHUB_STEP_SUMMARY + done + echo "" >> $GITHUB_STEP_SUMMARY + echo "
" >> $GITHUB_STEP_SUMMARY + + # Report build and test results + if [[ "${{ needs.build-and-test.result }}" == "failure" ]]; then + echo "## :x: Build or tests failed" >> $GITHUB_STEP_SUMMARY + echo "
" >> $GITHUB_STEP_SUMMARY + echo "Failed projects (click to expand)" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + failed_projects='${{ needs.build-and-test.outputs.failed_projects }}' + if [[ -n "$failed_projects" ]]; then + echo "$failed_projects" | jq -r '.[]' | while IFS=: read -r project failure_reason; do + echo "- **$project**" >> $GITHUB_STEP_SUMMARY + echo " - Failure reason: $failure_reason" >> $GITHUB_STEP_SUMMARY + done else - failed=true - failed_tests+=($projectDir) - echo "Tests failed for $projectDir. Continuing with the next program." + echo "No failed projects reported. This might indicate an unexpected error in the workflow." >> $GITHUB_STEP_SUMMARY fi - cd - > /dev/null - done - if [ "$failed" = true ]; then - echo "*****************************" - echo "Programs that failed testing:" - printf "%s\n" "${failed_tests[@]}" - exit 1 + echo "" >> $GITHUB_STEP_SUMMARY + echo "
" >> $GITHUB_STEP_SUMMARY + elif [[ "${{ needs.build-and-test.result }}" == "success" ]]; then + echo "## :white_check_mark: All builds and tests passed" >> $GITHUB_STEP_SUMMARY else - echo "All tests passed." + echo "## :warning: Build and test job was skipped or canceled" >> $GITHUB_STEP_SUMMARY fi - shell: bash diff --git a/.husky/pre-commit b/.husky/pre-commit index a16d8b1d..2312dc58 100644 --- a/.husky/pre-commit +++ b/.husky/pre-commit @@ -1,4 +1 @@ -#!/usr/bin/env sh -. "$(dirname "$0")/_/husky.sh" - npx lint-staged diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ed0528a3..bf3dab9c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -16,7 +16,7 @@ We welcome contributions in the form of code, documentation, bug reports, featur ## General coding and writing guidelines -Please follow the [Contributing and Style Guide from the Developer Content Repo](https://github.com/solana-foundation/developer-content/blob/main/CONTRIBUTING.md). +Please follow the [Contributing and Style Guide from the Developer Content Repo](https://github.com/solana-foundation/developer-content/blob/main/CONTRIBUTING.md). Specifically for code in this repo: @@ -43,7 +43,7 @@ Specifically for code in this repo: [Biome](https://biomejs.dev/). Execute the following command to format and lint your code at the root of this project before submitting a pull request: ```bash -pnpm check:fix +pnpm fix ``` 7. Some projects can be ignored from the building and testing process by adding the project name to the `.gitignore` file. diff --git a/basics/hello-solana/native/package.json b/basics/hello-solana/native/package.json index e772d7dc..76cd4def 100644 --- a/basics/hello-solana/native/package.json +++ b/basics/hello-solana/native/package.json @@ -1,6 +1,6 @@ { "scripts": { - "test": "pnpm ts-mocha -p ./tsconfig.json -t 1000000 ./tests/test.ts", + "test": "pnpm ts-mocha -p ./tsconfig.json -t 1000000 ./tests/index.test.ts", "build-and-test": "cargo build-sbf --manifest-path=./program/Cargo.toml --sbf-out-dir=./tests/fixtures && pnpm test", "build": "cargo build-sbf --manifest-path=./program/Cargo.toml --sbf-out-dir=./program/target/so", "deploy": "solana program deploy ./program/target/so/hello_solana_program.so" diff --git a/basics/hello-solana/native/tests/test.ts b/basics/hello-solana/native/tests/index.test.ts similarity index 100% rename from basics/hello-solana/native/tests/test.ts rename to basics/hello-solana/native/tests/index.test.ts diff --git a/basics/hello-solana/steel/package.json b/basics/hello-solana/steel/package.json index f054a368..7b16847c 100644 --- a/basics/hello-solana/steel/package.json +++ b/basics/hello-solana/steel/package.json @@ -3,7 +3,7 @@ "version": "1.0.0", "description": "hello world with steel framework for solana", "scripts": { - "test": "pnpm ts-mocha -p ./tsconfig.json -t 1000000 ./tests/test.ts", + "test": "pnpm ts-mocha -p ./tsconfig.json -t 1000000 ./tests/index.test.ts", "build-and-test": "cargo build-sbf --manifest-path=./program/Cargo.toml --sbf-out-dir=./tests/fixtures && pnpm test", "build": "cargo build-sbf --manifest-path=./program/Cargo.toml --sbf-out-dir=./program/target/so", "deploy": "solana program deploy ./program/target/so/hello_solana_program.so" diff --git a/basics/hello-solana/steel/tests/test.ts b/basics/hello-solana/steel/tests/index.test.ts similarity index 100% rename from basics/hello-solana/steel/tests/test.ts rename to basics/hello-solana/steel/tests/index.test.ts