Adding additional test suites. [skip-ci] #66
Workflow file for this run
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -------------------------------------------------------------------- | |
# | |
# Licensed to the Apache Software Foundation (ASF) under one or more | |
# contributor license agreements. See the NOTICE file distributed | |
# with this work for additional information regarding copyright | |
# ownership. The ASF licenses this file to You under the Apache | |
# License, Version 2.0 (the "License"); you may not use this file | |
# except in compliance with the License. You may obtain a copy of the | |
# License at | |
# | |
# http://www.apache.org/licenses/LICENSE-2.0 | |
# | |
# Unless required by applicable law or agreed to in writing, software | |
# distributed under the License is distributed on an "AS IS" BASIS, | |
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or | |
# implied. See the License for the specific language governing | |
# permissions and limitations under the License. | |
# | |
# -------------------------------------------------------------------- | |
# GitHub Actions Workflow: Apache Cloudberry Build Pipeline | |
# -------------------------------------------------------------------- | |
# Description: | |
# | |
# This workflow builds, tests, and packages Apache Cloudberry on | |
# Rocky Linux 9. It ensures artifact integrity, performs installation | |
# tests, validates key operations, and provides detailed test reports, | |
# including handling for ignored test cases. | |
# | |
# Workflow Overview: | |
# 1. **Check Skip**: | |
# - Dynamically determines if the workflow should run based on CI skip flags. | |
# - Evaluates the following fields for skip flags: | |
# - **Pull Request Events**: PR title and PR body. | |
# - **Push Events**: Commit message of the head commit. | |
# - Supports the following skip patterns (case-insensitive): | |
# - `[skip ci]` | |
# - `[ci skip]` | |
# - `[no ci]` | |
# - **Example Usage**: | |
# - Add `[skip ci]` to a commit message, PR title, or body to skip the workflow. | |
# | |
# 2. **Build Job**: | |
# - Configures and builds Apache Cloudberry. | |
# - Runs unit tests and verifies build artifacts. | |
# - Creates RPM packages, source tarballs, and logs. | |
# - **Key Artifacts**: RPM package, source tarball, build logs. | |
# | |
# 3. **RPM Install Test Job**: | |
# - Verifies RPM integrity and installs Cloudberry. | |
# - Validates successful installation. | |
# - **Key Artifacts**: Installation logs, verification results. | |
# | |
# 4. **Test Job (Matrix)**: | |
# - Executes a test matrix to validate different scenarios. | |
# - Creates a demo cluster and runs installcheck tests. | |
# - Parses and reports test results, including failed and ignored tests. | |
# - **Key Features**: | |
# - Regression diffs are displayed if found, aiding quick debugging. | |
# - Both failed and ignored test names are logged and reported. | |
# - **Key Artifacts**: Test logs, regression files, test summaries. | |
# | |
# **Matrix Tests**: | |
# - `ic-good-opt-off`: Basic regression tests with optimizer disabled. | |
# - `ic-expandshrink`: Cluster expansion/shrinking tests. | |
# - `ic-singlenode_isolation`: Isolation tests on a single-node setup. | |
# - `ic-singlenode_isolation2`: Additional isolation tests on single-node. | |
# - `ic-resgroup-v2`: Resource group version 2 tests. | |
# | |
# 5. **Report Job**: | |
# - Aggregates job results into a final report. | |
# - Sends failure notifications if any step fails. | |
# | |
# Execution Environment: | |
# - **Runs On**: Ubuntu-latest with Rocky Linux 9 containers. | |
# - **Resource Requirements**: | |
# - Disk: Minimum 20GB free space. | |
# - Memory: Minimum 8GB RAM. | |
# - CPU: Recommended 4+ cores. | |
# | |
# Triggers: | |
# - Push to `build-devel` branch. | |
# - Pull requests to `build-devel` branch. | |
# - Manual workflow dispatch. | |
# | |
# Container Images: | |
# - **Build**: `apache/incubator-cloudberry:cbdb-build-rocky9-latest` | |
# - **Test**: `apache/incubator-cloudberry:cbdb-test-rocky9-latest` | |
# | |
# Artifacts: | |
# - RPM Package (retention: ${{ env.LOG_RETENTION_DAYS }} days). | |
# - Source Tarball (retention: ${{ env.LOG_RETENTION_DAYS }} days). | |
# - Logs and Test Results (retention: ${{ env.LOG_RETENTION_DAYS }} days). | |
# - Regression Diffs (retention: ${{ env.LOG_RETENTION_DAYS }} days). | |
# | |
# Notes: | |
# - Supports concurrent job execution. | |
# - Includes robust skip logic for pull requests and pushes. | |
# - Handles ignored test cases, ensuring results are comprehensive. | |
# - Provides detailed logs and error handling for failed and ignored tests. | |
# -------------------------------------------------------------------- | |
name: Apache Cloudberry Build | |
on: | |
push: | |
branches: [main] | |
pull_request: | |
branches: [main] | |
workflow_dispatch: | |
concurrency: | |
group: ${{ github.workflow }}-${{ github.ref }} | |
cancel-in-progress: false | |
# Note: Step details, logs, and artifacts require users to be logged into GitHub | |
# even for public repositories. This is a GitHub security feature and cannot | |
# be overridden by permissions. | |
permissions: | |
# READ permissions allow viewing repository contents | |
contents: read # Required for checking out code and reading repository files | |
# READ permissions for packages (Container registry, etc) | |
packages: read # Allows reading from GitHub package registry | |
# WRITE permissions for actions includes read access to: | |
# - Workflow runs | |
# - Artifacts (requires GitHub login) | |
# - Logs (requires GitHub login) | |
actions: write | |
# READ permissions for checks API: | |
# - Step details visibility (requires GitHub login) | |
# - Check run status and details | |
checks: read | |
# READ permissions for pull request metadata: | |
# - PR status | |
# - Associated checks | |
# - Review states | |
pull-requests: read | |
env: | |
LOG_RETENTION_DAYS: 7 | |
jobs: | |
## ====================================================================== | |
## Job: check-skip | |
## ====================================================================== | |
check-skip: | |
runs-on: ubuntu-latest | |
outputs: | |
should_skip: ${{ steps.skip-check.outputs.should_skip }} | |
steps: | |
- id: skip-check | |
shell: bash | |
env: | |
EVENT_NAME: ${{ github.event_name }} | |
PR_TITLE: ${{ github.event.pull_request.title || '' }} | |
PR_BODY: ${{ github.event.pull_request.body || '' }} | |
run: | | |
# Default to not skipping | |
echo "should_skip=false" >> "$GITHUB_OUTPUT" | |
# Apply skip logic only for pull_request events | |
if [[ "$EVENT_NAME" == "pull_request" ]]; then | |
# Combine PR title and body for skip check | |
MESSAGE="${PR_TITLE}\n${PR_BODY}" | |
# Escape special characters using printf %s | |
ESCAPED_MESSAGE=$(printf "%s" "$MESSAGE") | |
echo "Checking PR title and body (escaped): $ESCAPED_MESSAGE" | |
# Check for skip patterns | |
if echo -e "$ESCAPED_MESSAGE" | grep -qEi '\[skip[ -]ci\]|\[ci[ -]skip\]|\[no[ -]ci\]'; then | |
echo "should_skip=true" >> "$GITHUB_OUTPUT" | |
fi | |
else | |
echo "Skip logic is not applied for $EVENT_NAME events." | |
fi | |
- name: Report Skip Status | |
if: steps.skip-check.outputs.should_skip == 'true' | |
run: | | |
echo "CI Skip flag detected in PR - skipping all checks." | |
exit 0 | |
## ====================================================================== | |
## Job: build | |
## ====================================================================== | |
build: | |
name: Build Apache Cloudberry | |
env: | |
JOB_TYPE: build | |
needs: check-skip | |
runs-on: ubuntu-latest | |
timeout-minutes: 120 | |
outputs: | |
build_timestamp: ${{ steps.set_timestamp.outputs.timestamp }} | |
container: | |
image: apache/incubator-cloudberry:cbdb-build-rocky9-latest | |
options: >- | |
--user root | |
-h cdw | |
steps: | |
- name: Skip Check | |
if: needs.check-skip.outputs.should_skip == 'true' | |
run: | | |
echo "Build skipped via CI skip flag" >> "$GITHUB_STEP_SUMMARY" | |
exit 0 | |
- name: Set build timestamp | |
if: needs.check-skip.outputs.should_skip != 'true' | |
id: set_timestamp # Add an ID to reference this step | |
run: | | |
timestamp=$(date +'%Y%m%d_%H%M%S') | |
echo "timestamp=$timestamp" | tee -a "$GITHUB_OUTPUT" # Use GITHUB_OUTPUT for job outputs | |
echo "BUILD_TIMESTAMP=$timestamp" | tee -a "$GITHUB_ENV" # Also set as environment variable | |
- name: Checkout Apache Cloudberry (apache/cloudberry) | |
if: needs.check-skip.outputs.should_skip != 'true' | |
uses: actions/checkout@v4 | |
with: | |
repository: apache/cloudberry | |
fetch-depth: 1 | |
- name: Checkout CI Build/Test Scripts (apache/cloudberry-devops-release) | |
if: needs.check-skip.outputs.should_skip != 'true' | |
uses: actions/checkout@v4 | |
with: | |
repository: apache/cloudberry-devops-release | |
ref: main | |
path: cloudberry-devops-release | |
- name: Move cloudberry-devops-release directory | |
if: needs.check-skip.outputs.should_skip != 'true' | |
run: | | |
set -eo pipefail | |
if ! mv "${GITHUB_WORKSPACE}"/cloudberry-devops-release "${GITHUB_WORKSPACE}"/..; then | |
echo "::error::Container initialization failed" | |
exit 1 | |
fi | |
- name: Cloudberry Environment Initialization | |
if: needs.check-skip.outputs.should_skip != 'true' | |
env: | |
LOGS_DIR: build-logs | |
run: | | |
set -eo pipefail | |
if ! su - gpadmin -c "/tmp/init_system.sh"; then | |
echo "::error::Container initialization failed" | |
exit 1 | |
fi | |
mkdir -p "${LOGS_DIR}/details" | |
chown -R gpadmin:gpadmin . | |
chmod -R 755 . | |
chmod 777 "${LOGS_DIR}" | |
df -kh / | |
rm -rf /__t/* | |
df -kh / | |
df -h | tee -a "${LOGS_DIR}/details/disk-usage.log" | |
free -h | tee -a "${LOGS_DIR}/details/memory-usage.log" | |
{ | |
echo "=== Environment Information ===" | |
uname -a | |
df -h | |
free -h | |
env | |
} | tee -a "${LOGS_DIR}/details/environment.log" | |
echo "SRC_DIR=${GITHUB_WORKSPACE}" | tee -a "$GITHUB_ENV" | |
- name: Generate Build Job Summary Start | |
if: needs.check-skip.outputs.should_skip != 'true' | |
run: | | |
{ | |
echo "# Build Job Summary" | |
echo "## Environment" | |
echo "- Start Time: $(date -u +'%Y-%m-%d %H:%M:%S UTC')" | |
echo "- OS Version: $(cat /etc/redhat-release)" | |
echo "- GCC Version: $(gcc --version | head -n1)" | |
} >> "$GITHUB_STEP_SUMMARY" | |
- name: Run Apache Cloudberry configure script | |
if: needs.check-skip.outputs.should_skip != 'true' | |
env: | |
SRC_DIR: ${{ github.workspace }} | |
run: | | |
set -eo pipefail | |
chmod +x "${SRC_DIR}"/../cloudberry-devops-release/build_automation/cloudberry/scripts/configure-cloudberry.sh | |
if ! time su - gpadmin -c "cd ${SRC_DIR} && SRC_DIR=${SRC_DIR} ${SRC_DIR}/../cloudberry-devops-release/build_automation/cloudberry/scripts/configure-cloudberry.sh"; then | |
echo "::error::Configure script failed" | |
exit 1 | |
fi | |
- name: Run Apache Cloudberry build script | |
if: needs.check-skip.outputs.should_skip != 'true' | |
env: | |
SRC_DIR: ${{ github.workspace }} | |
run: | | |
set -eo pipefail | |
chmod +x "${SRC_DIR}"/../cloudberry-devops-release/build_automation/cloudberry/scripts/build-cloudberry.sh | |
if ! time su - gpadmin -c "cd ${SRC_DIR} && SRC_DIR=${SRC_DIR} ${SRC_DIR}/../cloudberry-devops-release/build_automation/cloudberry/scripts/build-cloudberry.sh"; then | |
echo "::error::Build script failed" | |
exit 1 | |
fi | |
- name: Verify build artifacts | |
if: needs.check-skip.outputs.should_skip != 'true' | |
run: | | |
set -eo pipefail | |
echo "Verifying build artifacts..." | |
{ | |
echo "=== Build Artifacts Verification ===" | |
echo "Timestamp: $(date -u)" | |
if [ ! -d "/usr/local/cloudberry-db" ]; then | |
echo "::error::Build artifacts directory not found" | |
exit 1 | |
fi | |
# Verify critical binaries | |
critical_binaries=( | |
"/usr/local/cloudberry-db/bin/postgres" | |
"/usr/local/cloudberry-db/bin/psql" | |
) | |
echo "Checking critical binaries..." | |
for binary in "${critical_binaries[@]}"; do | |
if [ ! -f "$binary" ]; then | |
echo "::error::Critical binary missing: $binary" | |
exit 1 | |
fi | |
if [ ! -x "$binary" ]; then | |
echo "::error::Binary not executable: $binary" | |
exit 1 | |
fi | |
echo "Binary verified: $binary" | |
ls -l "$binary" | |
done | |
# Test binary execution | |
echo "Testing binary execution..." | |
if ! /usr/local/cloudberry-db/bin/postgres --version; then | |
echo "::error::postgres binary verification failed" | |
exit 1 | |
fi | |
if ! /usr/local/cloudberry-db/bin/psql --version; then | |
echo "::error::psql binary verification failed" | |
exit 1 | |
fi | |
echo "All build artifacts verified successfully" | |
} 2>&1 | tee -a build-logs/details/build-verification.log | |
- name: Create Source tarball, create RPM and verify artifacts | |
if: needs.check-skip.outputs.should_skip != 'true' | |
env: | |
CBDB_VERSION: 99.0.0 | |
BUILD_NUMBER: 1 | |
SRC_DIR: ${{ github.workspace }} | |
run: | | |
set -eo pipefail | |
{ | |
echo "=== Artifact Creation Log ===" | |
echo "Timestamp: $(date -u)" | |
# Create source tarball | |
echo "Creating source tarball..." | |
tar czf "${SRC_DIR}"/../apache-cloudberry-incubating-src.tgz -C "${SRC_DIR}"/.. ./cloudberry | |
mv "${SRC_DIR}"/../apache-cloudberry-incubating-src.tgz "${SRC_DIR}" | |
# Verify tarball contents | |
echo "Verifying source tarball contents..." | |
if ! tar tzf "${SRC_DIR}"/apache-cloudberry-incubating-src.tgz > /dev/null; then | |
echo "::error::Source tarball verification failed" | |
exit 1 | |
fi | |
# Create RPM | |
echo "Creating RPM package..." | |
rpmdev-setuptree | |
ln -s "${SRC_DIR}"/../cloudberry-devops-release/packaging/rpm/el/SPECS/apache-cloudberry-db-incubating.spec "${HOME}"/rpmbuild/SPECS/apache-cloudberry-db-incubating.spec | |
cp "${SRC_DIR}"/LICENSE /usr/local/cloudberry-db | |
"${SRC_DIR}"/../cloudberry-devops-release/scripts/build-rpm.sh --version "${CBDB_VERSION}" --release "${BUILD_NUMBER}" | |
# Get OS version and move RPM | |
os_version=$(grep -oP '(?<=^VERSION_ID=")[0-9]' /etc/os-release) | |
RPM_FILE="${HOME}"/rpmbuild/RPMS/x86_64/apache-cloudberry-db-incubating-"${CBDB_VERSION}"-"${BUILD_NUMBER}".el"${os_version}".x86_64.rpm | |
cp "${RPM_FILE}" "${SRC_DIR}" | |
# Get package information | |
echo "Package Information:" | |
rpm -qip "${RPM_FILE}" | |
# Verify critical files in RPM | |
echo "Verifying critical files in RPM..." | |
for binary in "bin/postgres" "bin/psql"; do | |
if ! rpm -qlp "${RPM_FILE}" | grep -q "${binary}$"; then | |
echo "::error::Critical binary '${binary}' not found in RPM" | |
exit 1 | |
fi | |
done | |
# Record checksums | |
echo "Calculating checksums..." | |
sha256sum "${RPM_FILE}" | tee -a build-logs/details/checksums.log | |
sha256sum "${SRC_DIR}"/apache-cloudberry-incubating-src.tgz | tee -a build-logs/details/checksums.log | |
echo "Artifacts created and verified successfully" | |
} 2>&1 | tee -a build-logs/details/artifact-creation.log | |
- name: Run Apache Cloudberry unittest script | |
if: needs.check-skip.outputs.should_skip != 'true' | |
env: | |
SRC_DIR: ${{ github.workspace }} | |
run: | | |
set -eo pipefail | |
chmod +x "${SRC_DIR}"/../cloudberry-devops-release/build_automation/cloudberry/scripts/unittest-cloudberry.sh | |
if ! time su - gpadmin -c "cd ${SRC_DIR} && SRC_DIR=${SRC_DIR} ${SRC_DIR}/../cloudberry-devops-release/build_automation/cloudberry/scripts/unittest-cloudberry.sh"; then | |
echo "::error::Unittest script failed" | |
exit 1 | |
fi | |
- name: Generate Build Job Summary End | |
if: always() | |
run: | | |
{ | |
echo "## Build Results" | |
echo "- End Time: $(date -u +'%Y-%m-%d %H:%M:%S UTC')" | |
} >> "$GITHUB_STEP_SUMMARY" | |
- name: Upload build logs | |
if: needs.check-skip.outputs.should_skip != 'true' | |
uses: actions/upload-artifact@v4 | |
with: | |
name: build-logs-${{ env.BUILD_TIMESTAMP }} | |
path: | | |
build-logs/ | |
retention-days: ${{ env.LOG_RETENTION_DAYS }} | |
- name: Upload Cloudberry RPM build artifacts | |
if: needs.check-skip.outputs.should_skip != 'true' | |
uses: actions/upload-artifact@v4 | |
with: | |
name: apache-cloudberry-db-incubating-rpm-build-artifacts | |
retention-days: ${{ env.LOG_RETENTION_DAYS }} | |
if-no-files-found: error | |
path: | | |
*.rpm | |
- name: Upload Cloudberry source build artifacts | |
if: needs.check-skip.outputs.should_skip != 'true' | |
uses: actions/upload-artifact@v4 | |
with: | |
name: apache-cloudberry-db-incubating-source-build-artifacts | |
retention-days: ${{ env.LOG_RETENTION_DAYS }} | |
if-no-files-found: error | |
path: | | |
apache-cloudberry-incubating-src.tgz | |
## ====================================================================== | |
## Job: rpm-install-test | |
## ====================================================================== | |
rpm-install-test: | |
name: RPM Install Test Apache Cloudberry | |
needs: [check-skip, build] | |
runs-on: ubuntu-latest | |
timeout-minutes: 120 | |
container: | |
image: apache/incubator-cloudberry:cbdb-test-rocky9-latest | |
options: >- | |
--user root | |
-h cdw | |
steps: | |
- name: Skip Check | |
if: needs.check-skip.outputs.should_skip == 'true' | |
run: | | |
echo "RPM install test skipped via CI skip flag" >> "$GITHUB_STEP_SUMMARY" | |
exit 0 | |
- name: Download Cloudberry RPM build artifacts | |
if: needs.check-skip.outputs.should_skip != 'true' | |
uses: actions/download-artifact@v4 | |
with: | |
name: apache-cloudberry-db-incubating-rpm-build-artifacts | |
path: ${{ github.workspace }}/rpm_build_artifacts | |
merge-multiple: false | |
- name: Cloudberry Environment Initialization | |
if: needs.check-skip.outputs.should_skip != 'true' | |
env: | |
LOGS_DIR: install-logs | |
run: | | |
set -eo pipefail | |
if ! su - gpadmin -c "/tmp/init_system.sh"; then | |
echo "::error::Container initialization failed" | |
exit 1 | |
fi | |
mkdir -p "${LOGS_DIR}/details" | |
chown -R gpadmin:gpadmin . | |
chmod -R 755 . | |
chmod 777 "${LOGS_DIR}" | |
df -kh / | |
rm -rf /__t/* | |
df -kh / | |
df -h | tee -a "${LOGS_DIR}/details/disk-usage.log" | |
free -h | tee -a "${LOGS_DIR}/details/memory-usage.log" | |
{ | |
echo "=== Environment Information ===" | |
uname -a | |
df -h | |
free -h | |
env | |
} | tee -a "${LOGS_DIR}/details/environment.log" | |
echo "SRC_DIR=${GITHUB_WORKSPACE}" | tee -a "$GITHUB_ENV" | |
- name: Verify RPM artifacts | |
if: needs.check-skip.outputs.should_skip != 'true' | |
id: verify-artifacts | |
run: | | |
set -eo pipefail | |
RPM_FILE=$(ls "${GITHUB_WORKSPACE}"/rpm_build_artifacts/apache-cloudberry-db-incubating*.rpm) | |
if [ ! -f "${RPM_FILE}" ]; then | |
echo "::error::RPM file not found" | |
exit 1 | |
fi | |
echo "rpm_file=${RPM_FILE}" >> "$GITHUB_OUTPUT" | |
echo "Verifying RPM artifacts..." | |
{ | |
echo "=== RPM Verification Summary ===" | |
echo "Timestamp: $(date -u)" | |
echo "RPM File: ${RPM_FILE}" | |
# Get RPM metadata and verify contents | |
echo "Package Information:" | |
rpm -qip "${RPM_FILE}" | |
# Get key RPM attributes for verification | |
RPM_VERSION=$(rpm -qp --queryformat "%{VERSION}" "${RPM_FILE}") | |
RPM_RELEASE=$(rpm -qp --queryformat "%{RELEASE}" "${RPM_FILE}") | |
echo "version=${RPM_VERSION}" >> "$GITHUB_OUTPUT" | |
echo "release=${RPM_RELEASE}" >> "$GITHUB_OUTPUT" | |
# Verify expected binaries are in the RPM | |
echo "Verifying critical files in RPM..." | |
for binary in "bin/postgres" "bin/psql"; do | |
if ! rpm -qlp "${RPM_FILE}" | grep -q "${binary}$"; then | |
echo "::error::Critical binary '${binary}' not found in RPM" | |
exit 1 | |
fi | |
done | |
echo "RPM Details:" | |
echo "- Version: ${RPM_VERSION}" | |
echo "- Release: ${RPM_RELEASE}" | |
# Calculate and store checksum | |
echo "Checksum:" | |
sha256sum "${RPM_FILE}" | |
} 2>&1 | tee -a install-logs/details/rpm-verification.log | |
- name: Install Cloudberry RPM | |
if: success() && needs.check-skip.outputs.should_skip != 'true' | |
env: | |
RPM_FILE: ${{ steps.verify-artifacts.outputs.rpm_file }} | |
RPM_VERSION: ${{ steps.verify-artifacts.outputs.version }} | |
RPM_RELEASE: ${{ steps.verify-artifacts.outputs.release }} | |
run: | | |
set -eo pipefail | |
if [ -z "${RPM_FILE}" ]; then | |
echo "::error::RPM_FILE environment variable is not set" | |
exit 1 | |
fi | |
{ | |
echo "=== RPM Installation Log ===" | |
echo "Timestamp: $(date -u)" | |
echo "RPM File: ${RPM_FILE}" | |
echo "Version: ${RPM_VERSION}" | |
echo "Release: ${RPM_RELEASE}" | |
# Clean install location | |
rm -rf /usr/local/cloudberry-db | |
# Install RPM | |
echo "Starting installation..." | |
if ! time dnf install -y "${RPM_FILE}"; then | |
echo "::error::RPM installation failed" | |
exit 1 | |
fi | |
echo "Installation completed successfully" | |
rpm -qi apache-cloudberry-db-incubating | |
echo "Installed files:" | |
rpm -ql apache-cloudberry-db-incubating | |
} 2>&1 | tee -a install-logs/details/rpm-installation.log | |
- name: Upload install logs | |
if: needs.check-skip.outputs.should_skip != 'true' | |
uses: actions/upload-artifact@v4 | |
with: | |
name: install-logs-${{ needs.build.outputs.build_timestamp }} | |
path: | | |
install-logs/ | |
retention-days: ${{ env.LOG_RETENTION_DAYS }} | |
- name: Generate Install Test Job Summary End | |
if: always() | |
shell: bash {0} | |
run: | | |
{ | |
echo "# Installed Package Summary" | |
echo "\`\`\`" | |
rpm -qi apache-cloudberry-db-incubating | |
echo "\`\`\`" | |
} >> "$GITHUB_STEP_SUMMARY" || true | |
## ====================================================================== | |
## Job: test | |
## ====================================================================== | |
test: | |
name: ${{ matrix.test }} | |
needs: [check-skip, build] | |
runs-on: ubuntu-latest | |
timeout-minutes: 120 | |
strategy: | |
fail-fast: false # Continue with other tests if one fails | |
matrix: | |
test: | |
- ic-good-opt-off | |
- ic-expandshrink | |
- ic-singlenode_isolation | |
- ic-singlenode_isolation2 | |
- ic-resgroup-v2 | |
include: | |
- test: ic-good-opt-off | |
make_target: installcheck-good | |
make_directory: --directory=src/test/regress | |
num_primary_mirror_pairs: 3 | |
pg_settings: | |
optimizer: "off" | |
- test: ic-expandshrink | |
make_target: installcheck-expandshrink | |
make_directory: --directory=src/test/isolation2 | |
num_primary_mirror_pairs: 3 | |
- test: ic-singlenode_isolation | |
make_target: installcheck-singlenode | |
make_directory: --directory=src/test/isolation | |
num_primary_mirror_pairs: 0 | |
- test: ic-singlenode_isolation2 | |
make_target: installcheck-singlenode | |
make_directory: --directory=src/test/singlenode_isolation2 | |
num_primary_mirror_pairs: 0 | |
- test: ic-resgroup-v2 | |
make_target: installcheck-resgroup-v2 | |
make_directory: --directory=src/test/isolation2 | |
num_primary_mirror_pairs: 3 | |
container: | |
image: apache/incubator-cloudberry:cbdb-build-rocky9-latest | |
options: >- | |
--privileged | |
--user root | |
--hostname cdw | |
--shm-size=2gb | |
--cgroupns=host | |
-v /sys/fs/cgroup:/sys/fs/cgroup:rw | |
steps: | |
- name: Skip Check | |
if: needs.check-skip.outputs.should_skip == 'true' | |
run: | | |
echo "Test ${{ matrix.test }} skipped via CI skip flag" >> "$GITHUB_STEP_SUMMARY" | |
exit 0 | |
- name: Use timestamp from previous job | |
if: needs.check-skip.outputs.should_skip != 'true' | |
run: | | |
echo "Timestamp from output: ${{ needs.build.outputs.build_timestamp }}" | |
- name: Checkout CI Build/Test Scripts (apache/cloudberry-devops-release) | |
if: needs.check-skip.outputs.should_skip != 'true' | |
uses: actions/checkout@v4 | |
with: | |
repository: apache/cloudberry-devops-release | |
ref: main | |
path: cloudberry-devops-release | |
- name: Move cloudberry-devops-release directory | |
if: needs.check-skip.outputs.should_skip != 'true' | |
run: | | |
set -eo pipefail | |
if ! mv "${GITHUB_WORKSPACE}"/cloudberry-devops-release "${GITHUB_WORKSPACE}"/..; then | |
echo "::error::Container initialization failed" | |
exit 1 | |
fi | |
- name: Cloudberry Environment Initialization | |
env: | |
LOGS_DIR: build-logs | |
run: | | |
set -eo pipefail | |
if ! su - gpadmin -c "/tmp/init_system.sh"; then | |
echo "::error::Container initialization failed" | |
exit 1 | |
fi | |
mkdir -p "${LOGS_DIR}/details" | |
chown -R gpadmin:gpadmin . | |
chmod -R 755 . | |
chmod 777 "${LOGS_DIR}" | |
df -kh / | |
rm -rf /__t/* | |
df -kh / | |
df -h | tee -a "${LOGS_DIR}/details/disk-usage.log" | |
free -h | tee -a "${LOGS_DIR}/details/memory-usage.log" | |
{ | |
echo "=== Environment Information ===" | |
uname -a | |
df -h | |
free -h | |
env | |
} | tee -a "${LOGS_DIR}/details/environment.log" | |
echo "SRC_DIR=${GITHUB_WORKSPACE}" | tee -a "$GITHUB_ENV" | |
- name: Setup cgroups | |
if: needs.check-skip.outputs.should_skip != 'true' | |
shell: bash | |
run: | | |
set -uxo pipefail | |
echo "Current mounts:" | |
mount | grep cgroup | |
CGROUP_BASEDIR=/sys/fs/cgroup | |
# 1. Basic setup with permissions | |
sudo chmod -R 777 ${CGROUP_BASEDIR}/ | |
sudo mkdir -p ${CGROUP_BASEDIR}/gpdb | |
sudo chmod -R 777 ${CGROUP_BASEDIR}/gpdb | |
sudo chown -R gpadmin:gpadmin ${CGROUP_BASEDIR}/gpdb | |
# 2. Enable controllers | |
sudo bash -c "echo '+cpu +cpuset +memory +io' > ${CGROUP_BASEDIR}/cgroup.subtree_control" || true | |
sudo bash -c "echo '+cpu +cpuset +memory +io' > ${CGROUP_BASEDIR}/gpdb/cgroup.subtree_control" || true | |
# 3. CPU settings | |
sudo bash -c "echo 'max 100000' > ${CGROUP_BASEDIR}/gpdb/cpu.max" || true | |
sudo bash -c "echo '100' > ${CGROUP_BASEDIR}/gpdb/cpu.weight" || true | |
sudo bash -c "echo '0' > ${CGROUP_BASEDIR}/gpdb/cpu.weight.nice" || true | |
sudo bash -c "echo 0-$(( $(nproc) - 1 )) > ${CGROUP_BASEDIR}/gpdb/cpuset.cpus" || true | |
sudo bash -c "echo '0' > ${CGROUP_BASEDIR}/gpdb/cpuset.mems" || true | |
# 4. Memory settings | |
sudo bash -c "echo 'max' > ${CGROUP_BASEDIR}/gpdb/memory.max" || true | |
sudo bash -c "echo '0' > ${CGROUP_BASEDIR}/gpdb/memory.min" || true | |
sudo bash -c "echo 'max' > ${CGROUP_BASEDIR}/gpdb/memory.high" || true | |
# 5. IO settings | |
echo "Available block devices:" | |
lsblk | |
sudo bash -c " | |
if [ -f \${CGROUP_BASEDIR}/gpdb/io.stat ]; then | |
echo 'Detected IO devices:' | |
cat \${CGROUP_BASEDIR}/gpdb/io.stat | |
fi | |
echo '' > \${CGROUP_BASEDIR}/gpdb/io.max || true | |
" | |
# 6. Fix permissions again after all writes | |
sudo chmod -R 777 ${CGROUP_BASEDIR}/gpdb | |
sudo chown -R gpadmin:gpadmin ${CGROUP_BASEDIR}/gpdb | |
# 7. Check required files | |
echo "Checking required files:" | |
required_files=( | |
"cgroup.procs" | |
"cpu.max" | |
"cpu.pressure" | |
"cpu.weight" | |
"cpu.weight.nice" | |
"cpu.stat" | |
"cpuset.cpus" | |
"cpuset.mems" | |
"cpuset.cpus.effective" | |
"cpuset.mems.effective" | |
"memory.current" | |
"io.max" | |
) | |
for file in "${required_files[@]}"; do | |
if [ -f "${CGROUP_BASEDIR}/gpdb/$file" ]; then | |
echo "✓ $file exists" | |
ls -l "${CGROUP_BASEDIR}/gpdb/$file" | |
else | |
echo "✗ $file missing" | |
fi | |
done | |
# 8. Test subdirectory creation | |
echo "Testing subdirectory creation..." | |
sudo -u gpadmin bash -c " | |
TEST_DIR=\${CGROUP_BASEDIR}/gpdb/test6448 | |
if mkdir -p \$TEST_DIR; then | |
echo 'Created test directory' | |
sudo chmod -R 777 \$TEST_DIR | |
if echo \$\$ > \$TEST_DIR/cgroup.procs; then | |
echo 'Successfully wrote to cgroup.procs' | |
cat \$TEST_DIR/cgroup.procs | |
# Move processes back to parent before cleanup | |
echo \$\$ > \${CGROUP_BASEDIR}/gpdb/cgroup.procs | |
else | |
echo 'Failed to write to cgroup.procs' | |
ls -la \$TEST_DIR/cgroup.procs | |
fi | |
ls -la \$TEST_DIR/ | |
rmdir \$TEST_DIR || { | |
echo 'Moving all processes to parent before cleanup' | |
cat \$TEST_DIR/cgroup.procs | while read pid; do | |
echo \$pid > \${CGROUP_BASEDIR}/gpdb/cgroup.procs 2>/dev/null || true | |
done | |
rmdir \$TEST_DIR | |
} | |
else | |
echo 'Failed to create test directory' | |
fi | |
" | |
# 9. Verify setup as gpadmin user | |
echo "Testing cgroup access as gpadmin..." | |
sudo -u gpadmin bash -c " | |
echo 'Checking mounts...' | |
mount | grep cgroup | |
echo 'Checking /proc/self/mounts...' | |
cat /proc/self/mounts | grep cgroup | |
if ! grep -q cgroup2 /proc/self/mounts; then | |
echo 'ERROR: cgroup2 mount NOT visible to gpadmin' | |
exit 1 | |
fi | |
echo 'SUCCESS: cgroup2 mount visible to gpadmin' | |
if ! [ -w ${CGROUP_BASEDIR}/gpdb ]; then | |
echo 'ERROR: gpadmin cannot write to gpdb cgroup' | |
exit 1 | |
fi | |
echo 'SUCCESS: gpadmin can write to gpdb cgroup' | |
echo 'Verifying key files content:' | |
echo 'cpu.max:' | |
cat ${CGROUP_BASEDIR}/gpdb/cpu.max || echo 'Failed to read cpu.max' | |
echo 'cpuset.cpus:' | |
cat ${CGROUP_BASEDIR}/gpdb/cpuset.cpus || echo 'Failed to read cpuset.cpus' | |
echo 'cgroup.subtree_control:' | |
cat ${CGROUP_BASEDIR}/gpdb/cgroup.subtree_control || echo 'Failed to read cgroup.subtree_control' | |
" | |
# 10. Show final state | |
echo "Final cgroup state:" | |
ls -la ${CGROUP_BASEDIR}/gpdb/ | |
echo "Cgroup setup completed successfully" | |
- name: "Generate Test Job Summary Start: ${{ matrix.test }}" | |
if: always() | |
run: | | |
{ | |
echo "# Test Job Summary: ${{ matrix.test }}" | |
echo "## Environment" | |
echo "- Start Time: $(date -u +'%Y-%m-%d %H:%M:%S UTC')" | |
if [[ "${{ needs.check-skip.outputs.should_skip }}" == "true" ]]; then | |
echo "## Skip Status" | |
echo "✓ Test execution skipped via CI skip flag" | |
else | |
echo "- OS Version: $(cat /etc/redhat-release)" | |
fi | |
} >> "$GITHUB_STEP_SUMMARY" | |
- name: Download Cloudberry RPM build artifacts | |
if: needs.check-skip.outputs.should_skip != 'true' | |
uses: actions/download-artifact@v4 | |
with: | |
name: apache-cloudberry-db-incubating-rpm-build-artifacts | |
path: ${{ github.workspace }}/rpm_build_artifacts | |
merge-multiple: false | |
- name: Download Cloudberry Source build artifacts | |
if: needs.check-skip.outputs.should_skip != 'true' | |
uses: actions/download-artifact@v4 | |
with: | |
name: apache-cloudberry-db-incubating-source-build-artifacts | |
path: ${{ github.workspace }}/source_build_artifacts | |
merge-multiple: false | |
- name: Verify downloaded artifacts | |
if: needs.check-skip.outputs.should_skip != 'true' | |
id: verify-artifacts | |
run: | | |
set -eo pipefail | |
SRC_TARBALL_FILE=$(ls "${GITHUB_WORKSPACE}"/source_build_artifacts/apache-cloudberry-incubating-src.tgz) | |
if [ ! -f "${SRC_TARBALL_FILE}" ]; then | |
echo "::error::SRC TARBALL file not found" | |
exit 1 | |
fi | |
echo "src_tarball_file=${SRC_TARBALL_FILE}" >> "$GITHUB_OUTPUT" | |
echo "Verifying SRC TARBALL artifacts..." | |
{ | |
echo "=== SRC TARBALL Verification Summary ===" | |
echo "Timestamp: $(date -u)" | |
echo "SRC TARBALL File: ${SRC_TARBALL_FILE}" | |
# Calculate and store checksum | |
echo "Checksum:" | |
sha256sum "${SRC_TARBALL_FILE}" | |
} 2>&1 | tee -a build-logs/details/src-tarball-verification.log | |
RPM_FILE=$(ls "${GITHUB_WORKSPACE}"/rpm_build_artifacts/apache-cloudberry-db-incubating*.rpm) | |
if [ ! -f "${RPM_FILE}" ]; then | |
echo "::error::RPM file not found" | |
exit 1 | |
fi | |
echo "rpm_file=${RPM_FILE}" >> "$GITHUB_OUTPUT" | |
echo "Verifying RPM artifacts..." | |
{ | |
echo "=== RPM Verification Summary ===" | |
echo "Timestamp: $(date -u)" | |
echo "RPM File: ${RPM_FILE}" | |
# Get RPM metadata and verify contents | |
echo "Package Information:" | |
rpm -qip "${RPM_FILE}" | |
# Get key RPM attributes for verification | |
RPM_VERSION=$(rpm -qp --queryformat "%{VERSION}" "${RPM_FILE}") | |
RPM_RELEASE=$(rpm -qp --queryformat "%{RELEASE}" "${RPM_FILE}") | |
echo "version=${RPM_VERSION}" >> "$GITHUB_OUTPUT" | |
echo "release=${RPM_RELEASE}" >> "$GITHUB_OUTPUT" | |
# Verify expected binaries are in the RPM | |
echo "Verifying critical files in RPM..." | |
for binary in "bin/postgres" "bin/psql"; do | |
if ! rpm -qlp "${RPM_FILE}" | grep -q "${binary}$"; then | |
echo "::error::Critical binary '${binary}' not found in RPM" | |
exit 1 | |
fi | |
done | |
echo "RPM Details:" | |
echo "- Version: ${RPM_VERSION}" | |
echo "- Release: ${RPM_RELEASE}" | |
# Calculate and store checksum | |
echo "Checksum:" | |
sha256sum "${RPM_FILE}" | |
} 2>&1 | tee -a build-logs/details/rpm-verification.log | |
- name: Install Cloudberry RPM | |
if: success() && needs.check-skip.outputs.should_skip != 'true' | |
env: | |
RPM_FILE: ${{ steps.verify-artifacts.outputs.rpm_file }} | |
RPM_VERSION: ${{ steps.verify-artifacts.outputs.version }} | |
RPM_RELEASE: ${{ steps.verify-artifacts.outputs.release }} | |
run: | | |
set -eo pipefail | |
if [ -z "${RPM_FILE}" ]; then | |
echo "::error::RPM_FILE environment variable is not set" | |
exit 1 | |
fi | |
{ | |
echo "=== RPM Installation Log ===" | |
echo "Timestamp: $(date -u)" | |
echo "RPM File: ${RPM_FILE}" | |
echo "Version: ${RPM_VERSION}" | |
echo "Release: ${RPM_RELEASE}" | |
# Clean install location | |
rm -rf /usr/local/cloudberry-db | |
# Install RPM | |
echo "Starting installation..." | |
if ! time dnf install -y "${RPM_FILE}"; then | |
echo "::error::RPM installation failed" | |
exit 1 | |
fi | |
echo "Installation completed successfully" | |
rpm -qi apache-cloudberry-db-incubating | |
} 2>&1 | tee -a build-logs/details/rpm-installation.log | |
- name: Extract source tarball | |
if: success() && needs.check-skip.outputs.should_skip != 'true' | |
env: | |
SRC_TARBALL_FILE: ${{ steps.verify-artifacts.outputs.src_tarball_file }} | |
SRC_DIR: ${{ github.workspace }} | |
run: | | |
set -eo pipefail | |
{ | |
echo "=== Source Extraction Log ===" | |
echo "Timestamp: $(date -u)" | |
echo "Starting extraction..." | |
if ! time tar zxf "${SRC_TARBALL_FILE}" -C "${SRC_DIR}"/.. ; then | |
echo "::error::Source extraction failed" | |
exit 1 | |
fi | |
echo "Extraction completed successfully" | |
echo "Extracted contents:" | |
ls -la "${SRC_DIR}/../cloudberry" | |
echo "Directory size:" | |
du -sh "${SRC_DIR}/../cloudberry" | |
} 2>&1 | tee -a build-logs/details/source-extraction.log | |
- name: Create Apache Cloudberry demo cluster | |
if: success() && needs.check-skip.outputs.should_skip != 'true' | |
env: | |
SRC_DIR: ${{ github.workspace }} | |
run: | | |
set -eo pipefail | |
{ | |
chmod +x "${SRC_DIR}"/../cloudberry-devops-release/build_automation/cloudberry/scripts/create-cloudberry-demo-cluster.sh | |
if ! time su - gpadmin -c "cd ${SRC_DIR} && NUM_PRIMARY_MIRROR_PAIRS='${{ matrix.num_primary_mirror_pairs }}' SRC_DIR=${SRC_DIR} ${SRC_DIR}/../cloudberry-devops-release/build_automation/cloudberry/scripts/create-cloudberry-demo-cluster.sh"; then | |
echo "::error::Demo cluster creation failed" | |
exit 1 | |
fi | |
} 2>&1 | tee -a build-logs/details/create-cloudberry-demo-cluster.log | |
- name: "Run Tests: ${{ matrix.test }}" | |
if: success() && needs.check-skip.outputs.should_skip != 'true' | |
env: | |
SRC_DIR: ${{ github.workspace }} | |
run: | | |
set -o pipefail | |
{ | |
# Construct PGOPTIONS from pg_settings | |
PG_OPTS="" | |
if [[ -n "${{ matrix.pg_settings.optimizer }}" ]]; then | |
PG_OPTS="$PG_OPTS -c optimizer=${{ matrix.pg_settings.optimizer }}" | |
fi | |
chmod +x "${SRC_DIR}"/../cloudberry-devops-release/build_automation/cloudberry/scripts/test-cloudberry.sh | |
if ! time su - gpadmin -c "cd ${SRC_DIR} && \ | |
MAKE_NAME='${{ matrix.test }}' \ | |
MAKE_TARGET='${{ matrix.make_target }}' \ | |
MAKE_DIRECTORY='${{ matrix.make_directory }}' \ | |
PGOPTIONS='${PG_OPTS}' \ | |
SRC_DIR='${SRC_DIR}' \ | |
${SRC_DIR}/../cloudberry-devops-release/build_automation/cloudberry/scripts/test-cloudberry.sh"; then | |
echo "::error::Test execution failed" | |
df -kh / | |
exit 1 | |
fi | |
} 2>&1 | tee -a build-logs/details/make-${{ matrix.test }}.log | |
- name: Parse Test Results | |
id: test-results | |
if: always() | |
shell: bash {0} | |
env: | |
SRC_DIR: ${{ github.workspace }} | |
run: | | |
set -o pipefail | |
{ | |
# Check if the workflow was cancelled | |
if [[ "${{ job.status }}" == "cancelled" ]]; then | |
echo "Workflow was cancelled - recording cancelled status" | |
{ | |
echo "status=cancelled" | |
echo "total_tests=0" | |
echo "failed_tests=0" | |
echo "passed_tests=0" | |
echo "ignored_tests=0" | |
} >> "$GITHUB_OUTPUT" | |
echo "Cancellation status recorded" | tee -a build-logs/details/parse-test-results-${{ matrix.test }}.log | |
exit 0 | |
fi | |
# Check for skipped tests | |
if [[ "${{ needs.check-skip.outputs.should_skip }}" == "true" ]]; then | |
echo "Tests skipped - setting default values" | |
{ | |
echo "status=skipped" | |
echo "total_tests=0" | |
echo "failed_tests=0" | |
echo "passed_tests=0" | |
echo "ignored_tests=0" | |
} >> "$GITHUB_OUTPUT" | |
echo "Skip status recorded" | tee -a build-logs/details/parse-test-results-${{ matrix.test }}.log | |
exit 0 | |
fi | |
# Ensure log file exists and has content | |
LOG_FILE="build-logs/details/make-${{ matrix.test }}.log" | |
if [[ ! -f "$LOG_FILE" ]]; then | |
echo "::warning::Test log file not found, using default values" | |
{ | |
echo "status=no_log" | |
echo "total_tests=0" | |
echo "failed_tests=0" | |
echo "passed_tests=0" | |
echo "ignored_tests=0" | |
} >> "$GITHUB_OUTPUT" | |
exit 0 | |
fi | |
# Check if log file is empty or contains only whitespace | |
if [[ ! -s "$LOG_FILE" ]] || ! grep -q '[^[:space:]]' "$LOG_FILE"; then | |
echo "::warning::Test log file is empty, using default values" | |
{ | |
echo "status=empty_log" | |
echo "total_tests=0" | |
echo "failed_tests=0" | |
echo "passed_tests=0" | |
echo "ignored_tests=0" | |
} >> "$GITHUB_OUTPUT" | |
exit 0 | |
fi | |
# Attempt to parse results | |
chmod +x "${SRC_DIR}"/../cloudberry-devops-release/build_automation/cloudberry/scripts/parse-test-results.sh | |
MAKE_NAME='${{ matrix.test }}' \ | |
"${SRC_DIR}"/../cloudberry-devops-release/build_automation/cloudberry/scripts/parse-test-results.sh \ | |
"$LOG_FILE" | |
status_code=$? | |
case $status_code in | |
0) # All tests passed | |
echo "All tests passed successfully" | |
;; | |
1) # Tests failed but parsed successfully | |
echo "Test failures detected but properly parsed" | |
# Don't exit here - let the workflow continue | |
;; | |
2) # Parse error or missing file | |
echo "::warning::Could not parse test results properly" | |
{ | |
echo "status=parse_error" | |
echo "total_tests=0" | |
echo "failed_tests=0" | |
echo "passed_tests=0" | |
echo "ignored_tests=0" | |
} >> "$GITHUB_OUTPUT" | |
exit 0 | |
;; | |
*) # Unexpected error | |
echo "::warning::Unexpected error during test results parsing" | |
{ | |
echo "status=unknown_error" | |
echo "total_tests=0" | |
echo "failed_tests=0" | |
echo "passed_tests=0" | |
echo "ignored_tests=0" | |
} >> "$GITHUB_OUTPUT" | |
exit 0 | |
;; | |
esac | |
} 2>&1 | tee -a build-logs/details/parse-test-results-${{ matrix.test }}.log | |
- name: Check and Display Regression Diffs | |
if: always() | |
run: | | |
# Search for regression.diffs recursively | |
found_file=$(find . -type f -name "regression.diffs" | head -n 1) | |
if [[ -n "$found_file" ]]; then | |
echo "Found regression.diffs at: $found_file" | |
cat "$found_file" | |
else | |
echo "No regression.diffs file found in the hierarchy." | |
fi | |
- name: "Generate Test Job Summary End: ${{ matrix.test }}" | |
if: always() | |
shell: bash {0} | |
run: | | |
{ | |
if [[ "${{ needs.check-skip.outputs.should_skip }}" == "true" ]]; then | |
echo "## Test Results - SKIPPED" | |
echo "- End Time: $(date -u +'%Y-%m-%d %H:%M:%S UTC')" | |
exit 0 | |
fi | |
echo "## Test Results" | |
echo "- End Time: $(date -u +'%Y-%m-%d %H:%M:%S UTC')" | |
# Check if job was cancelled | |
if [[ "${{ job.status }}" == "cancelled" ]]; then | |
echo "### Test Status" | |
echo "🚫 Test execution was cancelled" | |
echo "" | |
echo "### Execution Summary" | |
echo "Test run was interrupted and did not complete. No test results are available." | |
exit 0 | |
fi | |
# Add test execution results | |
echo "### Test Execution Summary" | |
echo "| Metric | Count |" | |
echo "|--------|-------|" | |
echo "| Total Tests | ${{ steps.test-results.outputs.total_tests }} |" | |
echo "| Passed Tests | ${{ steps.test-results.outputs.passed_tests }} |" | |
echo "| Failed Tests | ${{ steps.test-results.outputs.failed_tests }} |" | |
echo "| Ignored Tests | ${{ steps.test-results.outputs.ignored_tests }} |" | |
# Test status with emoji based on results | |
echo "### Test Status" | |
if [[ "${{ steps.test-results.outputs.status }}" == "passed" ]]; then | |
echo "✅ All ${{ steps.test-results.outputs.total_tests }} tests passed successfully" | |
else | |
echo "⚠️ ${{ steps.test-results.outputs.failed_tests }} of ${{ steps.test-results.outputs.total_tests }} tests failed" | |
# Add failed tests as a bullet list if there are failures | |
if [[ -n "${{ steps.test-results.outputs.failed_test_names }}" ]]; then | |
echo "" | |
echo "### Failed Tests" | |
echo "The following tests failed:" | |
echo "${{ steps.test-results.outputs.failed_test_names }}" | tr ',' '\n' | while read -r test; do | |
if [[ -n "$test" ]]; then | |
echo "* \`${test}\`" | |
fi | |
done | |
fi | |
fi | |
# Add ignored tests as a separate section | |
if [[ -n "${{ steps.test-results.outputs.ignored_test_names }}" ]]; then | |
echo "" | |
echo "### Ignored Tests" | |
echo "The following tests were ignored (results not critical):" | |
echo "${{ steps.test-results.outputs.ignored_test_names }}" | tr ',' '\n' | while read -r test; do | |
if [[ -n "$test" ]]; then | |
echo "* \`${test}\`" | |
fi | |
done | |
fi | |
} >> "$GITHUB_STEP_SUMMARY" || true | |
- name: Upload test logs | |
if: always() | |
uses: actions/upload-artifact@v4 | |
with: | |
name: test-logs-${{ matrix.test }}-${{ needs.build.outputs.build_timestamp }} | |
path: | | |
build-logs/ | |
retention-days: ${{ env.LOG_RETENTION_DAYS }} | |
- name: Upload test regression logs | |
if: >- | |
always() && ( | |
steps.test-results.outputs.status == 'failed' || | |
job.status == 'cancelled' || | |
steps.test-results.outputs.status == 'cancelled' | |
) | |
uses: actions/upload-artifact@v4 | |
with: | |
name: regression-logs-${{ matrix.test }}-${{ needs.build.outputs.build_timestamp }} | |
path: | | |
src/test/**/regression.out | |
src/test/**/regression.diffs | |
src/test/**/results/ | |
gpAux/gpdemo/datadirs/standby/log/ | |
gpAux/gpdemo/datadirs/qddir/demoDataDir-1/log/ | |
gpAux/gpdemo/datadirs/dbfast1/demoDataDir0/log/ | |
gpAux/gpdemo/datadirs/dbfast2/demoDataDir1/log/ | |
gpAux/gpdemo/datadirs/dbfast3/demoDataDir2/log/ | |
gpAux/gpdemo/datadirs/dbfast_mirror1/demoDataDir0/log/ | |
gpAux/gpdemo/datadirs/dbfast_mirror2/demoDataDir1/log/ | |
gpAux/gpdemo/datadirs/dbfast_mirror3/demoDataDir2/log/ | |
retention-days: ${{ env.LOG_RETENTION_DAYS }} | |
report: | |
name: Generate Apache Cloudberry Build Report | |
needs: [check-skip, build, rpm-install-test, test] | |
if: always() | |
runs-on: ubuntu-latest | |
steps: | |
- name: Generate Final Report | |
run: | | |
{ | |
echo "# Apache Cloudberry Build Pipeline Report" | |
if [[ "${{ needs.check-skip.outputs.should_skip }}" == "true" ]]; then | |
echo "## CI Skip Status" | |
echo "✅ CI checks skipped via skip flag" | |
echo "- Completion Time: $(date -u +'%Y-%m-%d %H:%M:%S UTC')" | |
else | |
echo "## Job Status" | |
echo "- Build Job: ${{ needs.build.result }}" | |
echo "- Test Job: ${{ needs.test.result }}" | |
echo "- Completion Time: $(date -u +'%Y-%m-%d %H:%M:%S UTC')" | |
if [[ "${{ needs.build.result }}" == "success" && "${{ needs.test.result }}" == "success" ]]; then | |
echo "✅ Pipeline completed successfully" | |
else | |
echo "⚠️ Pipeline completed with failures" | |
if [[ "${{ needs.build.result }}" != "success" ]]; then | |
echo "### Build Job Failure" | |
echo "Check build logs for details" | |
fi | |
if [[ "${{ needs.test.result }}" != "success" ]]; then | |
echo "### Test Job Failure" | |
echo "Check test logs and regression files for details" | |
fi | |
fi | |
fi | |
} >> "$GITHUB_STEP_SUMMARY" | |
- name: Notify on failure | |
if: | | |
needs.check-skip.outputs.should_skip != 'true' && | |
(needs.build.result != 'success' || needs.test.result != 'success') | |
run: | | |
echo "::error::Build/Test pipeline failed! Check job summaries and logs for details" | |
echo "Timestamp: $(date -u +'%Y-%m-%d %H:%M:%S UTC')" | |
echo "Build Result: ${{ needs.build.result }}" | |
echo "Test Result: ${{ needs.test.result }}" |