Skip to content

Fix trigger.dev preview environment archiving reliability #2186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 26, 2025

Conversation

devin-ai-integration[bot]
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot commented Jun 24, 2025

Fix trigger.dev preview environment archiving reliability

Summary

This PR implements the fixes proposed by Claude in issue #2181 to improve the reliability of trigger.dev preview environment archiving in GitHub Actions workflows.

Changes Made

1. Prevent Archive Cancellation

  • Modified cancel-in-progress setting to ${{ github.event.action != 'closed' }} instead of true
  • This prevents archiving operations from being cancelled by concurrent workflow runs

2. Enhanced Archive Step with Robust Error Handling

  • Replaced the simple archive command with a comprehensive solution that includes:
    • Branch name validation: Prevents errors from empty branch names
    • Retry logic: 3 attempts with 10-second delays to handle temporary failures
    • Verification: Confirms archiving succeeded by checking if environment still exists
    • Better logging: Detailed output for debugging archiving issues

3. Added Cleanup Fallback Step

  • New cleanup step that runs even if the main archiving fails
  • Ensures no stale preview environments remain
  • Uses always() condition to run regardless of previous step status

Problem Addressed

The original workflow had several potential failure scenarios:

  • Race conditions: Archiving operations could be cancelled by concurrent workflows
  • No error handling: Archive command could fail silently
  • No retry mechanism: Temporary failures (network issues, API limits) weren't handled
  • No verification: No confirmation that archiving actually succeeded
  • Stale environments: Failed archiving could leave environments running

Testing Transparency

What I Actually Checked

  • ✅ Applied both fixes exactly as specified in Claude's analysis
  • ✅ Verified YAML syntax is correct (no parsing errors)
  • ✅ Confirmed all environment variables and commands match existing patterns
  • ✅ Ensured proper indentation and structure in the workflow file

What I Did Not Check

  • ❌ Could not test the actual trigger.dev archiving functionality locally (requires secrets and live environment)
  • ❌ Could not verify the retry logic works with real API failures
  • ❌ Could not test the verification step with actual preview environments

Review Checklist

Please verify:

  • The retry logic parameters (3 attempts, 10-second delays) are appropriate for your use case
  • The verification command trigger preview list --branch works as expected in your environment
  • The cleanup fallback step doesn't interfere with other workflows
  • The logging output provides sufficient debugging information

Related Issues

Fixes #2181


Link to Devin run: https://app.devin.ai/sessions/daa3c06b19b54017bfc27c17ae8cfb6a

Requested by: [email protected]

- Prevent archiving operations from being cancelled by concurrent workflows
- Add retry logic with 3 attempts and 10-second delays
- Add branch name validation to prevent empty branch errors
- Add verification step to confirm archiving succeeded
- Add cleanup fallback step to handle stale environments
- Improve logging for better debugging

Addresses issues identified in #2181

Co-Authored-By: [email protected] <[email protected]>
Copy link

changeset-bot bot commented Jun 24, 2025

⚠️ No Changeset found

Latest commit: cb9e1c8

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link

vercel bot commented Jun 24, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
liam-app ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 26, 2025 2:59am
liam-erd-sample ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 26, 2025 2:59am
liam-storybook ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 26, 2025 2:59am
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
liam-docs ⬜️ Ignored (Inspect) Visit Preview Jun 26, 2025 2:59am

Copy link

supabase bot commented Jun 25, 2025

Updates to Preview Branch (devin/1750759330-fix-trigger-dev-archiving) ↗︎

Deployments Status Updated
Database Thu, 26 Jun 2025 02:48:28 UTC
Services Thu, 26 Jun 2025 02:48:28 UTC
APIs Thu, 26 Jun 2025 02:48:28 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks Status Updated
Configurations Thu, 26 Jun 2025 02:48:43 UTC
Migrations Thu, 26 Jun 2025 02:48:45 UTC
Seeding Thu, 26 Jun 2025 02:48:45 UTC
Edge Functions Thu, 26 Jun 2025 02:48:45 UTC

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

@MH4GF MH4GF marked this pull request as draft June 25, 2025 00:49
Copy link
Contributor

qodo-merge-for-open-source bot commented Jun 25, 2025

PR Reviewer Guide 🔍

(Review updated until commit 1d288dd)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ✅

2181 - PR Code Verified

Compliant requirements:

• Prevent archive operations from being cancelled by concurrent workflow runs
• Add error handling and retry logic for temporary failures
• Ensure no stale preview environments remain after PR closure
• Improve logging and debugging capabilities for archiving issues

Requires further human verification:

• Fix trigger.dev preview environment archiving reliability issues (requires testing with actual trigger.dev environment)

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Shell Quoting

The branch name variable is not properly quoted in the shell commands, which could cause issues if branch names contain special characters or spaces

      if pnpm --filter @liam-hq/jobs exec trigger preview archive --branch "${{ github.head_ref }}"; then
        echo "Successfully archived preview environment for branch: ${{ github.head_ref }}"

        # Verify archiving by checking if the environment still exists
        echo "Verifying archive operation..."
        sleep 5

        if pnpm --filter @liam-hq/jobs exec trigger preview list --branch "${{ github.head_ref }}" 2>/dev/null | grep -q "${{ github.head_ref }}"; then
          echo "Warning: Environment may still be active after archiving"
        else
          echo "Archive verification successful - environment no longer listed"
        fi

        exit 0
      else
        RETRY_COUNT=$((RETRY_COUNT + 1))
        if [ $RETRY_COUNT -lt $MAX_RETRIES ]; then
          echo "Archive failed, retrying in 10 seconds..."
          sleep 10
        else
          echo "Archive failed after $MAX_RETRIES attempts"
          exit 1
        fi
      fi
    done

- name: Cleanup stale environments
  if: github.event.action == 'closed' && always()
  shell: bash
  env:
    TRIGGER_ACCESS_TOKEN: ${{ secrets.TRIGGER_ACCESS_TOKEN }}
    TRIGGER_PROJECT_ID: ${{ vars.TRIGGER_PROJECT_ID }}
  run: |
    echo "Checking for any stale preview environments..."

    # List all preview environments and attempt to clean up any that match this branch
    echo "Ensuring no preview environments remain for branch: ${{ github.head_ref }}"

    # Attempt cleanup even if the previous step failed
    pnpm --filter @liam-hq/jobs exec trigger preview archive --branch "${{ github.head_ref }}" 2>/dev/null || true
Logic Issue

The cleanup step condition uses always() which will run even on workflow cancellation, but the step may not have access to required environment variables or context in all failure scenarios

if: github.event.action == 'closed' && always()
shell: bash

Copy link
Contributor

qodo-merge-for-open-source bot commented Jun 25, 2025

PR Code Suggestions ✨

Latest suggestions up to 1d288dd
Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Fix verification logic error handling

The verification logic is flawed because it treats the absence of output as
success, but the command might fail for other reasons. Add explicit error
handling to distinguish between "environment not found" and actual command
failures.

.github/workflows/trigger_dev_preview.yml [57-61]

-if pnpm --filter @liam-hq/jobs exec trigger preview list --branch "${{ github.head_ref }}" 2>/dev/null | grep -q "${{ github.head_ref }}"; then
-  echo "Warning: Environment may still be active after archiving"
+if pnpm --filter @liam-hq/jobs exec trigger preview list --branch "${{ github.head_ref }}" 2>/dev/null; then
+  if pnpm --filter @liam-hq/jobs exec trigger preview list --branch "${{ github.head_ref }}" 2>/dev/null | grep -q "${{ github.head_ref }}"; then
+    echo "Warning: Environment may still be active after archiving"
+  else
+    echo "Archive verification successful - environment no longer listed"
+  fi
 else
-  echo "Archive verification successful - environment no longer listed"
+  echo "Unable to verify archive status - list command failed"
 fi
  • Apply / Chat
Suggestion importance[1-10]: 7

__

Why: The suggestion correctly identifies a flaw in the verification logic where a failure of the pnpm ... list command would be misinterpreted as a successful archive. The proposed fix, while slightly inefficient, correctly separates the command success check from the output content check, improving the robustness of the workflow.

Medium
  • Update

Previous suggestions

Suggestions up to commit 6ee5efd
CategorySuggestion                                                                                                                                    Impact
Security
Add branch name sanitization

Branch name validation should also check for potentially dangerous characters
that could cause command injection or unexpected behavior in shell commands.

.github/workflows/trigger_dev_preview.yml [41-46]

 # Validate branch name
 BRANCH_NAME="${{ github.head_ref }}"
-if [[ -z "$BRANCH_NAME" ]]; then
-  echo "Error: Branch name is empty"
+if [[ -z "$BRANCH_NAME" ]] || [[ "$BRANCH_NAME" =~ [^a-zA-Z0-9/_-] ]]; then
+  echo "Error: Branch name is empty or contains invalid characters"
   exit 1
 fi
Suggestion importance[1-10]: 8

__

Why: This is a valuable security hardening suggestion. While the variable BRANCH_NAME is currently quoted, which mitigates command injection risk, explicitly validating the branch name format provides strong defense-in-depth. This prevents potential issues from unexpected characters or future code changes where quoting might be accidentally omitted.

Medium
General
Refine cleanup step condition

The always() condition will execute this step even when previous steps are
skipped or cancelled, which may not be intended. Consider using failure()
instead to only run cleanup when the archive step fails.

.github/workflows/trigger_dev_preview.yml [83-84]

 - name: Cleanup stale environments
-  if: github.event.action == 'closed' && always()
+  if: github.event.action == 'closed' && failure()
Suggestion importance[1-10]: 7

__

Why: The suggestion correctly points out that using always() causes the cleanup step to run even when the preceding archive step succeeds, which is redundant. Changing the condition to failure() improves the workflow's efficiency and logic by ensuring the cleanup step only runs when it's actually needed.

Medium
Fix verification pattern matching

The verification logic is flawed because grep -q "$BRANCH_NAME" will match any
line containing the branch name, even if it's in an error message or different
context. This could lead to false positives in verification.

.github/workflows/trigger_dev_preview.yml [64-68]

-if pnpm --filter @liam-hq/jobs exec trigger preview list --branch "$BRANCH_NAME" 2>/dev/null | grep -q "$BRANCH_NAME"; then
+if pnpm --filter @liam-hq/jobs exec trigger preview list --branch "$BRANCH_NAME" 2>/dev/null | grep -q "^$BRANCH_NAME\s"; then
   echo "Warning: Environment may still be active after archiving"
 else
   echo "Archive verification successful - environment no longer listed"
 fi
Suggestion importance[1-10]: 6

__

Why: The suggestion correctly identifies that using grep -q "$BRANCH_NAME" is too broad and could lead to false positives. The proposed change to grep -q "^$BRANCH_NAME\\s" makes the verification logic more robust by ensuring the branch name is matched at the beginning of a line and followed by a space, which is a more reliable check.

Low

- Remove branch name validation as requested by [email protected]
- Keep all other enhancements: retry logic, verification, cleanup step
- Use direct github.head_ref instead of BRANCH_NAME variable

Co-Authored-By: [email protected] <[email protected]>
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves the reliability of archiving preview environments on trigger.dev by preventing cancellation of concurrent workflows, adding robust retry logic with verification, and ensuring cleanup of stale environments.

  • Updated concurrency to conditionally prevent cancellation based on event action
  • Implemented retry logic with logging and verification for the archive command
  • Added a cleanup fallback step for stale preview environments

Comment on lines +57 to +63
if pnpm --filter @liam-hq/jobs exec trigger preview list --branch "${{ github.head_ref }}" 2>/dev/null | grep -q "${{ github.head_ref }}"; then
echo "Warning: Environment may still be active after archiving"
else
echo "Archive verification successful - environment no longer listed"
fi

exit 0
Copy link
Preview

Copilot AI Jun 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The verification step logs a warning when the preview environment is still detected but then exits with success. This could lead to false positives where failures are not caught; consider exiting with a non-zero status or adding additional retry logic when the environment is still active.

Suggested change
if pnpm --filter @liam-hq/jobs exec trigger preview list --branch "${{ github.head_ref }}" 2>/dev/null | grep -q "${{ github.head_ref }}"; then
echo "Warning: Environment may still be active after archiving"
else
echo "Archive verification successful - environment no longer listed"
fi
exit 0
MAX_VERIFY_RETRIES=3
VERIFY_RETRY_COUNT=0
while [ $VERIFY_RETRY_COUNT -lt $MAX_VERIFY_RETRIES ]; do
if pnpm --filter @liam-hq/jobs exec trigger preview list --branch "${{ github.head_ref }}" 2>/dev/null | grep -q "${{ github.head_ref }}"; then
echo "Environment still active after archiving. Retrying verification in 10 seconds..."
VERIFY_RETRY_COUNT=$((VERIFY_RETRY_COUNT + 1))
sleep 10
else
echo "Archive verification successful - environment no longer listed"
exit 0
fi
done
echo "Verification failed: Environment still active after maximum retries"
exit 1

Copilot uses AI. Check for mistakes.

fi
done

- name: Cleanup stale environments
Copy link
Preview

Copilot AI Jun 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The condition combining 'github.event.action == "closed"' with 'always()' in the cleanup step can be confusing. Adding a comment to clarify the intended behavior of running the cleanup step regardless of previous failures while ensuring it only applies when the event is 'closed' would improve maintainability.

Suggested change
- name: Cleanup stale environments
- name: Cleanup stale environments
# This step ensures cleanup of stale environments only when the event action is 'closed'.
# The 'always()' function ensures this step runs regardless of the success or failure of previous steps.

Copilot uses AI. Check for mistakes.

@MH4GF MH4GF self-assigned this Jun 25, 2025
Copy link
Member

@hoshinotsuyoshi hoshinotsuyoshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment on lines +88 to +89
# Attempt cleanup even if the previous step failed
pnpm --filter @liam-hq/jobs exec trigger preview archive --branch "${{ github.head_ref }}" 2>/dev/null || true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attempt cleanup even if the previous step failed

I understand this runs even if the previous step succeeded ? — which seems fine either way.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the motivation is to remove it completely.

@@ -36,4 +36,56 @@ jobs:
TRIGGER_ACCESS_TOKEN: ${{ secrets.TRIGGER_ACCESS_TOKEN }}
TRIGGER_PROJECT_ID: ${{ vars.TRIGGER_PROJECT_ID }}
run: |
pnpm --filter @liam-hq/jobs exec trigger preview archive --branch ${{ github.head_ref }}
set -e
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nits]
Looks good! Just a note — shell: bash already implies bash --noprofile --norc -eo pipefail {0}, so set -e might not be needed.

https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!! @claude Please implement this comment?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixing a problem with claude not working on #2216

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix it: cb9e1c8

GitHub Actions' shell: bash already includes error handling with -eo pipefail,
making the explicit set -e unnecessary.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Copy link

coderabbitai bot commented Jun 26, 2025

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@MH4GF MH4GF added this pull request to the merge queue Jun 26, 2025
Merged via the queue into main with commit 4a595a5 Jun 26, 2025
29 of 30 checks passed
@MH4GF MH4GF deleted the devin/1750759330-fix-trigger-dev-archiving branch June 26, 2025 06:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sometimes the trigger.dev preview environment is not archived properly
2 participants