Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor tests to be integration tests rather than E2E deployments #194

Open
1000hz opened this issue Oct 10, 2023 · 0 comments
Open

Refactor tests to be integration tests rather than E2E deployments #194

1000hz opened this issue Oct 10, 2023 · 0 comments

Comments

@1000hz
Copy link
Contributor

1000hz commented Oct 10, 2023

The way this action is currently being tested is primarily through a GitHub Actions workflow that deploys several Worker fixtures and passes as long as the action doesn't fail to complete.

This approach has a few severe drawbacks. Because deploying these Workers to our test account requires our API token, PRs submitted by external contributors don't have access to our secrets and these always fail to run. These failures also mask those that might potentially occur during other checks like code formatting. This results in us merging sometimes-broken code due to having no reliable confidence signal before merging.

Second, the current E2E structure makes it difficult to assert against expected behaviors of the action in an automated way. It's also cumbersome to construct tests for each scenario (and tear them down after the test completes, etc.) These E2E tests end up mostly testing the underlying behavior of wrangler rather than the behavior of the action itself.

We can improve our test coverage by moving towards integration testing for much of the behavior currently being tested (to varying degrees) in an E2E manner. To do this, we'll invoke the action's entrypoint in a vitest and assert that the expected subprocesses are spawned appropriately.

We can limit E2E tests to a few basic smoke tests that only run if: ${{ github.repository_owner == 'cloudflare' }} while allowing the rest of the test suite to run on external PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants