Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI test run against a Netflify staging instance #5

Open
ivelin opened this issue Mar 7, 2021 · 4 comments
Open

Add CI test run against a Netflify staging instance #5

ivelin opened this issue Mar 7, 2021 · 4 comments
Assignees
Labels
enhancement New feature or request

Comments

@ivelin
Copy link
Contributor

ivelin commented Mar 7, 2021

Is your feature request related to a problem? Please describe.
Currently the CI runs local tests, but not live tests against a Netlify functions pre-deployment environment.

Describe the solution you'd like
Levelarge the CI best practices from ambianic-ui. We run two separate CI workflow scripts of lighthouse tests against local and live netlify staging version of the new code in a PR or commit push.

@ivelin ivelin added the enhancement New feature or request label Mar 7, 2021
@vickywane
Copy link
Contributor

@ivelin To understand this better. Are you requesting tests are run against the local Netlify instance created using netlify-dev?

Currently, tests are run against deployed functions using their API endpoints from Netlify.

@ivelin
Copy link
Contributor Author

ivelin commented Mar 7, 2021

@vickywane good point. I overlooked that the tests are running against the actual deployed ambianic-functions on Netflify. The issue with this approach is that we deploy first (potentially buggy new code) which instantly affects users and then we run tests against it.

The tests must be focused on the new code in the specific PR.

I will correct my request. Ideally we would have two separate workflows:

  1. One that runs against a local CI copy of functions via netlify dev and catches any issues before deployment.
  2. Another one that runs against a preview deployment of the code changed in the given PR. See how our LH flow waits for a preview deployment and tests against it. The goal of point 2 is to catch any issues before these functions are deployed in production where all users are instantly affected.

@vickywane
Copy link
Contributor

This is quite an interesting edge case. It’s is important to note that we are using newman which run the postman scripts. For the scripts to be created they have to be given a live endpoint, i dont think this might be possible with a local endpoint.

Also netlify-dev changes the PORT where the functions are run each time it’s started.

I would give this a deeper look-into

@ivelin
Copy link
Contributor Author

ivelin commented Mar 7, 2021 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants