Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Needs smoke tests #484

Open
RobinDaugherty opened this issue Oct 3, 2020 · 2 comments
Open

Needs smoke tests #484

RobinDaugherty opened this issue Oct 3, 2020 · 2 comments

Comments

@RobinDaugherty
Copy link
Member

#480 and #481 fixed an issue that I introduced by referring to a constant that wasn't loaded yet.

CI didn't fail, even though every app failed the moment it tried to load.

I think the solution here is to add a smoke test that runs an app of some kind (Rails/Sinatra/pure Rack) and make sure the app starts. This actually might be more effective than all of the specs we have, which currently have to cover a lot of different cases (with Rails, each version of rails, etc).

@peaonunes
Copy link
Contributor

peaonunes commented Oct 3, 2020

That's a fair point. I cannot think in multiple ways to easily validate that, one idea I just had was:

  1. Rails app, or anything, with the dependency pointing to the development.
  2. A rake task that simply puts on stdout.
  3. A spec that navigates to the app and run the rake.

The rake would load the Rails env so if the gem was corrupted in any way the rake would not execute and would fail.
I'm just not 💯 sure on how to make the step 1 to work, but I would be happy to give it a try. What do you think?

@RobinDaugherty
Copy link
Member Author

I think this might be a very simple thing actually. I have various Rails and Rack apps I've built to manually test in. One or more of these could be downloaded in our CI, then:

  1. Start the app server in a background process
  2. Wait for the app server to listen on the configured port (and time out otherwise)
  3. Make a request to known-good and known-bad routes
  4. (nice-to-have) make an "internal" Better Errors request to get variables or run a console command
  5. Terminate the background process

This could be done using a build stage in Travis, or maybe just tacked onto the existing CI run, running on each combination of Ruby/Rails/binding_of_caller/haml.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants