Testing Guidelines (RSpec and Cucumber, etc.)
A .rubocop.yml file is provided with the project, configured to also check RSpec and Rails styles.
You can read about the RSpec styles expected here.
This can be used with an editor or IDE to check or autocorrect code in the tests; alternatively it is possible to run bundle exec rubocop from the command line, with the option to auto-fix file (bundle exec rubocop -a).
RSpec is configured to run with --order random by default.
This helps to find out if there is a global state changed by another example, and/or if some state has 'leaked' from one example to another.
This also can show if there are any dependencies between/among example. This can be a smell pointing to unnecessary or complex dependencies in the code.
More in general, examples should be isolated from one another, and only dependent on example groups, for example for global state. The random order option will still keep examples inside a group together, randomizing their order only inside the group.
For those situations where it makes sense to run the tests in order, run them with bin/rspec --order d; for example, to have a more documentation-oriented view of the examples, you could run: bin/rspec --format d --order d.
SHF tests use DatabaseCleaner to keep the test database clean between examples.
When and how the DatabaseCleaner is run is configured in the spec/rails_helper.rb file.
DatabaseCleaner should open a DB transaction before each example and reset it at the end.
This means that models created before the hook before(:each) a.k.a. before(:example) is fired or after the hook a.k.a. append_after(:example) is fired will persist in the database between tests, polluting the environment for the following tests. (DatabaseCleaner will not 'clean up' those models.)
Any operation that could interfere with a DB transaction could cause an inconsistent database state. If there is a statement in a test that explicitly works with a DB transaction, it could interfere with DatabaseCleaner and cause an inconsistent state.
before(:all) can change the state across many examples and tests in ways that are not obvious.
If you find that you need to use before(:all), consider that a code smell and look at the class (or object) being tested.
If you change the 'current' date/time, be sure to go back to the 'real time.' No matter how you implement this (with TimeCop or ActiveSupport test helpers), be sure to 'travel back.'
The best way to ensure this is to use a block. Ex:
# Timecop
Timecop.freeze(Time.utc(2019, 11, 28)) do
expect(subject.send_alert_this_day?(timing, { days: [1] }, paid_members_co)).to be_falsey
end
# ActiveSupport test time travel
travel_to Time.utc(2019, 11, 28) do
expect(subject.send_alert_this_day?(timing, { days: [1] }, paid_members_co)).to be_falsey
end
Tests should not depend on the outside world.
Use VCR and/or mocks (ex: RSpec 'doubles') to simulate any interaction with outside systems.
Do not test something already covered in a test. It is repeating code and so causes the same problems: you will have to be sure to update code in more than one place, etc.
A typical examples would be database operations in unit testing. We know the database works, we checked that in the model specs.
Faster tests make developers happy, slow tests tend to encourage a sloppier coverage. While it is usually possible to test using only a subset of examples during development, the full suite should be run at least once locally.
Consider using FactoryBot's build_stubbed method instead of create, for example, or RSpec stubs and doubles.
If it's hard to write simple tests then probably the code base is too interdependent and you might want to refactor it.
Some end-to-end-ish tests, like controller specs should only test that messages are sent correctly; in other cases a cucumber scenario might already fulfill the same objective.
When you are doing end-to-end tests, mock/stub as much as you possibly can while still ensuring you are testing the right things.
Ashley: Here's a really good list of testing guidelines from EvilMartians: https://github.com/evilmartians/terraforming-rails/blob/master/guides/flaky.md
- perhaps we can list some of the main guides that we also should use as well as point to it as 'further reading' or another resource
