Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary of changes
Asana Ticket: No ticket.
A bit of a reversion of #463. See that PR for background on what the
ExVCR
package is and why it was added as a tool as part of our Elixir unit testing. TLDR We tried it as a way to record V3 API responses that happened to be made in the course of unit tests, to prevent problems when the underlying data had changed yet a test expects a certain value.Ultimately it is probably useful for simple tests. However I (any maybe other devs, I don't know), often ran into problems running the test suite where I would get this error message for tests I was not actively working on or editing:
Which is fine when it happens infrequently... but it happens many times.
In a way the error message exposed a gap in my understanding of how I thought this should work - wouldn't it just append new responses to the existing cassette, ignore it in favor of what's already recorded, or overwrite the recording for me?
From what I can tell, the library has some open issues around cassettes (the recorded responses) possibly not being handled optimally:
And I'm a bit wary that ExVCR might have issues with other tools we use in our testing (see issues for Bypass).
I think part of the problem, is that for some of our tests, including a few we were already using ExVCR to assist with, there are many underlying requests made to the V3 API. For example loading a certain page might load a route with data derived from a query for a schedule which in turn might fetch a trip which then we'll follow up with a request for its vehicle. That's a lot of requests to coordinate, but an hour later that same test is going to fetch a different schedule, ... I'm theorizing that this is a use case that's beyond ExVCR's ability to gracefully handle, but is very common across our entire codebase.
There are other ways to improve our tests - ideally unit tests are isolated to the concept that's actually being tested, without being dependent on the details of the underlying data. Maybe the page load test only cares that there's a
%Trip{}
being passed to theconn
, and we shouldn't test for that by looking for a certain headway text/route name/etc on the page response.So overall I think it was a nice experiment but ExVCR seemed to be getting in the way more than it was helping. Open to counterarguments on that. This PR deletes everything related to our usage of the library (the
:exvcr
dependency itself, our ownexvcr_helpers
module, etc).General checks