OHHTTPStub Tests failing inconsistently

[chrisballinger]

The first run failed, but when run again with no changes it passed. My hypothesis is that OHHTTPStubs is not completely torn down cleanly, so when the tests are run in a different order they may fail.

Run 1 Failed

https://circleci.com/gh/Rightpoint/ios-template-output/76

Test Suite ProjectNameTests.xctest started
APIClientTests
✗ testAuthenticatedRequestWithCredentials, XCTAssertTrue failed - client did not make a refresh request
✗ testAuthenticatedRequestWithNoCredentials, XCTAssertNotNil failed
✓ testManyAuthenticatedRequestWithCredentials (0.042 seconds)
OAuthTests
✓ testAdaptBeforeAuthenticated (0.009 seconds)
✓ testOAuthLogin (0.012 seconds)
✓ testOAuthRefresh (0.016 seconds)
ProjectNameTests
✓ testUserDefaults (0.001 seconds)

ProjectNameTests.APIClientTests
testAuthenticatedRequestWithCredentials, XCTAssertTrue failed - client did not make a refresh request
/Users/distiller/project/app/ProjectNameTests/OAuth/APIClientTests.swift:72

        waitForExpectations(timeout: timeout, handler: nil)
        XCTAssert(authorized, "client did not make a refresh request")
    }

testAuthenticatedRequestWithNoCredentials, XCTAssertNotNil failed
/Users/distiller/project/app/ProjectNameTests/OAuth/APIClientTests.swift:41

        client.request(TestEndpoint()) { _, error in
            XCTAssertNotNil(error)
            expectation.fulfill()

 Executed 7 tests, with 2 failures (0 unexpected) in 0.147 (0.169) seconds

Run 2 Passed

https://circleci.com/gh/Rightpoint/ios-template-output/77

[12:48:05]: ▸ All tests
[12:48:05]: ▸ Test Suite ProjectNameTests.xctest started
[12:48:05]: ▸ APIClientTests
[12:48:05]: ▸ ✓ testAuthenticatedRequestWithCredentials (0.086 seconds)
[12:48:05]: ▸ ✓ testAuthenticatedRequestWithNoCredentials (0.007 seconds)
[12:48:05]: ▸ ✓ testManyAuthenticatedRequestWithCredentials (0.044 seconds)
[12:48:05]: ▸ OAuthTests
[12:48:05]: ▸ ✓ testAdaptBeforeAuthenticated (0.006 seconds)
[12:48:05]: ▸ ✓ testOAuthLogin (0.010 seconds)
[12:48:05]: ▸ ✓ testOAuthRefresh (0.013 seconds)
[12:48:05]: ▸ ProjectNameTests
[12:48:05]: ▸ ✓ testUserDefaults (0.001 seconds)
[12:48:05]: ▸ Executed 7 tests, with 0 failures (0 unexpected) in 0.168 (0.174) seconds

[mattwhitlock]

Should OHHTTPStub (and similar components) be pulled out of the base template and be a part of a swift package with its own tests? This one in particular has hampered development progress on the template. I spent a lot of time trying to get this fixed on a simple change to a template generation file.

[chrisballinger]

@mattwhitlock I think it should be ripped out. The OAuthClient bundled with the template isn't necessarily the best choice these days anyway.

I'm working on a "next gen" template that hopefully addresses many of the pain points of the existing template workflow.

[mattwhitlock]

Cool, I think you have lots of support to rip it out. On the next gen template, that sounds good. Keep us posted and I'm sure you'll have volunteers to help, test and give feedback. :) Also @janabethhart I'm sure is quite interested in the new template work.