From c088e13a05837afc866c993df411d4c4cca6cb2d Mon Sep 17 00:00:00 2001 From: Lawrence Forooghian Date: Tue, 28 Feb 2023 15:29:43 -0300 Subject: [PATCH] Add subscriber network connectivity tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Resolves #539. These are based on the corresponding tests in the Android codebase at commit f6d163f. They stick as closely as possible to the structure and behaviour of those tests, to which I have not applied much of a critical eye. The majority of the log messages and comments are simply copied across. The only structural difference between these tests and the Android ones is that I have removed the caller’s responsibility to call `SubscriberMonitor#close`. There are places in the Android implementation where we forget to call it, and also it isn’t called if an error is thrown by `waitForStateTransition`. By making the subscriber monitor responsible for its own cleanup, we remove these issues. When I started writing these tests, we were still targeting iOS 12 and above, which meant it was not possible to use Swift concurrency. Hence, all of the code is written using either completion handlers or blocking functions. Since then, we’ve decided to set our deployment target to iOS 13 (see #597), which means we could now use Swift concurrency. However, by that point I was well into the writing of these tests and did not want to re-structure them, especially since they require functionality that doesn’t exist out of the box in Swift concurrency (for example timeouts). We may wish to revisit at some point and switch to using Swift concurrency, but I don’t think it’s urgent. In order to generate the list of which faults to skip, I ran all of the tests and checked which ones failed. Whilst doing so, I encountered a crash in ably-cocoa, which I believe is already reported as ably/ably-cocoa#1380. In order to get past this issue, I tried using Marat’s fix from ably-cocoa branch `fix/1380-dispatch-ARTOSReachability_Callback` (commit 036be28). This removed the crashes. The list of skipped tests is the list of tests that failed when using this branch. It then actually turned out that the non-skipped tests do not exhibit this crash anyway, so I didn’t include any version change to ably-cocoa in this PR. We can deal with this crash when we try re-enabling the tests in #575, by which point hopefully the fix for ably-cocoa#1380 will have been released. (Internal thread re this crash and its appearance in these tests is [1].) [1] https://ably-real-time.slack.com/archives/CSQEKCE81/p1678802535684519 --- .../Helper/CombineSubscriberDelegate.swift | 87 ++++ .../CombineSubscriberDelegateTests.swift | 47 +++ .../Subscriber/Helper/SubscriberMonitor.swift | 354 ++++++++++++++++ .../Helper/SubscriberMonitorFactory.swift | 319 ++++++++++++++ ...scriberNetworkConnectivityTestsParam.swift | 75 ++++ .../Subscriber/Helper/TestResources.swift | 310 ++++++++++++++ .../SubscriberNetworkConnectivityTests.swift | 392 ++++++++++++++++++ .../Proxy/SDKTestProxyClient.swift | 3 +- Tests/SystemTests/Utils/MultipleErrors.swift | 32 ++ 9 files changed, 1618 insertions(+), 1 deletion(-) create mode 100644 Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/CombineSubscriberDelegate.swift create mode 100644 Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/CombineSubscriberDelegateTests.swift create mode 100644 Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberMonitor.swift create mode 100644 Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberMonitorFactory.swift create mode 100644 Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberNetworkConnectivityTestsParam.swift create mode 100644 Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/TestResources.swift create mode 100644 Tests/SystemTests/NetworkConnectivityTests/Subscriber/SubscriberNetworkConnectivityTests.swift create mode 100644 Tests/SystemTests/Utils/MultipleErrors.swift diff --git a/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/CombineSubscriberDelegate.swift b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/CombineSubscriberDelegate.swift new file mode 100644 index 000000000..bbdbf347f --- /dev/null +++ b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/CombineSubscriberDelegate.swift @@ -0,0 +1,87 @@ +import AblyAssetTrackingInternal +import AblyAssetTrackingSubscriber +import AblyAssetTrackingSubscriberTesting +import class Combine.CurrentValueSubject +import struct Combine.AnyPublisher + +extension SubscriberNetworkConnectivityTests { + /// A helper implementation of `SubscriberDelegate` which exposes Combine publishers which emit the events sent by a `Subscriber` to its delegate. + /// + /// The idea of this class is to allow us to keep the implementation of the subscriber `NetworkConnectivityTests` as similar to those of Android as possible. The Android `Subscriber` does not use a delegate pattern; rather, it uses [Kotlin flows](https://kotlinlang.org/docs/flow.html) to emit values over time. + /// + /// Furthermore, the Android version of the subscriber `NetworkConnectivityTests` relies on some specific behaviours of the flow types provided by Kotlin. This class reproduces those behaviours as much as is necessary to satisfy the tests. + class CombineSubscriberDelegate: SubscriberDelegate { + private let logHandler: InternalLogHandler + + /// Publishes the connection status values received by ``SubscriberDelegate.subscriber(sender:,didChangeAssetConnectionStatus:)``. + /// + /// This is meant to mimic Android’s `_trackableStates = MutableStateFlow(TrackableState.Offline())`. + /// + /// This publisher sends the latest received value to each new subscriber, to mimic the behaviour of a Kotlin `StateFlow`. Unlike a `StateFlow`, however, this publisher does not publish an initial value. + let trackableStates: AnyPublisher + private let trackableStatesSubject = CurrentValueSubject(nil) + + /// Publishes the resolution values received by ``SubscriberDelegate.subscriber(sender:,didUpdateResolution:)``. + /// + /// This is meant to mimic Android’s `_resolutions = MutableSharedFlow(replay = 1)`. + /// + /// This publisher sends the latest received value to each new subscriber, to mimic the behaviour of a Kotlin `SharedFlow` with `replay = 1`. + let resolutions: AnyPublisher + private let resolutionsSubject = CurrentValueSubject(nil) + + /// Publishes the resolution values received by ``SubscriberDelegate.subscriber(sender:,didUpdatePublisherPresence:)``. + /// + /// This is meant to mimic Android’s `_publisherPresence = MutableStateFlow(false)`. + /// + /// This publisher sends the latest received value to each new subscriber, to mimic the behaviour of a Kotlin `StateFlow`. Unlike a `StateFlow`, however, this publisher does not publish an initial value. + let publisherPresence: AnyPublisher + private let publisherPresenceSubject = CurrentValueSubject(nil) + + /// Publishes the location values received by ``SubscriberDelegate.subscriber(sender:,didUpdateEnhancedLocation:)``. + /// + /// This is meant to mimic Android’s `_enhancedLocations = MutableSharedFlow(replay = 1)`. + /// + /// This publisher sends the latest received value to each new subscriber, to mimic the behaviour of a Kotlin `SharedFlow` with `replay = 1`. + let locations: AnyPublisher + private let locationsSubject = CurrentValueSubject(nil) + + init(logHandler: InternalLogHandler) { + self.trackableStates = trackableStatesSubject.compactMap { $0 }.eraseToAnyPublisher() + self.resolutions = resolutionsSubject.compactMap { $0 }.eraseToAnyPublisher() + self.publisherPresence = publisherPresenceSubject.compactMap { $0 }.eraseToAnyPublisher() + self.locations = locationsSubject.compactMap { $0 }.eraseToAnyPublisher() + + self.logHandler = logHandler.addingSubsystem(Self.self) + } + + // MARK: SubscriberDelegate + + func subscriber(sender: Subscriber, didChangeAssetConnectionStatus status: ConnectionState) { + logHandler.debug(message: "Delegate received subscriber(sender:,didChangeAssetConnectionStatus:) - status \(status)", error: nil) + trackableStatesSubject.value = status + logHandler.debug(message: "Sent status \(status) to _trackableStates", error: nil) + } + + func subscriber(sender: Subscriber, didUpdateResolution resolution: Resolution) { + logHandler.debug(message: "Delegate received subscriber(sender:,didUpdateResolution:) - resolution \(resolution)", error: nil) + resolutionsSubject.value = resolution + logHandler.debug(message: "Sent resolution \(resolution) to _resolutions", error: nil) + } + + func subscriber(sender: Subscriber, didUpdatePublisherPresence isPresent: Bool) { + logHandler.debug(message: "Delegate received subscriber(sender:,didUpdatePublisherPresence:) - isPresent \(isPresent)", error: nil) + publisherPresenceSubject.value = isPresent + logHandler.debug(message: "Sent isPresent \(isPresent) to _publisherPresence", error: nil) + } + + func subscriber(sender: Subscriber, didUpdateEnhancedLocation locationUpdate: LocationUpdate) { + logHandler.debug(message: "Delegate received subscriber(sender:,didUpdateEnhancedLocation:) - locationUpdate \(locationUpdate)", error: nil) + locationsSubject.value = locationUpdate + logHandler.debug(message: "Sent locationUpdate to _locations", error: nil) + } + + func subscriber(sender: Subscriber, didFailWithError error: ErrorInformation) { + logHandler.error(message: "Delegate received subscriber(sender:,didFailWithError:", error: error) + } + } +} diff --git a/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/CombineSubscriberDelegateTests.swift b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/CombineSubscriberDelegateTests.swift new file mode 100644 index 000000000..d3d88a51b --- /dev/null +++ b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/CombineSubscriberDelegateTests.swift @@ -0,0 +1,47 @@ +import AblyAssetTrackingSubscriberTesting +import AblyAssetTrackingTesting +import Combine +import XCTest + +class CombineSubscriberDelegateTests: XCTestCase { + // The testing pattern is taken from + // https://heckj.github.io/swiftui-notes/#patterns-testing-and-debugging + + func testReplaysLastValueToAllNewSubscribers() { + let combineSubscriberDelegate = SubscriberNetworkConnectivityTests.CombineSubscriberDelegate( + logHandler: TestLogging.sharedInternalLogHandler + ) + + let subscriber = SubscriberMock() + + // Given... + // ...that the object under test has received an invocation of `subscriber(sender:,didChangeAssetConnectionStatus:)` + + combineSubscriberDelegate.subscriber(sender: subscriber, didChangeAssetConnectionStatus: .online) + + // When... + + var cancellables = Set() + let firstExpectation = expectation(description: "First subscriber gets value") + let secondExpectation = expectation(description: "Second subscriber gets value") + + // ...a subscriber is added to the object under test’s `trackableStates`... + combineSubscriberDelegate.trackableStates.sink { status in + XCTAssertEqual(status, .online) + firstExpectation.fulfill() + + // ...and, when that first subscriber receives a value, another subscriber is added to the object under test’s `trackableStates`... + combineSubscriberDelegate.trackableStates.sink { status in + XCTAssertEqual(status, .online) + secondExpectation.fulfill() + } + .store(in: &cancellables) + } + .store(in: &cancellables) + + // Then... + // ...both subscribers receive the connection status sent to the object under test’s `subscriber(sender:,didChangeAssetConnectionStatus:)` method. + + waitForExpectations(timeout: 10) + } +} diff --git a/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberMonitor.swift b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberMonitor.swift new file mode 100644 index 000000000..719e4bf0a --- /dev/null +++ b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberMonitor.swift @@ -0,0 +1,354 @@ +import Ably +import AblyAssetTrackingInternal +import AblyAssetTrackingSubscriber +import AblyAssetTrackingSubscriberTesting +import AblyAssetTrackingTesting +import Foundation +import protocol Combine.Publisher +import class Combine.AnyCancellable +import struct Combine.AnyPublisher + +extension SubscriberNetworkConnectivityTests { + /** + * Monitors Subscriber activity so that we can make assertions about any trackable state + * transitions expected and ensure side-effects occur. + */ + final class SubscriberMonitor { + struct SharedState { + fileprivate var combineSubscriberDelegate: CombineSubscriberDelegate + + init(combineSubscriberDelegate: CombineSubscriberDelegate) { + self.combineSubscriberDelegate = combineSubscriberDelegate + } + } + + private let sharedState: SharedState + private let logHandler: InternalLogHandler + private let subscriber: Subscriber + private let subscriberClientID: String + private let label: String + private let trackableID: String + private let expectedState: ConnectionState + private let failureStates: Set + private let expectedSubscriberPresence: Bool? + private let expectedPublisherPresence: Bool + private let expectedLocation: Location? + private let expectedPublisherResolution: Resolution? + private let expectedSubscriberResolution: Resolution? + private let timeout: TimeInterval + private let ably: ARTRealtime + private let subscriberResolutionPreferences: AnyPublisher + + private static let assertionsQueue = DispatchQueue(label: "com.ably.tracking.tests.SubscriberMonitor.assertions") + private static func runAsyncOnAssertionsQueue(logHandler: InternalLogHandler, block: @escaping () -> Void) { + logHandler.debug(message: "Dispatching block to assertionsQueue", error: nil) + assertionsQueue.async { + logHandler.debug(message: "Calling block on assertionsQueue", error: nil) + block() + logHandler.debug(message: "Block finished executing on assertionsQueue", error: nil) + } + } + + init(sharedState: SharedState, logHandler: InternalLogHandler, subscriber: Subscriber, subscriberClientID: String, label: String, trackableID: String, expectedState: ConnectionState, failureStates: Set, expectedSubscriberPresence: Bool?, expectedPublisherPresence: Bool, expectedLocation: Location?, expectedPublisherResolution: Resolution?, expectedSubscriberResolution: Resolution?, timeout: TimeInterval, subscriberResolutionPreferences: AnyPublisher) { + self.sharedState = sharedState + self.logHandler = logHandler.addingSubsystem(.named("SubscriberMonitor(\(label))")) + self.subscriber = subscriber + self.subscriberClientID = subscriberClientID + self.label = label + self.trackableID = trackableID + self.expectedState = expectedState + self.failureStates = failureStates + self.expectedSubscriberPresence = expectedSubscriberPresence + self.expectedPublisherPresence = expectedPublisherPresence + self.expectedLocation = expectedLocation + self.expectedPublisherResolution = expectedPublisherResolution + self.expectedSubscriberResolution = expectedSubscriberResolution + self.timeout = timeout + self.subscriberResolutionPreferences = subscriberResolutionPreferences + + let clientOptions = ARTClientOptions(key: Secrets.ablyApiKey) + clientOptions.clientId = "SubscriberMonitor-\(trackableID)" + clientOptions.logHandler = InternalARTLogHandler(logHandler: self.logHandler) + clientOptions.logLevel = .verbose + ably = ARTRealtime(options: clientOptions) + } + + // MARK: - State transition + + /** + * Performs the given async operation, then waits for expectations to + * be delivered (or not) before cleaning up. + */ + func waitForStateTransition( _ asyncOp: (InternalLogHandler, @escaping (Result) -> Void) -> Void) throws { + logHandler.logMessage(level: .debug, message: "waitForStateTransition called; calling asyncOp", error: nil) + + var errors = MultipleErrors() + + /* + We want the assertions to piggy-back on the same timeout as the "asyncOp and assertions complete" operation. This gives us the same behaviour as Android. + + To achieve this, we need to use a waiter context, since these operations take place on different threads (SubscriberMonitor.assertionsQueue and the main thread respectively). + */ + let waiterContext = Blocking.WaiterContext(logHandler: logHandler) + defer { waiterContext.cancel() } + + do { + try Blocking.run(label: "asyncOp and assertions complete", timeout: timeout, logHandler: logHandler, waiterContext: waiterContext) { (handler: @escaping (Result) -> Void) in + let asyncOpLogHandler = logHandler.addingSubsystem(.named("asyncOp")) + + asyncOp(asyncOpLogHandler) { [logHandler] result in + /* + Consider the following sequence of events: + + 1. asyncOp completes by calling `Dispatch.async`-ing its callback to the main queue (e.g. when asyncOp is an AAT operation, since AAT performs all public callbacks on the main queue) + 2. The subsequent assert* operations wait for various callbacks, some of are generated by ably-cocoa and hence (since DefaultAbly doesn’t specify a custom ARTClientOptions.dispatchQueue) `Dispatch.async`-ed to the main queue. + + If we were to perform 2 synchronously inside 1, then 2 would never complete since the callbacks which it is waiting for would never get executed (since they need 1 to first end and free up the main queue). Hence we perform 2 on a non-main queue in order to get out of 1’s Dispatch.async main queue callback. + */ + + SubscriberMonitor.runAsyncOnAssertionsQueue(logHandler: logHandler) { [weak self] in + guard let self else { + return + } + + switch result { + case .success: + logHandler.logMessage(level: .debug, message: "waitForStateTransition’s asyncOp succeeded", error: nil) + + do { + // These methods all tell Blocking.run to wait indefinitely, but — as mentioned above — since they share a waiter context with the "asyncOp and assertions complete" operation then if that operation’s timeout elapses these methods will fail, which is the behaviour we want. + + try self.assertStateTransition(waiterContext: waiterContext) + try self.assertSubscriberPresence(waiterContext: waiterContext) + try self.assertPublisherPresence(waiterContext: waiterContext) + try self.assertLocationUpdated(waiterContext: waiterContext) + try self.assertPublisherResolution(waiterContext: waiterContext) + try self.assertSubscriberPreferredResolution(waiterContext: waiterContext) + handler(.success) + } catch { + handler(.failure(error)) + } + case .failure(let error): + logHandler.logMessage(level: .error, message: "waitForStateTransition’s asyncOp failed", error: error) + handler(.failure(error)) + } + } + } + } + } catch { + errors.add(error) + } + + do { + try close() + } catch { + errors.add(error) + } + + try errors.check() + } + + // MARK: - Lifecycle + + /** + * Close any open resources used by this monitor, in a blocking fashion. + */ + private func close() throws { + if ably.connection.state == .closed { + return + } + + try Blocking.run(label: "Wait for Ably to close", timeout: 10, logHandler: logHandler) { (handler: @escaping ResultHandler) in + ably.connection.once(.closed) { _ in + handler(.success) + } + ably.close() + } + } + + // MARK: - Utility + + @discardableResult + private func first(_ publisher: any Publisher, label: String, waiterContext: Blocking.WaiterContext) throws -> Output { + var subscriptions: Set = [] + + return try withExtendedLifetime(subscriptions) { + try Blocking.run(label: label, timeout: nil, logHandler: logHandler, waiterContext: waiterContext) { (handler: (@escaping (Result) -> Void)) in + publisher + .eraseToAnyPublisher() + .first() + .sink { handler(.success($0)) } + .store(in: &subscriptions) + } + } + } + + // MARK: - Assertions + + private enum AssertionError: Error { + case generic(message: String) + } + + private func createGenericAssertionError(message: String) -> AssertionError { + let taggedMessage = logHandler.tagMessage(message) + return .generic(message: taggedMessage) + } + + private func assertStateTransition(waiterContext: Blocking.WaiterContext) throws { + logHandler.logMessage(level: .debug, message: "Awaiting state transition to \(expectedState)", error: nil) + + let publisher = sharedState.combineSubscriberDelegate.trackableStates + .compactMap { [failureStates, expectedState, logHandler] state in + SubscriberMonitor.receive( + state, + failureStates: failureStates, + expectedState: expectedState, + logHandler: logHandler + ) + } + + let success = try first(publisher, label: "Wait for state transition to \(expectedState)", waiterContext: waiterContext) + if !success { + throw createGenericAssertionError(message: "Wait for state transition to \(expectedState) did not result in success.") + } + } + + /** + * Maps received `ConnectionState` to a success/fail/ignore outcome for this test. + */ + private static func receive(_ state: ConnectionState, failureStates: Set, expectedState: ConnectionState, logHandler: InternalLogHandler) -> Bool? { + if failureStates.contains(state) { + logHandler.logMessage(level: .error, message: "(FAIL) Got state \(state)", error: nil) + return false + } + + if state == expectedState { + logHandler.logMessage(level: .debug, message: "(SUCCESS) Got state \(state)", error: nil) + return true + } + + logHandler.logMessage(level: .debug, message: "(IGNORED) Got state \(state)", error: nil) + return nil + } + + private func assertPublisherResolution(waiterContext: Blocking.WaiterContext) throws { + guard let expectedPublisherResolution else { + logHandler.debug(message: "(SKIP) expectedPublisherResolution = nil", error: nil) + return + } + + logHandler.debug(message: "(WAITING) expectedPublisherResolution = \(expectedPublisherResolution)", error: nil) + + try listenForExpectedPublisherResolution(waiterContext: waiterContext) + } + + /** + * Uses the subscribers presence state flow to listen for the expected + * publisher resolution change. + * + * This can happen at any time after the initial trackable state transition, + * and so we cannot rely on the first state we collect being the "newest" one. + */ + private func listenForExpectedPublisherResolution(waiterContext: Blocking.WaiterContext) throws { + let publisher = sharedState.combineSubscriberDelegate.resolutions.filter { [expectedPublisherResolution] in $0 == expectedPublisherResolution } + try first(publisher, label: "wait for matching publisher resolution", waiterContext: waiterContext) + } + + private func assertSubscriberPresence(waiterContext: Blocking.WaiterContext) throws { + guard let expectedSubscriberPresence else { + // not checking for subscriber presence in this test + logHandler.debug(message: "(SKIP) expectedSubscriberPresence = nil", error: nil) + return + } + + let subscriberPresent = try subscriberIsPresent(waiterContext: waiterContext) + if subscriberPresent != expectedSubscriberPresence { + logHandler.logMessage(level: .error, message: "(FAIL) subscriberPresent = \(subscriberPresent)", error: nil) + throw createGenericAssertionError(message: "Expected subscriberPresence: \(expectedSubscriberPresence) but got \(subscriberPresent)") + } else { + logHandler.debug(message: "(PASS) subscriberPresent = \(subscriberPresent)", error: nil) + } + } + + /** + * Perform a request to the Ably API to get a snapshot of the current presence for the channel, + * and check to see if the Subscriber's clientId is present in that snapshot. + */ + private func subscriberIsPresent(waiterContext: Blocking.WaiterContext) throws -> Bool { + let members = try Blocking.run(label: "subscriberIsPresent fetch present members", timeout: nil, logHandler: logHandler, waiterContext: waiterContext) { (handler: @escaping (Result<[ARTPresenceMessage], Error>) -> Void) in + let channel = ably.channels.get("tracking:\(trackableID)") + + channel.presence.get { members, error in + if let error { + handler(.failure(error)) + return + } + + handler(.success(members!)) + } + } + + logHandler.debug(message: "subscriberIsPresent fetched members \(members)", error: nil) + + return members.contains { $0.clientId == subscriberClientID && $0.action == .present } + } + + /** + * Assert that we eventually receive the expected publisher presence. + * + * This can happen at any time after the initial trackable state transition, + * and so we cannot rely on the first state we collect being the "new" one. + */ + private func assertPublisherPresence(waiterContext: Blocking.WaiterContext) throws { + logHandler.debug(message: "(WAITING): publisher presence -> \(String(describing: expectedPublisherPresence))", error: nil) + + let publisher = sharedState.combineSubscriberDelegate.publisherPresence.filter { [expectedPublisherPresence] in $0 == expectedPublisherPresence } + let presence = try first(publisher, label: "assertPublisherPresence wait for publisher presence", waiterContext: waiterContext) + + logHandler.debug(message: "(PASS): publisher presence was \(presence)", error: nil) + } + + /** + * Throw an assertion error if expectations about published location updates have not + * been meet in this test. + */ + private func assertLocationUpdated(waiterContext: Blocking.WaiterContext) throws { + guard let expectedLocation else { + // no expected location set - skip assertion + logHandler.debug(message: "(SKIP) expectedLocationUpdate = null", error: nil) + return + } + + logHandler.debug(message: "(WAITING) expectedLocationUpdate = \(expectedLocation)", error: nil) + + try listenForExpectedLocationUpdate(waiterContext: waiterContext) + } + + /** + * Use the subscriber location flow to listen for a location update matching the one we're expecting. + * + * These location updates may arrive at any time after the trackable transitions to online, so we therefore + * cannot rely on the first thing we find being the "newest" state and therefore must wait for a bit. + */ + private func listenForExpectedLocationUpdate(waiterContext: Blocking.WaiterContext) throws { + let publisher = sharedState.combineSubscriberDelegate.locations.filter { [expectedLocation] in $0.location == expectedLocation }.map(\.location) + + try first(publisher, label: "Wait for location \(String(describing: expectedLocation)) from subscriber", waiterContext: waiterContext) + } + + /** + * Assert that we receive the expected subscriber resolution. + */ + private func assertSubscriberPreferredResolution(waiterContext: Blocking.WaiterContext) throws { + guard let expectedSubscriberResolution else { + logHandler.debug(message: "(SKIPPED) expectedSubscriberResolution = nil", error: nil) + return + } + + logHandler.debug(message: "(WAITING) preferredSubscriberResolution = \(expectedSubscriberResolution)", error: nil) + + let publisher = subscriberResolutionPreferences.filter { $0 == expectedSubscriberResolution } + try first(publisher, label: "Wait for subscriber resolution preference \(expectedSubscriberResolution)", waiterContext: waiterContext) + } + } +} diff --git a/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberMonitorFactory.swift b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberMonitorFactory.swift new file mode 100644 index 000000000..5dc33ad24 --- /dev/null +++ b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberMonitorFactory.swift @@ -0,0 +1,319 @@ +import AblyAssetTrackingCore +import AblyAssetTrackingInternal +import AblyAssetTrackingSubscriber +import AblyAssetTrackingSubscriberTesting +import Foundation +import struct Combine.AnyPublisher + +extension SubscriberNetworkConnectivityTests { + class SubscriberMonitorFactory { + private let sharedState: SubscriberMonitor.SharedState + private let subscriber: Subscriber + private let logHandler: InternalLogHandler + private let trackableID: String + private let faultType: FaultTypeDTO + private let subscriberClientID: String + private let subscriberResolutionPreferences: AnyPublisher + + init(subscriber: Subscriber, combineSubscriberDelegate: CombineSubscriberDelegate, logHandler: InternalLogHandler, trackableID: String, faultType: FaultTypeDTO, subscriberClientID: String, subscriberResolutionPreferences: AnyPublisher) { + self.sharedState = .init(combineSubscriberDelegate: combineSubscriberDelegate) + self.subscriber = subscriber + self.logHandler = logHandler + self.trackableID = trackableID + self.faultType = faultType + self.subscriberClientID = subscriberClientID + self.subscriberResolutionPreferences = subscriberResolutionPreferences + } + + /** + * Construct `SubscriberMonitor` configured to expect appropriate state transitions + * for the given fault type while it is active. `label` will be used for logging captured transitions. + */ + func forActiveFault( + label: String, + locationUpdate: Location? = nil, + publisherResolution: Resolution? = nil, + publisherDisconnected: Bool = false, + subscriberResolution: Resolution? = nil + ) -> SubscriberMonitor { + let expectedState: ConnectionState + switch faultType { + case .fatal: + expectedState = .failed + case .nonfatal where publisherDisconnected: + expectedState = .offline + case .nonfatal: + expectedState = .online + case .nonfatalWhenResolved: + expectedState = .offline + } + + let failureStates: Set + switch faultType { + case .fatal: + failureStates = [.offline] + case .nonfatal, .nonfatalWhenResolved: + failureStates = [.failed] + } + + let expectedSubscriberPresence: Bool? + switch faultType { + case .nonfatal: + expectedSubscriberPresence = true + case .nonfatalWhenResolved: + expectedSubscriberPresence = nil + case .fatal: + expectedSubscriberPresence = false + } + + let expectedPublisherPresence: Bool + switch faultType { + case .nonfatal: + expectedPublisherPresence = !publisherDisconnected + case .nonfatalWhenResolved, .fatal: + expectedPublisherPresence = false + } + + let timeout: TimeInterval + switch faultType { + case let .fatal(failedWithinMillis: failedWithinMillis): + timeout = Double(failedWithinMillis) / 1000 + case let .nonfatal(resolvedWithinMillis: resolvedWithinMillis): + timeout = Double(resolvedWithinMillis) / 1000 + case let .nonfatalWhenResolved(offlineWithinMillis: offlineWithinMillis, _): + timeout = Double(offlineWithinMillis) / 1000 + } + + return SubscriberMonitor( + sharedState: sharedState, + logHandler: logHandler, + subscriber: subscriber, + subscriberClientID: subscriberClientID, + label: label, + trackableID: trackableID, + expectedState: expectedState, + failureStates: failureStates, + expectedSubscriberPresence: expectedSubscriberPresence, + expectedPublisherPresence: expectedPublisherPresence, + expectedLocation: locationUpdate, + expectedPublisherResolution: publisherResolution, + expectedSubscriberResolution: subscriberResolution, + timeout: timeout, + subscriberResolutionPreferences: subscriberResolutionPreferences + ) + } + + /** + * Construct `SubscriberMonitor` configured to expect appropriate state transitions + * for the given fault type while it is active but the subscriber is shutting down. + * + * `label` will be used for logging captured transitions. + */ + func forActiveFaultWhenShuttingDownSubscriber( + label: String, + locationUpdate: Location? = nil, + publisherResolution: Resolution? = nil, + publisherDisconnected: Bool = false, + subscriberResolution: Resolution? = nil + ) -> SubscriberMonitor { + let expectedState: ConnectionState + switch faultType { + case .fatal: + expectedState = .failed + case .nonfatal, .nonfatalWhenResolved: + expectedState = .offline + } + + let failureStates: Set + switch faultType { + case .fatal: + failureStates = [.offline] + case .nonfatal, .nonfatalWhenResolved: + failureStates = [.failed] + } + + let expectedSubscriberPresence: Bool + switch faultType { + case .nonfatal, .fatal: + expectedSubscriberPresence = false + case .nonfatalWhenResolved: + expectedSubscriberPresence = true + } + + let expectedPublisherPresence: Bool + switch faultType { + case .nonfatal: + expectedPublisherPresence = !publisherDisconnected + case .nonfatalWhenResolved, .fatal: + expectedPublisherPresence = false + } + + let timeout: TimeInterval + switch faultType { + case .fatal(let failedWithinMillis): + timeout = Double(failedWithinMillis) / 1000 + case .nonfatal(let resolvedWithinMillis): + timeout = Double(resolvedWithinMillis) / 1000 + case .nonfatalWhenResolved(let offlineWithinMillis, _): + timeout = Double(offlineWithinMillis) / 1000 + } + + return SubscriberMonitor( + sharedState: sharedState, + logHandler: logHandler, + subscriber: subscriber, + subscriberClientID: subscriberClientID, + label: label, + trackableID: trackableID, + expectedState: expectedState, + failureStates: failureStates, + expectedSubscriberPresence: expectedSubscriberPresence, + expectedPublisherPresence: expectedPublisherPresence, + expectedLocation: locationUpdate, + expectedPublisherResolution: publisherResolution, + expectedSubscriberResolution: subscriberResolution, + timeout: timeout, + subscriberResolutionPreferences: subscriberResolutionPreferences + ) + } + + /** + * Construct a `SubscriberMonitor` configured to expect appropriate transitions for + * the given fault type after it has been resolved. `label` is used for logging. + */ + func forResolvedFault( + label: String, + locationUpdate: Location? = nil, + publisherResolution: Resolution? = nil, + expectedPublisherPresence: Bool = true, + subscriberResolution: Resolution? = nil + ) -> SubscriberMonitor { + let expectedState: ConnectionState + if !expectedPublisherPresence { + expectedState = .offline + } else { + switch faultType { + case .fatal: + expectedState = .failed + case .nonfatal, .nonfatalWhenResolved: + expectedState = .online + } + } + + let failureStates: Set + switch faultType { + case .fatal: + failureStates = [.offline, .online] + case .nonfatal, .nonfatalWhenResolved: + failureStates = [.failed] + } + + let expectedSubscriberPresence: Bool + switch faultType { + case .fatal: + expectedSubscriberPresence = false + case .nonfatal, .nonfatalWhenResolved: + expectedSubscriberPresence = true + } + + let timeout: TimeInterval + switch faultType { + case .fatal(let failedWithinMillis): + timeout = Double(failedWithinMillis) / 1000 + case .nonfatal(let resolvedWithinMillis): + timeout = Double(resolvedWithinMillis) / 1000 + case .nonfatalWhenResolved(let offlineWithinMillis, _): + timeout = Double(offlineWithinMillis) / 1000 + } + + return SubscriberMonitor( + sharedState: sharedState, + logHandler: logHandler, + subscriber: subscriber, + subscriberClientID: subscriberClientID, + label: label, + trackableID: trackableID, + expectedState: expectedState, + failureStates: failureStates, + expectedSubscriberPresence: expectedSubscriberPresence, + expectedPublisherPresence: expectedPublisherPresence, + expectedLocation: locationUpdate, + expectedPublisherResolution: publisherResolution, + expectedSubscriberResolution: subscriberResolution, + timeout: timeout, + subscriberResolutionPreferences: subscriberResolutionPreferences + ) + } + + /** + * Construct a `SubscriberMonitor` configured to expect appropriate transitions for + * the given fault type after it has been resolved and the publisher is stopped. + * + * `label` is used for logging. + */ + func forResolvedFaultWithSubscriberStopped( + label: String, + locationUpdate: Location? = nil, + publisherResolution: Resolution? = nil, + subscriberResolution: Resolution? = nil + ) -> SubscriberMonitor { + let timeout: TimeInterval + switch faultType { + case .fatal(let failedWithinMillis): + timeout = Double(failedWithinMillis) / 1000 + case .nonfatal(let resolvedWithinMillis): + timeout = Double(resolvedWithinMillis) / 1000 + case .nonfatalWhenResolved(let offlineWithinMillis, _): + timeout = Double(offlineWithinMillis) / 1000 + } + + return SubscriberMonitor( + sharedState: sharedState, + logHandler: logHandler, + subscriber: subscriber, + subscriberClientID: subscriberClientID, + label: label, + trackableID: trackableID, + expectedState: .offline, + failureStates: [.failed], + expectedSubscriberPresence: false, + expectedPublisherPresence: true, + expectedLocation: locationUpdate, + expectedPublisherResolution: publisherResolution, + expectedSubscriberResolution: subscriberResolution, + timeout: timeout, + subscriberResolutionPreferences: subscriberResolutionPreferences + ) + } + + /** + * Construct a `SubscriberMonitor` configured to expect a Trackable to come + * online within a given timeout, and fail if the Failed state is seen at any point. + */ + func onlineWithoutFail( + label: String, + timeout: TimeInterval, + subscriberResolution: Resolution? = nil, + locationUpdate: Location? = nil, + publisherResolution: Resolution? = nil + ) -> SubscriberMonitor { + SubscriberMonitor( + sharedState: sharedState, + logHandler: logHandler, + subscriber: subscriber, + subscriberClientID: subscriberClientID, + label: label, + trackableID: trackableID, + expectedState: .online, + failureStates: [.failed], + expectedSubscriberPresence: true, + expectedPublisherPresence: true, + expectedLocation: locationUpdate, + expectedPublisherResolution: publisherResolution, + expectedSubscriberResolution: subscriberResolution, + timeout: timeout, + subscriberResolutionPreferences: subscriberResolutionPreferences + ) + } + } +} diff --git a/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberNetworkConnectivityTestsParam.swift b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberNetworkConnectivityTestsParam.swift new file mode 100644 index 000000000..68ec3f04c --- /dev/null +++ b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/SubscriberNetworkConnectivityTestsParam.swift @@ -0,0 +1,75 @@ +import AblyAssetTrackingTesting + +struct SubscriberNetworkConnectivityTestsParam: ParameterizedTestCaseParam { + static func fetchParams(_ completion: @escaping (Result<[SubscriberNetworkConnectivityTestsParam], Error>) -> Void) { + let logHandler = TestLogging.sharedInternalLogHandler.addingSubsystem(.typed(self)) + var proxyClient: SDKTestProxyClient? = SDKTestProxyClient(logHandler: logHandler) + + proxyClient!.getAllFaults { result in + let paramsResult = result.map { faultNames in + faultNames.map(SubscriberNetworkConnectivityTestsParam.init(faultName:)) + } + proxyClient = nil + completion(paramsResult) + } + } + + var faultName: String + + var methodNameComponent: String { + faultName + } + + /// If this returns `true`, then all test cases will be skipped for this parameter. + /// + /// This allows us to write test cases for faults that might not yet be handled by the SDK, and be sure that they compile. + /// + /// As faults become properly handled by the SDK, they should be removed here. And, as new faults are introduced which are not yet properly handled by the SDK, they should be added here. + var isSkipped: Bool { + [ + // Failures: + // test_faultBeforeStartingSubscriber_TcpConnectionRefused + // test_faultBeforeStoppingSubscriber_TcpConnectionRefused + // test_faultWhilstTracking_TcpConnectionRefused + "TcpConnectionRefused", + + // Failures: + // test_faultBeforeStartingSubscriber_TcpConnectionUnresponsive + // test_faultBeforeStoppingSubscriber_TcpConnectionUnresponsive + // test_faultWhilstTracking_TcpConnectionUnresponsive + "TcpConnectionUnresponsive", + + // Failures: + // test_faultBeforeStartingSubscriber_AttachUnresponsive + "AttachUnresponsive", + + // Failures: + // test_faultBeforeStartingSubscriber_DisconnectWithFailedResume + "DisconnectWithFailedResume", + + // Failures: + // test_faultBeforeStartingSubscriber_EnterFailedWithNonfatalNack + "EnterFailedWithNonfatalNack", + + // Failures: + // test_faultBeforeStartingSubscriber_UpdateFailedWithNonfatalNack + // test_faultWhilstTracking_UpdateFailedWithNonfatalNack + "UpdateFailedWithNonfatalNack", + + // Failures: + // test_faultBeforeStartingSubscriber_DisconnectAndSuspend + // test_faultBeforeStoppingSubscriber_DisconnectAndSuspend + // test_faultWhilstTracking_DisconnectAndSuspend + "DisconnectAndSuspend", + + // Failures: + // test_faultBeforeStoppingSubscriber_ReenterOnResumeFailed + // test_faultWhilstTracking_ReenterOnResumeFailed + "ReenterOnResumeFailed", + + // Failures: + // test_faultBeforeStartingSubscriber_EnterUnresponsive + "EnterUnresponsive" + ].contains(faultName) + } +} diff --git a/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/TestResources.swift b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/TestResources.swift new file mode 100644 index 000000000..a7637ecc2 --- /dev/null +++ b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/Helper/TestResources.swift @@ -0,0 +1,310 @@ +import Ably +import AblyAssetTrackingInternal +import AblyAssetTrackingInternalTesting +import AblyAssetTrackingPublisherTesting +@testable import AblyAssetTrackingSubscriber +import AblyAssetTrackingSubscriberTesting +import AblyAssetTrackingTesting +import XCTest +import class Combine.CurrentValueSubject +import struct Combine.AnyPublisher + +extension SubscriberNetworkConnectivityTests { + /// For the convenience of test cases, the methods of this class are blocking, except for those whose name contains `Async`. + final class TestResources { + private let proxyClient: SDKTestProxyClient + let faultSimulation: FaultSimulationDTO + let trackableID: String + private var ablyPublishing: AblyPublishing? + let logHandler: InternalLogHandler + private let subscriberClientID: String + private var subscriberTestEnvironment: SubscriberTestEnvironment? + + private struct AblyPublishing { + var defaultAbly: DefaultAbly + // To maintain a strong reference to prevent it being deallocated + private var delegate: AblyPublisherDelegate + + init(defaultAbly: DefaultAbly, delegate: AblyPublisherDelegate) { + self.defaultAbly = defaultAbly + self.delegate = delegate + } + } + + /// Provides a `Subscriber` instance and any other resources useful for testing it. + struct SubscriberTestEnvironment { + /// This subscriber has already been started. + /// + /// Its delegate is configured such that ``subscriberMonitorFactory`` can create monitors which can make assertions about delegate events. You should not change the value of its `delegate` property. + var subscriber: Subscriber + + /// A factory for creating instances of ``SubscriberMonitor`` for the subscriber contained in the ``subscriber`` property. + var subscriberMonitorFactory: SubscriberMonitorFactory + } + + private let subscriberResolutionsSubject = CurrentValueSubject(nil) + /// Re-publishes the latest received value to each new subscriber, to mimic behaviour of a Kotlin SharedFlow with replay 1. This lets us make the same assertions as in the Android version of these tests. + let subscriberResolutions: AnyPublisher + + enum TestResourcesError: Error { + case ablyPublishingDidNotComeOnline + } + + init(testCase: XCTestCase, proxyClient: SDKTestProxyClient, faultSimulation: FaultSimulationDTO, trackableID: String, logHandler: InternalLogHandler) { + self.proxyClient = proxyClient + self.faultSimulation = faultSimulation + self.trackableID = trackableID + self.logHandler = logHandler + + logHandler.logMessage(level: .info, message: "Created TestResources", error: nil) + + self.subscriberClientID = "AATNetworkConnectivityTests_Subscriber-\(trackableID)" + self.subscriberResolutions = subscriberResolutionsSubject.compactMap { $0 }.eraseToAnyPublisher() + } + + private func runBlocking(label: String, timeout: TimeInterval?, _ operation: (@escaping (Result) -> Void) -> Void) throws -> Success { + try Blocking.run(label: label, timeout: timeout, logHandler: logHandler, operation) + } + + private func shutdownSubscriber() throws { + guard let subscriberTestEnvironment else { + return + } + + defer { + self.subscriberTestEnvironment = nil + } + + try runBlocking(label: "Shut down subscriber", timeout: 10) { handler in + subscriberTestEnvironment.subscriber.stop(completion: handler) + } + } + + func tearDown() throws { + var errors = MultipleErrors() + + do { + try shutdownSubscriber() + } catch { + logHandler.error(message: "tearDown failed to shut down subscriber", error: error) + errors.add(error) + } + + do { + try runBlocking(label: "Shut down Ably publishing in tearDown", timeout: 10) { handler in + shutdownAblyPublishingAsync(handler) + } + } catch { + logHandler.error(message: "tearDown failed to shut down Ably publishing", error: error) + errors.add(error) + } + + do { + try runBlocking(label: "Clean up fault simulation \(faultSimulation.id)", timeout: 10) { handler in + proxyClient.cleanUpFaultSimulation(withID: faultSimulation.id, handler) + } + } catch { + logHandler.error(message: "tearDown failed to clean up fault simulation \(faultSimulation.id)", error: error) + errors.add(error) + } + + try errors.check() + } + + func enableFault() throws { + try runBlocking(label: "Enable fault simulation \(faultSimulation.id)", timeout: 10) { proxyClient.enableFaultSimulation(withID: faultSimulation.id, $0) } + } + + func enableFaultAsync(_ completionHandler: @escaping (Result) -> Void) { + proxyClient.enableFaultSimulation(withID: faultSimulation.id, completionHandler) + } + + func resolveFaultAsync(_ completionHandler: @escaping (Result) -> Void) { + proxyClient.resolveFaultSimulation(withID: faultSimulation.id, completionHandler) + } + + /// Fetches an Ably token suitable for creating a `Subscriber`. + private func fetchTokenAsync(_ completion: @escaping (Result) -> Void) { + let clientOptions = ARTClientOptions(key: Secrets.ablyApiKey) + clientOptions.clientId = subscriberClientID + let rest = ARTRest(options: clientOptions) + + rest.auth.requestToken { tokenDetails, error in + if let error { + completion(.failure(error)) + return + } + + let unwrappedTokenDetails = tokenDetails! + + let aatTokenDetails = TokenDetails( + token: unwrappedTokenDetails.token, + expires: unwrappedTokenDetails.expires!, + issued: unwrappedTokenDetails.issued!, + capability: unwrappedTokenDetails.capability!, + clientId: unwrappedTokenDetails.clientId! + ) + + completion(.success(aatTokenDetails)) + } + } + + func createSubscriber() throws -> SubscriberTestEnvironment { + try runBlocking(label: "Create subscriber test environment", timeout: 10) { handler in + createSubscriberAsync(handler) + } + } + + private func createSubscriberAsync(_ completion: @escaping (Result) -> Void) { + if let subscriberTestEnvironment { + completion(.success(subscriberTestEnvironment)) + } + + fetchTokenAsync { [weak self] result in + guard let self else { + return + } + + let token: TokenDetails + switch result { + case .success(let fetchedToken): + token = fetchedToken + case .failure(let error): + completion(.failure(error)) + return + } + + let connectionConfiguration = ConnectionConfiguration { _, completion in + completion(.success(.tokenDetails(token))) + } + + let host = Host( + realtimeHost: self.proxyClient.baseURL.host!, + port: self.faultSimulation.proxy.listenPort, + tls: false + ) + + let logHandler = self.logHandler + .addingSubsystem(.assetTracking) + .addingSubsystem(.named("subscriber")) + + let defaultAbly = DefaultAbly( + factory: AblyCocoaSDKRealtimeFactory(), + configuration: connectionConfiguration, + host: host, + mode: .subscribe, + logHandler: logHandler + ) + + let resolution = Resolution(accuracy: .balanced, desiredInterval: 1, minimumDisplacement: 0) + + let subscriber = DefaultSubscriber( + ablySubscriber: defaultAbly, + trackableId: self.trackableID, + resolution: resolution, + logHandler: logHandler + ) + + let combineSubscriberDelegate = CombineSubscriberDelegate(logHandler: logHandler) + subscriber.delegate = combineSubscriberDelegate + + let subscriberMonitorFactory = SubscriberMonitorFactory( + subscriber: subscriber, + combineSubscriberDelegate: combineSubscriberDelegate, + logHandler: logHandler, + trackableID: self.trackableID, + faultType: self.faultSimulation.type, + subscriberClientID: self.subscriberClientID, + subscriberResolutionPreferences: self.subscriberResolutions + ) + + let testEnvironment = SubscriberTestEnvironment( + subscriber: subscriber, + subscriberMonitorFactory: subscriberMonitorFactory + ) + self.subscriberTestEnvironment = testEnvironment + + subscriber.start { result in + switch result { + case .success: + completion(.success(testEnvironment)) + case .failure(let error): + completion(.failure(error)) + } + } + } + } + + private let ablyPublishingPresenceData = PresenceData(type: .publisher, resolution: .init(accuracy: .balanced, desiredInterval: 1, minimumDisplacement: 0)) + + func createAndStartPublishingAblyConnection() throws -> DefaultAbly { + if let ablyPublishing { + return ablyPublishing.defaultAbly + } + + // Configure connection options + let connectionConfiguration = ConnectionConfiguration(apiKey: Secrets.ablyApiKey, clientId: "AATNetworkConnectivityTests_ablyPublishing-\(trackableID)") + + // Connect to Ably + + let defaultAbly = DefaultAbly(factory: AblyCocoaSDKRealtimeFactory(), configuration: connectionConfiguration, host: nil, mode: .publish, logHandler: logHandler.addingSubsystem(.named("ablyPublishing"))) + + let delegateMock = AblyPublisherDelegateMock() + defaultAbly.publisherDelegate = delegateMock + + ablyPublishing = .init(defaultAbly: defaultAbly, delegate: delegateMock) + + try runBlocking(label: "Connect to Ably", timeout: 10) { handler in + defaultAbly.connect(trackableId: trackableID, presenceData: ablyPublishingPresenceData, useRewind: true, completion: handler) + } + + // The Android version of these tests then calls defaultAbly.subscribeForChannelStateChange and waits for that to emit an online state. Given that the above call to `connect` has succeeded, the channel is _already_ in an online state. But Android’s DefaultAbly implementation of subscribeForChannelStateChange re-emits the current channel state, which our implementation does not. So I’ve omitted this channel state change wait in the Swift tests since it would always fail. + + let stateChangeExpectation = XCTestExpectation(description: "Channel state set to online") + delegateMock.ablyPublisherDidChangeChannelConnectionStateForTrackableClosure = { _, state, _ in + if state == .online { + stateChangeExpectation.fulfill() + } + } + + // Listen for presence and resolution updates + delegateMock.ablyPublisherDidReceivePresenceUpdateForTrackablePresenceDataClientIdClosure = { [subscriberResolutionsSubject, logHandler] _, _, _, presenceData, _ in + if presenceData.type == .subscriber, let resolution = presenceData.resolution { + logHandler.debug(message: "ablyPublishing received subscriber resolution \(resolution), emitting on subscriberResolutions", error: nil) + subscriberResolutionsSubject.value = resolution + } + } + defaultAbly.subscribeForPresenceMessages(trackable: .init(id: trackableID)) + + return defaultAbly + } + + /** + * If the test has started up a publishing connection to the Ably + * channel, shut it down. + */ + func shutdownAblyPublishingAsync(_ completionHandler: @escaping (Result) -> Void) { + guard let ablyPublishing else { + completionHandler(.success) + return + } + + logHandler.debug(message: "Shutting down Ably publishing connection", error: nil) + + ablyPublishing.defaultAbly.close(presenceData: ablyPublishingPresenceData) { [weak self] result in + guard let self else { + return + } + + switch result { + case .success: + self.ablyPublishing = nil + self.logHandler.debug(message: "Ably publishing connection shutdown", error: nil) + completionHandler(.success) + case .failure(let error): + completionHandler(.failure(error)) + } + } + } + } +} diff --git a/Tests/SystemTests/NetworkConnectivityTests/Subscriber/SubscriberNetworkConnectivityTests.swift b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/SubscriberNetworkConnectivityTests.swift new file mode 100644 index 000000000..a5f48761f --- /dev/null +++ b/Tests/SystemTests/NetworkConnectivityTests/Subscriber/SubscriberNetworkConnectivityTests.swift @@ -0,0 +1,392 @@ +import AblyAssetTrackingCore +import AblyAssetTrackingInternal +@testable import AblyAssetTrackingSubscriber +import AblyAssetTrackingTesting +import XCTest + +final class SubscriberNetworkConnectivityTests: ParameterizedTestCase { + // Implementation of required `ParameterizedTestCase` method. + override class func fetchParams(_ completion: @escaping (Result<[SubscriberNetworkConnectivityTestsParam], Error>) -> Void) { + SubscriberNetworkConnectivityTestsParam.fetchParams(completion) + } + + private var _testResources: TestResources! + /// Convenience non-optional getter, to be used within test cases. + private var testResources: TestResources { + _testResources + } + + override func setUpWithError() throws { + guard !currentParam.isSkipped else { + throw XCTSkip("Subscriber tests are skipped for fault \(currentParam.faultName)") + } + + let trackableID = UUID().uuidString + + // Just log the first component of the UUID, for the sake of logs’ readability. + let truncatedTrackableID = trackableID.components(separatedBy: "-")[0] + let logHandler = TestLogging.sharedInternalLogHandler.addingSubsystem(.named("test-\(truncatedTrackableID)")) + let proxyClient = SDKTestProxyClient(logHandler: logHandler) + + let setUpLogHandler = logHandler.addingSubsystem(.named("setUp")) + let faultSimulation = try Blocking.run(label: "Create fault simulation \(currentParam.faultName)", timeout: 10, logHandler: setUpLogHandler) { handler in + proxyClient.createFaultSimulation(withName: currentParam.faultName, handler) + } + + _testResources = TestResources( + testCase: self, + proxyClient: proxyClient, + faultSimulation: faultSimulation, + trackableID: trackableID, + logHandler: logHandler + ) + } + + override func tearDownWithError() throws { + try _testResources?.tearDown() + } + + /** + * Test that Subscriber can handle the given fault occurring before a user + * starts the subscriber. + * + * We expect the subscriber to not throw an error. + */ + func parameterizedTest_faultBeforeStartingSubscriber() throws { + try testResources.enableFault() + + let subscriberTestEnvironment = try testResources.createSubscriber() + let defaultAbly = try testResources.createAndStartPublishingAblyConnection() + + let subscriber = subscriberTestEnvironment.subscriber + let subscriberMonitorFactory = subscriberTestEnvironment.subscriberMonitorFactory + + let locationUpdate = Location(coordinate: .init(latitude: 2, longitude: 2)) + let publisherResolution = Resolution(accuracy: .minimum, desiredInterval: 100, minimumDisplacement: 0) + let subscriberResolution = Resolution(accuracy: .maximum, desiredInterval: 2, minimumDisplacement: 0) + + try subscriberMonitorFactory.forActiveFault( + label: "[fault active] subscriber", + locationUpdate: nil, + publisherResolution: nil, + subscriberResolution: subscriberResolution + ) + .waitForStateTransition { [testResources] (logHandler, handler: @escaping (Result) -> Void) in + // Connect up a publisher to do publisher things + + defaultAbly.updatePresenceData( + trackableId: testResources.trackableID, + presenceData: .init(type: .publisher, resolution: publisherResolution) + ) { result in + switch result { + case .success: + defaultAbly.sendEnhancedLocation( + locationUpdate: .init(location: locationUpdate), + trackable: Trackable(id: testResources.trackableID) + ) { result in + switch result { + case .success: + logHandler.debug(message: "Sent enhanced location update on Ably channel", error: nil) + // While we're offline-ish, change the subscribers preferred resolutions + + // The Android version of this test uses the fire-and-forget sendResolutionPreference (which deprecates sendResolutionPreference but which we have not yet implemented in AAT Swift). + subscriber.resolutionPreference(resolution: subscriberResolution) { result in + switch result { + case .success: + handler(.success) + case .failure(let error): + handler(.failure(error)) + } + } + case .failure(let error): + handler(.failure(error)) + } + } + case .failure(let error): + handler(.failure(error)) + } + } + } + + // Resolve the fault and make sure everything comes through + try subscriberMonitorFactory.forResolvedFault( + label: "[fault resolved] subscriber", + locationUpdate: locationUpdate, + publisherResolution: publisherResolution, + subscriberResolution: subscriberResolution + ) + .waitForStateTransition { _, handler in + testResources.resolveFaultAsync(handler) + } + } + + /** + * Test that Subscriber can handle the given fault occurring after a user + * starts the subscriber and then proceeds to stop it. + * + * We expect the subscriber to stop cleanly, with no thrown errors. + */ + func parameterizedTest_faultBeforeStoppingSubscriber() throws { + let subscriberTestEnvironment = try testResources.createSubscriber() + let defaultAbly = try testResources.createAndStartPublishingAblyConnection() + + let subscriber = subscriberTestEnvironment.subscriber + let subscriberMonitorFactory = subscriberTestEnvironment.subscriberMonitorFactory + + // Assert the subscriber goes online + let locationUpdate = Location(coordinate: .init(latitude: 2, longitude: 2)) + let publisherResolution = Resolution(accuracy: .balanced, desiredInterval: 1, minimumDisplacement: 0) + let subscriberResolution = Resolution(accuracy: .balanced, desiredInterval: 1, minimumDisplacement: 0) + + try subscriberMonitorFactory.onlineWithoutFail( + label: "[no fault] subscriber online", + timeout: 10, + subscriberResolution: subscriberResolution, + locationUpdate: locationUpdate, + publisherResolution: publisherResolution + ) + .waitForStateTransition { (logHandler, handler: @escaping (Result) -> Void) in + defaultAbly.sendEnhancedLocation( + locationUpdate: .init(location: locationUpdate), + trackable: .init(id: testResources.trackableID) + ) { result in + switch result { + case .success: + logHandler.debug(message: "Sent enhanced location update on Ably channel", error: nil) + handler(.success) + case .failure(let error): + handler(.failure(error)) + } + } + } + + // Enable the fault, shutdown the subscriber + try subscriberMonitorFactory.forActiveFaultWhenShuttingDownSubscriber( + label: "[fault active] subscriber", + publisherResolution: publisherResolution, + subscriberResolution: subscriberResolution + ) + .waitForStateTransition { (_, handler: @escaping (Result) -> Void) in + testResources.enableFaultAsync { result in + switch result { + case .success: + subscriber.stop { result in + switch result { + case .success: + handler(.success) + case .failure(let error): + handler(.failure(error)) + } + } + case .failure(let error): + handler(.failure(error)) + } + } + } + + // Resolve the fault + try subscriberMonitorFactory.forResolvedFaultWithSubscriberStopped( + label: "[fault resolved] subscriber", + publisherResolution: publisherResolution, + subscriberResolution: subscriberResolution + ) + .waitForStateTransition { _, handler in + testResources.resolveFaultAsync(handler) + } + } + + /** + * Test that Subscriber can handle the given fault occurring whilst tracking. + * + * We expect that upon the resolution of the fault, location updates sent in + * the meantime will be received by the subscriber. + */ + func parameterizedTest_faultWhilstTracking() throws { + let subscriberTestEnvironment = try testResources.createSubscriber() + + let subscriber = subscriberTestEnvironment.subscriber + let subscriberMonitorFactory = subscriberTestEnvironment.subscriberMonitorFactory + + // Bring a publisher online and send a location update + let defaultAbly = try testResources.createAndStartPublishingAblyConnection() + let locationUpdate = Location(coordinate: .init(latitude: 2, longitude: 2)) + let publisherResolution = Resolution(accuracy: .balanced, desiredInterval: 1, minimumDisplacement: 0) + let subscriberResolution = Resolution(accuracy: .balanced, desiredInterval: 1, minimumDisplacement: 0) + + try subscriberMonitorFactory.onlineWithoutFail( + label: "[no fault] subscriber online", + timeout: 10, + subscriberResolution: subscriberResolution, + locationUpdate: locationUpdate, + publisherResolution: publisherResolution + ) + .waitForStateTransition { (logHandler, handler: @escaping (Result) -> Void) in + defaultAbly.sendEnhancedLocation( + locationUpdate: .init(location: locationUpdate), + trackable: .init(id: testResources.trackableID) + ) { result in + switch result { + case .success: + logHandler.debug(message: "Sent enhanced location update on Ably channel", error: nil) + handler(.success) + case .failure(let error): + handler(.failure(error)) + } + } + } + + // Add an active trackable while fault active + let secondLocationUpdate = Location(coordinate: .init(latitude: 3, longitude: 3)) + let secondPublisherResolution = Resolution(accuracy: .minimum, desiredInterval: 100, minimumDisplacement: 0) + let secondSubscriberResolution = Resolution(accuracy: .maximum, desiredInterval: 2, minimumDisplacement: 0) + + let activeFaultExpectedLocationUpdate: Location + switch testResources.faultSimulation.type { + case .nonfatal: + activeFaultExpectedLocationUpdate = secondLocationUpdate + case .fatal, .nonfatalWhenResolved: + activeFaultExpectedLocationUpdate = locationUpdate + } + + let activeFaultExpectedPublisherResolution: Resolution + switch testResources.faultSimulation.type { + case .nonfatal: + activeFaultExpectedPublisherResolution = secondPublisherResolution + case .fatal, .nonfatalWhenResolved: + activeFaultExpectedPublisherResolution = publisherResolution + } + + let activeFaultExpectedSubscriberResolution: Resolution + switch testResources.faultSimulation.type { + case .nonfatal: + activeFaultExpectedSubscriberResolution = secondSubscriberResolution + case .fatal, .nonfatalWhenResolved: + activeFaultExpectedSubscriberResolution = subscriberResolution + } + + try subscriberMonitorFactory.forActiveFault( + label: "[fault active] subscriber", + locationUpdate: activeFaultExpectedLocationUpdate, + publisherResolution: activeFaultExpectedPublisherResolution, + subscriberResolution: activeFaultExpectedSubscriberResolution + ) + .waitForStateTransition { (logHandler, handler: @escaping (Result) -> Void) in + // Start the fault + testResources.enableFaultAsync { [testResources] result in + switch result { + case .success: + // Connect up a publisher to do publisher things + + defaultAbly.updatePresenceData( + trackableId: testResources.trackableID, + presenceData: .init(type: .publisher, resolution: secondPublisherResolution) + ) { result in + switch result { + case .success: + defaultAbly.sendEnhancedLocation( + locationUpdate: .init(location: secondLocationUpdate), + trackable: .init(id: testResources.trackableID) + ) { result in + switch result { + case .success: + logHandler.debug(message: "Sent second enhanced location update on Ably channel", error: nil) + + // While we're offline-ish, change the subscribers preferred resolution + + // The Android version of this test uses the fire-and-forget sendResolutionPreference (which deprecates sendResolutionPreference but which we have not yet implemented in AAT Swift). + subscriber.resolutionPreference(resolution: secondSubscriberResolution) { result in + switch result { + case .success: + handler(.success) + case .failure(let error): + handler(.failure(error)) + } + } + case .failure(let error): + handler(.failure(error)) + } + } + case .failure(let error): + handler(.failure(error)) + } + } + case .failure(let error): + handler(.failure(error)) + } + } + } + + // Resolve the fault, wait for Trackable to move to expected state + + let thirdLocationUpdate = Location(coordinate: .init(latitude: 4, longitude: 4)) + let thirdPublisherResolution = Resolution(accuracy: .maximum, desiredInterval: 3, minimumDisplacement: 0) + + try subscriberMonitorFactory.forResolvedFault( + label: "[fault resolved] subscriber", + locationUpdate: thirdLocationUpdate, + publisherResolution: thirdPublisherResolution, + subscriberResolution: secondSubscriberResolution + ) + .waitForStateTransition { (logHandler, handler: @escaping (Result) -> Void) in + defaultAbly.updatePresenceData( + trackableId: testResources.trackableID, + presenceData: .init(type: .publisher, resolution: thirdPublisherResolution) + ) { [testResources] result in + switch result { + case .success: + defaultAbly.sendEnhancedLocation( + locationUpdate: .init(location: thirdLocationUpdate), + trackable: .init(id: testResources.trackableID) + ) { result in + switch result { + case .success: + logHandler.debug(message: "Sent third enhanced location update on Ably channel", error: nil) + + // Resolve the problem + testResources.resolveFaultAsync(handler) + case .failure(let error): + handler(.failure(error)) + } + } + case .failure(let error): + handler(.failure(error)) + } + } + } + + // Restart the fault to simulate the publisher going away whilst we're offline + + try subscriberMonitorFactory.forActiveFault( + label: "[fault active] publisher shutdown for disconnect test", + locationUpdate: thirdLocationUpdate, + publisherResolution: /* thirdPublisherResolution, */ nil, /* TODO I’m not convinced that thirdPublisherResolution is right. Since shutdownAblyPublishing leaves presence with a hardcoded presence data, and hence a hardcoded resolution (accuracy: .balanced, desiredInterval: 1, minimumDisplacement: 0) that’s the only resolution the subscriber can expect to definitely receive */ + publisherDisconnected: true, + subscriberResolution: secondSubscriberResolution + ) + .waitForStateTransition { (_, handler: @escaping (Result) -> Void) in + // Start the fault + testResources.enableFaultAsync { [testResources] result in + switch result { + case .success: + // Disconnect the publisher + testResources.shutdownAblyPublishingAsync(handler) + case .failure(let error): + handler(.failure(error)) + } + } + } + + // Resolve the fault one last time and check that the publisher is offline + + try subscriberMonitorFactory.forResolvedFault( + label: "[fault resolved] subscriber publisher disconnect test", + locationUpdate: thirdLocationUpdate, + expectedPublisherPresence: false, + subscriberResolution: secondSubscriberResolution + ) + .waitForStateTransition { _, handler in + // Resolve the problem + testResources.resolveFaultAsync(handler) + } + } +} diff --git a/Tests/SystemTests/Proxy/SDKTestProxyClient.swift b/Tests/SystemTests/Proxy/SDKTestProxyClient.swift index d499ba008..931c00c6d 100644 --- a/Tests/SystemTests/Proxy/SDKTestProxyClient.swift +++ b/Tests/SystemTests/Proxy/SDKTestProxyClient.swift @@ -3,7 +3,8 @@ import AblyAssetTrackingInternal /// A client for communicating with an instance of the SDK test proxy server. Provides methods for creating and managing proxies which are able to simulate connectivity faults that might occur during use of the Ably Asset Tracking SDKs. class SDKTestProxyClient { - private let baseURL: URL + /// The base URL of the SDK test proxy server. + public let baseURL: URL private let logHandler: InternalLogHandler private let urlSession = URLSession(configuration: .default) diff --git a/Tests/SystemTests/Utils/MultipleErrors.swift b/Tests/SystemTests/Utils/MultipleErrors.swift new file mode 100644 index 000000000..a06321453 --- /dev/null +++ b/Tests/SystemTests/Utils/MultipleErrors.swift @@ -0,0 +1,32 @@ +/// Maintains a list of errors and provides a way to throw any contained errors. +/// +/// Useful in the situation where you want to perform a sequence of operations, continuing even if one fails, and subsequently report on the result of these operations once they’ve all been attempted. +struct MultipleErrors { + private var errors: [Error] = [] + + enum MultipleErrorsError: Error { + case multipleErrors([Error]) + } + + /// Adds an error to the list. + mutating func add(_ error: Error) { + errors.append(error) + } + + /// Throws any errors in the list. + /// + /// - If the list of errors is empty, this does nothing. + /// - If the list of errors contains a single error, this throws that error. + /// - If the list of errors contains multiple errors, this throws a ``MultipleErrorsError.multipleErrors`` describing those errors. + func check() throws { + if errors.isEmpty { + return + } + + if errors.count == 1 { + throw errors[0] + } + + throw MultipleErrorsError.multipleErrors(errors) + } +}