Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add iOS test making sure app functioning when API is down #6133

Conversation

niklasberglund
Copy link
Contributor

@niklasberglund niklasberglund commented Apr 16, 2024

Implemented test verifying that app is still functioning and can connect to a relay when the API is down. The simulation of API being down is not optimal. We have discussed it and decided to go for this way of simulating it for now. Might be improved in the future.

To test the changes run the test testAppStillFunctioningWhenAPIDown under ConnectivityTests. To simulate the the API being down all access methods are disabled and then a dummy access method is created to make API unreachable.

Duo to the extra setup and teardown steps the test function violates function_body_length(function body shouldn't be over 50 lines of code) so it is disabled for this test function.


This change is Reviewable

Copy link

linear bot commented Apr 16, 2024

@niklasberglund niklasberglund added the iOS Issues related to iOS label Apr 16, 2024
@niklasberglund niklasberglund force-pushed the write-test-that-makes-sure-connecting-works-without-the-api-ios-463 branch from df6b921 to 17dc511 Compare April 16, 2024 15:25
Copy link
Collaborator

@pinkisemils pinkisemils left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 8 of 26 files at r1, all commit messages.
Reviewable status: 8 of 26 files reviewed, 1 unresolved discussion (waiting on @niklasberglund)


ios/MullvadVPN/Containers/Navigation/CustomNavigationController.swift line 29 at r1 (raw file):

    override func viewDidLayoutSubviews() {
        super.viewDidLayoutSubviews()
        navigationItem.backBarButtonItem?.accessibilityIdentifier = "asdasd"

Is this a good accessibility identifier name?

@niklasberglund niklasberglund force-pushed the write-test-that-makes-sure-connecting-works-without-the-api-ios-463 branch from 17dc511 to d5eb797 Compare April 19, 2024 11:32
Copy link
Contributor Author

@niklasberglund niklasberglund left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 7 of 26 files reviewed, 1 unresolved discussion (waiting on @pinkisemils)


ios/MullvadVPN/Containers/Navigation/CustomNavigationController.swift line 29 at r1 (raw file):

Previously, pinkisemils (Emīls Piņķis) wrote…

Is this a good accessibility identifier name?

Was probably testing something out and forgot to remove 😄 Have removed the accessibility identifier, it was unused

Copy link
Contributor

@rablador rablador left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 23 of 26 files at r1, 3 of 3 files at r2, all commit messages.
Reviewable status: all files reviewed, 6 unresolved discussions (waiting on @niklasberglund and @pinkisemils)


ios/MullvadVPN/Coordinators/Settings/APIAccess/Cells/SwitchCellContentView.swift line 80 at r2 (raw file):

        switchContainer.transform = CGAffineTransform(scaleX: 0.85, y: 0.85)

        if let accessibilityIdentifier = actualConfiguration.accessibilityIdentifier {

accessibilityIdentifier can be optional, so no need to check for existence.


ios/MullvadVPN.xcodeproj/project.pbxproj line 3638 at r2 (raw file):

				85C7A2E82B89024B00035D5A /* SettingsTests.swift */,
				85D039972BA4711800940E7F /* SettingsMigrationTests.swift */,
				8588F5C42BCD7FB000374D0B /* XCUIElement+Extensions.swift */,

Seems to be a duplicate.


ios/MullvadVPNUITests/ConnectivityTests.swift line 46 at r2 (raw file):

    // swiftlint:disable function_body_length
    /// Test that the app is functioning when API is down. To simulate API being down we create a dummy access

Nit: "... method"?


ios/MullvadVPNUITests/ConnectivityTests.swift line 127 at r2 (raw file):

    /// Toggle enabled switch for all existing access methods. It is a precondition that the app is currently showing API access view.
    private func toggleAllAccessMethodsEnabledSwitch() {

Nit: The name is a little ambiguous. Reading the code and comment it's obvious what it does, but the name could suggest that you want to toggle all switches to be enabled.


ios/MullvadVPNUITests/Test base classes/BaseUITestCase.swift line 56 at r2 (raw file):

    }

    func allowLocalNetworkAccess() {

This and the function below do almost the same thing. Are both necessary?

@niklasberglund niklasberglund force-pushed the write-test-that-makes-sure-connecting-works-without-the-api-ios-463 branch 4 times, most recently from fad5b40 to 76646dc Compare April 23, 2024 07:16
Copy link
Contributor Author

@niklasberglund niklasberglund left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 21 of 26 files reviewed, 6 unresolved discussions (waiting on @pinkisemils and @rablador)


ios/MullvadVPN.xcodeproj/project.pbxproj line 3638 at r2 (raw file):

Previously, rablador (Jon Petersson) wrote…

Seems to be a duplicate.

Ah yes. Removed duplicate reference.


ios/MullvadVPNUITests/ConnectivityTests.swift line 46 at r2 (raw file):

Previously, rablador (Jon Petersson) wrote…

Nit: "... method"?

Yes 😄 Added " method"


ios/MullvadVPNUITests/ConnectivityTests.swift line 127 at r2 (raw file):

Previously, rablador (Jon Petersson) wrote…

Nit: The name is a little ambiguous. Reading the code and comment it's obvious what it does, but the name could suggest that you want to toggle all switches to be enabled.

Agree, "enabled" is very close to "enable". It is the switch for toggling the "enable method" though. Would adding "method" in the name and documentation make it more clear? toggleAllAccessmethodsEnableAccessMethodSwitch() and documentation saying Toggle enable method switch for all existing access methods. It is a precondition that the app is currently showing API access view.
image.png


ios/MullvadVPNUITests/Test base classes/BaseUITestCase.swift line 56 at r2 (raw file):

Previously, rablador (Jon Petersson) wrote…

This and the function below do almost the same thing. Are both necessary?

Hmmmm good point. Actually its not used, only the conditional allowLocalNetworkAccessIfAsked() is used by test cases. Same goes for allowAddVPNConfigurations and allowAddVPNConfigurationsIfAsked.

The idea was that it's good to strictly make sure it is shown when we know it should be. But we never know.

Copy link
Contributor

@rablador rablador left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 5 of 5 files at r3, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @niklasberglund and @pinkisemils)


ios/MullvadVPNUITests/ConnectivityTests.swift line 127 at r2 (raw file):

Previously, niklasberglund (Niklas Berglund) wrote…

Agree, "enabled" is very close to "enable". It is the switch for toggling the "enable method" though. Would adding "method" in the name and documentation make it more clear? toggleAllAccessmethodsEnableAccessMethodSwitch() and documentation saying Toggle enable method switch for all existing access methods. It is a precondition that the app is currently showing API access view.
image.png

I think perhaps adding rewording to [...]Switches would make it clearer, since we're toggling more than one.

@niklasberglund niklasberglund force-pushed the write-test-that-makes-sure-connecting-works-without-the-api-ios-463 branch 2 times, most recently from 556d916 to fd684f4 Compare April 29, 2024 15:48
Copy link
Contributor Author

@niklasberglund niklasberglund left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 24 of 26 files reviewed, 2 unresolved discussions (waiting on @pinkisemils and @rablador)


ios/MullvadVPN/Coordinators/Settings/APIAccess/Cells/SwitchCellContentView.swift line 80 at r2 (raw file):

Previously, rablador (Jon Petersson) wrote…

accessibilityIdentifier can be optional, so no need to check for existence.

Ah yes. Fixed


ios/MullvadVPNUITests/ConnectivityTests.swift line 127 at r2 (raw file):

Previously, rablador (Jon Petersson) wrote…

I think perhaps adding rewording to [...]Switches would make it clearer, since we're toggling more than one.

Changed to switches plural

@niklasberglund niklasberglund force-pushed the write-test-that-makes-sure-connecting-works-without-the-api-ios-463 branch from fd684f4 to dbcb169 Compare April 30, 2024 07:54
Copy link
Contributor

@rablador rablador left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 2 files at r4, 4 of 4 files at r5, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @pinkisemils)

@niklasberglund niklasberglund force-pushed the write-test-that-makes-sure-connecting-works-without-the-api-ios-463 branch from dbcb169 to 7e6d57e Compare May 6, 2024 11:39
Copy link
Contributor

@rablador rablador left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r6, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @pinkisemils)

Copy link
Collaborator

@pinkisemils pinkisemils left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @niklasberglund)


ios/MullvadVPN.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved line 0 at r6 (raw file):
This file should be reverted.

@niklasberglund niklasberglund force-pushed the write-test-that-makes-sure-connecting-works-without-the-api-ios-463 branch from 7e6d57e to 128ae6d Compare May 10, 2024 07:46
Copy link
Contributor Author

@niklasberglund niklasberglund left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 25 of 26 files reviewed, 1 unresolved discussion (waiting on @pinkisemils and @rablador)


ios/MullvadVPN.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved line at r6 (raw file):

Previously, pinkisemils (Emīls Piņķis) wrote…

This file should be reverted.

Reverted now

Copy link
Collaborator

@pinkisemils pinkisemils left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r7, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

@niklasberglund niklasberglund force-pushed the write-test-that-makes-sure-connecting-works-without-the-api-ios-463 branch from 128ae6d to 1d765a0 Compare May 10, 2024 09:36
Copy link
Collaborator

@pinkisemils pinkisemils left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 4 files at r8, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

Copy link
Contributor

@rablador rablador left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r7, 4 of 4 files at r8, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @niklasberglund)


ios/MullvadVPNUITests/ConnectivityTests.swift line 61 at r8 (raw file):

            .tapApplyButton()

        // Select the first country, its first city and its first relay

This is not reliable since ownership and relay list can change in the future.

Copy link
Contributor Author

@niklasberglund niklasberglund left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @rablador)


ios/MullvadVPNUITests/ConnectivityTests.swift line 61 at r8 (raw file):

Previously, rablador (Jon Petersson) wrote…

This is not reliable since ownership and relay list can change in the future.

What type of changes would break the test? The test connects to any(the first) Mullvad-owned relay the applies filter for rented only so that the connected relay is not part of the filter.

Copy link
Contributor

@rablador rablador left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @niklasberglund)


ios/MullvadVPNUITests/ConnectivityTests.swift line 61 at r8 (raw file):

Previously, niklasberglund (Niklas Berglund) wrote…

What type of changes would break the test? The test connects to any(the first) Mullvad-owned relay the applies filter for rented only so that the connected relay is not part of the filter.

Ah, my bad, I missed that you filter owned relays first. Nevermind!

Copy link
Contributor

@rablador rablador left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

@niklasberglund niklasberglund force-pushed the write-test-that-makes-sure-connecting-works-without-the-api-ios-463 branch from 1d765a0 to abed837 Compare May 13, 2024 09:00
Copy link
Collaborator

@pinkisemils pinkisemils left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r9, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved

@pinkisemils pinkisemils merged commit 53364a8 into main May 13, 2024
7 checks passed
@pinkisemils pinkisemils deleted the write-test-that-makes-sure-connecting-works-without-the-api-ios-463 branch May 13, 2024 09:10
Copy link

🚨 End to end tests failed. Please check the failed workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
iOS Issues related to iOS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants