UI Test workflows #2515
UI Test workflows #2515
Conversation
github-actions
bot
added
the
bot: not in app board
kshann
removed
the
bot: not in app board
There was a problem hiding this comment.
Halle changed UI Tests xcscheme in #2458 to include building and running tests-server so these changes will have to be ported here too.
There was a problem hiding this comment.
There was a problem hiding this comment.
Yes, thanks for catching this. The scheme test section has to be set to review with debug executable turned off, but this also means that accessibility loses the app PID after the first test case during a local run, so I also probably need to port your solution with setting the XCUIApplication() to the review bundle ID, and then I need to run all of the tests a couple of times to confirm that everything is going to be sturdy.
There was a problem hiding this comment.
I'm checking this out right now, and with some luck will have confirmed this in not very long.
There was a problem hiding this comment.
@kshann, do the tests all run normally in CI with this PR when you set the UI Tests scheme test target to review with debug executable unchecked, or do you also see anomalies?
Task/Issue URL: https://app.asana.com/0/0/1205717523425891/f Tech Design URL: CC: @ayoy **Description**: 13/14 flows from this PR #2515 Have left asana notifications commented out for now. **Steps to test this PR**: 1. Example run https://github.com/duckduckgo/macos-browser/actions/runs/8660861087 2. If you want to run `gh workflow run "ui_tests.yml" --ref "kieran/ui-tests-macos1314"` <!-- Tagging instructions If this PR isn't ready to be merged for whatever reason it should be marked with the `DO NOT MERGE` label (particularly if it's a draft) If it's pending Product Review/PFR, please add the `Pending Product Review` label. If at any point it isn't actively being worked on/ready for review/otherwise moving forward (besides the above PR/PFR exception) strongly consider closing it (or not opening it in the first place). If you decide not to close it, make sure it's labelled to make it clear the PRs state and comment with more information. --> --- ###### Internal references: [Pull Request Review Checklist](https://app.asana.com/0/1202500774821704/1203764234894239/f) [Software Engineering Expectations](https://app.asana.com/0/59792373528535/199064865822552) [Technical Design Template](https://app.asana.com/0/59792373528535/184709971311943) [Pull Request Documentation](https://app.asana.com/0/1202500774821704/1204012835277482/f)
|
This PR has been inactive for more than 7 days and will be automatically closed 7 days from now. |
|
This PR has been closed after 14 days of inactivity. Feel free to reopen it if you plan to continue working on it or have further discussions. |
Task/Issue URL: https://app.asana.com/0/0/1205717523425891/f
Tech Design URL:
CC: @ayoy
Description:
This runs UI tests across macos 11/12/13/14.
MacOS11 has some problems with the current set of tests. I think it's better we merge this and monitor how other tests run in MacOS 11 and debug as we go as 12/13/14 run well and will provide feedback.
Currently will not notify asana until we begin running more tests but can be uncommented when that is required.
Sample workflows MacOS 13/14 MacOS 12
Happy with schedules?
Steps to test this PR:
Internal references:
Pull Request Review Checklist
Software Engineering Expectations
Technical Design Template
Pull Request Documentation