[Testing] Fix for flaky UITests in CI that occasionally fail - 6 #28747

ghost · 2025-04-02T05:09:26Z

This pull request includes several changes to the test cases in the src/Controls/tests/TestCases.Shared.Tests/Tests/Issues directory to address issues with inconsistent keyboard visibility in iOS CI environments and to improve test reliability. The most significant changes are:

Issue25473 - VerifyEntryClearButtonVisibilitySetToWhileEditing

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs: Added conditional compilation directives to handle inconsistent keyboard visibility in iOS CI environments by cropping the screenshot when running on iOS. This change affects the VerifyEntryClearButtonVisibilitySetToWhileEditing and VerifyEntryClearButtonVisibilitySetToNever methods. [1] [2]

Issue19500 - TextInEditorShouldScroll

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue19500.cs: Added a retry mechanism for scrolling in the TextInEditorShouldScroll test to address flakiness on MacCatalyst.

Copilot

Pull Request Overview

This PR aims to improve the reliability of flaky UITests running in iOS CI environments by addressing inconsistent keyboard visibility and residual text issues in test cases.

In Issue25473 tests, conditional compilation directives have been added to crop the screenshot for iOS.
In the shared UITest, a call to App.ClearText for the SearchBar has been inserted to clear residual text before entering new input.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/_IssuesUITest.cs	Added App.ClearText in NavigateToIssue to ensure the search bar is empty.
src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs	Added conditional directives for cropping screenshots in iOS to address keyboard visibility flakiness.

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs

…473.cs Co-authored-by: Copilot <[email protected]>

rmarinho · 2025-04-02T09:53:46Z

/azp run

azure-pipelines · 2025-04-02T09:54:09Z

Azure Pipelines successfully started running 3 pipeline(s).

rmarinho · 2025-04-02T11:05:39Z

Seems some testing still failing ..

https://dev.azure.com/xamarin/public/_build/results?buildId=139445&view=ms.vss-test-web.build-test-results-tab

rmarinho · 2025-04-03T09:49:58Z

/azp run

azure-pipelines · 2025-04-03T09:50:19Z

Azure Pipelines successfully started running 3 pipeline(s).

PureWeen · 2025-04-03T14:28:04Z

/azp run MAUI-UITests-public

azure-pipelines · 2025-04-03T14:28:15Z

Azure Pipelines successfully started running 1 pipeline(s).

rmarinho · 2025-04-03T16:08:33Z

Can we pull fix for EntryClearButtonColorShouldMatchTextColor from #28780

PureWeen · 2025-04-03T21:59:02Z

All tests related to Issue25473 and Issue19500 are passing

jsuarezruiz · 2025-04-04T09:05:55Z

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue19500.cs


+#if MACCATALYST // In Catalyst scroll down is not effective so here we retry one more time to address the flakyness.
+			Thread.Sleep(500);
+			App.ScrollDown("editor");


Modifying the swipePercentage parameter helps to fix the problem?

@jsuarezruiz, This test failure is not due to the swipe area percentage. In the failure scenario, the swipe action did not initiate at all. Therefore, we have implemented a retry mechanism instead of increasing the swipe percentage. Please let me know if you still have any concerns regarding this approach.

…r/Footer After Changing ItemsLayout in CV2 (#28890) * [Testing] Fix for flaky UITests in CI that occasionally fail - 7 (#28764) * improve stability to flaky tests * Updated changes * Update CarouselViewUITests.LoopNoFreeze.cs * Update EntryClearButtonColorShouldMatchTextColor.png * Update CommunityToolkit.Maui reference in sample content template (#28510) * Update CommunityToolkit.Maui & Mvvm reference in template * Fix MAUI Toolkit breaking changes * Debump MVVM Toolkit * Update MauiApp.1.csproj * Update package version in MauiApp.1.csproj * Add framework-specific validation flags and update package version * Swap TextValidationBehavior Flags for frameworks * [Testing] Fix for flaky UITests in CI that occasionally fail - 6 (#28747) * Fixed flaky tests related to entry control * Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Co-authored-by: Copilot <[email protected]> * Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Co-authored-by: Copilot <[email protected]> * fixed 19500 test * reverted the common changes * Fix typo in Issue19500.cs comments --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Shane Neuville <[email protected]> * Update dependencies from https://github.com/dotnet/xharness build 20250403.2 (#28819) Microsoft.DotNet.XHarness.CLI , Microsoft.DotNet.XHarness.TestRunners.Common , Microsoft.DotNet.XHarness.TestRunners.Xunit From Version 9.0.0-prerelease.25167.9 -> To Version 9.0.0-prerelease.25203.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Update Issue19509.cs (#28807) * Resave mac tabbedpage images (#28803) * Fixed ListView leak caused by not disposed ContextActionsCell (#28702) * Fixed something that looks like copy/paste error * Fixed memory leak caused by not disposed ContextActionsCell * [Testing] Fix for flaky UITests in CI that occasionally fail - 8 (#28828) * improve stability to flaky test * Update Issue7678.cs * Fixed the TargetInvocationException Occurs When Selecting Header/Footer After Changing ItemsLayout in CV2 * Included the test case and sample for the fix * Added the test case for footer --------- Co-authored-by: HarishKumarSF4517 <[email protected]> Co-authored-by: Gerald Versluis <[email protected]> Co-authored-by: Anandhan Rajagopal <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Shane Neuville <[email protected]> Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Vitaly Knyazev <[email protected]>

bhavanesh2001 · 2025-04-11T11:53:29Z

@anandhan-rajagopal TextInEditorShouldScroll is still flaky in mac

…r/Footer After Changing ItemsLayout in CV2 (dotnet#28890) * [Testing] Fix for flaky UITests in CI that occasionally fail - 7 (dotnet#28764) * improve stability to flaky tests * Updated changes * Update CarouselViewUITests.LoopNoFreeze.cs * Update EntryClearButtonColorShouldMatchTextColor.png * Update CommunityToolkit.Maui reference in sample content template (dotnet#28510) * Update CommunityToolkit.Maui & Mvvm reference in template * Fix MAUI Toolkit breaking changes * Debump MVVM Toolkit * Update MauiApp.1.csproj * Update package version in MauiApp.1.csproj * Add framework-specific validation flags and update package version * Swap TextValidationBehavior Flags for frameworks * [Testing] Fix for flaky UITests in CI that occasionally fail - 6 (dotnet#28747) * Fixed flaky tests related to entry control * Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Co-authored-by: Copilot <[email protected]> * Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Co-authored-by: Copilot <[email protected]> * fixed 19500 test * reverted the common changes * Fix typo in Issue19500.cs comments --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Shane Neuville <[email protected]> * Update dependencies from https://github.com/dotnet/xharness build 20250403.2 (dotnet#28819) Microsoft.DotNet.XHarness.CLI , Microsoft.DotNet.XHarness.TestRunners.Common , Microsoft.DotNet.XHarness.TestRunners.Xunit From Version 9.0.0-prerelease.25167.9 -> To Version 9.0.0-prerelease.25203.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Update Issue19509.cs (dotnet#28807) * Resave mac tabbedpage images (dotnet#28803) * Fixed ListView leak caused by not disposed ContextActionsCell (dotnet#28702) * Fixed something that looks like copy/paste error * Fixed memory leak caused by not disposed ContextActionsCell * [Testing] Fix for flaky UITests in CI that occasionally fail - 8 (dotnet#28828) * improve stability to flaky test * Update Issue7678.cs * Fixed the TargetInvocationException Occurs When Selecting Header/Footer After Changing ItemsLayout in CV2 * Included the test case and sample for the fix * Added the test case for footer --------- Co-authored-by: HarishKumarSF4517 <[email protected]> Co-authored-by: Gerald Versluis <[email protected]> Co-authored-by: Anandhan Rajagopal <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Shane Neuville <[email protected]> Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Vitaly Knyazev <[email protected]>

…r/Footer After Changing ItemsLayout in CV2 (#28890) * [Testing] Fix for flaky UITests in CI that occasionally fail - 7 (#28764) * improve stability to flaky tests * Updated changes * Update CarouselViewUITests.LoopNoFreeze.cs * Update EntryClearButtonColorShouldMatchTextColor.png * Update CommunityToolkit.Maui reference in sample content template (#28510) * Update CommunityToolkit.Maui & Mvvm reference in template * Fix MAUI Toolkit breaking changes * Debump MVVM Toolkit * Update MauiApp.1.csproj * Update package version in MauiApp.1.csproj * Add framework-specific validation flags and update package version * Swap TextValidationBehavior Flags for frameworks * [Testing] Fix for flaky UITests in CI that occasionally fail - 6 (#28747) * Fixed flaky tests related to entry control * Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Co-authored-by: Copilot <[email protected]> * Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Co-authored-by: Copilot <[email protected]> * fixed 19500 test * reverted the common changes * Fix typo in Issue19500.cs comments --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Shane Neuville <[email protected]> * Update dependencies from https://github.com/dotnet/xharness build 20250403.2 (#28819) Microsoft.DotNet.XHarness.CLI , Microsoft.DotNet.XHarness.TestRunners.Common , Microsoft.DotNet.XHarness.TestRunners.Xunit From Version 9.0.0-prerelease.25167.9 -> To Version 9.0.0-prerelease.25203.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Update Issue19509.cs (#28807) * Resave mac tabbedpage images (#28803) * Fixed ListView leak caused by not disposed ContextActionsCell (#28702) * Fixed something that looks like copy/paste error * Fixed memory leak caused by not disposed ContextActionsCell * [Testing] Fix for flaky UITests in CI that occasionally fail - 8 (#28828) * improve stability to flaky test * Update Issue7678.cs * Fixed the TargetInvocationException Occurs When Selecting Header/Footer After Changing ItemsLayout in CV2 * Included the test case and sample for the fix * Added the test case for footer --------- Co-authored-by: HarishKumarSF4517 <[email protected]> Co-authored-by: Gerald Versluis <[email protected]> Co-authored-by: Anandhan Rajagopal <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Shane Neuville <[email protected]> Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Vitaly Knyazev <[email protected]>

…r/Footer After Changing ItemsLayout in CV2 (dotnet#28890) * [Testing] Fix for flaky UITests in CI that occasionally fail - 7 (dotnet#28764) * improve stability to flaky tests * Updated changes * Update CarouselViewUITests.LoopNoFreeze.cs * Update EntryClearButtonColorShouldMatchTextColor.png * Update CommunityToolkit.Maui reference in sample content template (dotnet#28510) * Update CommunityToolkit.Maui & Mvvm reference in template * Fix MAUI Toolkit breaking changes * Debump MVVM Toolkit * Update MauiApp.1.csproj * Update package version in MauiApp.1.csproj * Add framework-specific validation flags and update package version * Swap TextValidationBehavior Flags for frameworks * [Testing] Fix for flaky UITests in CI that occasionally fail - 6 (dotnet#28747) * Fixed flaky tests related to entry control * Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Co-authored-by: Copilot <[email protected]> * Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Co-authored-by: Copilot <[email protected]> * fixed 19500 test * reverted the common changes * Fix typo in Issue19500.cs comments --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Shane Neuville <[email protected]> * Update dependencies from https://github.com/dotnet/xharness build 20250403.2 (dotnet#28819) Microsoft.DotNet.XHarness.CLI , Microsoft.DotNet.XHarness.TestRunners.Common , Microsoft.DotNet.XHarness.TestRunners.Xunit From Version 9.0.0-prerelease.25167.9 -> To Version 9.0.0-prerelease.25203.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Update Issue19509.cs (dotnet#28807) * Resave mac tabbedpage images (dotnet#28803) * Fixed ListView leak caused by not disposed ContextActionsCell (dotnet#28702) * Fixed something that looks like copy/paste error * Fixed memory leak caused by not disposed ContextActionsCell * [Testing] Fix for flaky UITests in CI that occasionally fail - 8 (dotnet#28828) * improve stability to flaky test * Update Issue7678.cs * Fixed the TargetInvocationException Occurs When Selecting Header/Footer After Changing ItemsLayout in CV2 * Included the test case and sample for the fix * Added the test case for footer --------- Co-authored-by: HarishKumarSF4517 <[email protected]> Co-authored-by: Gerald Versluis <[email protected]> Co-authored-by: Anandhan Rajagopal <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Shane Neuville <[email protected]> Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Vitaly Knyazev <[email protected]>

…r/Footer After Changing ItemsLayout in CV2 (#28890) * [Testing] Fix for flaky UITests in CI that occasionally fail - 7 (#28764) * improve stability to flaky tests * Updated changes * Update CarouselViewUITests.LoopNoFreeze.cs * Update EntryClearButtonColorShouldMatchTextColor.png * Update CommunityToolkit.Maui reference in sample content template (#28510) * Update CommunityToolkit.Maui & Mvvm reference in template * Fix MAUI Toolkit breaking changes * Debump MVVM Toolkit * Update MauiApp.1.csproj * Update package version in MauiApp.1.csproj * Add framework-specific validation flags and update package version * Swap TextValidationBehavior Flags for frameworks * [Testing] Fix for flaky UITests in CI that occasionally fail - 6 (#28747) * Fixed flaky tests related to entry control * Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Co-authored-by: Copilot <[email protected]> * Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Co-authored-by: Copilot <[email protected]> * fixed 19500 test * reverted the common changes * Fix typo in Issue19500.cs comments --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Shane Neuville <[email protected]> * Update dependencies from https://github.com/dotnet/xharness build 20250403.2 (#28819) Microsoft.DotNet.XHarness.CLI , Microsoft.DotNet.XHarness.TestRunners.Common , Microsoft.DotNet.XHarness.TestRunners.Xunit From Version 9.0.0-prerelease.25167.9 -> To Version 9.0.0-prerelease.25203.2 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> * Update Issue19509.cs (#28807) * Resave mac tabbedpage images (#28803) * Fixed ListView leak caused by not disposed ContextActionsCell (#28702) * Fixed something that looks like copy/paste error * Fixed memory leak caused by not disposed ContextActionsCell * [Testing] Fix for flaky UITests in CI that occasionally fail - 8 (#28828) * improve stability to flaky test * Update Issue7678.cs * Fixed the TargetInvocationException Occurs When Selecting Header/Footer After Changing ItemsLayout in CV2 * Included the test case and sample for the fix * Added the test case for footer --------- Co-authored-by: HarishKumarSF4517 <[email protected]> Co-authored-by: Gerald Versluis <[email protected]> Co-authored-by: Anandhan Rajagopal <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Shane Neuville <[email protected]> Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Vitaly Knyazev <[email protected]>

Fixed flaky tests related to entry control

b8f5180

dotnet-policy-service bot added community ✨ Community Contribution partner/syncfusion Issues / PR's with Syncfusion collaboration labels Apr 2, 2025

ghost added the area-testing Unit tests, device tests label Apr 2, 2025

ghost marked this pull request as ready for review April 2, 2025 05:10

Copilot AI review requested due to automatic review settings April 2, 2025 05:10

ghost self-requested a review as a code owner April 2, 2025 05:10

ghost requested review from jsuarezruiz and tj-devel709 April 2, 2025 05:10

Copilot AI reviewed Apr 2, 2025

View reviewed changes

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Outdated Show resolved Hide resolved

src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25473.cs Outdated Show resolved Hide resolved

Anandhan Rajagopal and others added 3 commits April 2, 2025 10:41

Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25…

7d3285e

…473.cs Co-authored-by: Copilot <[email protected]>

Update src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue25…

c50930d

…473.cs Co-authored-by: Copilot <[email protected]>

fixed 19500 test

456c09e

ghost requested a review from rmarinho April 2, 2025 09:19

reverted the common changes

7b280ea

Fix typo in Issue19500.cs comments

acc31dd

PureWeen added this to the .NET 9 SR6 milestone Apr 3, 2025

PureWeen added this to MAUI SDK Ongoing Apr 3, 2025

github-project-automation bot moved this to Todo in MAUI SDK Ongoing Apr 3, 2025

PureWeen approved these changes Apr 3, 2025

View reviewed changes

rmarinho approved these changes Apr 3, 2025

View reviewed changes

github-project-automation bot moved this from Todo to Approved in MAUI SDK Ongoing Apr 3, 2025

jsuarezruiz reviewed Apr 4, 2025

View reviewed changes

PureWeen merged commit e41df8e into dotnet:main Apr 4, 2025
74 of 79 checks passed

github-project-automation bot moved this from Approved to Done in MAUI SDK Ongoing Apr 4, 2025

github-actions bot locked and limited conversation to collaborators May 12, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Testing] Fix for flaky UITests in CI that occasionally fail - 6 #28747

[Testing] Fix for flaky UITests in CI that occasionally fail - 6 #28747

Uh oh!

ghost commented Apr 2, 2025 •

edited by PureWeen

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

rmarinho commented Apr 2, 2025

Uh oh!

azure-pipelines bot commented Apr 2, 2025

Uh oh!

rmarinho commented Apr 2, 2025

Uh oh!

rmarinho commented Apr 3, 2025

Uh oh!

azure-pipelines bot commented Apr 3, 2025

Uh oh!

PureWeen commented Apr 3, 2025

Uh oh!

azure-pipelines bot commented Apr 3, 2025

Uh oh!

rmarinho commented Apr 3, 2025

Uh oh!

PureWeen commented Apr 3, 2025

Uh oh!

jsuarezruiz Apr 4, 2025

Uh oh!

ghost Apr 4, 2025

Uh oh!

Uh oh!

bhavanesh2001 commented Apr 11, 2025

Uh oh!

Uh oh!

[Testing] Fix for flaky UITests in CI that occasionally fail - 6 #28747

[Testing] Fix for flaky UITests in CI that occasionally fail - 6 #28747

Uh oh!

Conversation

ghost commented Apr 2, 2025 • edited by PureWeen Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue25473 - VerifyEntryClearButtonVisibilitySetToWhileEditing

Issue19500 - TextInEditorShouldScroll

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

rmarinho commented Apr 2, 2025

Uh oh!

azure-pipelines bot commented Apr 2, 2025

Uh oh!

rmarinho commented Apr 2, 2025

Uh oh!

rmarinho commented Apr 3, 2025

Uh oh!

azure-pipelines bot commented Apr 3, 2025

Uh oh!

PureWeen commented Apr 3, 2025

Uh oh!

azure-pipelines bot commented Apr 3, 2025

Uh oh!

rmarinho commented Apr 3, 2025

Uh oh!

PureWeen commented Apr 3, 2025

Uh oh!

jsuarezruiz Apr 4, 2025

Choose a reason for hiding this comment

Uh oh!

ghost Apr 4, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bhavanesh2001 commented Apr 11, 2025

Uh oh!

Uh oh!

ghost commented Apr 2, 2025 •

edited by PureWeen

Loading