Rebuild the Xamarin.TestCloud nuget for .NET 6

[JohnHDev]

Describe the solution you'd like
We need a new build of the Xamarin.TestCloud nuget for .NET 6. This is to allow .NET Maui projects to use UI tests on iOS.
.NET Maui is now in RC and is a supported product byMicrosoft, so dependencies like this need to be updated.

Describe alternatives you've considered
There isn't an alternative other than dropping Xamarin UITest and AppCenter altogether.

[AlexeyBudynkov]

@JohnHDev, hello. You mean Xamarin.TestCloud.Agent nuget package or Xamarin.UITest nuget package?

[JohnHDev]

It is the testcloud agent nuget, the one that is required by iOS projects. At the moment we can't migrate to .NET Maui due to this.

[AlexeyBudynkov]

@JohnHDev Okay, thank you for your request. We created a task for that.

[JohnHDev]

Excellent thank you!
Any idea when this might be available?

[AlexeyBudynkov]

Excellent thank you! Any idea when this might be available?

It's in our plans, but we don't have ETA for now.

[JohnHDev]

Thanks Alexey.
If there is anything you can do to push this forward, that would be appreciated. Migrating to .NET Maui is a high priority and we need to get all our dependencies updated, including this one. We have extensive UI testing that will need to support .NET 6, and all we need to do that is this updated nuget. I would do it myself if it were open source.

[AlexeyBudynkov]

Thanks Alexey. If there is anything you can do to push this forward, that would be appreciated. Migrating to .NET Maui is a high priority and we need to get all our dependencies updated, including this one. We have extensive UI testing that will need to support .NET 6, and all we need to do that is this updated nuget. I would do it myself if it were open source.

We will reply to you when we'll finish the estimation

[kdharani]

Our organization has a plan to migrate to MAUI soon. I upvote this feature.

[IlyaBausovAkvelon]

This feature will be added in the new release soon.

[JohnHDev]

@IlyaBausovAkvelon any news on this pls?

[Slm0nB]

We would love this feature :-)

[JohnHDev]

Any update on this? .NET MAUI is now GA and we still can't run UI tests on iOS.
'Soon' obviously means different things to different people...

[dgerding]

Just weighing in with my strong, enthusiastic hope for this feature to arrive soon :-) thanks

[johnathan-s]

A client is deciding on a tech stack.
I cannot advise to the client to go with .net MAUI until there is a clear and clean automated test solution to testing .net MAUI.
Please put this at the top of the priority list.
Thanks

[dgerding]

Yes, please make this a top priority. We can't move to Maui without a replacement for automated ux testing.

[JohnHDev]

@AlexeyBudynkov it is now 3 months since we asked for this and we still can't create UI tests for iOS.
Would you advise dropping Xamarin UITest and AppCenter all together? We need better support than this.

If this isn't going to happen can you ask for it to be made open source pls, without seeing the source I suspect we might be able to do this ourselves in a day or 3.

[akamenev]

Hi @JohnHDev, thank you for your patience! Unfortunately, this feature takes more time to implement than we expected. It is currently in development and we will update everyone here once it is available.

[dgerding]

Hi @akamenev, we need some kind of eta, please. This year? Thanks

[dgerding]

Hi @akamenev, please provide an answer about Maui test support with some kind of real timeline or date estimations. thanks!

[mnxamdev]

Are we able to build .Net MAUI apps in App Center Build yet?

[AlexeyBudynkov]

@mnxamdev Hello. We're still working on it

[LarsNymand]

Upvote for .net maui support

[pm90pl]

BrowserStack - but that cloud supports only Appium

[bvbellomo]

I don't know of any alternatives to AppCenter for iOS beside Apple's store, but would be interested if anyone had one.

[akrol95]

Any update on this?

[vvechkanov]

@akrol95 We are still working on that task. Unfortunately no ETA yet. Thank you for your patience.

[miszu]

+1 for that, we invested quite a lot of time in these UI Tests and now simply can't run them. Happy to beta test a package early :)

[evgenyvalavin]

any news?

[MartinZikmund]

This is very important one

[weitzhandler]

This one is a blocker

[krzbbaranowski]

My team is considering replacing TestCloud with local machines. I wonder how many teams will do the same.
We are almost done with the migration to .NET 7 so we can't wait for the new version of TestCloud indefinitely...

My biggest concern is the fact that MS team has no ETA.

[JohnHDev]

@krzbbaranowski what will you replace the test framework with? Ideally looking for a replacement for Xamarin.UITest in its entirety, but still be able to create UI tests with C#.

[johnathan-s]

To any who have ended up on this thread that have an alternative, please post them here. Thanks

…

On Tue, Apr 4, 2023 at 11:27 PM JohnHDev ***@***.***> wrote: @krzbbaranowski <https://github.com/krzbbaranowski> what will you replace the test framework with? Ideally looking for a replacement for Xamarin.UITest in its entirety, but still be able to create UI tests with C#. — Reply to this email directly, view it on GitHub <#2416 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMUG7OIZT7OZPJSI3ZMMSBTW7QARNANCNFSM5UHPUV4Q> . You are receiving this because you commented.Message ID: ***@***.***>

[joshardt]

To any who have ended up on this thread that have an alternative, please post them here. Thanks
…

Appium with BrowserStack in C# with NUnit. No backdoors though.

[e1T]

Our team has started the transition to Appium with C# and xUnit. We've added up an interface to abstract platform-specific drivers so we can use a single code base for the UI tests. Benefit of Appium is that it works with Xamarin Forms app and our future MAUI version as well.
We're also planning on implementing parallel testing on multiple devices locally.

[JohnHDev]

@e1T How does Appium deal with automation on iOS? Xamarin UITest requires the iOS app is built with the Xamarin.TestCloud nuget installed with an initialiser for the library, is there not the same requirement with Appium?

Looks like we will most likely go the same route as you.

[e1T]

The iOS (or Android) app project doesn't need to include references to Appium itself. With Appium you can automate any application. For iOS it uses a Web Driver Agent installed on the device which runs the commands sent from the Appium server.
The architecture looks like this:

[anxkha]

I was able to successfully get my application to have a basic test run against it today with Xamarin.UITest 4.1.3 in an NUnit 3 project targeting .NET 7. I'm not sure where the disconnect is between the MS dev team working on this and their inability to post a meaningful update in this issue, but I suspect this has been fixed on .NET 6/7 for a little while now. I don't have any super complicated test cases to try out with this at this time, but I'm curious if others can try Xamarin.UITest 4.1.3 and let me/us all know if your more advanced tests just work!

[JohnHDev]

@anxkha I presume you are referring to a UI test with an Android app, the issue here is that to run a UI test on iOS we need to install the TestCloud nuget, which we can't yet without it being rebuilt. Let us know though! :)

[anxkha]

@JohnHDev Ah yes, that's a fair point my bad on not fully understanding some of the concerns here (there's confusion earlier on in the thread about which package specifically). Our app is pretty specialized and only on Android so I was only concerned about that. That's disappointing that TestCloud hasn't been updated yet :(

[JohnHDev]

@vvechkanov any news? we are about to invest resources into moving from Xamarin UITest to Appium. We need to do this to be able to move to .NET Maui before Xamarin is deprecated.

If it is no longer supported just say so. At the moment Microsoft is just wasting our time.

[amrara]

Any news?
Running Android or iOS build on .NET 6, with XamarinUITest 4.1.3 on App Center still not possible for me.
WARNING: Testing tests built for .NET 6 and above is not yet supported on Test Cloud.
Any workaround please?

[IlyaBausovAkvelon]

Hi All!
Thanks for your patience. The new TestCloudAgent with .NET 6 and above support will be released tomorrow.

[JohnHDev]

@IlyaBausovAkvelon thank you for the update.
I have just tried the new Xamarin.TestCloud nuget on a .NET Maui project and I am now able to successfully write UI tests that include iOS.
Please congratulate the team on a job well done. It is unfortunate it has taken a year (- 6 days) to complete.

[JohnHDev]

@IlyaBausovAkvelon I have spotted an issue with the nuget, release builds appear to be including the nuget and they fail on uploading to TestFlight. The Calabash.Start() code is wrapped in a ifdef for DEBUG, but I have worked around the issue with the following in the .csproj:

<ItemGroup Condition="'$(TargetFramework)' == 'net7.0-ios' AND $(Configuration) == 'Debug'"> <PackageReference Include="Xamarin.TestCloud.Agent" Version="0.23.2" /> </ItemGroup>

the error in the upload is:
[08:13:08]: [altool] detail = "The app references non-public selectors in Payload/AppDev.xxx.app/AppDev.xxx: setOrientation:animated:";

[evgenyvalavin]

Same issue with non-public API Usage from App Store Connect:

ITMS-90338: Non-public API usage - The app references non-public selectors in [ProjectName]: setOrientation:animated:. If method names in your source code match the private Apple APIs listed above, altering your method names will help prevent this app from being flagged in future submissions. In addition, note that one or more of the above APIs may be located in a static library that was included with your app. If so, they must be removed. For further information, visit the Technical Support Information at http://developer.apple.com/support/technical/

ITMS-90809: Deprecated API Usage - New apps that use UIWebView are no longer accepted. Instead, use WKWebView for improved security and reliability. Learn more (https://developer.apple.com/documentation/uikit/uiwebview).

[JohnHDev]

@IlyaBausovAkvelon any comments on this? ideally need a fix, having to put conditions in the csproj isn't really ideal.

[IlyaBausovAkvelon]

@JohnHDev could you please clarify do you need Release configuration to test app on TestFlight?
Does TestFlight accept Debug builds?

Answering your question: This is by design. We don't think that calabash and TestCloud.Agent package should be in release configurations. I think in your case it's better to customize the project file.

@DmitriyKirakosyan

[JohnHDev]

@IlyaBausovAkvelon The builds we are uploading to TestFlight are release builds.

The Xamarin.Calabash.Start(); code is wrapped in an IFDEF DEBUG && IOS, so isn't being called on a release build.

However, something in the nuget means it is being included in the Release build when it should be linked out if there aren't any references.

[evgenyvalavin]

What if it's actually a dotnet trimmer's (Linker) issue? Looks like it's the trimmer's responsibility to not include assemblies wrapped in DEBUG or AD-HOC etc.
.NET 7 Linker has lots of issues which were not released yet: dotnet/linker#3160

[JohnHDev]

What if it's actually a dotnet trimmer's (Linker) issue? Looks like it's the trimmer's responsibility to not include assemblies wrapped in DEBUG or AD-HOC etc.
.NET 7 Linker has lots of issues which were not released yet: dotnet/linker#3160

I believe you are correct, I have updated Xamarin iOS projects to the latest Xamarin.TestCloud nuget and Im not seeing the same issue with Release builds.

Im going to close this issue as the TestCloud nuget has now been updated to support .NET 6.