Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial Support for .xctestrun File Format Version 2 #559

Open
wants to merge 35 commits into
base: main
Choose a base branch
from

Conversation

bahrimootaz
Copy link
Contributor

@bahrimootaz bahrimootaz commented Feb 7, 2025

Description :
Enable the execution of XCTest cases using .xctestrun configuration files in version 2 format, expanding compatibility and aligning with latest Xcode standards.

Context :
Currently, go-ios supports .xctestrun files with format version 1. However, Xcode has introduced version 2 of the .xctestrun file format, which brings structural changes and enhancements. To maintain compatibility with modern XCTest setups and improve usability for customers, we need to support this newer format.

Reference: new structure of the .xctestrun is documented here

Testing :
In this MR we test did test the following scenarios :

  • contains only a single XCTest 🟢
  • contains multiple XCTest targeting the same app 🟢
  • contains multiple XCTest targeting different apps 🟢

var results []TestSuite
for _, target := range xcTestTargets {
suites, err := RunTestWithConfig(ctx, target)
if err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would effectivelly stop test executions if one has an error. I don't know if that's a good idea.

Copy link
Contributor Author

@bahrimootaz bahrimootaz Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we have tests that are erroring => no it won't stop execution of other tests if exists.
but if the failure because of something we do in the testmanagerd, it will stop execution => which is expected.

Copy link
Collaborator

@dmissmann dmissmann Feb 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there are two test targets and the first one returns an error here it won't execute the second one. So it would stop the execution.

I guess we can return both, the partial results and the error

var results []TestSuite
var targetErrors []error
for _, target := range xcTestTargets {
	suites, err := RunTestWithConfig(ctx, target)
	if err != nil {
		targetErrors = append(targetErrors, err)
		continue
	}
	...
}
return results, errors.join(targetErrors...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmissmann nice idea, we implemented this

ios/testmanagerd/xctestrunutils.go Outdated Show resolved Hide resolved
ios/testmanagerd/xctestrunutils.go Outdated Show resolved Hide resolved
ios/testmanagerd/xctestrunutils.go Outdated Show resolved Hide resolved
ios/testmanagerd/xctestrunutils.go Outdated Show resolved Hide resolved
ios/testmanagerd/xctestrunutils.go Show resolved Hide resolved
@bahrimootaz bahrimootaz changed the title Full Support for .xctestrun File Format Version 2 Partial Support for .xctestrun File Format Version 2 Feb 10, 2025
@bahrimootaz bahrimootaz changed the title Partial Support for .xctestrun File Format Version 2 Initial Support for .xctestrun File Format Version 2 Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants