-
Notifications
You must be signed in to change notification settings - Fork 6k
Document MSBuild integration #38846
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Document MSBuild integration #38846
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
c3c9794
Document MSBuild integration
nohwnd 6102e70
Merge branch 'main' of https://github.com/dotnet/docs into dotnet-tes…
nohwnd 06f3b0c
Lint
nohwnd b337569
Apply suggestions from code review
nohwnd 47f501c
Some fixes
nohwnd 93fd6f0
Merge branch 'dotnet-test-integration' of https://github.com/nohwnd/d…
nohwnd f79d4c2
Apply suggestions from code review
nohwnd 448f2b4
Whitespace
nohwnd 6a44017
fix
nohwnd 93875ce
Fix intro
nohwnd 15e049a
add to toc
nohwnd 85adea0
Fix intro
nohwnd 00be165
Apply suggestions from code review
nohwnd 16dae60
Update docs/core/testing/unit-testing-mstest-runner-integrations.md
nohwnd 46ac84b
Update docs/core/testing/unit-testing-mstest-runner-integrations.md
nohwnd a67b71f
Merge branch 'main' into dotnet-test-integration
nohwnd ee12863
Apply suggestions from code review
Evangelink File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
157 changes: 157 additions & 0 deletions
157
docs/core/testing/unit-testing-mstest-runner-integrations.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,157 @@ | ||
--- | ||
title: Use MSTest runner with `dotnet test` | ||
description: Learn how to run MSTest runner tests through dotnet test. | ||
author: nohwnd | ||
ms.author: jajares | ||
ms.date: 12/20/2023 | ||
--- | ||
|
||
# Use MSTest runner with `dotnet test` | ||
|
||
This article describes how to use `dotnet test` to run tests when using MSTest runner, and the various options that are available to configure the MSBuild output produced when running tests through MSTest runner. | ||
|
||
This article shows how to use `dotnet test` to run all tests in a solution (_*.sln_) that uses MSTest runner. | ||
|
||
## `dotnet test` integration | ||
|
||
[dotnet test](https://learn.microsoft.com/dotnet/core/tools/dotnet-test) command is a way to run tests from solutions, projects or already built assemblies. [MSTest runner](https://learn.microsoft.com/dotnet/core/testing/unit-testing-mstest-runner-intro) hooks up into this infrastructure to provide a unified way to run tests. Especially when migrating from VSTest to MSTest runner. | ||
|
||
### `dotnet test` integration - VSTest mode | ||
|
||
MSTest runner provides a compatibility layer to work with `dotnet test` seamlessly. This layer requires [Microsoft.Testing.Platform.MSBuild](https://nuget.org/packages/Microsoft.Testing.Platform.MSBuild) NuGet package. This package is automatically installed as a dependency of [MSTest](https://nuget.org/packages/MSTest) and [MSTest.TestAdapter](https://nuget.org/packages/MSTest.TestAdapter) package. | ||
|
||
Tests can be run by running: | ||
|
||
```dotnetcli | ||
dotnet test | ||
``` | ||
|
||
This layer runs test through VSTest and integrates with it on VSTest Test Framework Adapter level. | ||
|
||
### `dotnet test` - MSTest runner mode | ||
|
||
By default, VSTest is used to run MSTest runner tests. You can enable a full MSTest runner by specifying the `<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>` setting in your project. This setting disables VSTest. | ||
|
||
```xml | ||
<Project Sdk="Microsoft.NET.Sdk"> | ||
|
||
<PropertyGroup> | ||
<TargetFramework>net8.0</TargetFramework> | ||
<ImplicitUsings>enable</ImplicitUsings> | ||
<Nullable>enable</Nullable> | ||
|
||
<IsPackable>false</IsPackable> | ||
<IsTestProject>true</IsTestProject> | ||
|
||
<OutputType>Exe</OutputType> | ||
<EnableMSTestRunner>true</EnableMSTestRunner> | ||
|
||
<!-- Add this to your project file. --> | ||
<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport> | ||
|
||
</PropertyGroup> | ||
|
||
<!-- ... --> | ||
|
||
</Project> | ||
``` | ||
|
||
In this mode additional parameters to the run are not provided directly through commandline. They need to be provided as MSBuild property `TestingPlatformCommandLineArguments`: | ||
|
||
```dotnetcli | ||
dotnet test -p:TestingPlatformCommandLineArguments=" --minimum-expected-tests 10 " | ||
``` | ||
|
||
## Additional MSBuild options | ||
nohwnd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
The MSBuild integration provides options that can be specified in user project or through global properties on command line, such as `-p:TestingPlatformShowTestsFailure=true`. | ||
|
||
These are the available options: | ||
|
||
### Show failure per test | ||
|
||
By default test failures are summarized into a _.log_ file, and a single failure per test project is reported to MSBuild. | ||
|
||
> [!CAUTION] | ||
> Enabling this option adds overhead to test execution and may degrade performance. | ||
|
||
To show errors per failed test, specify `-p:TestingPlatformShowTestsFailure=true` on commandline, or add `<TestingPlatformShowTestsFailure>true</TestingPlatformShowTestsFailure>` property to your project file. | ||
|
||
On command line: | ||
|
||
```dotnetcli | ||
dotnet test -p:TestingPlatformShowTestsFailure=true | ||
``` | ||
|
||
Or in project file: | ||
|
||
```xml | ||
<Project Sdk="Microsoft.NET.Sdk"> | ||
|
||
<PropertyGroup> | ||
<TargetFramework>net8.0</TargetFramework> | ||
<ImplicitUsings>enable</ImplicitUsings> | ||
<Nullable>enable</Nullable> | ||
|
||
<IsPackable>false</IsPackable> | ||
<IsTestProject>true</IsTestProject> | ||
|
||
<OutputType>Exe</OutputType> | ||
<EnableMSTestRunner>true</EnableMSTestRunner> | ||
|
||
<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport> | ||
|
||
<!-- Add this to your project file. --> | ||
<TestingPlatformShowTestsFailure>true</TestingPlatformShowTestsFailure> | ||
|
||
</PropertyGroup> | ||
|
||
<!-- ... --> | ||
|
||
</Project> | ||
``` | ||
|
||
### Show complete platform output | ||
|
||
By default, all console output that the underlying test executable writes is captured and hidden from the user. This includes the banner, version information, and formatted test information. | ||
|
||
> [!CAUTION] | ||
> Enabling this option adds overhead to test execution and may degrade performance. | ||
|
||
To show this information together with MSBuild output use`<TestingPlatformCaptureOutput>false</TestingPlatformCaptureOutput>`. | ||
nohwnd marked this conversation as resolved.
Show resolved
Hide resolved
nohwnd marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
This option doesn't impact how the testing framework captures user output written by `Console.WriteLine` or other similar ways to write to the console. | ||
|
||
On command line: | ||
|
||
```dotnetcli | ||
dotnet test -p:TestingPlatformCaptureOutput=true | ||
``` | ||
|
||
Or in project file: | ||
|
||
```xml | ||
<Project Sdk="Microsoft.NET.Sdk"> | ||
|
||
<PropertyGroup> | ||
<TargetFramework>net8.0</TargetFramework> | ||
<ImplicitUsings>enable</ImplicitUsings> | ||
<Nullable>enable</Nullable> | ||
|
||
<IsPackable>false</IsPackable> | ||
<IsTestProject>true</IsTestProject> | ||
|
||
<OutputType>Exe</OutputType> | ||
<EnableMSTestRunner>true</EnableMSTestRunner> | ||
|
||
<TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport> | ||
|
||
<!-- Add this to your project file. --> | ||
<TestingPlatformCaptureOutput>false</TestingPlatformCaptureOutput> | ||
|
||
</PropertyGroup> | ||
|
||
<!-- ... --> | ||
|
||
</Project> | ||
``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.