Document MSTestAnalysisMode property (#44632)

Youssef1313 · Evangelink · gewarren · web-flow · commit 8400125d032f · 2025-02-05T21:02:22.000Z
* Document MSTestAnalysisMode property

* Address review comments

* Add info on finding the globalconfig

* Doc in MSBuild reference for .NET SDK projects

* Update docs/core/testing/mstest-analyzers/overview.md

Co-authored-by: Amaury Levé &lt;amauryleve@microsoft.com&gt;

* Update analyzer doc pages for special treatments regarding MSTestAnalysisMode

* Apply suggestions from code review

Co-authored-by: Genevieve Warren &lt;24882762+gewarren@users.noreply.github.com&gt;

* Update docs/core/testing/mstest-analyzers/overview.md

Co-authored-by: Genevieve Warren &lt;24882762+gewarren@users.noreply.github.com&gt;

---------

Co-authored-by: Amaury Levé &lt;amauryleve@microsoft.com&gt;
Co-authored-by: Genevieve Warren &lt;24882762+gewarren@users.noreply.github.com&gt;
diff --git a/docs/core/project-sdk/msbuild-props.md b/docs/core/project-sdk/msbuild-props.md
@@ -1457,6 +1457,7 @@ The following MSBuild properties are documented in this section:
 - [TestingPlatformShowTestsFailure](#testingplatformshowtestsfailure)
 - [TestingExtensionsProfile](#testingextensionsprofile)
 - [UseVSTest](#usevstest)
+- [MSTestAnalysisMode](#mstestanalysismode)
 
 ### IsTestProject
 
@@ -1561,6 +1562,10 @@ For more information, see [MSTest runner profile](../testing/unit-testing-mstest
 
 Set the `UseVSTest` property to `true` to switch from the MSTest runner to the [VSTest](/visualstudio/test/vstest-console-options) runner when using the [MSTest project SDK](../testing/unit-testing-mstest-sdk.md).
 
+### MSTestAnalysisMode
+
+This property decides which analyzers are enabled at which severity. For more information, see [MSTest code analysis](../testing/mstest-analyzers/overview.md).
+
 ## Hosting-related properties
 
 The following MSBuild properties are documented in this section:
diff --git a/docs/core/testing/mstest-analyzers/includes/disabled-in-all.md b/docs/core/testing/mstest-analyzers/includes/disabled-in-all.md
@@ -0,0 +1,2 @@
+> [!NOTE]
+> This rule is opt-in. It's not enabled even when using `<MSTestAnalysisMode>All</MSTestAnalysisMode>`. For more information about `MSTestAnalysisMode`, see [MSTest code analysis](../overview.md).
diff --git a/docs/core/testing/mstest-analyzers/mstest0003.md b/docs/core/testing/mstest-analyzers/mstest0003.md
@@ -16,16 +16,16 @@ dev_langs:
 ---
 # MSTEST0003: Test methods should have valid layout
 
-| Property                            | Value                                              |
-|-------------------------------------|----------------------------------------------------|
-| **Rule ID**                         | MSTEST0003                                         |
-| **Title**                           | Test methods should have valid layout              |
-| **Category**                        | Usage                                              |
-| **Fix is breaking or non-breaking** | Breaking                                           |
-| **Enabled by default**              | Yes                                                |
-| **Default severity**                | Warning                                            |
-| **Introduced in version**           | 3.2.0                                              |
-| **Is there a code fix**             | Yes                                                |
+| Property                            | Value                                                                                     |
+|-------------------------------------|------------------------------------------------------------------------------------------|
+| **Rule ID**                         | MSTEST0003                                                                               |
+| **Title**                           | Test methods should have valid layout                                                    |
+| **Category**                        | Usage                                                                                    |
+| **Fix is breaking or non-breaking** | Breaking                                                                                 |
+| **Enabled by default**              | Yes                                                                                      |
+| **Default severity**                | Warning (escalated to Error when setting `MSTestAnalysisMode` to `Recommended` or `All`) |
+| **Introduced in version**           | 3.2.0                                                                                    |
+| **Is there a code fix**             | Yes                                                                                      |
 
 ## Cause
 
diff --git a/docs/core/testing/mstest-analyzers/mstest0015.md b/docs/core/testing/mstest-analyzers/mstest0015.md
@@ -40,6 +40,8 @@ Ensure that the test method isn't ignored.
 
 Do not suppress a warning from this rule. If you ignore this rule, test method will be ignored.
 
+[!INCLUDE [disabled-in-all](includes/disabled-in-all.md)]
+
 ## Suppress a warning
 
 If you just want to suppress a single violation, add preprocessor directives to your source file to disable and then re-enable the rule.
diff --git a/docs/core/testing/mstest-analyzers/mstest0019.md b/docs/core/testing/mstest-analyzers/mstest0019.md
@@ -40,6 +40,8 @@ Replace the constructor call with a `[TestInitialize]` method.
 
 You usually don't want to suppress warnings from this rule if you decided to opt-in for it.
 
+[!INCLUDE [disabled-in-all](includes/disabled-in-all.md)]
+
 ## Suppress a warning
 
 If you just want to suppress a single violation, add preprocessor directives to your source file to disable and then re-enable the rule.
diff --git a/docs/core/testing/mstest-analyzers/mstest0020.md b/docs/core/testing/mstest-analyzers/mstest0020.md
@@ -40,6 +40,8 @@ Replace `[TestInitialize]` returning `void` by constructors.
 
 You usually don't want to suppress warnings from this rule if you decided to opt-in for it.
 
+[!INCLUDE [disabled-in-all](includes/disabled-in-all.md)]
+
 ## Suppress a warning
 
 If you just want to suppress a single violation, add preprocessor directives to your source file to disable and then re-enable the rule.
diff --git a/docs/core/testing/mstest-analyzers/mstest0021.md b/docs/core/testing/mstest-analyzers/mstest0021.md
@@ -40,6 +40,8 @@ Replace `[TestCleanup]` method by `Dispose` or `DisposeAsync` pattern.
 
 You usually don't want to suppress warnings from this rule if you decided to opt-in for it.
 
+[!INCLUDE [disabled-in-all](includes/disabled-in-all.md)]
+
 ## Suppress a warning
 
 If you just want to suppress a single violation, add preprocessor directives to your source file to disable and then re-enable the rule.
diff --git a/docs/core/testing/mstest-analyzers/mstest0022.md b/docs/core/testing/mstest-analyzers/mstest0022.md
@@ -40,6 +40,8 @@ Replace `Dispose` or `DisposeAsync` methods with `[TestCleanup]`.
 
 You usually don't want to suppress warnings from this rule if you decided to opt-in for it.
 
+[!INCLUDE [disabled-in-all](includes/disabled-in-all.md)]
+
 ## Suppress a warning
 
 If you just want to suppress a single violation, add preprocessor directives to your source file to disable and then re-enable the rule.
diff --git a/docs/core/testing/mstest-analyzers/overview.md b/docs/core/testing/mstest-analyzers/overview.md
@@ -15,6 +15,55 @@ ms.date: 12/20/2023
 
 The rules are organized into categories such as performance usage...
 
+Starting with MSTest.TestFramework 3.7, the MSTest.Analyzers NuGet package is a dependency of the framework. For earlier versions, you need to use `MSTest` metapackage or add a package reference for `MSTest.Analyzers` explicitly.
+
+## MSTestAnalysisMode
+
+Starting with MSTest 3.8, an MSBuild property named `MSTestAnalysisMode` is available to determine which analyzers are enabled at which severity.
+
+> [!TIP]
+> To see which rules are enabled at which severity for each mode, you can navigate to the package of the version of interest in NuGet cache, locate the `globalconfigs` directory, and open the `.globalconfig` file corresponding to the analysis mode.
+> For more information on finding the NuGet cache directory, see [Managing the global packages, cache, and temp folders](/nuget/consume-packages/managing-the-global-packages-and-cache-folders). From that directory, locate `mstest.analyzers` directory, then the version (3.8 and higher), then `globalconfigs`.
+> Alternatively, you can download the NuGet package of the version of interest from `nuget.org` and view it in NuGet Package Explorer (Windows app), or view directly in the [web app version of NuGet Package Explorer](https://nuget.info/packages/MSTest.Analyzers/).
+
+The available values for this property:
+
+- [`None`](#none)
+- [`Default`](#default)
+- [`Recommended`](#recommended)
+- [`All`](#all)
+
+### `None`
+
+This value sets all analyzers to `none` severity, disabling all of them. You can then enable individual analyzers using `.editorconfig` or `.globalconfig` files.
+
+### `Default`
+
+This setting follows the default documented behavior for each rule.
+
+- Rules that are enabled by default will use their default severity.
+- Rules that are disabled by default will use `none` severity.
+
+> [!NOTE]
+> Rules that are enabled by default as warnings are violations that are expected to cause issues at run time.
+
+### `Recommended`
+
+This is the mode we expect most developers to use. Rules that are enabled by default with Info (`suggestion`) severity are escalated to warnings. Moreover, certain rules might be escalated to errors in both `Recommended` and `All` modes. For example, [MSTEST0003: Test methods should have valid layout](mstest0003.md) is escalated to error in `Recommended` and `All` modes.
+
+### `All`
+
+This mode is more aggressive than `Recommended`. All rules are enabled as warnings. As mentioned for `Recommended` mode, certain rules might be escalated to errors in both `Recommended` and `All` modes. For example, [MSTEST0003: Test methods should have valid layout](./mstest0003.md) is escalated to error in `Recommended` and `All` modes.
+
+> [!NOTE]
+> The following rules are completely opt-in and are not enabled in `Default`, `Recommended`, or `All` modes:
+>
+> - [MSTEST0015: Test method should not be ignored](mstest0015.md)
+> - [MSTEST0019: Prefer TestInitialize methods over constructors](mstest0019.md)
+> - [MSTEST0020: Prefer constructors over TestInitialize methods](mstest0020.md)
+> - [MSTEST0021: Prefer Dispose over TestCleanup methods](mstest0021.md)
+> - [MSTEST0022: Prefer TestCleanup over Dispose methods](mstest0022.md)
+
 ## Categories
 
 ### Design rules

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,2 @@`
	`1`	`+> [!NOTE]`
	`2`	+> This rule is opt-in. It's not enabled even when using `<MSTestAnalysisMode>All</MSTestAnalysisMode>`. For more information about `MSTestAnalysisMode`, see [MSTest code analysis](../overview.md).