Skip to content

Commit 0551fc7

Browse files
Add testing platform architecture (#41735)
* Add testing platform architecture * Fix links and png * Fix nits * More link fixing * Apply suggestions from code review Co-authored-by: David Pine <[email protected]> * Apply suggestions from code review Co-authored-by: David Pine <[email protected]> * Use csharp code slugs instead of cs * Apply suggestions from code review Co-authored-by: David Pine <[email protected]> * Apply suggestions from code review Co-authored-by: David Pine <[email protected]> * Apply suggestions from code review Co-authored-by: David Pine <[email protected]> * Update docs/core/testing/unit-testing-platform-architecture-services.md Co-authored-by: David Pine <[email protected]> * Update docs/core/testing/unit-testing-platform-architecture-services.md Co-authored-by: David Pine <[email protected]> * Update docs/core/testing/unit-testing-platform-architecture-extensions.md Co-authored-by: David Pine <[email protected]> * Apply suggestions from code review Co-authored-by: David Pine <[email protected]> * Do not use "we" * Refactoring * Mermaid links + pngs * Refactor * Update message bus * Fix links and image name --------- Co-authored-by: David Pine <[email protected]>
1 parent f78e116 commit 0551fc7

12 files changed

+2090
-3
lines changed
38.3 KB
Loading
34.4 KB
Loading
20.6 KB
Loading
30.1 KB
Loading
71.6 KB
Loading
Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
---
2+
title: Microsoft.Testing.Platform capabilities overview
3+
description: Learn about Microsoft.Testing.Platform capabilities concept.
4+
author: MarcoRossignoli
5+
ms.author: mrossignoli
6+
ms.date: 07/11/2024
7+
---
8+
9+
# Microsoft.Testing.Platform capabilities
10+
11+
In the context of the testing platform, a *capability* refers to the *potential to perform a specific action or provide specific information*. It's a means for the testing framework and extensions to *declare* their *ability* to *operate* in a certain manner or provide specific information to the *requesters*.
12+
13+
The *requesters* can be any component involved in a test session, such as the platform, an extension, or the testing framework itself.
14+
15+
The primary objective of the capability system is to facilitate effective communication among the components involved in a test session, enabling them to exchange information and meet their respective needs accurately.
16+
17+
## Guided example
18+
19+
Let's consider a hypothetical example to demonstrate the necessity of a capability system.
20+
21+
> [!NOTE]
22+
> This example is purely for illustrative purposes and isn't currently implemented within the testing platform or any testing framework.
23+
24+
Imagine a situation where you have an extension that requires the testing framework to execute no more than one test at a time. Furthermore, after each test, the extension needs to know the CPU usage for that specific test.
25+
26+
To accommodate the preceding scenario, you need to inquire from the testing framework if:
27+
28+
1. It has the capability to execute only one test at a time.
29+
2. It can provide information regarding the amount of CPU consumed by each test.
30+
31+
How can the extension determine if the testing framework has the ability to operate in this mode and provide CPU usage information for a test session? In the testing platform, this capability is represented by an implementation the `Microsoft.Testing.Platform.Capabilities.ICapability` interface:
32+
33+
```csharp
34+
// Base capabilities contracts
35+
36+
public interface ICapability
37+
{
38+
}
39+
40+
public interface ICapabilities<TCapability>
41+
where TCapability : ICapability
42+
{
43+
IReadOnlyCollection<TCapability> Capabilities { get; }
44+
}
45+
46+
// Specific testing framework capabilities
47+
48+
public interface ITestFrameworkCapabilities : ICapabilities<ITestFrameworkCapability>
49+
{
50+
}
51+
52+
public interface ITestFrameworkCapability : ICapability
53+
{
54+
}
55+
```
56+
57+
As you can see, the `ICapability` interface is a *marker* interface because it can represent *any capability*, and the actual implementation will be context dependent. You'll also observe the `ITestFrameworkCapability`, which inherits from `ICapability` to classify the capability. The capability system's generic nature allows for convenient grouping by context. The `ITestFrameworkCapability` groups all the capabilities implemented by the [testing framework](./unit-testing-platform-architecture-extensions.md#create-a-testing-framework). The `ICapabilities<TCapability>` interface reveals the *set* of all capabilities implemented by an extension. Similarly, for the base one, there's a context-specific testing framework called `ITestFrameworkCapabilities`. The `ITestFrameworkCapabilities` is provided to the platform during the [testing framework registration](./unit-testing-platform-architecture-extensions.md#register-a-testing-framework) process.
58+
59+
To create a capability that addresses the aforementioned scenario, you define it as follows:
60+
61+
```csharp
62+
public interface IDisableParallelismCapability : ITestFrameworkCapability
63+
{
64+
bool CanDisableParallelism { get; }
65+
bool CanProvidePerTestCpuConsumption { get; }
66+
void Enable();
67+
}
68+
```
69+
70+
If the testing framework implements this interface, at runtime, the following can be queried:
71+
72+
* Verify if the testing framework has the ability to turn off parallelism `CanDisableParallelism = true`.
73+
* Determine if the testing framework can supply CPU usage data `CanProvidePerTestCPUConsumption = true`.
74+
* Request the testing adapter to activate this mode by invoking the `Enable()` method before the test session commences.
75+
76+
The hypothetical code fragment inside the extension could be something like:
77+
78+
```csharp
79+
IServiceProvider provider = null; // TODO: Get IServiceProvider.
80+
var capabilities = serviceProvider.GetRequiredService<ITestFrameworkCapabilities>();
81+
82+
// Utilize the `GetCapability` API to search for the specific capability to query.
83+
var capability = capabilities.GetCapability<IDisableParallelismCapability>();
84+
if (capability is null)
85+
{
86+
// Capability not supported...
87+
}
88+
else
89+
{
90+
capability.Enable();
91+
if (capability.CanDisableParallelism)
92+
{
93+
// Do something...
94+
}
95+
96+
if (capability.CanProvidePerTestCpuConsumption)
97+
{
98+
// Do something...
99+
}
100+
}
101+
```
102+
103+
The preceding example illustrates how the capability infrastructure enables a powerful mechanism for communicating abilities between the components involved in a test session. While the sample demonstrates a capability specifically designed for the testing framework, any component can expose and implement extensions that inherit from `ICapability`.
104+
105+
It's evident that not all details can be communicated through an interface. Considering the previous example, what should the extension expect if the `CanProvidePerTestCpuConsumption` is supported? What kind of custom information is expected to be transmitted via the [IMessageBus](./unit-testing-platform-architecture-services.md#the-imessagebus-service) by the testing framework? The solution is **documentation of the capability**. It's the responsibility of the capability *owner* to design, ship, and document it as clearly as possible to assist implementors who want to effectively *collaborate* with the extension that requires the specific capability.
106+
107+
For instance, the TRX report extension enables the testing framework to implement the necessary capability to accurately generate a TRX report. The extension to register is included in the <https://www.nuget.org/packages/Microsoft.Testing.Extensions.TrxReport> NuGet package, but the capability to implement is found in the *contract only*<https://www.nuget.org/packages/Microsoft.Testing.Extensions.TrxReport.Abstractions> NuGet package.
108+
109+
In conclusion, let's summarize the primary aspects of the capability system:
110+
111+
* It's essential for facilitating clear and stable communication between components.
112+
* All capabilities should inherit from `ICapability` or an interface that inherits from it, and are exposed through a collection with the `ICapabilities` interface.
113+
* It aids in the evolution of features without causing breaking changes. If a certain capability isn't supported, appropriate action can be taken.
114+
* The responsibility of designing, shipping, and documenting the usage of a capability lies with the *capability owner*. The testing platform can also *own* a capability in the same way as any other extension.
115+
116+
## Framework capabilities
117+
118+
The platform exposes a specialized interface named `ITestFrameworkCapability` that is the base of all capabilities exposed for test frameworks. These capabilities are provided when [registering the test framework to the platform](./unit-testing-platform-architecture-extensions.md#register-a-testing-framework).
119+
120+
### `IBannerMessageOwnerCapability`
121+
122+
An optional [test framework capability](#framework-capabilities) that allows the test framework to provide the banner message to the platform. If the message is `null` or if the capability is n't present, the platform will use its default banner message.
123+
124+
This capability implementation allows you to abstract away the various conditions that the test framework may need to consider when deciding whether or not the banner message should be displayed.
125+
126+
The platform exposes the [`IPlatformInformation` service](./unit-testing-platform-architecture-services.md#the-iplatforminformation-service) to provide some information about the platform that could be useful when building your custom banner message.

0 commit comments

Comments
 (0)