How do I enable FCC on some solutions but not others

[bb-avi]

I have a legacy solution (monolith) that has way too many tests to do code coverage locally.
But our new solutions (microservices) are small and I want FCC to run on them.

I have been unable to get the finecodecoverage-settings.xml file to work as expected.

Here is what I have in the finecodecoverage-settings.xml file that is in the same folder as the solution:

<FineCodeCoverage topLevel="true">
  <Enabled>True</Enabled>
</FineCodeCoverage>

• When FCC is disabled in Tools > Settings then I set the Enabled value to True and still tests do not run.
• When FCC is enabled in Tools > Settings then I set the Enabled value to False and still tests run.

It also doesn't seem to matter whether or not I use topLevel="true". I just found that in the documentation and thought I'd give it a try.
This still doesn't work whether or not I use my custom .runsettings file.

What am I doing wrong?

[tonyhallett]

Note that Visual Studio Tools / Options / Fine Code Coverage Enabled false will always prevent any coverage.

Using project settings / finecodecoverage-settings.xml works for old style coverage and for when fcc generates runsettings behind the scenes.

I am assuming that you are providing runsettings for one or more projects. The presence of the data collector will result in coverage being collected unless you set the enabled attribute to false.

I have changed the FCC code that intercepts the user runsettings file. It will then apply the enabled attribute as false - if all the projects that are associated with the runsettings file are disabled through either if the two methods outlined.

Unfortunately this change has resulted in some failing tests so I will need to look at these before I push the changes through.

[bb-avi]

Thank you for your quick reply. My legacy solution does not have a .runsettings file but my new ones do - so that inclusions/exclusions can be specified per-solution. With or without a .runsettings file, setting enabled to true in options and false in the finecodecoverage-settings.xml file did not disable FCC.

[tonyhallett]

There was a problem with the code that may have been the causes of your issue. Please update visual studio to see changes.

If you still encounter it note that the settings are determined as follows.

a) Begin with those set in vs
b) Overwrite or possibly merge for string arrays - project settings have highest precedence. Settings from finecodecoverage-settings.xml files have precedence based upon the file depth. ( We start at the project directory then ascend to topLevel - then reverse them - then apply in order )

[tonyhallett]

No runsettings used here.

DemoFCC.-.Microsoft.Visual.Studio.-.Experimental.Instance.2023-07-02.17-21-20.mp4

[bb-avi]

Thank you for your time.
I have no issue anymore.

I think I dug deeper than I needed to and assumed if FCC ran anything at all then the root problem I was trying to overcome would still persist. Since I didn't describe the root problem here I think it may be helpful to others to describe it.

When FCC is enabled and I run unit tests in my legacy .NET 4.8 solution FCC causes the NUnit 3 test adapter to crash with the following exception: System.Security.VerificationException: Operation could destabilize the runtime.

Full Stack Trace

NUnit Adapter 3.16.1.0: Test execution started
Running selected tests in C:\git\dotnet\G2Web\apiTest\bin\Debug\apiTest.dll
Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests
The NUnit 3 driver encountered an error while executing reflected code.
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, IGrouping`2 testCases, TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 309
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 152
Innerexception: System.Security.VerificationException: Operation could destabilize the runtime.
   at NUnit.Framework.Api.FrameworkController..ctor(String assemblyNameOrPath, String idPrefix, IDictionary settings) in D:\a\1\s\src\NUnitFramework\framework\Api\FrameworkController.cs:line 68

When I disabled FCC in Tools > Options the problem went away. But that meant every time I loaded a solution I wanted to use FCC on, I had to enable FCC in Tools > Options. And every time I loaded my legacy solution I had to disable FCC again.

To fix this I had to:

1. Enable FCC in Tools > Options
2. Add a finecodecoverage-settings.xml file in the legacy solution that contains the following:

<FineCodeCoverage>
  <Enabled>false</Enabled>
</FineCodeCoverage>

After doing this the crash no longer occurs and FCC partially runs.

By partially runs, I mean it will still add <RunSettingsFilePath></RunSettingsFilePath> to the test project file, if it doesn't exist, and the FCC output will look like the following:

Fine Code Coverage 7/3/2023 9:09:09 AM: Ms code coverage
Fine Code Coverage 7/3/2023 9:09:30 AM: No cobertura files for ms code coverage.
Fine Code Coverage 7/3/2023 9:09:30 AM: ================================== DONE ==================================

These are acceptable side-effects for me so I'm moving forward.

Thanks again for your help @tonyhallett !

[bb-avi]

BTW: To overcome the error System.Security.VerificationException: Operation could destabilize the runtime I had to limit what assemblies were included/excluded in the finecodecoverage-settings.xml file. For MsCodeCoverage it looks like this:

<FineCodeCoverage topLevel="true">
  <Enabled>true</Enabled>
  <ModulePathsInclude>
    MyProjectUnderTest1.dll
    MyProjectUnderTest2.dll
  </ModulePathsInclude>
  <ModulePathsExclude>
  </ModulePathsExclude>
</FineCodeCoverage>