Integrate with vstest Test Platform Protocols to enhance performance and consolidate code

[Issafalcon]

Using the VS Test Platform API (https://github.com/microsoft/vstest/blob/main/docs/RFCs/0007-Editors-API-Specification.md) to run test discovery and running tests will enable consolidation of code for each test runner currently in this adapter.

Using a listener in Nvim over a port for dotnet vstest will allow async updates for test discovery and streaming of test results straight from the Test Platform.

This will improve performance significantly.

[616b2f]

It may be worth also investigate the abstraction provided by vstest see (https://github.com/microsoft/vstest) or more specific for test discovery (https://github.com/microsoft/vstest/blob/main/docs/RFCs/0002-Test-Discovery-Protocol.md) this should allow test discovery in an runner agnostic way, in theory :D.

I currently looking into it because I wanna implement it in Build Server Protocol Implementation I write for dotnet and try to solve the same problem. Because I currently use your plugin for tests, I thought you maybe interested in it.

PS: vstest sounds like an Visual Studio only thing, but it's really not in fact.

[Issafalcon]

It's an interesting thought, and thanks for bringing it to my attention @616b2f !

I wonder how we could go about passing and receiving messages from the API via the TCP connection to the test platform port. Looking at the docs briefly, there are other considerations about mapping the structure of the protocol messages back to the discovered tests in the adapter.

If all that could work, then it could really consolidate the neotest-dotnet adapter code, and make it less liable to breaking changes from the various test runners!

It's not something I will have time to invest in looking at any time soon though, so it's something to stick on the roadmap. If you make any progress with your work and can offer any tips, that would be great!

[kostya9]

From what I could find, vstest's API is based on actual DLLs, which are non-trivial to find and attribute to specific test files.

I am exploring using omnisharp LSP instead of vstest (since omnisharp already does that work and integrates with the vstest platform), do you see that being the path forward and having a chance of being merged? I am currently working in my branch here main...kostya9:neotest-dotnet:omnisharp_tests.

The latest version can query omnisharp for tests, build the tree correctly, and integrates into existing functionality of running tests. Basically, I get the data from the o#/v2/discovertests LSP endpoint and add some postprocessing.

[Issafalcon]

Hi @kostya9 . Thanks for looking into this! I was also having the same battle trying to find a good way of integrating with vstest test platform.

I also (in the early days of this plugin) used Omnisharp as the way of discovering and running tests, but found a few issues with it (can't quite remember the specifics off hand). However, given the rise of the Roslyn LSP, and others like c_sharp_language_server etc, it doesn't feel right to make Omnisharp the LSP dependency of this plugin. It would rule out using this for users who want to use one of the other LSPs (I use roslyn lsp myself now).

I have recently come across GustavEikaas/easy-dotnet.nvim#74. This, is, I think, the way forward, an will be something I will try as soon as I get time.

[616b2f]

I didn't knew that it's possible to run F# scripts with the dotnet cli tool directly, the solution you linked is great, that's basically what I am also using in BSP.

I tried to make it work with neotest (kind of neotest-bsp adapter) but the interface for the adapter is designed around the idea that neotest does the file discovery and the adapter's responsibility is to parse those and provide test cases that it finds there. While the vstest approach is to find the test cases inside an assembly, you can still find from which file the test case is coming from but you can't ask to parse specific file, it's really on an assembly level and it handles all the test file discovery on it's own.

It's not a show stopper but it's not fitting the design of neotest adapters, at least this is how I currently understood the design.