MultipleAssertions

[mizady]

I created a branch on the TestFramework to add Assert.Multiple so that we can have multiple assertions and get the multiple failures instead of stopping at the first failure. I went ahead and did a draft pull request since I wanted to check to see if this would have any conflicts or problems with how the CI/CD pipeline runs. Some insight from someone who has more experience with the CI/CD pipeline for nanoframework and how the TestFramework interacts there would be welcomed. If there isn't any issues then I'll be happy to mark the pull request ready for review.

nanoframework/nanoFramework.TestFramework#291

This is what the usage looks like, I'm intending on adding this to the sample in the Samples repository as well.

Assert.Multiple( () => Assert.IsTrue(false), () => Assert.IsFalse(true) );

This should yield a MultipleAssertionException containing the two exceptions generated by the two failing assertions. I confirmed this using a temporary test project. Using Console.Writeline to print the multiple exceptions since without it, you only get the string printed stating that multiple assertions failed, and using the OutputHelper would have resulted in a circular dependency.

[josesimoes]

Hey Lance!

Adding new Assert methods wont' cause any issues on the pipelines, as any failure would have the same outcome: not 100% of the test are passing, so it will be 🔴 .

As for the need to have a multiple assert, as I was reading through your description above was thinking if it wouldn't be easier to change the code to keep processing all the asserts in a unit test method instead of stopping at the first one (which I think is what is happening now and probably has prompted you to look into this).
I follow your point on the improvement of getting all the details on all assert failures instead of stopping at the 1st one and fix and push and get the next one.
This is to pursue the effort to keep things as simple and small as possible and only add new stuff is really needed.
But nothing against adding this, if that's the preference of the community.

[mizady]

I did think about the aspect of whether it was necessary. I would say technically it is not, however I'm fairly certain it would save time in some parts of development. Probably not a massive amount of time as I wouldn't go back to change any current tests unless already having to work on it. When doing the Uri parser debugging I realized that having Assert.Multiple would have saved rerunning the tests over and over again in some cases. But I do understand the need to keep things small, especially if the test framework is actually run on the hardware. All in all, probably not worth the trade of space for potentially reduced processing and probably only a minute or two in some cases of saved time rerunning tests.