Setup custom failExceptions list within allureConfig.json file, NUnit

[alexsavenok-savenduk]

Hello,
Does anybody tried to setup "failExceptions" section for NUnit + C# using the latest Allure version?
{ "allure": { "directory": "allure-results", "failExceptions": [ "NUnit.Framework.AssertionException", "MyCustomAssertionException", "OpenQA.Selenium.NoSuchElementException" ] } }

This is how I tried to work with it, but failed. Report still displays tests with NoSuchElementException as broken, not failed.

[delatrie]

Hello, @alexsavenok-savenduk !

Well, it's a little bit more complicated with NUnit.

The thing is (and we need to document it correctly) that if an unhandled exception has occurred during a test's execution, NUnit doesn't give us the instance of that exception. We only have access to the message and the stack trace, which we can't check against failExceptions. Hence, in Allure NUnit, failExceptions only affects steps and fixtures but not tests.

But NUnit allows us to distinguish failed assertions from unhandled exceptions that aren't assertion-related, and we stick to that distinction to tell apart broken tests from failed ones (and to be honest, that's our go-to in such situations, to keep closer to what the framework reports to us).

That means you just need to be explicit about what you expect from the system under test. I.e., turn the following:

IWebElement button = driver.FindElement(By.Id("submit"));

Into the following:

IWebElement button = null;
Assert.That(() => { button = driver.FindElement(By.Id("submit")); }, Throws.Nothing); // explicitly tells that the system under test must contain the element.

With a couple of helper functions, you might even write it as compactly as you used to:

public static class NUnitSeleniumExtensions
{
    public static IWebElement FindElementAsserted(this IWebDriver driver, By by)
    {
        IWebElement element = null;
        Assert.That(() => { element = driver.FindElement(by); }, Throws.Nothing);
        return element;
    }
}

class MyTestClass
{
    IWebDriver driver;

    [SetUp]
    public void SetUp()
    {
        /* Initialize this.driver */
    }

    [TearDown]
    public void TearDown()
    {
        /* Dispose of this.driver */
    }

    [Test]
    public void MyTestMethod()
    {
        var element = driver.FindElementAsserted(By.Id("submit"));
    }
}

Ultimately, that makes total sense: failed means a failed assertion. So, write one, and you'll be good.

I hope that will help.

[alexsavenok-savenduk]

Got it! Thanks Maksim for your help!

[alexsavenok-savenduk]

Will try to apply options which you described above.