Test Levels

Ginger unit test are marked with Levels (0-5) and each marked with AAA - Arrange, Act Assert

• Level 1-3 - runs for each push

• Level 4 - run once nightly only

• Level 5 - Runs weekly

• Level 0: Temporary level if we want the build machine to run our test first and if it fail stop other tests, DO NOT check in to master only to sub branch

• Level 1: For core test of items which are stand alone classes and requires very minimal setup to test the method

  • tests only basic core test runs in less than 500 ms, can run in parallel – no dependency – no Mutex

• Level 2: Plugins, Driver, Standalone actions, GingerConsole integration between components

• Level 3: Ginger UI POM include UI testing launch browser, test app etc, can have Mutex – sequential

• Level 4: Slower Longer tests – test durability, parallelism - run nightly only not during regular build Check Performance only, speed of execution, can be long test running the same action many times Requires internet connection - going to public web sites, for example to test Github check-in

Running full Ginger UI and performing different activities like create new solution, add Agent, automate and more, same like a regular user will do these tests include visual testing where we compare baseline screen to current screen and doing bitmap diff with several algorithms to compare

• Level 5 Longevity Tests - these test execute run sets and keep Ginger running for hours while measuring Memory, CPU, Disk consumption. These tests execution takes 36 hours and they are executed once a week on the development branch
• Performance Tests -
• Require specific component to be installed and user/pass to access it like SVN server
• Amdocs internal - We have a large amount of automation solution of our customers and we verify the new build is working on then without impact

Expected test level per project

• GingerCoreCommonTest - Level 1
• GingerTest - Level 3 and up

Test are executed in order and stop if fails, for example if test in Level 1 fails it will not continue to test level 2 and build will be marked as failed

Test Class

• Test class name should end with Test
• Mark the test class with the test level, do not mark specific methods
• Name the test method describing what the test does, do not use underscore '_' in test method
• Keep the test method code minimal
• More asserts per test method is better no limit
• Assert should have comments
• Use Test resources methods to get test data
• Output data
• Data Driver TestMethod input from CSV, or DataRow
• Run sequential test using Mutex

For example

public class StandAlonePluginTest
{
    [ClassInitialize]
    public static void ClassInit(TestContext TC)
    {
        // put class initialization code here, will be exeucted once 
    }

    [ClassCleanup]
    public static void ClassCleanup()
    {
		// put class cleanup code here if needed 
    }

    [TestInitialize]
    public void TestInitialize()
    {
        // Code that will be executed before each test method start
    }

    [TestCleanup]
    public void TestCleanUp()
    {
        // Code which will be executed after the test method
    }

[Level1]
[TestMethod,Timeout(60000)]
public void VariableRandomNumberMin5Max10Interval1()
{
    //Arrange
    VariableRandomNumber VRN = new VariableRandomNumber();
    VRN.Min = 5;
    VRN.Max = 10;
    VRN.Interval = 1;

    //Act
    VRN.GenerateAutoValue();

    //Assert
    Assert.IsTrue(decimal.Parse(VRN.Value) >= 5, "vn.Value>=5");
    Assert.IsTrue(decimal.Parse(VRN.Value) <= 10, "vn.Value<=10");
}