NUnit vs. XUnit vs. MSTest: Comparing Unit Testing Frameworks In C#

One of the most challenging things to do is ‘making the right choice.’ Arriving at a decision becomes even more complicated when there are multiple options in front of you☺. The same is the case with choosing a testing framework for .NET Core. The three major C# Unit testing frameworks are MSTest, NUnit, and xUnit.Net. You should select the most appropriate test framework that suits your project requirements. In this blog, we will see a detailed comparison between NUnit vs. XUnit vs. MSTest.

We have earlier covered all the three frameworks independently and recommend you to refer to those articles for detailed information on these test frameworks. In this article, we will briefly go through the features and compare the frameworks (NUnit vs. XUnit vs. MSTest) so that you can arrive at a decision to select the best-suited framework.

All three frameworks use annotations/attributes to inform the underlying framework about how the source code should be interpreted. Attributes are meta-tags that give more information about the test methods and classes defined in the source code.

We would be demonstrating NUnit vs. XUnit vs. MSTest comparison using cross browser testing using the Selenium framework. When choosing a test framework, you should check the in-house expertise and check whether the framework is well-suited for the ‘type’ of project your team is working on!

NUnit

NUnit is an open-source testing framework ported from JUnit. The latest version of NUnit is NUnit3 that is rewritten with many new features and has support for a wide range of .NET platforms.

NUnit has been downloaded more than 126 million times from NuGet.org. This shows the popularity of NUnit within the .NET user community. As of writing this article, there were close to 24,000 questions tagged as NUnit on Stackoverflow. It is a popular test framework used for Selenium automation testing. If you are planning to perform Test-Driven Development (TDD) with C#, you should have a close look at the NUnit framework.

Automation testing in NUnit can be performed using the Visual Studio GUI and the NUnit console (NUnit.ConsoleRunner). NUnit framework uses an attribute/annotation style system like other test frameworks; shown below are the most popular NUnit attributes used for Selenium automation testing.

The attributes [Test] and [TestCase] are not extensible as those attributes are sealed.

Using NUnit framework, tests can be executed serially as well as parallel. Parallel test execution is possible at assembly, class, or method level. When parallelism is enabled, by default four tests can be executed in parallel. As Parallel test execution is supported in NUnit, it becomes a compelling option for Selenium automation testing and cross browser testing.

XUnit

xUnit.Net is an open-source testing framework based on the .NET framework. ‘x’ stands for the programming language, e.g., JUnit, NUnit, etc. The creators of NUnit created xUnit as they wanted to build a better framework rather than adding incremental features to the NUnit framework.

xUnit is created with more focus on the community; hence it is easy to expand upon. You can download xUnit from NuGet gallery. So far, there are close to 7,500 questions tagged as xUnit on Stackoverflow.

Below are some of the core reasons why the creators of NUnit decided to reconsider the framework and built xUnit.Net.

As of writing this article, the latest version of xUnit was 2.4.1. The framework follows a unique style of testing, and tags like [Test] & [TestFixture], a part of the NUnit framework, are no longer used in the xUnit framework.

The popular attributes [SetUp] and [TearDown] are also not a part of the xUnit framework. As per the creators of NUnit (and xUnit), usage of [SetUp] and [TearDown] led to code duplication, and they wanted to implement the same features in a much more optimized manner in xUnit. For initialization, constructor of the test class is used, whereas, for de-initialization, IDisposable interface is used. This also encourages writing much cleaner tests. This makes this C# unit testing framework a much better option when it comes to Selenium automation testing as it is more robust and extensible.

As far as NUnit vs. XUnit vs. MSTest is concerned, the biggest difference between xUnit and the other two test frameworks (NUnit and MSTest) is that xUnit is much more extensible when compared to NUnit and MSTest. The [Fact] attribute is used instead of the [Test] attribute. Non-parameterized tests are implemented under the [Fact] attribute, whereas the [Theory] attribute is used if you plan to use parameterized tests.

In NUnit and MSTest, the class that contains the tests is under the [TestClass] attribute. This was not a very robust approach hence [TestClass] attribute was also removed in xUnit. Instead, intelligence is built in the xUnit framework so that it can locate the test methods, irrespective of the location of the tests.

Shown below are the most popular attributes/annotations used in the xUnit framework:

MSTest

MSTest is the default test framework that is shipped along with Visual Studio. The initial version of MSTest (V1) was not open-source; however, MSTest V2 is open-source. The project is hosted on GitHub. Like other test frameworks, it can also be used for data driven testing. You can download MSTest V2 from Nuget.org.

As of writing this article, there are close to 10,000 questions with the tag MSTest on Stackoverflow. Below are the most commonly used MSTest attributes.

MSTest V2 has cross-platform support and can be extended using custom test attributes & custom asserts. Parallel test execution is supported by the MSTest framework, where parallelism is possible at the Method level or Class level.

The test framework has evolved from being an “in-box” testing framework to a framework that is gaining wide adoption from the developer community.

NUnit vs. XUnit vs. MSTest

We now compare the C# unit testing frameworks from an attribute usage point of view along with a simple example, which demonstrates the code flow.

Attributes in test frameworks

Irrespective of the C# unit testing framework, attributes are used to describe class, methods, properties, etc. Attributes are meta-tags that provide additional information about the implementation under that particular tag.

To make the NUnit vs. XUnit vs. MSTest comparison clearer, we cover the important attributes in each of these frameworks. The NUnit vs. XUnit vs. MSTest attributes comparison will also help in porting the test implementation from one test framework to another.

To demonstrate the usage of attributes and the sequence in which the test code is executed, we take the example of simple test implementation.

NUnit Test Framework

In the example shown above, the implementation under [SetUp] and [TearDown] attributes will be called for each test case.

using NUnit.Framework;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;

namespace NUnit_Test
{
    class NUnit_Demo
    {
        [SetUp]
        public void Initialize()
        {
            Console.WriteLine("Inside SetUp");
        }

        [TearDown]
        public void DeInitialize()
        {
            Console.WriteLine("Inside TearDown");
        }

        public class TestClass1
        {
            [OneTimeSetUp]
            public static void ClassInitialize()
            {
                Console.WriteLine("Inside OneTimeSetUp");
            }

            [OneTimeTearDown]
            public static void ClassCleanup()
            {
                Console.WriteLine("Inside OneTimeTearDown");
            }
        }

        [Test, Order(1)]
        public void Test_1()
        {
            Console.WriteLine("Inside TestMethod Test_1");
        }

        [Test, Order(2)]
        public void Test_2()
        {
            Console.WriteLine("Inside TestMethod Test_2");
        }
    }
}

Shown below is the execution log:

Inside SetUp
Inside TestMethod Test_1
Inside TearDown

Inside SetUp
Inside TestMethod Test_2
Inside TearDown

xUnit Test Framework

As seen in the example shown below, the [SetUp] and [TearDown] attributes from the NUnit framework are not present. Instead, a constructor is used for initialization, and a Dispose method is used for de-initialization.

xUnit creates a new instance of the test class for every test. In some time-critical test scenarios, execution of creation & cleanup code may take time and make the tests slower. Class fixtures can be used in such situations where a fixture instance is created before any of the tests are executed, and once the tests have completed execution, cleanup of the fixture object by calling Dispose (if it is present).

In the example, we make use of Class Fixtures which means that even though there are two test classes, initialization, and cleanup code will be called only once.

using System;
using Xunit;

namespace xUnit_Test
{
    public class xUnit_Tests : IDisposable
    {
        public xUnit_Tests()
        {
            Console.WriteLine("Inside SetUp Constructor");
        }

        public void Dispose()
        {
            Console.WriteLine("Inside CleanUp or Dispose method");
        }
    }

    public class UnitTest_1 : IClassFixture<xUnit_Tests>
    {
        [Fact]
        public void Test_1()
        {
            Console.WriteLine("Inside Test_1");
        }
    }
    public class UnitTest_2 : IClassFixture<xUnit_Tests>
    {
        [Fact]
        public void Test_2()
        {
            Console.WriteLine("Inside Test_2");
        }
    }
}

Shown below is the output log:

Inside SetUp Constructor
Inside Test_1

Inside Test_2
Inside CleanUp or Dispose method

MSTest Test Framework

The test code only contains Console.WriteLine, which is put to trace the execution flow. Most of the widely used MSTest attributes are used in the demonstrated test code.

using Microsoft.VisualStudio.TestTools.UnitTesting;

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;

namespace MsTest
{
    [TestClass]
    public class Initialize
    {
        [AssemblyInitialize]
        public static void AssemblyInitialize(TestContext context)
        {
            Console.WriteLine("Inside AssemblyInitialize");
        }
    }

    public class DeInitialize
    {
        [AssemblyCleanup]
        public static void AssemblyCleanup()
        {
            Console.WriteLine("Inside AssemblyCleanup");
        }
    }

    [TestClass]
    public class TestClass1
    {
        [ClassInitialize]
        public static void ClassInitialize(TestContext context)
        {
            Console.WriteLine("Inside ClassInitialize");
        }

        [ClassCleanup]
        public static void ClassCleanup()
        {
            Console.WriteLine("Inside ClassCleanup");
        }

        [TestMethod]
        public void Test_1()
        {
            Console.WriteLine("Inside TestMethod Test_1");
        }
    }

    [TestClass]
    public class TestClass2
    {
        [TestInitialize]
        public void TestInitialize()
        {
            Console.WriteLine("Inside TestInitialize");
        }

        [TestMethod]
        public void Test_2()
        {
            Console.WriteLine("Inside TestMethod Test_2");
        }

        [TestCleanup]
        public void TestCleanup()
        {
            Console.WriteLine("Inside TestCleanup");
        }
    }
}

Shown below is the execution log:

Inside ClassInitialize
Inside TestMethod Test_1
Inside AssemblyInitialize

Inside TestInitialize
Inside TestMethod Test_2
Inside TestCleanup

The implementation under [ClassInitialize] & [ClassCleanup] attributes are called respectively once before the methods are executed and once after the execution of the test methods. The implementation under [TestInitialize] and [TestCleanup] attributes are respectively called once before & after executing tests in each class.

Core Differences between NUnit, XUnit, and MSTest

xUnit framework is different from NUnit and MSTest frameworks on multiple fronts. Let’s look at the core differences:

1. Isolation of Tests

xUnit framework provides much better isolation of tests in comparison to NUnit and MSTest frameworks. For each test case, the test class is instantiated, executed, and is discarded after the execution. This ensures that the tests can be executed in any order as there is reduced/no dependency between the tests. Executing each test as a separate instance minimizes the chances of one test causing the other tests to fail!

There is also an option to share setup/cleanup code that can be shared between different tests without the need to share object instances. One way of doing this is via Class Fixtures which we demonstrated earlier. xUnit has excellent documentation on how to share contexts between tests.

2. Extensibility

When we do NUnit vs. XUnit vs. MSTest, extensibility plays an important role in choosing a particular test framework. The choice might depend on the needs of the project, but in some scenarios, extensibility can turn the tables around for a particular test framework.

When compared to MSTest and NUnit frameworks, xUnit framework is more extensible since it makes use of [Fact] and [Theory] attributes. Many attributes that were present in NUnit framework e.g. [TestFixture], [TestFixtureSetup], [TestFixtureTearDown] [ClassCleanup], [ClassInitialize], [TestCleanup], etc. are not included in the xUnit framework.

Introduction of the [Theory] attribute for parameterized tests is one of the prime examples of the extensibility of the framework. This also makes the implementation of custom functionality much easier in the xUnit framework.

3. Initialization and De-initialization

The NUnit uses [SetUp], [TearDown] pairs whereas MSTest uses [TestInitialize], [TestCleanup] pairs for setting up the activities related to initialization & de-initialization of the test code.

On the other hand, xUnit uses the class constructor for the implementation of steps related to test initialization and IDisposable interface for the implementation of steps related to de-initialization.

xUnit starts a new instance per test, whereas, in NUnit & MSTest frameworks, all the tests execute in the same Fixture/Class.

4. Assertion mechanism

xUnit framework makes use of Assert.Throws instead of [ExpectedException] which is used in NUnit and MSTest. The drawback of using [ExpectedException] is that the errors might not be reported if they occur in the wrong part of the code. For example, if assert has to be raised for Security Exception, but Authentication Exception occurs, [ExpectedCondition] will not raise assert.

On the other hand, Assert.Throws raises assert even if the exception is generic. This ensures that assert is raised even after the exception is raised.

5. Parallel test execution

All the three C# unit testing frameworks support parallel test execution and are well-suited for Selenium automation testing as throughput plays a major role in automation testing. Below are the ways in which parallelism can be achieved in each of the test frameworks.

• NUnit – Parallelism is possible at the level of Children (child tests are executed in parallel with other tests), Fixtures (descendants of test till the level of Test Fixtures can execute in parallel), Self (test itself can be executed in parallel with other tests), and All (test & its descendants can execute in parallel with others at the same level).

[Parallelizable(ParallelScope.Children)]
[Parallelizable(ParallelScope.Fixtures)]
[Parallelizable(ParallelScope.Self)]
[Parallelizable(ParallelScope.All)]

• xUnit – Parallelism by putting the test classes into a single test collection or by executing an ‘n’ number of threads in parallel.

[assembly: CollectionBehavior(CollectionBehavior.CollectionPerAssembly)]

[assembly: CollectionBehavior(MaxParallelThreads = n)]

• MSTest – Parallelism at Method level as well as Class level.

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)]

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.ClassLevel)]

Note: Workers indicate the number of threads that can be executed in parallel.

NUnit vs. XUnit vs. MSTest – Demonstration using Remote Selenium Grid

For the initial rounds of cross browser testing can be performed on the local Selenium as it aids in verifying the functionalities of the test code on ‘certain’ combinations of browsers & operating systems. Selenium automation testing can be performed using either of the three C# unit testing frameworks.

Automation testing using the local Selenium Grid is a good start to cross browser testing activity, but local testing may not be sufficient to attain good test coverage. Having in-house infrastructure with machines having different web browsers and operating systems installed is not a scalable & economic idea.

When cross browser testing is performed using local Selenium WebDriver, the activity can hit a roadblock as you cannot perform Selenium automation testing using innumerable combinations of web browsers, operating systems, and devices. Instead of the local Selenium Grid, the capabilities of the remote Selenium Grid on the cloud can be leveraged so that testing is more fool-proof.

LambdaTest is a cross browser testing platform on the cloud where you can verify the test code on 2000+ combinations of browsers, devices, and operating systems. The effort to port the code from the local Selenium Grid to LambdaTest’s remote Selenium Grid is not much as major changes are infrastructure-related.

After creating an account on LambdaTest, make a note of the user-name and access-key from the Profile Section. The Dashboard consists of all the relevant information about the tests, status, logs, video recordings of the tests, etc. LambdaTest capabilities generator is used to generate the desired browser capabilities against which the cross browser tests will be performed. Shown below are the capabilities for Safari on the macOS Mojave platform:

DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.SetCapability("user","Your Lambda Username")
capabilities.SetCapability("accessKey","Your Lambda Access Key")
capabilities.SetCapability("build", "your build name");
capabilities.SetCapability("name", "your test name");
capabilities.SetCapability("platform", "MacOS Mojave");
capabilities.SetCapability("browserName", "Safari");
capabilities.SetCapability("version","12.0");

Implementation (using NUnit C# unit testing framework)

To demonstrate the usage of Remote Selenium WebDriver on LambdaTest Grid, we implement the below test case:

Below are the browser and operating system combinations on which the test has to be performed.

Browser	Browser version	Platform/Operating System
Chrome	72.0	Windows 10
Internet Explorer	11.0	Windows 10
Safari	11.0	macOS High Sierra
Microsoft Edge	18.0	Windows 10

The browser capabilities are generated using the LambdaTest capabilities generator. The combination of user-name & access-key is used to access LambdaTest’s remote Selenium Grid.

using System;
using OpenQA.Selenium;
using OpenQA.Selenium.Remote;
using NUnit.Framework;
using System.Threading;
using System.Collections.Generic;

namespace ParallelLTSelenium
{
    [TestFixture("chrome", "72.0", "Windows 10")]
    [TestFixture("internet explorer", "11.0", "Windows 10")]
    [TestFixture("Safari", "11.0", "macOS High Sierra")]
    [TestFixture("MicrosoftEdge", "18.0", "Windows 10")]
    [Parallelizable(ParallelScope.All)]
    public class ParallelLTTests
    {
        ThreadLocal<IWebDriver> driver = new ThreadLocal<IWebDriver>();
        private String browser;
        private String version;
        private String os;

        public ParallelLTTests(String browser, String version, String os)
        {
            this.browser = browser;
            this.version = version;
            this.os = os;
        }

        [SetUp]
        public void Init()
        {
            String username = "user-name";
            String accesskey = "access-key";
            String gridURL = "@hub.lambdatest.com/wd/hub";

            DesiredCapabilities capabilities = new DesiredCapabilities();

            capabilities.SetCapability("user", username);
            capabilities.SetCapability("accessKey", accesskey);
            capabilities.SetCapability("browserName", browser);
            capabilities.SetCapability("version", version);
            capabilities.SetCapability("platform", os);

            driver.Value = new RemoteWebDriver(new Uri("https://" + username + ":" + accesskey + gridURL), capabilities, TimeSpan.FromSeconds(600));

            System.Threading.Thread.Sleep(2000);
        }

        [Test]
        public void LT_ToDo_Test()
        {
            {
                String test_url_1 = "https://lambdatest.github.io/sample-todo-app/";
                driver.Value.Url = test_url_1;
                String itemName = "Adding item to the list";

                System.Threading.Thread.Sleep(2000);

                // Click on First Check box
                IWebElement firstCheckBox = driver.Value.FindElement(By.Name("li1"));
                firstCheckBox.Click();

                // Click on Second Check box
                IWebElement secondCheckBox = driver.Value.FindElement(By.Name("li2"));
                secondCheckBox.Click();

                // Enter Item name
                IWebElement textfield = driver.Value.FindElement(By.Id("sampletodotext"));
                textfield.SendKeys(itemName);

                // Click on Add button
                IWebElement addButton = driver.Value.FindElement(By.Id("addbutton"));
                addButton.Click();

                // Verified Added Item name
                IWebElement itemtext = driver.Value.FindElement(By.XPath("/html/body/div/div/div/ul/li[6]/span"));
                String getText = itemtext.Text;

                // Check if the newly added item is present or not using
                // Condition constraint (Boolean)
                Assert.That((itemName.Contains(getText)), Is.True);

                /* Perform wait to check the output */
                System.Threading.Thread.Sleep(2000);
            }
        }

        [TearDown]
        public void Cleanup()
        {
            bool passed = TestContext.CurrentContext.Result.Outcome.Status == NUnit.Framework.Interfaces.TestStatus.Passed;
            try
            {
                // Logs the result to Lambdatest
                ((IJavaScriptExecutor)driver.Value).ExecuteScript("lambda-status=" + (passed ? "passed" : "failed"));
            }
            finally
            {
                // Terminates the remote webdriver session
                driver.Value.Quit();
            }
        }
    }
}

Parallel test execution is enabled using the Parallelizable attribute, and test combinations are passed via TestFixtures. The actual logic of the test code is under the [Test] attribute. Shown below is the snapshot from the Automation Tab on LambdaTest and Visual Studio:

Implementation (using xUnit C# unit testing framework)

For demonstrating parallel test execution using xUnit framework on remote Selenium Grid, we test the following scenarios:

Test Case 1 – LamdaTest ToDo App

1. Navigate to the to-do app https://lambdatest.github.io/sample-todo-app/ using the Firefox WebDriver.
2. Mark the first two items as Done, i.e., Check those two items.
3. Add a new item – Adding an item to the list.
4. Click the Add button to add that new item to the list.

Browsers on which cross browser testing is performed are:

Test Case 1

Browser	Browser version	Platform/Operating System
Chrome	72.0	Windows 10
Microsoft Edge	18.0	Windows 10
Firefox	70.0	macOS High Sierra
Safari	12.0	macOS Mojave

Test Case 2 & 3 – Google Search for LambdaTest

1. Navigate to Google.com.
2. Search for LambdaTest.
3. Quit the browser window.

Both the test cases are the same, but the execution will be performed on different web browsers.

Test Case 2

Browser	Browser version	Platform/Operating System
Chrome	72.0	Windows 10
Microsoft Edge	18.0	Windows 10
Firefox	70.0	macOS High Sierra

Test Case 3

Browser	Browser version	Platform/Operating System
Microsoft Edge	18.0	Windows 10
Firefox	70.0	macOS High Sierra
Safari	12.0	macOS Mojave

using Xunit;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Firefox;
using System.Threading.Tasks;
using System.Text.RegularExpressions;
using OpenQA.Selenium.Support.UI;
using OpenQA.Selenium.Interactions;
/* For using Remote Selenium WebDriver */
using OpenQA.Selenium.Remote;

[assembly: CollectionBehavior(MaxParallelThreads = 4)]

namespace ParallelLTSelenium
{
    public class GlobalVar
    {
        static int _globalValue;
        public static int GlobalValue
        {
            get
            {
                return _globalValue;
            }
            set
            {
                _globalValue = value;
            }
        }

        public static String username = "user-name";
        public static String accesskey = "access-key";
        public static String gridURL = "@hub.lambdatest.com/wd/hub";
        public static DesiredCapabilities capabilities = new DesiredCapabilities();
        public static IWebDriver driver1;
    }

    public class ParallelLTTests : IDisposable
    {
        public ParallelLTTests()
        {
            GlobalVar.capabilities.SetCapability("user", GlobalVar.username);
            GlobalVar.capabilities.SetCapability("accessKey", GlobalVar.accesskey);
        }

        public void Dispose()
        {
            // Closure handled in each test case
        }
    }

    public class UnitTest_1 : IClassFixture<ParallelLTTests>
    {
        [Theory]
        [InlineData("chrome", "72.0", "Windows 10")]
        [InlineData("MicrosoftEdge", "18.0", "Windows 10")]
        [InlineData("Firefox", "70.0", "Windows 10")]
        [InlineData("Safari", "12.0", "macOS Mojave")]

        public void LT_ToDo_Test(String browser, String version, String os)
        {
            String itemName = "Yey, Let's add it to list";
            IWebDriver driver;

            GlobalVar.capabilities.SetCapability("browserName", browser);
            GlobalVar.capabilities.SetCapability("version", version);
            GlobalVar.capabilities.SetCapability("platform", os);
            GlobalVar.capabilities.SetCapability("build", "[xUnit - 1] LT ToDoApp using Xunit in Parallel on LambdaTest");
            GlobalVar.capabilities.SetCapability("name", "[xUnit - 1] - LT ToDoApp using Xunit in Parallel on LambdaTest");

            driver = new RemoteWebDriver(new Uri("https://" + GlobalVar.username + ":" + GlobalVar.accesskey + GlobalVar.gridURL), GlobalVar.capabilities, TimeSpan.FromSeconds(2000));
            driver.Url = "https://lambdatest.github.io/sample-todo-app/";

            Assert.Equal("Sample page - lambdatest.com", driver.Title);
            // Click on First Check box
            IWebElement firstCheckBox = driver.FindElement(By.Name("li1"));
            firstCheckBox.Click();

            // Click on Second Check box
            IWebElement secondCheckBox = driver.FindElement(By.Name("li2"));
            secondCheckBox.Click();

            // Enter Item name
            IWebElement textfield = driver.FindElement(By.Id("sampletodotext"));
            textfield.SendKeys(itemName);

            // Click on Add button
            IWebElement addButton = driver.FindElement(By.Id("addbutton"));
            addButton.Click();

            // Verified Added Item name
            IWebElement itemtext = driver.FindElement(By.XPath("/html/body/div/div/div/ul/li[6]/span"));
            String getText = itemtext.Text;
            Assert.True(itemName.Contains(getText));

            /* Perform wait to check the output */
            System.Threading.Thread.Sleep(2000);

            Console.WriteLine("LT_ToDo_Test Passed");

            driver.Close();
            driver.Quit();
        }
    }
    public class UnitTest_2 : IClassFixture<ParallelLTTests>
    {
        [Theory]
        [InlineData("chrome", "72.0", "Windows 10")]
        [InlineData("MicrosoftEdge", "18.0", "Windows 10")]
        [InlineData("Firefox", "70.0", "Windows 10")]

        public void Google_Test_1(String browser, String version, String os)
        {
            IWebDriver driver;

            GlobalVar.capabilities.SetCapability("browserName", browser);
            GlobalVar.capabilities.SetCapability("version", version);
            GlobalVar.capabilities.SetCapability("platform", os);
            GlobalVar.capabilities.SetCapability("build", "[xUnit - 2] - Google search (1) using XUnit in Parallel on LambdaTest");
            GlobalVar.capabilities.SetCapability("name", "[xUnit - 2] - Google search (1) using XUnit in Parallel on LambdaTest");

            driver = new RemoteWebDriver(new Uri("https://" + GlobalVar.username + ":" + GlobalVar.accesskey + GlobalVar.gridURL), GlobalVar.capabilities, TimeSpan.FromSeconds(2000));
            driver.Url = "https://www.google.com";

            System.Threading.Thread.Sleep(2000);

            IWebElement element = driver.FindElement(By.XPath("//*[@id='tsf']/div[2]/div[1]/div[1]/div/div[2]/input"));

            element.SendKeys("LambdaTest");

            /* Submit the Search */
            element.Submit();

            /* Perform wait to check the output */
            System.Threading.Thread.Sleep(2000);

            Console.WriteLine("Google_Test Passed");

            driver.Close();
            driver.Quit();
        }
    }

    public class UnitTest_3 : IClassFixture<ParallelLTTests>
    {
        [Theory]
        [InlineData("chrome", "72.0", "Windows 10")]
        [InlineData("MicrosoftEdge", "18.0", "Windows 10")]
        [InlineData("Firefox", "70.0", "macOS High Sierra")]
        [InlineData("Safari", "12.0", "macOS Mojave")]

        public void Google_Test_2(String browser, String version, String os)
        {
            IWebDriver driver;

            GlobalVar.capabilities.SetCapability("browserName", browser);
            GlobalVar.capabilities.SetCapability("version", version);
            GlobalVar.capabilities.SetCapability("platform", os);
            GlobalVar.capabilities.SetCapability("build", "[xUnit - 2] - Google Search (2) using XUnit in Parallel on LambdaTest");
            GlobalVar.capabilities.SetCapability("name", "[xUnit - 2] - Google Search (2) using XUnit in Parallel on LambdaTest");

            driver = new RemoteWebDriver(new Uri("https://" + GlobalVar.username + ":" + GlobalVar.accesskey + GlobalVar.gridURL), GlobalVar.capabilities, TimeSpan.FromSeconds(2000));
            driver.Url = "https://www.google.com";

            System.Threading.Thread.Sleep(2000);
            IWebElement element = driver.FindElement(By.XPath("//*[@id='tsf']/div[2]/div[1]/div[1]/div/div[2]/input"));

            element.SendKeys("LambdaTest");

            /* Submit the Search */
            element.Submit();

            /* Perform wait to check the output */
            System.Threading.Thread.Sleep(2000);

            Console.WriteLine("Google_Test Passed");

            driver.Close();
            driver.Quit();
        }
    }
}

The setup/initialization code is a part of the constructor, whereas the implementation under IDisposable [i.e., public void Dispose()] can be used to perform the de-initialization. For simplification, de-initialization is moved to the individual test cases. Shown below is the output snapshot from LambdaTest and Visual Studio.

Implementation (using MSTest C# unit testing framework)

To demonstrate the usage of parallelism with the MSTest framework, we implement the test cases used for the xUnit framework. The browser and operating system combinations for testing are also the same.

using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
/* For using Remote Selenium WebDriver */
using OpenQA.Selenium.Remote;
using System;
using System.Threading;

[assembly: Parallelize(Workers = 5, Scope = ExecutionScope.MethodLevel)]

namespace ParallelLTSelenium
{
    [TestClass]
    public class ParallelLTTests
    {
        IWebDriver driver;

        String username = "user-name";
        String accesskey = "access-key";
        String gridURL = "@hub.lambdatest.com/wd/hub";

        DesiredCapabilities capabilities;

        [TestInitialize]
        public void setupInit()
        {
            capabilities = new DesiredCapabilities();

            capabilities.SetCapability("user", username);
            capabilities.SetCapability("accessKey", accesskey);
        }

        [DataTestMethod]
        [DataRow("chrome", "72.0", "Windows 10")]
        [DataRow("MicrosoftEdge", "18.0", "Windows 10")]
        [DataRow("Firefox", "70.0", "macOS High Sierra")]
        [DataRow("Safari", "12.0", "macOS Mojave")]

        [TestMethod]
        public void LT_ToDo_Test(String browser, String version, String os)
        {
            String itemName = "Yey, Let's add it to list";

            capabilities.SetCapability("browserName", browser);
            capabilities.SetCapability("version", version);
            capabilities.SetCapability("platform", os);
            capabilities.SetCapability("build", "LT ToDoApp using MsTest in Parallel on LambdaTest");
            capabilities.SetCapability("name", "LT ToDoApp using MsTest in Parallel on LambdaTest");

            driver = new RemoteWebDriver(new Uri("https://" + username + ":" + accesskey + gridURL), capabilities, TimeSpan.FromSeconds(2000));

            driver.Url = "https://lambdatest.github.io/sample-todo-app/";

            Assert.AreEqual("Sample page - lambdatest.com", driver.Title);
            // Click on First Check box
            IWebElement firstCheckBox = driver.FindElement(By.Name("li1"));
            firstCheckBox.Click();

            // Click on Second Check box
            IWebElement secondCheckBox = driver.FindElement(By.Name("li2"));
            secondCheckBox.Click();

            // Enter Item name
            IWebElement textfield = driver.FindElement(By.Id("sampletodotext"));
            textfield.SendKeys(itemName);

            // Click on Add button
            IWebElement addButton = driver.FindElement(By.Id("addbutton"));
            addButton.Click();

            // Verified Added Item name
            IWebElement itemtext = driver.FindElement(By.XPath("/html/body/div/div/div/ul/li[6]/span"));
            String getText = itemtext.Text;
            Assert.IsTrue(itemName.Contains(getText));

            /* Perform wait to check the output */
            System.Threading.Thread.Sleep(2000);

            Console.WriteLine("LT_ToDo_Test Passed");
        }

        [DataTestMethod]
        [DataRow("chrome", "72.0", "Windows 10")]
        [DataRow("MicrosoftEdge", "18.0", "Windows 10")]
        [DataRow("Firefox", "70.0", "macOS High Sierra")]

        [TestMethod]
        public void Google_Test_1(String browser, String version, String os)
        {
            capabilities.SetCapability("browserName", browser);
            capabilities.SetCapability("version", version);
            capabilities.SetCapability("platform", os);
            capabilities.SetCapability("build", "Google search (1) using MsTest in Parallel on LambdaTest");
            capabilities.SetCapability("name", "Google search (1) using MsTest in Parallel on LambdaTest");

            driver = new RemoteWebDriver(new Uri("https://" + username + ":" + accesskey + gridURL), capabilities, TimeSpan.FromSeconds(2000));

            //System.Threading.Thread.Sleep(2000);

            driver.Url = "https://www.google.com";

            IWebElement element = driver.FindElement(By.XPath("//*[@id='tsf']/div[2]/div[1]/div[1]/div/div[2]/input"));

            element.SendKeys("LambdaTest");

            /* Submit the Search */
            element.Submit();

            /* Perform wait to check the output */
            System.Threading.Thread.Sleep(2000);

            Console.WriteLine("Google_Test Passed");

        }

        [DataTestMethod]
        [DataRow("chrome", "72.0", "Windows 10")]
        [DataRow("MicrosoftEdge", "18.0", "Windows 10")]
        [DataRow("Firefox", "70.0", "macOS High Sierra")]
        [DataRow("Safari", "12.0", "macOS Mojave")]

        [TestMethod]
        public void Google_Test_2(String browser, String version, String os)
        {
            capabilities.SetCapability("browserName", browser);
            capabilities.SetCapability("version", version);
            capabilities.SetCapability("platform", os);
            capabilities.SetCapability("build", "Google Search (2) using MsTest in Parallel on LambdaTest");
            capabilities.SetCapability("name", "Google Search (2) using MsTest in Parallel on LambdaTest");

            driver = new RemoteWebDriver(new Uri("https://" + username + ":" + accesskey + gridURL), capabilities, TimeSpan.FromSeconds(2000));

            //System.Threading.Thread.Sleep(2000);

            driver.Url = "https://www.google.com";

            IWebElement element = driver.FindElement(By.XPath("//*[@id='tsf']/div[2]/div[1]/div[1]/div/div[2]/input"));

            element.SendKeys("LambdaTest");

            /* Submit the Search */
            element.Submit();

            /* Perform wait to check the output */
            System.Threading.Thread.Sleep(2000);

            Console.WriteLine("Google_Test Passed");

        }

        [TestCleanup]
        public void Cleanup()
        {
            if (driver != null)
                driver.Quit();
        }
    }
}

Each test case is under a separate [TestMethod] attribute. Since we have implemented three tests (LT_ToDo_Test, Google_Test_1, and Google_Test_2), three concurrent sessions will be executed till the time the number of pending tests are less than three. The [DataRow] attribute is used to create parameterized tests.

Shown below is the output snapshot from the Automation tab on LambdaTest and Visual Studio:

Conclusion

Each of the .NET frameworks that we have discussed so far has evolved over a period of time. All these frameworks are well-suited for cross browser testing as they have support for Selenium. MSTest V2 is a giant leap in terms of usability when compared to MSTest V1. The xUnit framework is more sophisticated as it is more community-focused, and the creators had immense learning while building the NUnit framework.

To conclude, the NUnit vs. XUnit vs. MSTest comparison is more about which C# test framework is well-suited to perform your job and the test expertise that your team possesses. If I had an option to select a framework, I would go with xUnit since it is more extensible and has fewer attributes, making the code clean & easy to maintain.

What is your favorite C# test framework, do leave your choice & relevant explanation in the comments section.

Comments

[jeikabu]

Very thorough- I haven't compared them in a while. Originally used nunit, but have only used xunit the last few years. Some of their choices were perhaps controversial and uncomfortable at first, but they were made for the right reasons.

[Sukhpinder Singh]

Loved every single bit of information about this post. Lots of information about all 3 frameworks. I also prefer xUnit.