How to filter NUnit tests by category using "dotnet test"

Question

I have a project that has a

[TestFixture, Category("Oracle")]

and a

[TestFixture, Category("OracleOdbc")]

with a couple of tests which I would like to execute separately using dotnet test.

Here's what I tried after some Googling:

1. dotnet test MyProject.csproj --where "cat==Oracle" but this switch does not exists anymore.
2. dotnet test MyProject.csproj --filter Category="Oracle" yields 0 applicable tests: No test is available in ....

Then, I've stumbled over this article and although it describes MSTest (and NUnit has the CategoryAttribute and not a TestCategoryAttribute), I've tried

3. dotnet test MyProject.csproj --filter TestCategory="Oracle"

Bingo. This time all "Oracle" tests were executed. But now comes the confusing part. If I run dotnet test MyProject.csproj --filter TestCategory="OracleOdbc", all tests are being executed, including "Oracle" and "OracleOdbc". This makes me wonder if TestCategroy is the proper way to go for NUnit or if this is a bug.

I'm using .NET Command Line Tools (2.1.2) and the project references are:

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />
<PackageReference Include="NUnit" Version="3.8.1" />
<PackageReference Include="NUnit3TestAdapter" Version="3.9.0" />
<PackageReference Include="TeamCity.VSTest.TestAdapter" Version="1.0.7" />

BTW, I don't know if it matters but my test project is multi-targeting netcoreapp2.0 and net462.

Answer 1

This might not be very helpful, but it seems to be working for me correctly. I created the projects using the dotnet-cli.

First I installed the NUnit3 test adapter instructions from here. This only needs to be run once on each machine so you don't need to do it again if you have already run it.

dotnet new -i NUnit3.DotNetNew.Template

Then I created my solution, created my test project and added the test project to the solution.

dotnet new sln -n Solution
dotnet new nunit -n TestProject -o tests\TestProject
dotnet sln add tests\TestProject\TestProject.csproj

Then I updated UnitTest1.cs to include two test fixtures, one with the category Oracle and one with the category OracleOdbc.

using NUnit.Framework;

namespace Tests
{
    [TestFixture]
    [Category("Oracle")]
    public class OracleTests
    {
        [Test]
        public void OracleTest()
        {
            Assert.Fail();
        }
    }

    [TestFixture]
    [Category("OracleOdbc")]
    public class OracleOdbcTests
    {
        [Test]
        public void OracleOdbcTest()
        {
            Assert.Fail();
        }
    }
}

Then I can specify which category I choose to run.

dotnet test tests/TestProject/TestProject.csproj --filter TestCategory="Oracle"

or

dotnet test tests/TestProject/TestProject.csproj --filter TestCategory="OracleOdbc"

both run only one test and the message shows it is the correct test that fails.

Using DotNet-Cli version 2.1.4 and NUnit3TestAdapter version 3.9.0

Answer 2

In Nunit Framework, Category Attribute can be on the level of the Method.

Example:

public class NUnitTest 
    {
        [Test]
        [Category("CategoryA")] 
        public void TestMethod1()
        {
        }

        [Test]
        [Category("CategoryB")] 
        public void TestMethod2()
        {
        }

    }

and command is:

dotnet test --filter TestCategory=CategoryA     #Runs tests which are annotated with [Category("CategoryA")].

Also, there are many options on the level of method and others For more details: read