NUnit Extensibility and AutoFixture

As a software consultant we can’t always choose the libraries we use on a client site, and so the story begins.

I fell in love with a library called AutoFixture-it really simplifies the arrange phase when writing unit tests. There are many patterns on how to use AutoFixture, but the one I prefer keeps test code free of direct AutoFixture dependencies.

Mark Seemann, the author of AutoFixture, also wrote an extension for xUnit.net called AutoDataAttribute which supplies auto-generated arguments to a parameterised unit test:

[Theory, AutoData]
public void IntroductoryTest(int expectedNumber, MyClass sut)
{
    int result = sut.Echo(expectedNumber);
    Assert.Equal(expectedNumber, result);
}

Extending AutoDataAttribute allows you to customise the internal fixture that generates arguments.

public class TestConventionsAttribute : AutoDataAttribute
{
    public TestConventionsAttribute()
        : base(new Fixture().Customize(new AutoNSubstituteCustomization()))
    {
    }
}

I wanted the same experience in NUnit. After reading several blog posts and the NUnit documentation on extensibility I was more confused than when I started.

Where to begin? NUnit has a TheoryAttribute, but the implementation differs hugely from xUnit. The NUnit DataAttribute is property-only, so it wasn’t even an option to extend.

After more investigation I found TestCaseAttribute and examples of parameterised unit tests in NUnit:

[TestCase(12, 3, 4)]
[TestCase(12, 2, 6)]
[TestCase(12, 4, 3)]
public void DivideTest(int n, int d, int q)
{
    Assert.AreEqual(q, n / d);
}

This was closer to what I wanted, so I created an AutoTestCaseAttribute inheriting TestCaseAttribute. After many attempts to provide arguments dynamically I discovered the property I needed to set was internal-and intentionally so. #sadface

More reading led me to TestCaseSourceAttribute, which lets you define parameters elsewhere:

static int[] EvenNumbers = new int[] { 2, 4, 6, 8 };

[Test, TestCaseSource("EvenNumbers")]
public void TestMethod(int num)
{
    Assert.IsTrue(num % 2 == 0);
}

This looked ideal, but wasn’t meant to be. The arguments passed must be statically defined and extending the attribute hit the same issues as TestCaseAttribute.

The conclusion: I needed to write an NUnit add-in that implements ITestCaseProvider2.

public interface ITestCaseProvider
{
    bool HasTestCasesFor(MethodInfo method);
    IEnumerable GetTestCasesFor(MethodInfo method);
}

public interface ITestCaseProvider2 : ITestCaseProvider
{
    bool HasTestCasesFor(MethodInfo method, Test suite);
    IEnumerable GetTestCasesFor(MethodInfo method, Test suite);
}

Below is where the magic happens:

public class AutoTestCaseProvider : ITestCaseProvider2
{
    ...

    public bool HasTestCasesFor(MethodInfo method)
    {
        return Reflect.HasAttribute(method, AutoFixtureNUnitFramework.AutoTestCaseAttribute, false);
    }

    ...

    public IEnumerable GetTestCasesFor(MethodInfo method, Test parentSuite)
    {
        Type[] parameterTypes = method.GetParameters().Select(o => o.ParameterType).ToArray();

        ArrayList parameterList = new ArrayList();

        var attributes = Reflect.GetAttributes(method, AutoFixtureNUnitFramework.AutoTestCaseAttribute, false);

        foreach (TestCaseDataAttribute attr in attributes)
        {
            foreach (var arguments in attr.GetArguments(method, parameterTypes))
            {
                ParameterSet parms = new ParameterSet();
                parms.Arguments = arguments;

                parameterList.Add(parms);
            }
        }

        return parameterList;
    }

    ...
}

Using reflection, NUnit.Framework identifies any unit tests with AutoTestCaseAttribute applied, and calls GetArguments. That method uses the configured fixture to generate arguments for the parameterised test.

public abstract class TestCaseDataAttribute : Attribute
{
    ...

    public abstract IEnumerable<object[]> GetArguments(MethodInfo method);

    ...
}

public class AutoTestCaseAttribute : TestCaseDataAttribute
{
    ...

    public override IEnumerable<object[]> GetArguments(MethodInfo method)
    {
        if (method == null)
        {
            throw new ArgumentNullException("method");
        }

        var specimens = new List<object>();
        foreach (var p in method.GetParameters())
        {
            CustomizeFixture(p);

            var specimen = Resolve(p);
            specimens.Add(specimen);
        }

        return new[] { specimens.ToArray() };
    }

    ...
}

An example of the final result:

[TestFixture]
public class Tests
{
    [Test, AutoTestCase]
    public void IntroductoryTest(int expectedNumber, MyClass sut)
    {
        int result = sut.Echo(expectedNumber);
        Assert.Equal(expectedNumber, result);
    }
}

As I write this I have submitted a pull request and I’m awaiting a code review. Hopefully the code will be merged and set free for the world to use soon.