Testing test assembly

I recently discovered that almost 50 unit tests don’t run for Mail.dll email library test suite.

Mail.dll uses NUnit as its unit testing framework, and with over 3000 tests it’s hard to spot that some are missing. The problem was that most (of those 50 that didn’t run) test methods where missing TestAttribute and 2 test classes were missing TestFixtureAttribute.

I decided that it’s time to create 2 unit tests that guard my unit test assembly.

First one checks if all public methods, in classes marked with TestFixtureAttribute, have TestAttribute applied.

Of curse we are using reflection for that.

It would be nice if we could just double click on the trace output and be taken, by VisualStudio, to the source file. Unfortunately it’s not possible, with standard .NET reflection, to get source code file name and line. I decided against using Mono.Cecil for that, especially that with R# it’s easy enough to navigate to method name.

[Test]
public void Methods_InTestFixtureTypes_HaveTestAttribute()
{
    int count = 0;
    foreach (Type type in this.GetType().Assembly.GetTypes())
    {
        if (Attribute.IsDefined(type, typeof(TestFixtureAttribute)))
        {
            foreach (MethodInfo method in type.GetMethods())
            {
                if (method.IsPublic
                    && method.IsSpecialName == false
                    && method.DeclaringType == type
                    && !Attribute.IsDefined(method, typeof(TearDownAttribute))
                    && !Attribute.IsDefined(method, typeof(SetUpAttribute))
                    && !Attribute.IsDefined(method, typeof(TestAttribute))
                    )
                {
                    Trace.WriteLine(type.FullName + "." + method.Name);
                    count++;
                }
            }
        }
    }
    Assert.AreEqual(0, count);
}

Second test checks if all public classes, that have at least one method marked with TestAttribute, have TestFixtureAttribute applied.

[Test]
public void Types_WithTestMethods_HaveTestFixtureAttribute()
{
    int count = 0;
    foreach (Type type in this.GetType().Assembly.GetTypes())
    {
        bool hasAtLeastOneTestMethod = new List<MethodInfo>(type.GetMethods()).Find(
            method => Attribute.IsDefined(method, typeof(TestAttribute))) != null;

        if (hasAtLeastOneTestMethod
            && Attribute.IsDefined(type, typeof(TestFixtureAttribute)) == false)
        {
            Trace.WriteLine(type.FullName);
            count++;
        }
    }
    Assert.AreEqual(0, count);
}

Some of you might argue that we shouldn’t be using Trace class inside the unit test. StringBuilder, WriteLine and
Assert.AreEqual(object expected, object actual, string message) are your friends in such case.

Tags: Unit testing