Dennis Burton's Develop Using .NET

Change is optional. Survival is not required.
Tags: tdd

In my last post, I mentioned the new ValuesAttribute that can be used as a test factory to generate a series of tests with many permutations of parameters. Looking into that feature led me to look at the feature set in NUnit 2.5 which is currently in Alpha 4. Some of these features address scenarios that I have run into in my test code. I wanted to mention them here so others could start benefiting as well.

The source of the new feature set that I am pulling from is the current release notes.

Movin' on up

One of the items that I think is most important is not a new feature, just a change in location. The change in mind set is what stands out as important to me. The Is, Has, Text, and List constraints have been moved into the NUnit.Framework namespace. They were formerly off in the more obscure NUnit.Framework.SyntaxHelpers namespace. The assertions that are created using these constraints line up much better with BDD and have a more readable feel to them. I am happy to see them becoming part of the mainstream namespace. If you are not familiar with the constraint model, check out code below.

// Classic model
Assert.AreEqual(expected, actual);
// Constraint model
Assert.That(actual, Is.EqualTo(expected));

I think the second reads much more like the English phrase that the constraint would represent. Check out the docs; you will find that almost everything you can do with the "classic" model can also be done in the constraint model.

Chained Setup and TearDown

Test code should be crafted with the same level of care as production intent code. This means that it should be DRY and carefully designed. Many times you will be testing a related set of classes, leading to base classes in your test code. Don't be afraid to capture common test code functionality in base classes. Prior to this release, the Setup and TearDown attributes could be applied in your base class, but if you applied the Setup or Teardown attributes in your derived class, the calls were not chained. You had to remember to call the base class version from the derived class. In 2.5, the base Setup method will be call prior to the derived Setup method by the framework. One more win for automation versus developer discipline.

New constraints

Chris Marinos recently blogged about disliking the [ExpectedException] style of tests and wrote a helper class to support the syntax of Throws.Exception. As of the 2.5 release Throws.Exception is included. This allows for much more focused testing. Putting the ExpectedExceptionAttribute on a method simply says that somewhere in the method the exception will get thrown. The exception may well be thrown 2 lines prior to the call you were intending to test. This yields a false positive test result. Throws.Exception (and its set of related Throws) takes a delegate so that you can check a specific call for the presence of an exception.

[Test]
[ExpectedException]
public void PassesButShouldnt()
{
    int importantPreWork = int.Parse("abcd");
    DoStuff();
}

[Test]
public void FailsProperly()
{
    int importantPreWork = int.Parse("abcd"); 
    Assert.Throws<Exception>( DoStuff );
}
public void DoStuff()
{ throw new Exception("Something is terribly wrong"); }

In the example, the PassesButShouldnt test passes due to the expected exception being thrown on the Parse call. Unfortunately, the test was intended to ensure that DoBadStuff threw an exception. The FailsProperly test more accurately checks that only the DoBadStuff call throws an exception. In addition, it also reads better as it places the constraint on the same line as the call just like any other Assert method.

Attribute testing

With the Has.Attribute constraint, you can verify an object (yes, object not class) is decorated with an attribute. A common example of where this could be used is validating that a class has been marked with the SerializableAttribute.

[Serializable]
public class testclass {}

[Test]
public void Test()
{ Assert.That(new testclass(), Has.Attribute<SerializableAttribute>()); }

I have to admit, when I first saw the Has.Attribute constraint, I thought for sure one of my test base classes was going to get a bit lighter. One of the applications that I work on has a pluggable architecture and uses attributes to determine what should be exposed to the end user. I had to code up a method that used reflection to determine if the items that were supposed to be exposed did in fact contain the attribute that exposed them. I was somewhat disappointed to see that it only seemed to work with instances of objects and not Types or MemberInfos. This did get me thinking, however, that my approach of putting this in a base class may not have been as elegant as learning about extending NUnit. Perhaps that will be a post in the future. I should call out that this was in the NUnit blog as a feature added in Alpha 1, however, it is the only feature listed here that has not made it to the documentation yet. That could mean that it just has not been documented yet, or it could indicate that it is not yet ready for prime time.

Range Testing

Is.InRange allows you to use a more concise syntax for expressing assertions that required both a GreaterThan and LessThan for validating an item is within bounds.

[Test]
public void RangeTesting()
{
    // old way
    Assert.That(3, Is.GreaterThanOrEqualTo(1) & Is.LessThanOrEqualTo(5));
    // new way
    Assert.That(3, Is.InRange(1, 5));

    // NOTE: InRange is inclusive
    Assert.That(5, Is.InRange(1, 5));
    Assert.That(1, Is.InRange(1, 5));
}

New Assertions

In some tests you are able to determine early in the test that the expected behavior has been met. Rather than throwing in a return statement or creating triangular code, the new Assert.Pass method is available for early termination with passing results. There is also a new inconclusive result state that can be set by calling Assert.Inconclusive. I have no idea how this can be used; I cannot think of any test scenarios that I have had to implement where I really was looking for "Assert...oh...I don't know." It seems there is a failure in the system at this point, even if that failure point is not understanding the requirements. I would take this as an indicator to go clarify requirements with the client and write a better test.

Specialized Assertions

CollectionAssert has picked up a set of IsOrdered constraints. This constraint uses the IComparable interface to verify increasing or decreasing order of all of the items in a collection. This is another one of those features that I have implemented in a test utilities class. I am happy to remove those lines of code.

Previous versions of NUnit had a FileAssert class to verify that your application was generating an output file. DirectoryAssert has been added as a compliment to FileAssert. The DirectoryAssert.IsWithin method (or IsNotWithin) will crawl from the directory specified, including all of its subdirectories, to verify that the expected directory is present. You can also use IsEmpty to verify if a file was output. You are, of course, still responsible for determining that the correct output was written.

Generic Support

There are several type constraints in NUnit that take typeof(class) as a parameter. The following constraints have been updated to include a version that takes a generic parameter specifying the type.

Assert.IsInstanceOf<T>(object actual);
Assert.IsNotInstanceOf<T>(object actual);
Assert.IsAssignableFrom<T>(object actual);
Assert.IsNotAssignableFrom<T>(object actual);
Assert.Throws<T>(TypeSnippet code);

The TestFixtureAttribute also picked up a generic version. Use this to test a class that takes a generic type parameter. Specify this attribute multiple times on a class, each time specifying a different type for the generic parameter. For each type specification, a new test instance is created and executed. This is useful when you were trying to exercise a class under test with a couple of different types. It does seem like this is the wrong level of abstraction for a test, but I am sure it is a reaction to some level of demand.

Try it out

Every one has a different tolerance for using things that have not been marked as released, yet. Early releases of NUnit traditionally have been very solid. Much like GMail still being in beta, it seems the open source community drop releases much more often, but keeps them flagged with alpha and beta. I have already started using these features to produce cleaner, more readable test code.

OpenID
Please login with either your OpenID above, or your details below.
Name
E-mail
(will show your gravatar icon)
Home page

Comment (Some html is allowed: a@href@title, strike) where the @ means "attribute." For example, you can use <a href="" title=""> or <blockquote cite="Scott">.  

Enter the code shown (prevents robots):

Live Comment Preview

Dennis Burton

View Dennis Burton's profile on LinkedIn
Follow me on twitter
Rate my presentations
Google Code repository

Community Events

Windows Azure Boot Camp Lansing GiveCamp