I was recently in a talk with a couple of people where I am currently consulting and the question came up about what type of unit testing was best. Should one use TDD or BDD, or perhaps TDD with a BDD style? Or is it okay to write tests after the class is coded? So many questions.
Here is my short answer about testing for most places I have been involved: Just start testing. That sounds like a pretty bad answer, perhaps, but I find most organizations simply aren’t testing. At a previous gig, I came in and attempted to sell the unit test idea and was told there was no time in the schedule to allow for testing. Common theme.
At my current assignment, I currently have about 100 tests on my code. The number will increase as defects are found (rule: every defect requires a test) and I and the test team complete analysis of the rabbit holes in the code. The team I am closely aligned to has 0 useful tests (a few tests, but full integration and very limited to happy path). Another group has 6 tests, all of which are either useless, badly named, or empty. Actually all three in all but one case. Same theme.
Not Enough Time
The normal throw back is the time involved in writing tests: We can’t afford the extra time it takes to write tests. First, I think this is a bogus answer. Yes, there is an initial learning curve cost, but if your organization has no time for learning new technology, you really should not have your own developers. Technology is being vomited out at a rapid pace and organizations that do not factor in time to learn get farther and farther behind. And, yes, there is some time writing tests, but it is really minimal when you add all of the factors. Examine these:
- Testing first (TDD/BDD), and to an extent testing early (just after coding classes), helps the developer find design flaws before too much code is written, saving rewrite time. This, alone, is often enough to make up for the “extra” time of writing the tests.
- Testing first or early allows for safe refactoring of code. This improves both quality and maintainability. This does not save time, per se, but it does stop developers from wasting time seeking bugs out created when refactoring blindly.
- Testing adds a safe haven for maintenance and new features. Developers can ensure their changes do not destroy current functionality. This increase productivity in two ways. First, the developer feels safe coding and will code more quickly. Second, any bugs created by “fixing” or adding code can quickly be resolved.
I worked on one assignment recently where I had to take over a code base. The developer had already burned 5 weeks of time and the design decisions created a path where there was quite a few more weeks to solve the problem. Taking over, I rewrote most of the code and flipped the design to a more domain driven design. I reused algorithms, but the entire design was scrapped. Development time about 2 weeks, including tests.
During the development, it was discovered the number of lines in the output file were much larger than projected. The idea of spinning the entire object graph into memory had to be scrapped and lines had to be written to file as soon as the final answer for the line was determined. Major change. But, with tests surrounding the code, the refactor took less than a week. I can only imagine how long it would have taken with the original base.
Note that tests are not a magic bullet that saved this project all by themselves. The original design separated out the business objects in a way that described the output file very well. Because of this design, moving from walking the object graph to passing a writer was much easier. There was also a difference in experience between the developers.
The important point here is the tests were written without a major time impact. And, because of the tests, a major design change had little impact on delivery. The correct argument, based on this project, is “you don’t have time NOT to test”.
How should I test?
This is a question that is hard to answer. There are all sorts of opinions on the best way, or even the right way, to test. With some purists, any deviation from the pure TDD or BDD way of testing is blasphemy. So, I will share my general “model” of testing.Determine the classes in your domain. Focusing on what you are building, from a business perspective, what do the objects look like?
- Write tests on the domain objects. These tests will “always” pass, so some consider them useless. I disagree, as you can reuse the tests when you start unit testing behavior, as most of your behavior alters state of at least one object.
- Determine the boundaries in your code. The question here is “where am I calling another library?”. You don’t have to do this perfect, but you should separate out your data access, domain objects, mapping data to domain objects, and core business functionality in your first pass. These are your boundaries.
- Write a “contract” library for each library that sits on a boundary. This is critical for mocking.
- Figure where you need to start. I generally will look at one of two areas: the area where I am the foggiest about the design or the area that has the greatest impact on the code. I then write tests based on the requirements.
- I generally skip the “it does not compile” step, but this is important when developing the discipline of TDD or BDD. I also often skip the “fail” step, going to first green. Yes, I know this is “wrong”, especially to the purist, but initial pass is most often 1 line of code, and I have yet to be bitten.
- Build up a battery of tests around the code base, building functionality up slowly.
How about the style of test? I prefer this type of model:
- Spin up the unit tested in class initialize and run the code
- Create a test for each piece of state it changes
Now, this is a general answer, as there are times when a single test is a more applicable approach. The important part is having a consistent way of testing, even if you blend testing styles based on different factors.
Testing Styles and Methodologies
I will now take a moment to run through a few different “styles” of testing, based on different methodologies. This is the most dangerous part of this post, as some purist will likely disagree with something I say here. 
Test Driven Development (TDD) Style
NOTE: TDD is a methodology where tests are written first to design the code. This section does not describe TDD. It merely covers a common style of organizing and writing tests.
As a style, this generally means writing a test class like so:
[TestClass()]
public class LogonInfoTest
{
[TestMethod()]
public void ConstructorTest()
{
string userId = "IMontoya";
string password = "P@ssw0rd";
LogonInfo logonInfo = new LogonInfo(userId, password);
Assert.AreEqual<string>(userId, logonInfo.UserId,
"The UserId was not correctly initialized.");
Assert.AreEqual<string>(password, logonInfo.Password,
"The Password was not correctly initialized.");
}
[TestMethod]
[ExpectedException(typeof(ArgumentException),
"A userId of null was inappropriately allowed.")]
public void NullUserIdInConstructor()
{
LogonInfo logonInfo = new LogonInfo(null, "P@ss0word");
}
[TestMethod]
[ExpectedException(typeof(ArgumentException),
"A empty userId was inappropriately allowed.")]
public void EmptyUserIdInConstructor()
{
LogonInfo logonInfo = new LogonInfo("", "P@ss0word");
}
}
The above taken from http://goo.gl/HinNT.
The basics here are the class identifies the the class being tested, while the methods identify what is being tested. There are other styles of organization, but most TDD examples on the web use the following “rules”:
- Name of Test Class: {ClassBeingTested}Tests – Example: DomainMapperTests
- Name of Test Method: {MethodBeingTested}Test
When I used this type of style, I would add _Pass and _Fail to have both a pass and fail test for each method being tested. I then changed my style to have more information on why there is a pass or fail. Eventually, my ruleset moved me to a a more BDD style.
Test Driven Approach with BDD styling
This is a hybrid I have been known to use, as true BDD (next section) requires more than merely writing unit tests. Using this style fits with the spin up in initialize, run numerous tests on state, style of test coding. Here is an example, using the same logon methodology from the TDD section.
[TestClass()]
public class when_logging_in_with_valid_credentials
{
private static LogonInfo _logonInfo;
private static string _userId;
private static string _password;
[ClassInitialize]
public void ClassInitialize()
{
_userId = “IMontoya”;
_password = “P@ssw0rd”;
_logon = new LogonInfo(userId, password);
}
[TestMethod()]
public void should_have_userId_of_IMontoya()
{
Assert.AreEqual<string>(_userId, _logonInfo.UserId,
“The UserId is not IMontoya. It is not initialized properly”);
}
[TestMethod()]
public void should_have_password_of_P@ssword()
{
Assert.AreEqual<string>(_password, _logonInfo.Password,
“The password is not P@ssword. It is not initialized properly”);
}
}
NOTE: While I am not against this type of test, it really does very little for us, by itself, other than show the values are properly set. In my normal development methodology, I would have these types of tests on the state objects only and then reuse the tests. The test above illustrates the same code being tested as the previous section, and nothing more.
Notice that the you would need to set up a separate class for the failures. This means more classes are developed, although the amount of total code is about the same.
The important takeaway is the class describes what you are testing and the tests describe the expectations based on that test. In general, you will end up with more test classes using the BDD styling. As a basic rule, think TDD = fewer classes, more methods and BDD styling = more classes, fewer methods.
One reason I like the BDD styling method is I can write tests for the domain models (always pass, I know) and then reuse the test methods as I test behavior. Another is the method name explicitly spells out the expectation, so I can, in most cases, find the location of the error to set a breakpoint without walking code. Great time savings.
I watched Eric Hexter and Greg Long talk about testing yesterday and they used a similar naming standard for test methods, but spun up the class in each method. While I would head in a different direction (class initialize to spin class up), it was a very effective session. The important point here is the implementation can be different, but testing is the key. If you are not testing because of a disagreement with styling, change style rather than avoid testing.
Behavior Driven Design (BDD) Style
Unlike TDD, it is hard to talk about BDD merely as a style. As illustrated in the last section, you can use a BDD style with a unit test framework, but truly practicing BDD requires a test runner that allows you to fully express the behavior in a predictable way (as far as testing goes). There are a couple of BDD frameworks out there for .NET.
- mSpec
- SpecFlow
- SpecUnit.NET
Due to ease of use, I prefer SpecFlow. There are some issues I have with the current implementation, primarily with helping stub in the steps files (there is a kludge when using nUnit), but I really can’t bitch too much, as it is open source and I can contribute. SpecFlow works with Gherkin, a language used with a testing tool called Cucumber (test tool for Ruby). The basics of Gherkin is you use the keywords GIVEN, WHEN and THEN, as such.
GIVEN {a precondition} – precondition
WHEN {I do something} – condition/event
THEN {I expect something} – expectations
You can add AND to have multiple preconditions, conditions and expectations. An example based on the SpecFlow bowling application example:
Scenario: Bowling a strike
Given a new frame
When I knock down all pins
Then my score should not be added up until the next two rolls are completed
And the value for the frame box should be “X”
SpecFlow will use a style where the object is spun up first, with all preconditions. The condition/event is then fired off and each of the expectations are tested.
I would love to take the original tests and describe them in Gherkin, but there is little value to the type of test, as there is no real behavior. It is nothing more than “I set the values correctly”.
Summary
This was a real short whirlwind tour of TDD, TDD frameworks with BDD styling and BDD. I did not even touch on some other testing methodologies. If there are any takeaways you should get it is this:
- The idea that testing takes longer is a myth. There is certainly learning curve, but that is true of anything new. There is also a small bit of time creating tests, but it is most often negligible, after you understand testing, and there is no impact when you consider the entire schedule. One might argue there is more time taken, but I would argue that time should have been taken making sure you were developing the right stuff anyway.
- The main difference on different testing types is the focus of the test. TDD style allows easy testing of state only, behavior only or behavior and state. BDD focuses on behavior, making state only testing a bit of a kludge. In most cases, you still will test the state of the system as related to behavior.
- If you have a question on which style, rather than spend a lot of time determining the path, why not go ahead and start testing? You can always adapt your strategy over time and refactor tests. It is far safer than changing code blind.
Hope this helps you on your journey. I will likely hit quite a few more posts on testing over the next few months.
Peace and Grace,
Greg
Twitter: @gbworld
