Unit Testing Basics

I was recently in a talk with a couple of people where I am currently consulting and the question came up about what type of unit testing was best. Should one use TDD or BDD, or perhaps TDD with a BDD style? Or is it okay to write tests after the class is coded? So many questions.

Here is my short answer about testing for most places I have been involved: Just start testing. That sounds like a pretty bad answer, perhaps, but I find most organizations simply aren’t testing. At a previous gig, I came in and attempted to sell the unit test idea and was told there was no time in the schedule to allow for testing. Common theme.

At my current assignment, I currently have about 100 tests on my code. The number will increase as defects are found (rule: every defect requires a test) and I and the test team complete analysis of the rabbit holes in the code. The team I am closely aligned to has 0 useful tests (a few tests, but full integration and very limited to happy path). Another group has 6 tests, all of which are either useless, badly named, or empty. Actually all three in all but one case. Same theme.

Not Enough Time

The normal throw back is the time involved in writing tests: We can’t afford the extra time it takes to write tests. First, I think this is a bogus answer. Yes, there is an initial learning curve cost, but if your organization has no time for learning new technology, you really should not have your own developers. Technology is being vomited out at a rapid pace and organizations that do not factor in time to learn get farther and farther behind. And, yes, there is some time writing tests, but it is really minimal when you add all of the factors. Examine these:

1. Testing first (TDD/BDD), and to an extent testing early (just after coding classes), helps the developer find design flaws before too much code is written, saving rewrite time. This, alone, is often enough to make up for the “extra” time of writing the tests.
2. Testing first or early allows for safe refactoring of code. This improves both quality and maintainability. This does not save time, per se, but it does stop developers from wasting time seeking bugs out created when refactoring blindly.
3. Testing adds a safe haven for maintenance and new features. Developers can ensure their changes do not destroy current functionality. This increase productivity in two ways. First, the developer feels safe coding and will code more quickly. Second, any bugs created by “fixing” or adding code can quickly be resolved.

I worked on one assignment recently where I had to take over a code base. The developer had already burned 5 weeks of time and the design decisions created a path where there was quite a few more weeks to solve the problem. Taking over, I rewrote most of the code and flipped the design to a more domain driven design. I reused algorithms, but the entire design was scrapped. Development time about 2 weeks, including tests.

During the development, it was discovered the number of lines in the output file were much larger than projected. The idea of spinning the entire object graph into memory had to be scrapped and lines had to be written to file as soon as the final answer for the line was determined. Major change. But, with tests surrounding the code, the refactor took less than a week. I can only imagine how long it would have taken with the original base.

Note that tests are not a magic bullet that saved this project all by themselves. The original design separated out the business objects in a way that described the output file very well.  Because of this design, moving from walking the object graph to passing a writer was much easier. There was also a difference in experience between the developers.

The important point here is the tests were written without a major time impact. And, because of the tests, a major design change had little impact on delivery. The correct argument, based on this project, is “you don’t have time NOT to test”.

How should I test?

This is a question that is hard to answer. There are all sorts of opinions on the best way, or even the right way, to test. With some purists, any deviation from the pure TDD or BDD way of testing is blasphemy. So, I will share my general “model” of testing.Determine the classes in your domain. Focusing on what you are building, from a business perspective, what do the objects look like?

1. Write tests on the domain objects. These tests will “always” pass, so some consider them useless. I disagree, as you can reuse the tests when you start unit testing behavior, as most of your behavior alters state of at least one object.
2. Determine the boundaries in your code. The question here is “where am I calling another library?”. You don’t have to do this perfect, but you should separate out your data access, domain objects, mapping data to domain objects, and core business functionality in your first pass. These are your boundaries.
3. Write a “contract” library for each library that sits on a boundary. This is critical for mocking.
4. Figure where you need to start. I generally will look at one of two areas: the area where I am the foggiest about the design or the area that has the greatest impact on the code. I then write tests based on the requirements.
5. I generally skip the “it does not compile” step, but this is important when developing the discipline of TDD or BDD. I also often skip the “fail” step, going to first green. Yes, I know this is “wrong”, especially to the purist, but initial pass is most often 1 line of code, and I have yet to be bitten.
6. Build up a battery of tests around the code base, building functionality up slowly.

How about the style of test? I prefer this type of model:

1. Spin up the unit tested in class initialize and run the code
2. Create a test for each piece of state it changes

Now, this is a general answer, as there are times when a single test is a more applicable approach. The important part is having a consistent way of testing, even if you blend testing styles based on different factors.

Testing Styles and Methodologies

I will now take a moment to run through a few different “styles” of testing, based on different methodologies. This is the most dangerous part of this post, as some purist will likely disagree with something I say here.

Test Driven Development (TDD) Style

NOTE: TDD is a methodology where tests are written first to design the code. This section does not describe TDD. It merely covers a common style of organizing and writing tests.

As a style, this generally means writing a test class like so:

   [TestClass()]
   public class LogonInfoTest
   {
      [TestMethod()]
      public void ConstructorTest()
      {
         string userId = "IMontoya";

         string password = "P@ssw0rd";

         LogonInfo logonInfo = new LogonInfo(userId, password);

         Assert.AreEqual<string>(userId, logonInfo.UserId,
            "The UserId was not correctly initialized.");
         Assert.AreEqual<string>(password, logonInfo.Password,
            "The Password was not correctly initialized.");
      }


      [TestMethod]
      [ExpectedException(typeof(ArgumentException),
         "A userId of null was inappropriately allowed.")]
      public void NullUserIdInConstructor()
      {
         LogonInfo logonInfo = new LogonInfo(null, "P@ss0word");
      }

      [TestMethod]
      [ExpectedException(typeof(ArgumentException),
         "A empty userId was inappropriately allowed.")]
      public void EmptyUserIdInConstructor()
      {
         LogonInfo logonInfo = new LogonInfo("", "P@ss0word");
      }
  }

The above taken from http://goo.gl/HinNT.

The basics here are the class identifies the the class being tested, while the methods identify what is being tested. There are other styles of organization, but most TDD examples on the web use the following “rules”:

• Name of Test Class: {ClassBeingTested}Tests – Example: DomainMapperTests
• Name of Test Method: {MethodBeingTested}Test

When I used this type of style, I would add _Pass and _Fail to have both a pass and fail test for each method being tested. I then changed my style to have more information on why there is a pass or fail. Eventually, my ruleset moved me to a a more BDD style.

Test Driven Approach with BDD styling

This is a hybrid I have been known to use, as true BDD (next section) requires more than merely writing unit tests. Using this style fits with the spin up in initialize, run numerous tests on state, style of test coding. Here is an example, using the same logon methodology from the TDD section.

[TestClass()]
public class when_logging_in_with_valid_credentials
{

  private static LogonInfo _logonInfo;
  private static string _userId;
  private static string _password;
  [ClassInitialize]
  public void ClassInitialize()
  {
    _userId = “IMontoya”;
    _password = “P@ssw0rd”;
    _logon = new LogonInfo(userId, password);
  }

  [TestMethod()]
  public void should_have_userId_of_IMontoya()
  {
    Assert.AreEqual<string>(_userId, _logonInfo.UserId,
    “The UserId is not IMontoya. It is not initialized properly”);
  }

  [TestMethod()]
  public void should_have_password_of_P@ssword()
  {
    Assert.AreEqual<string>(_password, _logonInfo.Password,
    “The password is not P@ssword. It is not initialized properly”);
  }
}

NOTE: While I am not against this type of test, it really does very little for us, by itself, other than show the values are properly set. In my normal development methodology, I would have these types of tests on the state objects only and then reuse the tests. The test above illustrates the same code being tested as the previous section, and nothing more.

Notice that the you would need to set up a separate class for the failures. This means more classes are developed, although the amount of total code is about the same.

The important takeaway is the class describes what you are testing and the tests describe the expectations based on that test. In general, you will end up with more test classes using the BDD styling. As a basic rule, think TDD = fewer classes, more methods and BDD styling = more classes, fewer methods.

One reason I like the BDD styling method is I can write tests for the domain models (always pass, I know) and then reuse the test methods as I test behavior. Another is the method name explicitly spells out the expectation, so I can, in most cases, find the location of the error to set a breakpoint without walking code. Great time savings.

I watched Eric Hexter and Greg Long talk about testing yesterday and they used a similar naming standard for test methods, but spun up the class in each method. While I would head in a different direction (class initialize to spin class up), it was a very effective session. The important point here is the implementation can be different, but testing is the key. If you are not testing because of a disagreement with styling, change style rather than avoid testing.

Behavior Driven Design (BDD) Style

Unlike TDD, it is hard to talk about BDD merely as a style. As illustrated in the last section, you can use a BDD style with a unit test framework, but truly practicing BDD requires a test runner that allows you to fully express the behavior in a predictable way (as far as testing goes). There are a couple of BDD frameworks out there for .NET.

• mSpec
• SpecFlow
• SpecUnit.NET

Due to ease of use, I prefer SpecFlow. There are some issues I have with the current implementation, primarily with helping stub in the steps files (there is a kludge when using nUnit), but I really can’t bitch too much, as it is open source and I can contribute. SpecFlow works with Gherkin, a language used with a testing tool called Cucumber (test tool for Ruby). The basics of Gherkin is you use the keywords GIVEN, WHEN and THEN, as such.

GIVEN {a precondition} – precondition
WHEN {I do something} – condition/event
THEN {I expect something} – expectations

You can add AND to have multiple preconditions, conditions and expectations. An example based on the SpecFlow bowling application example:

Scenario: Bowling a strike
Given a new frame
When I knock down all pins
Then my score should not be added up until the next two rolls are completed
And the value for the frame box should be “X”

SpecFlow will use a style where the object is spun up first, with all preconditions. The condition/event is then fired off and each of the expectations are tested.

I would love to take the original tests and describe them in Gherkin, but there is little value to the type of test, as there is no real behavior. It is nothing more than “I set the values correctly”.

Summary

This was a real short whirlwind tour of TDD, TDD frameworks with BDD styling and BDD. I did not even touch on some other testing methodologies. If there are any takeaways you should get it is this:

1. The idea that testing takes longer is a myth. There is certainly learning curve, but that is true of anything new. There is also a small bit of time creating tests, but it is most often negligible, after you understand testing, and there is no impact when you consider the entire schedule. One might argue there is more time taken, but I would argue that time should have been taken making sure you were developing the right stuff anyway.
2. The main difference on different testing types is the focus of the test. TDD style allows easy testing of state only, behavior only or behavior and state. BDD focuses on behavior, making state only testing a bit of a kludge. In most cases, you still will test the state of the system as related to behavior.
3. If you have a question on which style, rather than spend a lot of time determining the path, why not go ahead and start testing? You can always adapt your strategy over time and refactor tests. It is far safer than changing code blind.

Hope this helps you on your journey. I will likely hit quite a few more posts on testing over the next few months.

Peace and Grace,
Greg

Twitter: @gbworld

TSA Scanners

There has been a lot of controversy over the new scanning procedures lately. And, like good stewards of the realm, the media has jumped into to protect the government “underdog” from the onslaught of the “ignorant Neanderthals” ignoring “good science”. The question, however, is whether or not we have properly labeled the sides.

There are two basic areas of concern for the masses who are against the new procedures. The first is health concerns over being scanned, primarily from backscatter X-ray radiation. The second is the privacy concerns.

Health Concerns

Are we getting too much radiation when we submit to scans? The government states no. The amount  of radiation from a single scan is estimated at .05 to .1 mrem (micro rems), which is 1000 times less than a standard X-Ray. CBS News then states a variety of things that will give you more radiation.

• Living in a city
• Sleeping next to a partner
• Flying
• Drinking water

There are two problems with the comparisons. First, we have no choice over many of the activities that give us radiation. We can’t stop living on this earth or drinking liquids. But, we do not have to submit to scans unless the government forces us to do it to fly. Second, the amount of radiation is under debate.

The figure quoted is a whole body radiation estimate from the device. If backscatter used the same energy as a standard X-Ray, your full body would get the dose mentioned. But backscatter radiation is not equivalent, as it uses lower energy and does not penetrate the body. A group of scientists in California have estimated the amount of radiation on the skin is likely to be 20 times higher, if not more. Considering the limited depth of the X-Ray, I would not be surprised to find this concentrated area receiving much higher doses. Until there is a peer reviewed study, all figures should be considered suspect.

The government estimates 1 additional cancer death per 200 million scans, so there is an awareness of a risk, albeit a very minor risk (except to that one person?). One additional death means many additional individuals coming down with cancer. The question is how many additional cancer deaths if the radiation figure is off by a factor of 20 … or more?

There are also scanners that use millimeter wave technology, or radio waves. The assumption is these scanners are safer, as they do not use radiation. Probably correct, but there are detractors from this idea. Scientists in California have shown that while radio waves cannot knock DNA base pairs or sequences out of whack, like radiation, they can do other types of DNA damage, like ripping the DNA strands apart. The setup of the millimeter wave scanners are different enough they could not cause this type of damage, but are there other unhidden types of damage?

There is also a topic I have not seen covered by the media and that is the fact that the scanners are set a low resolution. This leaves an option of increasing resolution; perhaps not with the current machines (not sure), but what if we decide we need more? Increasing resolution means increasing the beam, which means increasing the risk.

The risk appears to be very low. In fact, I would agree with the apologists that it probably is low, but I question whether or not it is necessary. More about this in the section on Effectiveness and Reasoning.

Privacy Concerns

One image that is being bandied about the Internet is this one:

Click photo to read article on this photo

This is stated to be a TSA image reversed in Photoshop. The gun in the back gives it enough validity for the average person to buy into what I see as a ruse. The problem is neither of the technologies present would capture the hair. NOTE: The photo shoot these pix were faked from is available here (NSFW).

But, even with the scans not able to capture this good of a picture, the scans are good enough to capture some detail which could embarrass someone. TSA has stated the images cannot be saved and later disseminated, but there are scans from the same type of scanners, albeit NOT in an airport, that have been released on the Internet. These scans are low resolution, but the resolution can be turned up.

I personally don’t have a lot of concern on the privacy front. I already have people scanning all of my possessions and occasionally rifling through my possessions. If privacy was a major concern, I feel the people should have revolted a long time ago. I will once again ask, however, whether or not it is necessary.

Effectiveness and Reasoning

If you have not seen the video of Adam Savage speaking in Seattle, you should take a look. On a recent flight, he states he accidentally carried on two extremely long razor blades which were not caught by the scanners. Others are envisioning terrorists molding explosives to their bodies, which would be missed by the scanners.

This leads one to wonder if the device is effective. And, if not, why burden the public with any risk?

This is not really about safety, but the appearance of safety. Certainly, the scans do catch some things. But the scans are only as good as the scanner, and they miss a lot. I have made it through security with full bottles of water, although they did catch my slightly oversized tube of toothpaste once (God forbid someone walking on with really clean teeth).

What we are observing is security theater. Make it look good and hopefully you will deter enough people that mean the public harm. At the same time, you will keep the average member of the public feeling safe enough to fly and ensure we have an airline industry to get us to our destinations.

When the security theater is merely an inconvenience, then it is probably fine. The deterrent is probably worth an extra half hour, right? But when you add inconvenience to a 1 in X chance of cancer and potential invasion of privacy, is it still worth it? If it catches people who bring very large razor blades on planes, then we might say yes … but they missed that one. Maybe next time?

The sheeple of the United States have acquiesced to the scans overall. I talked to a gentleman in the Austin airport the day before Thanksgiving who was ridiculing anyone who would not walk through the scans. I presented him with some contrary evidence that was backed by scientists, so he probably thinks I am one of the idiots now. Then again, perhaps I gave him a bit of food for thought.

My Thoughts

I am not going to go through the scanners. I don’t feel there is a huge risk, but I don’t feel there is a need to burden myself with any additional risk, no matter how small. If I go through the scanner, the government won’t make my plane arrive 4 minutes faster to offset the additional radiation. And they can’t shield me from radiation in my home town, or wherever I visit. There are simply no ways to offset this additional risk through some type of radiation “trade”.

Until there is a peer reviewed scientific study of the scanners, it is unlikely I will ever submit. And, even then, I don’t need to risk being one of the 1 in 200 million, no matter how small that risk is.

If the scans stop terrorists from blowing up planes, perhaps they can be a good thing. To date, I have heard of no incidents where the scanners have caught anyone. And, if they do, I would like some evidence the person would not have been caught by other means already in place, however, before giving any stamp of approval to a scanner.

I have serious issues with the idea that a handful of people dying of cancer each year is worth the security theater and appearance of safety. Until we show a case where it works, and previous technology would not work, I don’t see my mind changing on this one. Even then, I have to question how many such attempts thwarted by the scanners equal the number of projected deaths.

Peace and Grace,
Greg

Twitter: @gbworld