Refactoring to TDD


Okay, so a few of you might think I am cracked right now. The rest of you are reading this, not really realizing the irony in the title. Think about it for a second. Can you really refactor code to be Test Driven? No, but TDD is a word that people recognize, and you can refactor your development paradigm to TDD. That is what we are talking about here.

What I mean when I say “refactor to TDD”, in this post is refactoring from informal integration tests or no tests to unit testing. In this particular post I am moving from an environment where there is no formal unit testing done by the developers in the organization to formal unit testing. In this environment, any testing done is integration testing, exercising all layers of the application. And, much of it is not automated. Instead, wrappers are written around the code to test output.

As an example, you will see a “test” coded in a separate windows forms application (or console application, etc):

    public Form1()
    {
        InitializeComponent();
    }

    private void TestButton1_Click(object sender, EventArgs e)
    {
        string name = "Greg";
        string message = MessageGenerator.GetMessage(name);

        if (message != "Hello Greg!")
        {
            MessageBox.Show("GetMessage() failed and did not return 'Hello Greg!'");
        }
        else
        {
            MessageBox.Show("GetMessage() test succeeded with 'Hello Greg!'");
        }
    }

Now, let’s assume the GetMessage() routine is in a middle tier class and looks like this:

public static string GetMessage(string name)
{
    StringBuilder builder = new StringBuilder();
    builder.Append("Hello ");
    builder.Append(name);
    builder.Append('!');

    return builder.ToString();
}

So far this is simple stuff. But let’s suppose we get some change orders.

New Requirements

In the new requirements, we pull a random quote from a database on another line. So, the GetMessage() routine starts to look more like this:

public static string GetMessage(string name)
{
    StringBuilder builder = new StringBuilder();
    builder.Append("Hello ");
    builder.Append(name);
    builder.Append('!');
    builder.Append("rn");
    builder.Append(RandomQuoteGenerator.GetRandomQuote());

    return builder.ToString();
}

GetRandomQuote() is the routine that calls the database. It looks something like this (noting that this is not really good form, it is to illustrate refactoring and moving to a TDD type paradigm):

public string GetRandomQuote()
{
    //Bad that this is hardcoded, but showing the proper way to 
    //get max from table == bloated blog entry
    int max = 1000;

    Random random = new Random((int)DateTime.Now.Ticks);
    int id = random.Next(1, tableMax);

    SqlConnection connection = new SqlConnection(ConnectionString);
    SqlCommand command = new SqlCommand("GetRandomQuote");
    command.CommandType = CommandType.StoredProcedure;

    string returnValue;

    try
    {
        connection.Open();
        returnValue = command.ExecuteScalar().ToString();
    }
    finally
    {
        connection.Dispose();
    }

    return returnValue;
}

The important part is our test becomes rather munged up, as we have to do something more like this:

private void TestButton1_Click(object sender, EventArgs e)
{
    string name = "Greg";
    string message = MessageGenerator.GetMessage(name);
    string[] messageSplit = message.Split("rn".ToCharArray());

    if (messageSplit[0] != "Hello Greg!")
    {
        MessageBox.Show("GetMessage() failed and did not return 'Hello Greg!'");
    }
    else
    {
        if (message[message.Length - 1].ToString().Length > 0)
        {
            MessageBox.Show("GetMessage() test succeeded with 'Hello Greg!' and a message.");
        }
        else
        {
            MessageBox.Show("GetMessage() has 'Hello Greg!', but does not have a database message!");
        }
    }
}

That is getting pretty nasty. Now, let’s try to move to automated testing.

Refactoring: The Dual Edged Sword

As we move to automated tests, we have a bit of chicken/egg problem. We need to refactor some of our code so it is testable, but we need tests to be refactor safely. So we need to refactor and test at the same time, but we can’t do both.

The solution is to use a unit test framework to set up integration tests. In this case, we already have windows forms testing (not great, but better than nothing), we cna use the tests and retool them to at least test the first line.

[TestClass()]
public class when_creating_a_hello_message_for_greg
{
    [TestMethod()]
    public void first_line_equals_hello_greg()
    {
        string name = "Greg"; 
        string expected = "Hello Greg!"; 
        string actual = MessageGenerator.GetMessage(name);
        string[] splitActual = actual.Split("rn".ToCharArray());

        Assert.AreEqual(expected, splitActual[0], "First line does not equal 'Hello Greg!'");
    }
}

That works. Note I use a BDD type of naming convention, although BDD purists are probably ready to throttle me. I do this, as I can easily see which methods failed and understand what is failing. The message tells me what went wrong, but I don’t have to see the message to know.

I now need to refactor the test a bit to test a second line.

[TestClass()]
public class when_creating_a_hello_message_for_greg
{
    private static string[] messageSplit;

    private string[] GetMessageSplit()
    {
        if (messageSplit == null)
        {
            string name = "Greg";
            string actual = MessageGenerator.GetMessage(name);

            messageSplit = actual.Split("rn".ToCharArray());
        }

        return messageSplit;
    }

    [TestMethod()]
    public void first_line_equals_hello_greg()
    {
        string expected = "Hello Greg!";
        string[] splitActual = GetMessageSplit();
        Assert.AreEqual(expected, splitActual[0], "First line does not equal 'Hello Greg!'");
    }

    [TestMethod()]
    public void second_line_equals_a_random_quote()
    {
        string[] splitActual = GetMessageSplit();

        Assert.IsTrue(splitActual[splitActual.Length - 1].ToString().Length > 0, "Does not contain a random quote!");
    }
}

So, we have both lines tested, but it is automated. Note that the code is not really unit tested, however, as we are still calling the database. But, since we have a test, even if it does exercise too much code, we can refactor and at least have some semblance of sanity, with tests to show our code does not break.

Inversion of Control

To move to true unit tests, we have to invert control. To do this, I have to do a few things at once. This is a good time to check in the code, including the tests, so we have a point to return to. In Team Foundation Server, we can simply shelf the changes.

  1. Create an interface for the random quote database bits
  2. Invert the GetMessage() routine to take the interface

We are not moving to a unit test yet, as refactoring is ALWAYS done in small steps. But we do have to accomplish this task to get to unit testing. First, here is the interface:

public interface IRandomQuoteRepository
{
    string GetRandomQuote();
}

Now, since our routine is embedded in the same class, we have to move it out and adhere to the interface. It looks like this (yes, it is ugly, but it is merely to illustrate our steps):

public class RandomQuoteRepository : IRandomQuoteRepository
{
    public string ConnectionString { get; private set; }

    public RandomQuoteRepository(string connectionString)
    {
        ConnectionString = connectionString;
    }

    #region IRandomQuoteRepository Members
    public string GetRandomQuote()
    {
        //Bad that this is hardcoded, but showing the proper way to 
        //get max from table == bloated blog entry
        int max = 1000;

        Random random = new Random((int)DateTime.Now.Ticks);
        int id = random.Next(1, tableMax);

        SqlConnection connection = new SqlConnection(ConnectionString);
        SqlCommand command = new SqlCommand("GetRandomQuote");
        command.CommandType = CommandType.StoredProcedure;

        string returnValue;

        try
        {
            connection.Open();
            returnValue = command.ExecuteScalar().ToString();
        }
        finally
        {
            connection.Dispose();
        }

        return returnValue;
    }
    #endregion
}

Our GetMessage then changes like this:

public static string GetMessage(string name)
{
    RandomQuoteRepository repository = new RandomQuoteRepository(ConnectionString);

    StringBuilder builder = new StringBuilder();
    builder.Append("Hello ");
    builder.Append(name);
    builder.Append('!');
    builder.Append("rn");
    builder.Append(repository.GetRandomQuote());

    return builder.ToString();
}

The next step is to test again, to make sure moving the code out works. You have done one refactor, so bozo check your work. Sure, you are still going to the database, but that is the cost. Refactor and test. Refactor and test. Get this rhythm down.

Once you are actually using TDD, you switch to Red … Green … Refactor. So we start with the test:

private string[] GetMessageSplit()
{
    if (messageSplit == null)
    {
        IRandomQuoteRepository repository = new RandomQuoteRepository("");

        string name = "Greg";
        string actual = MessageGenerator.GetMessage(name, repository);

        messageSplit = actual.Split("rn".ToCharArray());
    }

    return messageSplit;
}

No compile. That is close enough to a red. Now, just enough code to make it work. This is a simple change:

public static string GetMessage(string name, IRandomQuoteRepository repository)
{
    StringBuilder builder = new StringBuilder();
    builder.Append("Hello ");
    builder.Append(name);
    builder.Append('!');
    builder.Append("rn");
    builder.Append(repository.GetRandomQuote());

    return builder.ToString();
}

You are now inverted. The “UI” (test project in this case) feeds the repository as an interface. Note that you are still integration testing, as the database is still called.

Mocking so we can Unit Test

I am not getting into mock frameworks at this time, but we will make a very simple mock that always returns “A stitch in time saves nine!”. It has to adhere to the IRandomQuoteRepository interface, so it looks like this:

public class MockRandomQuoteRepository : IRandomQuoteRepository
{

    #region IRandomQuoteRepository Members

    public string GetRandomQuote()
    {
        return "A stitch in time saves nine!";
    }

    #endregion
}

Pretty simple, but watch what we can do. We no longer have to hit the database if we simply change our repository to the mock in our test class:

private string[] GetMessageSplit()
{
    if (messageSplit == null)
    {
        IRandomQuoteRepository repository = new MockRandomQuoteRepository();

        string name = "Greg";
        string actual = MessageGenerator.GetMessage(name, repository);

        messageSplit = actual.Split("rn".ToCharArray());
    }

    return messageSplit;
}

Even more important, since the output is no longer from a database, it need not be random, only act as if it is random. So we can test our conditions and make sure the last line is correct. This is a simple test refactor:

[TestMethod()]
public void second_line_equals_a_stitch_in_time_saves_nine()
{
    string expected = "A stitch in time saves nine!";
    string[] splitActual = GetMessageSplit();

    Assert.AreEqual(expected, splitActual[splitActual.Length - 1] , "Second line is not 'A stitch in time saves nine!'");
}

Now we are unit testing and life is grand. To move to TDD, we would take an additional step and start writing tests before every new feature. There is really no way to refactor to this, but you do have to refactor to unit testing before you can adopt TDD. I will have to leave TDD for another post.

Summary

Here is the sequence of events. They are in this order as you cannot effectively refactor WITHOUT tests. To recode blindly is both dangerous and stupid, just like driving drunk, except, in general, nobody dies. On critical medical systems, however, someone could actually die. You should treat all of your software as if it were a critical medical system. Kapish?

If anyone wants to comment about the basic idea of how to move from forms testing or no testing to unit tests, I would welcome comments. If you want to bitch about my naming convention or the fact the samples are not all best practices, please don’t waste your time. The whole purpose of this particular post is moving to unit testing.

Peace and Grace,
Greg

Twitter: @gbworld

Visual Studio 2010: RTM paid installs over Trial


If you have not checked yet, there are trial versions for Visual Studio 2010 for those who do not have an MSDN disk. If you go this route, you can easily upgrade once you have your Visual Studio install disk from MSDN. Here are the steps:

  1. Insert Disk
  2. Start installer
  3. Select Change or Remove Microsoft Visual Studio 2010
  4. Click next after the setup loads
  5. On the maintenance page click the Activate button that appears on the lower left side
  6. It should check the disk and send in the activation. That is it

This is the easiest way after downloading the MSDN ISO image or using the MSDN DVD (if you are working with an actual disk). You can also enter the key, if you desire, by going to Help >> Register Product and entering the key for Visual Studio. As the maintenance mode is extremely easy, I opted away from this route. 🙂

Peace and Grace,
Greg

Twitter: @gbworld

Visual Studio released. Microsoft reinvents download manager


I just got a chance to go to the MSDN site and see that Visual Studio is the hot topic of the day. This is not surprising, as the launch is happening out at DevConnections in Las Vegas. What is interesting is the various rebranding exercises I see and change of applications.

The first noticeable change is the rearrangement of the MSDN site. I wonder if this is only for the Visual Studio 2010 launch or a long term commitment to move all the left side links (downloads, etc) down the page to make room for more content.

image 

Once I got into MSDN downloads, I noticed some changes. First, I was asked to install a new downloader technology from Akamai. The new downloader appears to work with both ActiveX (default with IE) and Java, so I would imagine it increases your options to download. The new client looks like the screen shot below.

image

Interesting.

Here are the links:

Downloading in a hotel is a royal pain, so I will have these bits downloaded sometime this week. 🙂

Peace and Grace,
Greg

Twitter: @gbworld

Scanxiety


I am sitting in a room in Austin, Texas. My youngest daughter, Miranda, went in for tests today. The doctor has recommended she start going to the survivor clinic and she will only have to see him once a year. It is a good feeling. Yet, deep in the back of my mind I have a small bit of uneasiness as I still don’t have results from the scans. We call this scanxiety.

If it were in Miriam Webster’s Dictionary, it would appear like this:

Main Entry: scanx·i·ety
Pronunciation: skaŋ-ˈzī-ə-tē
Function: noun
Inflected Form(s): plural scanx·i·eties
Etymology: Middle English scannen, from Late Latin scandere, from Latin, to climb; akin to Middle Irish sceinnid he springs, Sanskrit skandati he leaps and Latin anxietas, from anxius

Date: 14th century to circa 1525

1 painful or apprehensive uneasiness of mind in parents over medical scans performed on their children
2 : an abnormal and overwhelming sense of apprehension and fear often marked by physiological signs (as sweating, tension, and increased pulse), by doubt concerning the reality and nature of the outcome of a scan, and by self-doubt about one’s capacity to cope with it

The odds are on my side. We are past the normal median mark for recurrence for Ewing’s Sarcoma patients. We also have a child who had favorable age,  along with placement and “staging” of her cancer (“staging” in quotes, as Ewing’s is not staged like most cancers). Yet there still hangs a small amount of uneasiness at what might come back.

The level of scanxiety is increased in cases with parents of children that have cancers very likely to recur. In some cases, the scanxiety level is so high it overwhelms.

You want to feel a bit of what it feels like. Imagine the picture below is your child. The picture was taken in November of 2007 and is a picture of Julian Avery, who was diagnosed with Medulloblastoma in March of 2007. Cancer finally murdered him on January 19, 2008 and he earned his wings. Forever 4. He just wanted to be 5. The cropping of the picture makes you long to see what is outside of the boundaries, much like a parent waiting for news about whether or not the cancer had come back. Cement the picture in your mind, and imagine it was your own child.

cancer14_024

If that is not enough to get the picture through, examine these two pictures below:

24119395_124518526611 24119395_124518535113

Above is the outcome none of us wish on any parent, not even our worst enemies (although you find you have fewer and fewer enemies after going through this route, as you find the things that used to make you hate are inconsequential). And it is both possible and perhaps very probable with many children. While 75-80% of children with cancer survive in the United States, about 2750 children are housed in stone gardens every year, their parents robbed of their laughter, their cries. About 2750 families lose a child, a brother, a sister. About 2750 wish they had one more day, one more week, one more month, one more year, one lifetime.

And the words each parent fears is “the cancer is back”. Imagine it for a moment and you will understand about 1% of what scanxiety is about.

Peace and Grace,
Greg

Twitter: @gbworld

IT perspectives: Strong indications you are NOT getting it (Agile)


I have had some time lately to think about the state of development in many companies today and decided it would be a time to start talking about some of the clues I see in organization after organization that they are not quite getting it. When I say getting it, I mean they have adopted the correct path, but don’t quite understand the paradigm.

Paradigm shifts are natural parts of innovation. When someone truly looks at anything from an innovative perspective, it almost always requires throwing away old thought patterns and shifting to a new way of looking at the world. In Microsoft development, the shift from COM >> .NET (and especially ASP >> ASP.NET) required a huge paradigm shift in thinking. Unfortunately, it was still possible to hang around your old “loser friends” and not make the shift. And, it has served me nicely, as I have cleaned up a great many ASP.NET apps that were actually ASP written using ASP.NET tools.

In this post, I would like to focus on some key clues that you are not “getting it” in Agile development.

Agile Development

Agile is the buzzword everyone wants to hang their hat on. You see job description after job description touting Agile. But when you get engaged with a company, you find many of them are not really doing Agile, and most are not doing Agile well. They are still working a waterfall methodology with a bit of time boxing, some stand up meetings, and other artifacts of one of the Agile methodologies. When you examine closely, you find very little that is Agile about their implementation of the methodology.

As an example, let’s look at the average corporate entity that adopts Scrum. They will generally bring in some consulting firm to teach them the artifacts of scrum, adopt a scrum template in Team System (assuming Team Foundation Server is their source repository), and start having sprint kickoff meetings, stand up meetings and burn down meetings. They will start talking about chickens and pigs and adopt the nomenclature of Scrum. But, they most often don’t really “get scrum”. Here are a few common clues that they don’t get it.

Developer Interruptions During the Sprint

The idea of time boxing an iteration (sprint) is to keep developers active and on task. The limited time for a sprint (2-3 weeks in most companies) means you can adjust priorities rather quickly, but adjusting in the middle of an iteration is wrong. In the waterfall methodology, it was often necessary to stop a developer or a team and move them to another task in midstream. This was due to the all or nothing nature of the waterfall methodology. As business changed, you often needed to change priorities and since everyone was in a long term stream, you had to dam up the river and get people going down side routes.

With Agile, you are, at most, a few weeks out of changing developer priorities. The only reason you should have to stop an iteration is you find you are building the wrong software. And, if that happens, you failed to plan, which means you planned to fail. It is expensive to interrupt a developer on task.

Once you adopt an Agile methodology, the PM part of the team should not only be managing the plan (through standup meetings), but also keeping ahead of what is going on in the next iteration. This leads us to the next clue an organization is not “getting it” when it comes to Agile.

Meetings that Take Too Long

The goal of a standup meeting is to get a status update and determine future flow. The format is:

  1. What did you get done since the last meeting?
  2. What are you going to do before the next meeting?
  3. Do you have any impediments?

So one developer’s contribution might go like this:

Developer: Yesterday, I got the menu roughed in, but I still have a bit more work to get it done. I am having a strange problem getting the site map to work and some nodes are not showing up.

Scrummaster: John has experience with the site map and some of the problems associated with it. I am going to have him pair with you for a couple of hours this morning and get through that roadblock.

At this rate, you can burn through the entire team in a matter of 5 to 10 minutes on most mornings, with a few extra minutes if you have to figure out who can help on a particular task. You will note that I have taken the assumption the shop does not pair program 100% of the time. I have done this, as I find few shops that have completely bought into the idea of pair programming.

By the same token, the sprint kickoff meetings should not be all day affairs. If your PM guys are one iteration ahead, then the kickoff meetings should be more focused on doling out the tasks than determining what tasks need to be done in an iteration. Certainly some new tasks may have popped up, but if the meeting is an all day affair, you have failed to plan again.

Throwing Code Over the Wall

In many traditional shops, it is quite common to see developers as people who develop … ONLY … and the QA team as the testers. The problem with this approach is you often see code with very bad flaws that should have been caught by dev pitched back over the wall by QA. You end up with this ongoing volleyball match where the code is the ball and the developers and QA people are the players. It is completely non-productive.

From the development side, you should be able to determine that your software is really working before sending to QA. This is done through unit testing and ensuring proper code coverage. With .NET 4.0 and Visual Studio 2010, there is a new tool called Pex (and Moles) that can be used to fill out code coverage with fringe conditions, so we have less and less excuses to not code tests. Add tools like Code contracts, and you not only end up covered, but you run the gamut when it comes to expressing intent of the code. It is fine if you do not practice TDD (some purists may cringe at that statement), but it is not fine to leave all of the testing to QA. Their job is quality, which means catching the conditions a developer might not think of, not testing to see if you actually did your job.

You have to decide whether you want to be a computer artist or a computer scientist. Anyone who codes blind, without proper tests, is practicing a form of computer art. One might even say voodoo art. In order for coding to be science, you have to have proof the code works as expected. Let’s delve into this with something personal for me.

In 2007, my youngest daughter was diagnosed with cancer. At 3 years old, with non-metastatic Ewing’s Sarcoma of the soft tissue, she was in a group where 100% of the participants who had her treatment protocol lived. The problem with taking that as a scientific fact, is the group had only 4 members, which makes it, statistically, insignificant. The larger group, which was statistically significant, had about a 70% survival rate if it was non-metastatic down to < 20% if it had metastasized to the lungs.

If we take this back to software, creating a small app to run the methods, or even adding a single test to significant methods, is better than nothing, but it is statistically insignificant as a measure of code quality. To have a proper method, you need high code coverage, at minimum, and preferably tests that hit fringe conditions. Without these tests, you are blindly asserting code quality … and most likely wasting both your time and that of QA.

Final Words

I would like to say the above points represent the exception rather than the rule, but that is not my experience. More often than not, people DO NOT get it, especially when there is a rather large paradigm shift in thinking (waterfall to Agile, for example). Some may argue that not “getting it” is less damaging than not “moving to it”, but I am not convinced that is always true. Sometimes bad Agile practices (waterfall mentality with an Agile methodology) are far more damaging than sticking to a waterfall methodology.

This is a good start on some of the practices that show an organization is not “getting it”. Other topics we will explore are not getting the paradigm shift from ASP to ASP.NET (or COM to .NET), not getting applications, and the like. I had envisioned getting into one post, but I need to keep this to few enough words that people will actually read it. 🙂

Peace and Grace,
Greg

Twitter: @gbworld

When does Visual Studio 2010 RTM?


This has been an interesting year. I currently have a nice VHD with Windows Server 2008 R2 installed that I dual boot with in Windows 7 (honestly one of the best features in Windows 7 for a betaware guy like me).

In previous release timeframes, we have downloaded the RTM bits long before the public could buy a shrink wrap copy. In fact, the general has been launch in Spring (around March) with a MSDN download in November. This time it could not happen due to performance issue, which forced additional builds prior to RTM. I have been anxiously awaiting the bits.

I wish I could tell you “go to MSDN and download”, but I can’t or that you can get involved in some Microsoft program and download, but I can’t. I wonder if the TAP folks are doing better. I would assume so, but I just don’t know, as I don’t currently have access to customer facing programs.

If you have been in a cave, VS 2010 is set to launch next week at DevConnections. Perhaps this is the reason the release has been held so close to the vest. I imagine there is some deal in place to ensure the bits do not leak prior to the launch so Microsoft can proudly hand out the bits to those who paid for the conference.

The good news is the launch is only a few days away and it is highly likely we can download from MSDN about that time. Until then, I am playing with my beta. 🙂

I felt like stating they were already released, but I think most of you know it is April Fool’s Day. 🙂

Peace and Grace,
Greg

Twitter: @gbworld