The TDD Session at DevLink – Anatomy of a Train Wreck

If you paid attention to #DevLink on Twitter, you already know about the lunch session on TDD. If not, here is a brief synopsis of the events, taken from the eyes of one of the participants.

It is not my goal to point fingers here or to try to get all of the events documented. I, instead, would like to show how a train wreck happens with the hope you can notice the signs and avoid one.

How This All Began

I was not involved in the beginning of the TDD session. It was incubated in an Open Spaces session I was not participating in. I was in a Open Spaces session on SOA, run by James Bender,  and only heard of the TDD lunch session when Alan Stevens joined in at the end of our session.

It is my understanding that Michael Woods was talking to a fellow DevLinker (is it okay to use DevLink in this manner?) about TDD. The idea, at that time, was to have five or six guys around a laptop talking about TDD to show the value. I am not sure how the idea expanded, but I am imagining that Alan Stevens and a few others seeing the value of expanding the idea into a demo. Pinging Alan today, I go the idea that the “incubation” was extremely fast.

By the time Alan came down to photograph and participate in our session, the idea was already a demo in one of the session rooms, with a projected, etc.

The Players

Here is the list of players, in no particular order:

Gregory A. Beamer (me) – I am a proponent of TDD (Test Driven Development) and am actively looking at how BDD (Business Driven Development) fits into testing. I am not a purist on TDD, although I am passionate about tests.

Alan Stevens – Alan is a proponent of TDD, but even a more active proponent of community. Alan was responsible for the Open Spaces sessions at DevLink.

Corey Haines – Corey is a strong proponent of both TDD and BDD. From conversations, it is evident Corey is more of a purist than I am.

Steve Harman – Steve is a guy who, according to others, “goes from 0 to 60 in 2 seconds”. Steve is passionate about TDD and BDD and speaks on a very deep level about things he is passionate about. I would love to attend one of his 300 level sessions some day.

Jim Holmes – This was my first opportunity to meet Jim Holmes. After the “fiasco”, Jim was facilitating the post mortem discussion between Michael, Steve, myself and a few others. Jim also helped referee the session.

Michael Wood – Michael was involved in the original discussion (five or six people and a laptop) and was also involved in some of the facilitation of the post mortem.

Mike Eaton – Mike acted as the primary referee in the session and attempted to keep it under control.

The Session

I walked into the session late, so I did not see the first few minutes. The session used the example of a DevLink type of lunch where X number of people had to be fed. This example was chosen at the session rather than set up before hand.

From the post mortem, it was my understanding that there was no pre-session meeting (in the hall?) so the session just flowed.

There was also a question as to what the session’s goal was. Here are different ideas I heard about the session:

Demo to teach beginners something about TDD

Open session demoing TDD and showing the variety of thoughts people have about TDD

Session illustrating the right way to do TDD for those who do not know TDD

While these goals might seem complementary, the overarching goals for a “true” beginner’s session and a session showing the right way are different, as is the goal for an open session with a demo.

The Train Wreck

The first clue of a train wreck came when the test class was renamed for the second or third time. The first confirmation of a train wreck came when the participants were informed that “#DevLink”Twitter group was abuzz on the session. Some of the nicer comments:

Good idea, poor execution on the #devlink TDD demo. If this is aimed at "new to TDD," there’s WAY too much religion, not enough fundamentals.

Macs don’t support TDD – so I’ve learned at #devlink open spaces – also you flip the bird a lot when doing TDD

If one went down mechanics and mindsets, there was the Mac, which was really less a problem once you got on it than previously thought. There was also the religion: how pure should you worship your *DD. And, there was a lot of minutia, especially surrounding the naming of classes and methods.

The biggest “sin” of the session was we lost the core audience. Although not everyone was aware, the session was touted as a “beginner’s TDD” session. Talking to Sarah Dutkiewicz after the session, I found out she walked completely confused prior to my walking in.

Lessons Learned

The first lesson we can learn his is you have to have a goal in mind. The goal has to be communicated amongst the participants. If we would have had a “quick two minute meeting in the hall”, prior to the session, the wreck could have been avoided.

Second, you should keep the number of people in list of participants down to a reasonable number. And, while you might want various points of view in the mix, make sure that all are willing to focus on the same goal. I am not sure this would have been a problem with anyone in this group. While we do have different views on TDD, everyone in the group is a consummate professional and willing to tone down a bit for a common goal.

Third, while it is nice to make up scenarios on the fly, the tried and true examples are often the best place to start. As Steve suggested after the session, the typical “canonical” bank transaction scenario would have been a better place to start, as every person in the room would have recognized that scenario and the participants would have been familiar with it.

Fourth. Even if it is an open session, there has to be a “boss” or a “referee”. This is the person who owns the show and is responsible for keeping control of the session and getting the players back on track. The players have to walk in agreeing that the referee has the power to pull things back into place.

Summary

It would be easy, as some tweets have suggested, defining this train wreck by the personalities involved. But this would not do justice to the reality that even the most consummate professional can get off track. It is my hope that

Meshing Out – What I took home from DevLink

Before getting into this entry too deep, I wanted to congratulate John Keller and crew for another successful event. I also wanted to congratulate Alan Stevens and crew for the wonderful Open Spaces sessions. I did not attend but one regular session, but the Open Spaces more than made up for missing the regular sessions.

 

I ended up trying to sign on a bit too late, so I had to hork a pass from someone who was not attending on day 2. I will remember to get my pass earlier next year and avoid the long lines. 🙂 Thanks to Rebel Bailey for lending me his pass on Saturday.

 

Mesh is Cool

I attended Jeff Blankenburg’s impromptu session on new stuff from Microsoft. Although PhotSynth, Sea Dragan and Popfly are kewl, I have seen them enough times to get very little out of that part of the session. Then Jeff showed Live Mesh. Here is somethign that immediately fits my needs.

For my work, I am often having to sync up more than one device. Now I have 5 GB on the web to sync up with. I already have my current project shared as a folder and I am currently downloading it to my laptop. In addition, I love the fact I can control my home machine from my laptop without having to load a lot of extraneous software. And only people in my mesh can do this, and only those with a proper logon. Very cool.

Where I see this going … anyone at Microsoft listening … is merging in Live Meeting so I can use the Mesh to immediately go to a sharing session or share, from a meeting, to my mesh. Company meshes. Group meshes. People will pay for this stuff. Trust me.

Microsoft is already hinting at an SDK for Mesh so I can create apps for it, so the future is nearly here.

Impromptu can mean a train wreck

The TDD session crashed. It was not a total disaster, but it needs to be done better next time. I have an entry I have completed on this, but I am awaiting a review of the article from Alan Stevens before putting it onto this blog. My goal on this one is dissecting the train wreck and providing a constructive post mortem. The last thing I want is to inadvertently tick someone off.

Open Spaces are fun and productive

I did not think an open sesssion would be as useful as it was. I spent most of my time in these sessions, firing thoughts off some of the best in the industry … and loved it. I will gladly do this again.

Final Thoughts

DevLink is an inexpensive alterative to the big name conferences. It is shaping up to be a premiere conference, despite the low price. Hats off to the guys who put this together and make it purr. You guys really deserve a big hand. Next time can we holder it closer to downtown again? 😉

Peace and Grace,
Greg

Stupid SQL Server 2008 Tricks

THe title is just a nod to David Letterman with his Stupid Pet Tricks or Stupid Human Tricks segments. It is probably not the best title, as will be revelead. A better title:

 

SQL Server 2008 Management Studio renders SQL Server 2005 Management Studio useless when working with projects

 

But "Stupid SQL Server 2008 Tricks" sure fits on the title line much better.

 

Since I have already released the gist of the "problem", I will explain the symptoms.

 

Reproing the Problem

Yesterday, I created a new SQL Server project in SQL Server 2005 manager. After I was finished, the solution explorer showed nothing. Zip, zilch, nada. I then recreated the project, with the same name and got an error.

 

 

If you want to get the same error, the repro steps are very easy:

 

1. Install SQL Server 2008 as a stand alone (not an upgrade)

2. Open SQL Server 2005 Management Studio

3. Create a project called Test and allow it to create the solution (default)

4. When it appears to do nothing, create a project called Test and allow it to create a solution (default)

 

You can alter steps 3 and 4 and not create the solution folder. As long as the setting is teh same the second time, you will get the error.

 

I then looked in Windows Explorer and yes, there was a project there (Test.ssmssqlproj), but there was no solution file. Strange. So, I tried to open the project file. Still nothing in SQL Management studio.

 

But, then I had an epiphany. Try opening the project in SQL Server 2008 Management Studio. Yes, it works.

 

Incompatibility of Tools

What this boils down to is there are some incompatibility with the underlying assemblies/DLLs that SQL Management studio uses to work its magic. In addition to this incompatibility, I have found a few more.

When working with a SQL Server 2005 instance, in SQL 2008 Management Studio, you cannot alter the definition of a table. Attempting to alter a SQL 2005 table in SQL 2008 Management Studio, even one you created with SQL 2008 Management Studio, fails with this largely useless error:

Now, looking at this, you might think. Yeah, the error box seems useless, but it allows you to save it as a text file. That sounds like it might be a cool feature, if you are thinking, as I did, that the text file is a drop and recreate script for the table. If you were thinking that, you and I were both wrong. Here is what the text file contains:

/*
   Friday, August 22, 20089:17:05 AM
   User:
   Server: BNA-GBEAMER2
   Database: UnitWarehouse
   Application:
*/

SimHistory

Now if you first thought is "that is a completely useless text file", you are I are tracking. If I were doing a lot of work and saved this file off and then came back a few weeks later to look at the file (unlikely, but let’s play along), would it mean ANYTHING to me? I can picture this "Greg, what the *#&% were you doing on August 22 at 9:17 AM with the SimHistory table?" (I only know this because I know my schema). Worse, imagine someone else looking at this text file. It might make sense if the file contained something like this:

/*
   Friday, August 22, 20089:17:05 AM
   User:
   Server: BNA-GBEAMER2
   Database: UnitWarehouse
   Application: 
   Object: SimHistory
*/

Attempt to alter object in Management Studio failed as saving changes to SQL Server 2005 objects is not permitted.

While saving off this file is still useless to me, it at least logs something.

What This Means

To me, the interesting point of this finding is it means I have three choices with SQL Server 2008.

1. Switch all of my databases to 2008 and only use SQL Server 2008 tools
2. Get rid of SQL 2008 completely until I am ready for it
3. Use SQL 2008 MS for my projects and SQL 2005 MS for my visual schema work

Number 1 is not an option until I know SQL 2008 databases can be attached on SQL 2005, as I have SQL 2005 database servers. I would suspect there is some incompatibility here due to the error altering schema. I could be wrong, as this just could be a tool/database impedence match. I do not have the time to investigate, so I will be pessimistic and safe.

Number 2 is a risky option now that I have a SQL 2008 instance installed, as I may end up with a hefty reinstall for 2005 to fix the tools. I cannot afford this time either, although not installing 2008 at all might have been a good option at one time. Unfortunately, I have to investigate 2008 for my job.

Number 3 is the sanest option right now, although it does require keeping both open. As long as I have 2008 open with a solution, it is easy to tell them apart. Since I am only working with it for projects, this is not a horrible ‘trade off’.

Peace and Grace,
Greg

Microsoft Heartland Influencer’s Summit

I ended up going to the Heartland Influencer’s Summit today. My main reason for going was to talk to Brian Prince, the Heartland Architect Evangelist, which I accomplished later. It was a productive talk. I left with a bit more of an open mind about a few things (nothing related directly to the Summit) and a thought about User Interface.

 

Much of the Summit was under a "gentleman’s NDA", so I can’t go into details about all of hte specifics, but that is not he purpose of this entry.

Paradigms 

The first thing of note is how extensive paradigms are. I have, more than once, caught myself in paradigms in thinking. In general, I find I am more susceptible to getting caught this way in things I find appealing. If I have an affinity towards something, I am less likely to challenge my thought process.

 

Let me get a bit more specific, with no names, as they are not important. In the Summit, they gave away desk light sabers as gifts. Here is a picture of mine sitting on my desk (yes, that is a Michelob Porter in the background – best option in the "cheaper" beers).

 

 

Alan Stevens and I both assembled ours (typical geeks) and a conversation about Star Wars ensued. It was generally agreed that episodes 1, 2 and 3 sucked and that midichlorians were stupid. Then the comment burst forth that there were "certain universal myths that were not exclusive to Christianity" (paraphrased, not a true quote). While I understand the nature in which the comment was made, the bulk of the evidence of the virgin birth/savior being a Universal myth comes from a single source about the Mithras cult which is largely conjecture on the part of the author. It is very possible, if not likely, that these stories do have a Universal nature, but the statement was stated with such force it revealed the speakers own paradigm.

 

There is nothing wrong with being in a paradigm. The comment just struck me as ironic after our previous conversations about how, using higher level constructs, one did not get caught on syntax. I am not mentioning names as I might have been stuck in my own paradigm at the moment and there is no reason to start a peeing contest over a statement.

 

What is important is many developers are stuck in paradigms without realizing it, as they start to see syntax as a programming construct and not an artifact of language. I believe both the author of the statement and myself agree that there are paradigms in programming, but it is interesting how easy it is to fail to realize we are in one even after we have advanced. It is another thing I have to keep myself in check over as we move forward.

Bad UI

Now this second one may come across as strange as it is in no way related to programming, except in abstract (abstractions are good). Look at this picture:

 

And here is an "enhanced" shot to illustrate the "bad UI".

First, notice three things. In red, there is a motion sensor. And, in green, there is a short distance between the pipes in the curve of the faucet (green). I should have taken a picture from the side, but the water pours about 2 inches away from the back of the sink. In blue, note the water.

 

There are a couple of things wrong. First, the motion sensor points straight out. This causes you to have to hold your hands up above the bowl to use it. Added to this is the fact that the water pours so close to the back of the sink. Combined, this causes a lot of water to pour out the back of the sink and eventually end up at the front, where it can deposit on the pants of anyone who stands close to the sink.

 

In the process of discussing UI (we are talking about a bunch of geeks talking about things "in nature"), we did determine a possible advantage to this design. Anyone who pees horizontally on his pants can blame it on the sink.

 

Fish bowl

One positive take away I got from this Summit was the fish bowl. This, to me, is almost as neat as sliced bread. Okay, old and tried analogy. As neat as scrum meetings done properly. Another geek analogy. ARRRRGGGHHHH!!!

The fishbowl is designed to facilitate a bunch of people on a topic without becomming a free for all. You set up six chairs in the middle of the room. One is for the facilitator, who makes sure the conversation remains on topic and the other five are for participants. One chair must always remain empty. This is important, as it always leaves space for others to enter the conversation. When a person enters, one of the current people must leave. The last rule is nobody outside the inner circle can talk (like chickens at a scrum meeting?). I was suprised at how well this flowed. We got through some topics that might otherwise degrade without any conflict or too many tangents. This will become another repertoire item for me. Very cool.

Well, that is it for now.

Peace and Grace,
Greg

VB to C# Conversion programs (altered review)

On August 8, I compared a demo of Instant C# (all I have) versus VB Conversions. It was a review prompted by the needs of a friend. Once I found some conversion issues with Instant C#, I posted a review despite it being a demo. One of the employees of the company added a comment and I have since added to the review. The review has been updated at the original URL.

 

Dave, the employee mentioned, has indicated there are some updates for Instant C# forthcoming, so I am willing to review the changed procuct, along with any VB Conversions changes, once they are ready.

 

This is actually a bit of a waste of my time, as I have no need of either product. I am willing to spend the time, however, in the interest of both fairness and providing you, the reader, with the most current information. I am opinionated, but I try to be as fair as I possibly can and I truly wish both parties the best of luck should the opportunity arise to review both again.

 

Peace and Grace,

Greg

The Politics of Poverty

I saw a commentary by Glenn Beck on the CNN website today about poverty. Unfortunately his commentary rails on Democrats in city hall, where I find that the bigger story is the idea that hand-outs do not help most people get out of poverty. While this might be considered a hallmark of Democratic policy, I feel the fact that these mayors are pushing hand out type programs is more important than their party. I imagine this was Beck’s intent, but it gets lost in the rail.

Poverty and Change

In life, the biggest changes are most often precipitated by some form of challenge. I am sure there are exceptions to this rule, but looking over my life, as well as lives of others I know, it is the moments of extreme challenge that have brought about the most extreme change. If there were programs to focus people on positive outcomes through their hardships rather than programs to relieve their pain by paying them, I believe we would see a much more radical change in the poverty level.

I have recently spent some time with the Total Transformation Program. If you are not familiar, this program is heavily advertised on the radio here in Nashville. The program, created by Child Behavioral Specialist, James Lehman, shows parents how to alter their child’s behavior in positive ways by changing the way we respond to acting out. One of the primary realizations I have through the program is my children’s acting out is due to their inability to cope because they do not have proper problem solving skills. One of the things I do that exacerbates the situation is picking up after them.

Now, these sound like very different beasts, but there are some very astounding similarities, when you look underneath the hood. When I pick up my daughters’ things, I am perpetuating the myth they are incapable of solving the problem and making it a reality. I relieve some of their pain (by removing their need to clean up) and I also remove the challenge of having to solve the problem. In like manner, when our primary focus is on paying away the pain of the poor, we remove the challenge of having to solve the problem.

Teaching Men to Fish

The following has become a mantra in Conservative circles:

Give a man a fish, feed him for a day
Teach a man a fish, feed him for a lifetime

On the surface, this is the key to poverty and change. We have to get people learning to fish if we want them to rise from the ashes of their former life. But, this is an oversimplification, as starving men do not learn. We have to both give men fish and teach them to fish to solve the problem. This means there are certain types of “handouts” that need to exist.

The end run here is both Democrats and Republicans, or if you like, Liberals and Conservatives are right. One might have a slightly higher rate of success with their program alone, but I am sure we will find that neither type of program by itself is as effective as both together.

Peace and Grace,
Greg

Refactoring the Disconnected LINQ to SQL Repository

I am bit blog happy tonight, but I am on a roll. In this post, I want to cover refactoring. To do this, I will be refactoring the solution I created in the last few posts (Repository Pattern in LINQ to SQL).  In our last post, I had the following routine to get the Primary Key for a LINQ to SQL object.

        private string GetPrimaryKeyName(T entity)

        {

            Type type = entity.GetType();

            foreach (PropertyInfo prop in type.GetProperties())

            {

                object[] attributes = prop.GetCustomAttributes(true);

 

                foreach (object o in attributes)

                {

                    ColumnAttribute attribute = (ColumnAttribute)o;

                    string name = o.ToString();

 

                    if (attribute.IsPrimaryKey)

                        return prop.Name;

                }

            }

 

            return null;

        }

To me, this is a lot of work for finding a primary key column. There has to be an easier way. But, before I can refactor, I need a test. Actually a suite of tests, as I want nothing to change, but here are a couple of tests that call the save method, which requires the name of a primary key.

[TestMethod()]

public void SaveNewSimStatusType()

{

    IDataContextFactory dataContextFactory = new SimStatusTypeDataContextFactory();

    IRepository<SimStatusType> target = new SqlRepository<SimStatusType>(dataContextFactory);

    string expectedName = "This is a test";

    int expectedValue = _simStatusTypeId + 1;

 

    SimStatusType entity = new SimStatusType();

    entity.SimStatusTypeId = 0;

    entity.SimStatusTypeName = expectedName;

 

    SimStatusType actual = target.Save(entity);

    isDirty = true;

 

    Assert.AreEqual(expectedName, actual.SimStatusTypeName, "name is different");

}

And

/// <summary>

///A test for Insert

///</summary>

[TestMethod()]

public void UpdateASimStatusType()

{

    IDataContextFactory dataContextFactory = new SimStatusTypeDataContextFactory();

    IRepository<SimStatusType> target = new SqlRepository<SimStatusType>(dataContextFactory);

    string expectedName = "This is a test";

    string changedName = "This is a second test";

    int simStatusTypeId;

 

    //Create entity

    SimStatusType entity = new SimStatusType();

    entity.SimStatusTypeId = 0;

    entity.SimStatusTypeName = expectedName;

    SimStatusType actual = target.Insert(entity);

    simStatusTypeId = actual.SimStatusTypeId;

 

    isDirty = true;

 

    Assert.AreEqual(expectedName, actual.SimStatusTypeName, "name is different");

 

    actual.SimStatusTypeName = changedName;

    target.Save(actual);

 

    Assert.AreEqual(changedName, actual.SimStatusTypeName, "name is different");

}

Now, I really need to refactor these tests a bit, as the warning messages tell me little and the names of the routines are not quite explicit enough for me, but that is another blog entry. Feel free to critique the tests, if you must, as I am more than happy to learn things.

When I run these routines, they come up green, which means the test is passing. I then look into how one might pull a primary key from a LINQ to SQL object. For my first refactor, I find that you can get the primary key off of the DataContext mapping. So, I change my GetPrimaryKeyName() routine to this:

private string GetPrimaryKeyName(T entity)

{

    using (System.Data.Linq.DataContext context = _dataContextFactory.Context)

    {

        MetaTable table = context.Mapping.GetTable(typeof(T));

        return table.RowType.IdentityMembers[0].Name;

    }

}

That is huge reduction in the number of lines necessary. Run tests. Still green. Okay, so I lucked out this time. But, I am now wrapping a context into a context, as the call has the context on it. Here is the save method, as it stands now.

public T Save(T entity)
{
    using (System.Data.Linq.DataContext context = _dataContextFactory.Context)
    {
        T databaseEntity;
        string idName = GetPrimaryKeyName(entity);
        int id = (int)GetIdValue(entity, idName);

        //Then
        if(0 == id)
        {
            //This is a save
            return Insert(entity);
        }
        else
        {
            databaseEntity = context.GetTable<T>().First(s => s == entity);
            LoadDatabaseEntity(entity, ref databaseEntity, idName);
        }

        context.SubmitChanges();
        return databaseEntity;
    }
}

This one just screams refactor, as I only need the context for the extent of the routine. The first is to call the GetPrimaryKey from outside of the context call. The other is to move the metadata pull into the save routine. Of the two, I am opting for the later until I see need for this elsewhere. While I do see a possibility, I am going to follow the rules of refactoring and only refactor as needed. Thus, a separate routine is overkill. My save routine gets change to this:

public T Save(T entity)
{
    using (System.Data.Linq.DataContext context = _dataContextFactory.Context)
    {
        T databaseEntity;
        string idName = _dataContextFactory.Context.Mapping.GetTable(typeof(T))
                        .RowType.IdentityMembers[0].Name;
        //string idName = GetPrimaryKeyName(entity);
        int id = (int)GetIdValue(entity, idName);

        //Then
        if(0 == id)
        {
            //This is a save
            context.GetTable<T>().InsertOnSubmit(entity);
            context.SubmitChanges();
            return entity;
        }
        else
        {
            databaseEntity = context.GetTable<T>().First(s => s == entity);
            LoadDatabaseEntity(entity, ref databaseEntity, idName);
        }

        context.SubmitChanges();
        return databaseEntity;
    }
}

I have now deleted the GetPrimaryKeyName() routine. Now, you might say, "but you said you would likely resurrect this code. Isn’t it better to just comment it out?" I am a bit torn on this one, but I am inclined to say no, as I can use the refactoring tools in either Visual Studio or Resharper (great product if you do not have it) to extract this method later. While this is not necessary in this routine (1 line), if I had multiple lines it would be prudent to delete now and extract later, as I would likely find some improvements in the code. I then end up copying and pasting changes, which is not a good idea when you have refactoring tools.

Just remember, the basic rule here is red … green … refactor. You then aim for green again. In my case, the change was so simple it did not break everything, but I have had cases where a refactor broke code. This is why tests are mandatory. If you are not using unit tests you should ask yourself "why am I not using them?"

Peace and Grace,
Greg

Improving the Disconnected Repository using LINQ to SQL

I have worked through the problem a bit further. It is still a bit nasty with the generics, but it works better. And I can now use any type of primary key. If you need to get an understanding of the entire process, look at my last entry.

In order to figure out the primary key for a LINQ to SQL object, you have to query properties of the object through reflection. Here is a generic method that will return the primary key. Note that all of my tables contain primary keys, so you have to work with nulls if you do not. I am also only using integers for my primary keys, so I optimistically assume an integer for the object. I still end up with one unbox in the mix, but it works fairly well.

To get the primary key, you first get the type. You then get every PropertyInfo on the type and finally check each attribute on each property. It goes like so:

private string GetPrimaryKeyName(T entity)
{
    Type type = entity.GetType();
    foreach(PropertyInfo prop in type.GetProperties())
    {
        object[] attributes = prop.GetCustomAttributes(true);

        foreach(object o in attributes)
        {
            ColumnAttribute attribute = (ColumnAttribute) o;
            string name = o.ToString();

            if (attribute.IsPrimaryKey)
                return prop.Name;
        }
    }

    return null;
}

The null is thrown in to make the routine compile. I could have done the same by doing the following:

private string GetPrimaryKeyName(T entity)
{
    Type type = entity.GetType();
    string returnVal = null;

    foreach (PropertyInfo prop in type.GetProperties())
    {
        object[] attributes = prop.GetCustomAttributes(true);

        foreach (object o in attributes)
        {
            ColumnAttribute attribute = (ColumnAttribute)o;
            string name = o.ToString();

            if (attribute.IsPrimaryKey)
            {
                returnVal = prop.Name;
                break;
            }
        }
    }

    return returnVal;}

I am not as fond of this syntax, but it will accomplish the same thing. As long as you have a primary key on the table, the LINQ to SQL object will have the following type of attribute:

[Column(Storage="_SimId", AutoSync=AutoSync.OnInsert, DbType="Int NOT NULL IDENTITY", IsPrimaryKey=true, IsDbGenerated=true)]

I can then get the value of the primary key, as an object, using the following routine.

private object GetIdValue(T entity, string idName)
{
    Type type = entity.GetType();
    PropertyInfo pi = type.GetProperty(idName);
    MethodInfo mi = pi.GetGetMethod();
    return mi.Invoke(entity, null);
}

To invoke this, I use the name I have pulled from the GetPrimaryKeyName() routine. This reduces down my main routine from this:

Type t1 = entity.GetType();
//This is a dangerous assumption
string idName = t1.Name + "Id";
PropertyInfo pi = t1.GetProperty(idName);
MethodInfo mi = pi.GetGetMethod();
object id = mi.Invoke(entity, null);

To this:

string idName = GetPrimaryKeyName(entity);
int id = (int)GetIdValue(entity, idName);

 

I find that much cleaner and more likely to show my intent. In addition, the primary key is no longer constrained to {TableName} + "Id". This is much cleaner. I can now offload the loading of the database entity object, as well. Here is my routine.

private void LoadDatabaseEntity(T entity, ref T databaseEntity, string idName)
{
    Type t1 = entity.GetType();
    Type t2 = databaseEntity.GetType();

    foreach (PropertyInfo prop in t1.GetProperties())
    {
        bool containsName = prop.Name.Contains(t1.Name);
        bool containsId = prop.Name.ToLower().Contains("id");

        if (prop.Name.ToLower() != idName)
        {
            PropertyInfo prop2 = t2.GetProperty(prop.Name);

            //How to get property value
            MethodInfo m1 = prop.GetGetMethod();
            object o = m1.Invoke(entity, null);

            MethodInfo m2 = prop2.GetSetMethod();
            m2.Invoke(databaseEntity, new object[] { o });
        }
    }
}

There is still one thing I would like to refactor and that is sending in the databaseEntity object by reference. It feels a lot like cheating, but with the way the LINQ context works, I will have to play around with it a bit prior to moving away from by reference. I am still loading each property. If I were building this completely from scratch, I might look into a way to flag each field as dirty to avoid setting each field, but this is not extremely dangerous as long as I do not do anything stupid prior to attempting to save an object. There is that danger, of course, and the generic implementation makes it difficult to add safety for everything (like violating the foreign key constraint). I am willing to take this risk, as I have the need to expose these objects to another language. I will check input when someone uses the web service to avoid danger. A bit more weight on the web service call, but I am not writing for Google scale here.

Here is the entire Save method, with the calls to the methods above:

public T Save(T entity)
{
    using (System.Data.Linq.DataContext context = _dataContextFactory.Context)
    {
        T databaseEntity;
        string idName = GetPrimaryKeyName(entity);
        int id = (int)GetIdValue(entity, idName);

        //Then
        if(0 == id)
        {
            //This is a save
            return Insert(entity);
        }
        else
        {
            databaseEntity = context.GetTable<T>().First(s => s == entity);
            LoadDatabaseEntity(entity, ref databaseEntity, idName);
        }

        context.SubmitChanges();
        return databaseEntity;
    }
}

There are a few tradeoffs here, since the implementation is generic. For those who don’t like searching the properties through reflection, you can create your own specific repository for the Save() method. I am still leaving the Insert() method separate, for times when I know I am just inserting, but I could just as easily refactor to this:

                if(0 == id)
                {
                    //This is a new object
                    context.GetTable<T>().InsertOnSubmit(entity);
                    context.SubmitChanges();
                    return entity;
                }

I am going to think about this, as I am not sure Insert() by itself serves any purpose. Since I have already gotten a query about showing my entire class, I will say that this request is forthcoming. I want to make sure I have the best way of doing the things I am doing here. Here are a couple of things I am going to look at tonight:

More efficient way of returning the primary key

Some way to properly handle all primary key types

Peace and Grace,
Greg

Repository Pattern in LINQ to SQL (Disconnected)

I have been working on a LINQ to SQL data access layer. This kind of goes against my grain, as I am not overly thrilled about LINQ to SQL as a DAL, but it solves an immediate problem. First, let’s explain the parameters that are important to this DAL.

1. DAL must be generic or code generated. Do not desire to create a DAL for each type of object, only exceptions
2. Objects have to be generated without a lot of work (code gen or drag and drop). This is just a time measure.
3. Objects have to be expressed simply so they can be duplicated in other languages (Java in this case),
4. Objects have to be served up via web services.

There are more, but these three will help weed things out. Here are some things I have tried.

• SubSonic and SubStage – SubStage generates the code properly, both for the DAL and the objects. It is okay on point 3, if it would properly run on a web service. Unfortunately, the way null fields are expressed cause a failure.
• ADO.NET Entity Framework – This is fine on all requirements, except 3. The Entity model uses a strange construct (from a web service standpoint) for joins. Instead of exposing the foreign key field, it creates an object. This is great for multi-field keys, but fails miserably for keeping it simple for those using another language.
•  DataSets – Fails on requirement #1 at present. I am sure I could work up a generic implementation, but it would take a bit too much time to figure it out. There are examples with LINQ to SQL to work from (mentioned later).
• Objects loaded with DataSets. A bit easier to genericize, but I have to either write a code gen for the objects or it fails requirement #2.
• LINQ to SQL with all tables in a single LINQ to SQL DBML. This also fails on point 3, as it adds properties for the join. It is possible the developers in Java could avoid these, however.

I have now opted for a single DBML per table in the database. In many ways, this is like the Active Record pattern, although I am using a Repository (a "well duh" for those who have read the title of this blog entry) to serve up the objects. This is primary for testability.

The Repository Pattern

For those not familiar, the Repository Pattern is designed to wrap a data source in a common interface, allowing you to do your CRUD against a very simple model. In general, you will create a single interface and multiple repositories, one for each class type. With generics, however, you have the option to create a generic repository and only create specific repositories where absolutely needed.

Here is an interface for my Repository.

public interface IRepository<T> where T : class
{
    List<T> All();
    List<T> FindAll(Func<T, bool> exp);
    List<T> Where(Func<T, bool> exp);

    T Single(Func<T, bool> exp);
    T First(Func<T, bool> exp);
    T Last(Func<T, bool> exp);
    T Save(T entity);

    T Insert(T entity);
    void Destroy(T entity);
    void Delete(T entity);
    void UnDelete(T entity);

    void MarkForDeletion(T entity);
    T CreateInstance();

    /// <summary>Persist the data context.</summary>
    void SaveAll();
}
There are a few constructs here that are purely LINQ to SQL, but you can see the basics of how an interface can be used across multiple types of entities. If you get rid of the Func<T, bool> portions  and change your query methodology, you can use the same type of repository pattern for other types of objects.

Repository Pattern with LINQ to SQL (one DBML per table)

First, let me explain the one DBML per table. This simple means I have created a DBML for each table and then dragged only that table on the surface. It would be much easier to create a single DBML for the database, or at least one per set of functionality, but that adds complexity to the objects that would have to be expressed in other languages, even though they would never be used in the other language. Thus, I created one DBML for each table.

 From Code Project, I found Liam McClennan’s entry on how to create a Repository in LINQ to SQL. Here is his repository.

public class Repository<T> : IRepository<T>
  where T : class
{
  protected IDataContextFactory _dataContextFactory;

  public IEnumerable<T> All()
  {
    return GetTable;
  }

  public IEnumerable<T> FindAll(Func<T, bool> exp)
  {
    return GetTable.Where<T>(exp);
  }

  public T Single(Func<T, bool> exp)
  {
    return GetTable.Single(exp);
  }

  public T First(Func<T, bool> exp)
  {
    return GetTable.First(exp);
  }

  public virtual void MarkForDeletion(T entity)
  {
    _dataContextFactory.Context.GetTable<T>().DeleteOnSubmit(entity);
  }

  public virtual T CreateInstance()
  {
    T entity = Activator.CreateInstance<T>();
    GetTable.InsertOnSubmit(entity);
    return entity;
  }
  public void SaveAll()
  {
    _dataContextFactory.SaveAll();
  }

  public Repository(IDataContextFactory dataContextFactory)
  {
    _dataContextFactory = dataContextFactory;
  }

  #region Properties

  private string PrimaryKeyName
  {
    get { return TableMetadata.RowType.IdentityMembers[0].Name; }
  }

  private System.Data.Linq.Table<T> GetTable
  {
    get { return _dataContextFactory.Context.GetTable<T>(); }
  }

  private System.Data.Linq.Mapping.MetaTable TableMetadata
  {
    get { return _dataContextFactory.Context.Mapping.GetTable(typeof(T)); }
  }

  private System.Data.Linq.Mapping.MetaType ClassMetadata
  {
    get { return _dataContextFactory.Context.Mapping.GetMetaType(typeof(T)); }
  }

  #endregion
}

The good here is this Repository contains a lot of methods I need. The bad thing is this type of Repository has to keep an open context. It would be fine for those of you who want to use a Repository on a windows application, but this fails miserably when we need a disconnected type of Repository, like I need for a web service and you might want for a web site.

I found the answer for this issue from Andrew Siemer’s blog entry.

I simply change this routine:

public IEnumerable<T> All()
{
    return context.GetTable<T>().ToList();
}

For this:

public List<T> All()
{
    using (System.Data.Linq.DataContext context = _dataContextFactory.Context)
    {
        return context.GetTable<T>().ToList();
    }
}

Now, it works disconnected. No problems.

There is one thing I have found that does not work. Andrew uses the following to save an object.

public void SaveRespondent(Respondent respondent)
{
    using (ProjectDataContext dc = conn.GetContext())
    {
        if (respondent.RespondentID > 0)
        {
            dc.Respondents.Attach(respondent, true);
        }
        else
        {
            dc.Respondents.InsertOnSubmit(respondent);
        }
        dc.SubmitChanges();
    }
}

If you notice, the Repository here is not generic. I also found that the Attach method does not work when using generics. Here is the error.

An entity can only be attached as modified without original state if it declares a version member or does not have an update check policy

I am sure this (error on Attach) is why Liam put the SaveAll() method on the DataContext factory instead of on the generic repository, as shown here:

  public void SaveAll()
  {
    _dataContextFactory.SaveAll();
  }

I don’t like this at all, as a factory is not a repository. It breaks the pattern. This gives me two options. The first is to create a repository per object. This adds a lot of maintenance for me. The second is to figure out how to genericize the Save method.

Generic Save

My first run was to create an Insert method, as I got the test that ran an Insert working fairly easily. In Liam’s version, he creates an object off context, so it is seen as a new object when saved back.  Like so:

  public virtual T CreateInstance()
  {
    T entity = Activator.CreateInstance<T>();
    GetTable.InsertOnSubmit(entity);
    return entity;
  }

I am creating my objects out of context, so this will not work. My version is more like this:

public T Insert(T entity)
{
    using (System.Data.Linq.DataContext context = _dataContextFactory.Context)
    {
        //Need something to update instead
        context.GetTable<T>().InsertOnSubmit(entity);
        context.SubmitChanges();

        return entity;
    }
}

This works great for the new entity. It will not work for an update. I then looked back at Andrew’s version, which uses an Attach() and I end up with an error that I have to timestamp the object or set something in the LINQ object. I will have to look at this direction, as my generic implementation looks a bit ugly. As mentioned above, the error message is:

An entity can only be attached as modified without original state if it declares a version member or does not have an update check policy.

Here is my solution, at least until I can refactor it down. First, I get find the ID column of the table. In order to do this, I am seeking for a column with the name of {TableName} + "Id". This works in my database, as the primary key for Sim is SimId, for SimStatusType is SImStatusTypeId. It will not work for all tables, so my next attempt will be to seek for the Column attribute on the LINQ class and look for IsPrimaryKey = true. Here is how I find the Id column:

//Get types
Type t1 = entity.GetType();

//First find the entity ID
//NOTE: This will only work with primary key = TableName + "Id" (case sensitive)
string idName = t1.Name + "Id";
PropertyInfo pi = t1.GetProperty(idName);
MethodInfo mi = pi.GetGetMethod();
object id = mi.Invoke(entity, null);

I now have the value for the ID I am changing. To get this to update, I update every field on a database version of the entity, except the primary key.

databaseEntity = context.GetTable<T>().First(s => s == entity);

 

Type t2 = databaseEntity.GetType();

foreach (PropertyInfo prop in t1.GetProperties())
{
    if (prop.Name.ToLower() != idName)
    {
        PropertyInfo prop2 = t2.GetProperty(prop.Name);

        //How to get property value
        MethodInfo m1 = prop.GetGetMethod();
        object o = m1.Invoke(entity, null);

        MethodInfo m2 = prop2.GetSetMethod();
        m2.Invoke(databaseEntity, new object[] { o });
    }
}

To make the Update() method a Save() method, I simply query whether the id is 0 or not. This is a bit kludgy, as I can have values in my database that are 0. Unfortunately, I can think of no way around this at present. Since 0 is only found in Type tables and always indicates UNKNOWN, which will never change, there is not too much danger here. Here is my complete Save routine.

public T Save(T entity)
{
  using (System.Data.Linq.DataContext context = _dataContextFactory.Context)
  {
    T databaseEntity;
    //Get types
    Type t1 = entity.GetType();

    //First find the entity ID
    //NOTE: This will only work with primary key = TableName + "Id" (case sensitive)
    string idName = t1.Name + "Id";
    PropertyInfo pi = t1.GetProperty(idName);
    MethodInfo mi = pi.GetGetMethod();
    object id = mi.Invoke(entity, null);

    //Then
    if(0 == (int)id)
    {
      //This is a save
      return Insert(entity);
    }
    else
    {
      databaseEntity = context.GetTable<T>().First(s => s == entity);

      Type t2 = databaseEntity.GetType();

      foreach (PropertyInfo prop in t1.GetProperties())
      {
        bool containsName = prop.Name.Contains(t1.Name);
        bool containsId = prop.Name.ToLower().Contains("id");

        if (prop.Name.ToLower() != idName)
        {
          PropertyInfo prop2 = t2.GetProperty(prop.Name);

          //How to get property value
          MethodInfo m1 = prop.GetGetMethod();
          object o = m1.Invoke(entity, null);

          MethodInfo m2 = prop2.GetSetMethod();
          m2.Invoke(databaseEntity, new object[] { o });
        }
      }
    }

    context.SubmitChanges();
    return databaseEntity;
  }
}

Still TODO:

In order to complete this, I would rather get the primary key using attributes. This allows the model to be used by those who do not use {TableName} + "Id" as their primary key. This will make it more flexible.  I would also like to figure out how to make the Attach() method work using generics. I have the error message, so it is a matter of hammering at it. Thus far, Google did little for the error message, so I am not sure Attach() can be conquered.

Peace and Grace,
Greg

More In LINQ to SQL with the Repository Pattern

I have figured out how to get the Repository where it does not require an active DataContext all the time. And, it works fine when run against my unit tests, so I am fairly certain it will work for me. The question I have now is whether it is the best solution. Here are my options:

 

1. Use a LINQ to SQL class with all of my entites (much like a data set)

I get two pros out of this. First, I can use a single data context, which is nice. Where this helps is avoiding too many factories:

 

IDataContextFactory dataContextFactory = new SimDataContextFactory();

 

versus

 

IDataContextFactory simFacotry= new SimDataContextFactory();

IDataContextFactory simStatusTypeFactory = new SimStatusTypeDataContextFactory();

IDataContextFactory simUploadFactory = new SimUploadDataContextFactory();

 

Second, I get generic implementation of my factory, as in the following method:

 

public IEnumerable<T> All()
{
    using (System.Data.Linq.DataContext context = _dataContextFactory.Context)
    {
        return context.GetTable<T>().ToList();
    }
}

 

The code here is adapted from Liam McClennan’s LINQ Repository project., with some changes I implemented due to things I read in Andrew Siemer’s blog entry. I will have to take this a bit further, as the ToList() is mandatory here, as the context goes out of scope. This means I end up having to cast as a List<T>, despite returning a list. As in the following:

 

IDataContextFactory dataContextFactory = new SimDataContextFactory();
IRepository<SimStatusType> target = new Repository<SimStatusType>(dataContextFactory);
List<SimStatusType> actual = (List<SimStatusType>)target.All();

 

A simple refactor will fix this, after I think about the implications. Currently, I see no downside to switching the interface to use List<T> instead of IEnumable<T>.

 

The downside here was mentioned in my last post. I am going to have to serve up the following object to a web service.

 

<s:complexType name="Sim">
  <s:sequence>
    <s:element minOccurs="1" maxOccurs="1" name="SimId" type="s:int" />
    <s:element minOccurs="1" maxOccurs="1" name="SimStatusTypeId" type="s:int" />
    <s:element minOccurs="1" maxOccurs="1" name="SimUploadId" nillable="true" type="s:int" />
    <s:element minOccurs="1" maxOccurs="1" name="MobileCountryId" nillable="true" type="s:int" />
    <s:element minOccurs="1" maxOccurs="1" name="MobileNetworkId" nillable="true" type="s:int" />
    <s:element minOccurs="0" maxOccurs="1" name="SimNumber" type="s:string" />
    <s:element minOccurs="0" maxOccurs="1" name="IMSI" type="s:string" />
    <s:element minOccurs="0" maxOccurs="1" name="MSISDN" type="s:string" />
    <s:element minOccurs="1" maxOccurs="1" name="CreatedOn" type="s:dateTime" />
    <s:element minOccurs="1" maxOccurs="1" name="CreatedByUserId" type="s:int" />
    <s:element minOccurs="1" maxOccurs="1" name="ModifiedOn" nillable="true" type="s:dateTime" />
    <s:element minOccurs="1" maxOccurs="1" name="ModifiedByUserId" nillable="true" type="s:int" />
    <s:element minOccurs="0" maxOccurs="1" name="SimUpload" type="tns:SimUpload" />
    <s:element minOccurs="0" maxOccurs="1" name="SimStatusType" type="tns:SimStatusType" />
  </s:sequence>
</s:complexType>

 

This is not a big deal if I stay in the Microsoft world, but may bite me, as I have a vendor with developers who code in Java and Flex. The Flex part does not scare me much for the transfer as they can use .NET Services. They will end up pruning off the useless stuff, however.

 

Pros

• Objects created for you
• Generic data transport
• Single Context

 Cons

• Constructs in WSDL that may cause complexity in other languages

2. Use a DBML per class

The benefit here is I avoid the extra tns: elements in my web service. I still have the advantage of generic implementation, but I lose the ability to use a single DataContext. This has a potential impact on scalability, as shown below.

 

The dual context is not a huge concern, necessarily, but the problem will continue to grow larger as we deal with more and more objects that save.

Pros

• Objects created for you
• Generic data transport
• Objects represented well for WSDL

 Cons

• Multiple data contexts to save complex sets of objects

The big question here is how much impact do multiple contexts place on scale. In all likelihood, very little, as I do not see many scenarios with multiple objects. It does definitely impact maintainability, however, as I have to create multiple objects for each table, as not all of the code can be completely generic. Over time, I may back off on this statement, but it is the case at present.

 3. Custom Objects

This is actually a set of scenarios. Here are the pros and cons:

Pros

• Objects represented well in WSDL
• Full control of data transport

 Cons

• Objects have to be coded
• Most likely non-generic (not usre yet)

There are  many ways I can tackle this one. I can set up my objects, for example, off of a DataSet. Done this way, I can stil end up with objects that transport well, while taking advantage of the ease of loading and saving using DataSets.

Summary

This is an age old issue. If you can buy software to do your work for you, it costs far less than hiring someone to hand code it. But, you trade some amouint of control when you do this. The other option is to custom code everything, giving you full control, but control comes at a cost, as time is money. Generally you cannot sell your implementation, so it is pure sunk costs. Of the two, I prefer to lean more towards the side of buying software and tailoring it. In order to get the best of both worlds, this means finding frameworks, not components.

Ah, but Frameworks present a problem. Microsoft desires to hit as many scenarios as possible, so the box is left pretty wide. But this means I am carrying a lot of weight I probably do not need. So, even with Frameworks, I end up with a less than perfect implementation.

The answer, at this time, is probably to go with LINQ with individual DBML bits for each table. Not optimal, but it solves the most problems. The objects are simple, so they can translate well to Java. The objects are written for me, so I avoid a lot of time with boilerplate code. The methodology is well documented, evne if I am stepping outside of its box a bit, so it should be easy enough to maintain. The main con here is how to keep scale issues from biting me. As I have worked with LINQ quite a bit, I think I have a bit of a grasp on this problem.

I need to go back through my posts and see if I posted this, but the biggest gotcha with LINQ, as a data transport, is trying to work with too much data at one time. LINQ sucks when it comes to real time systems that handle tons of data. I have no intention on using LINQ for systems that need this much data, so it is not a problem. Cementing the disconnected LINQ objects will take me far on my path to using it as a generic data transport.

Most likely, I will pitch it in the long run, as I do not see LINQ being the best DAL or ORM (it can be used as a pseudo version of either). Today, however, it speeds up my development efforts. And, if I use a single DBML per table (essentially using an Active Record type of pattern with a Repository pattern to get the data), I can switch to my own objects fairly easily and without altering my web services.

Peace and Grace,
Greg