Subsonic, part deux


I have now finished my database schema, so I played with setting up my data access using SubSonic. Thus far, I am pretty impressed.
 
First, if you need a repository, you can do something like this:

provider = new SqlDataProvider();
provider.DefaultConnectionString =
"{connectionstring here}";
ISubSonicRepository
repository = new SubSonicRepository(provider);

What this gives you is the ability to set up a mock, by deriving from ISubSonicRepository. I like this and will use it rather extensively in my test code. Thus far, I have written most of my tests against the respository. Yeah, I know this is not necessary, but it is one of the best ways to learn how to use it.
 
Here is the code to get a Backend with the id of 1:

Backend

backend = repository.Get<Backend>(1);

And, to add a new one, I simply create and save:

Backend backend = new Backend();
backend.BackendName =
"My Backend";
ISubSonicRepository repository = new SubSonicRepository(provider);
repository.Save(backend);

You gotta love generics. The one thing I am not sure I like is the way collections are implemented, as I will have to mock up each collection for my code instead of the Repository. Here is a load everything for the backend collection (to load a dropdownlist, for example):

BackendCollection coll = new BackendCollection();
coll.Load();

To me, that breaks the entire idea of having a Repository. I will have to think it through before making a firm decision, however. Perhaps this is something left over from the Active Record implementation?
 
Here are a few tests. Realize that these were to learn SubSonic and break many rules for tests I would write for my own stuff. ALso realize these are more Integration tests than Unit tests, as they actually manipulate real data. They are a good start for understanding how the repository works and will help in future posts when I am mocking the Repository. If you are a purist, please cut me some slack until I show some real tests. Then you can skewer me all you want.
 
namespace Microtrak.MultiProgrammer.Data.Tests.VisualStudio
{
 
/// <summary>
  /// Summary description for RepositoryTests
  /// </summary>
  [TestClass]
 
public class RepositoryTests
  {
   
private static SqlDataProvider provider;

    public RepositoryTests()
    {
     
//
      // TODO: Add constructor logic here
      //
    }

    private TestContext testContextInstance;

    /// <summary>
    ///Gets or sets the test context which provides
    ///information about and functionality for the current test run.
    ///</summary>
    public TestContext TestContext
    {
     
get
      {
       
return testContextInstance;
      }
     
set
      {
        testContextInstance =
value;
      }
    }

    #region Additional test attributes
   
//
    // You can use the following additional attributes as you write your tests:
    //
    // Use ClassInitialize to run code before running the first test in the class
    [ClassInitialize()]
   
public static void MyClassInitialize(TestContext testContext)
    {
      provider =
new SqlDataProvider();
      provider.DefaultConnectionString =
"server=(local);database=UnitWarehouse;UIS=sa;PWD=pass@word1;";
    }
    

    #endregion

    [TestMethod]
   
public void GetBackendForDallasColo()
    {
     
//expected
      int backendId = 1;
     
string backendName = "Dallas Colo";

      ISubSonicRepository repository = new SubSonicRepository(provider);

      Backend backend = repository.Get<Backend>(1);

      Assert.AreEqual(backendId, backend.BackendId, "Backend Id is different");
     
Assert.AreEqual(backendName, backend.BackendName, "Backend name is different");

    }

    [TestMethod]
   
public void GetAllBackends()
    {
     
BackendCollection coll = new BackendCollection();
      coll.Load();

      Assert.IsTrue(coll.Count == 4, "There are not four items in the collection");
    }

    [TestMethod]
   
public void AddtoCollection()
    {
     
Backend backend = new Backend();
      backend.BackendName =
"My Backend";
     
ISubSonicRepository repository = new SubSonicRepository(provider);
      repository.Save(backend);
    }

  }
}

 
 
Well, that is it for now. I will shared more as I get into my own tests.
 

Update on Miranda


Some things in life just suck. Does not matter how you slice a sh*t sandwich, it is still sh*t. Other things don’t suck as much. We have a situation that sucks, but I am not sure if it really sucks or just sort of sucks. And doctors wonder why so many parents go to complementary or alternative medicine.
 
As many of you know, my daughter Miranda has been fighting cancer since some time last year (diagnosed with Ewing’s Sarcoma on September 1, 2007). In early September, they removed some nodules from her lungs (or at least we thought they removed all of them). Apparently, they did not remove one of them. During the treatment, it did not shrink, which led them to believe that the nodule was not tumorous, as the main tumor reacted fine. On her CT scan yesterday (my birthday), the nodule has grown from 2mm to 4mm. Her oncologist did not call until late, as he and the radiologist pulled out all of the film and examined the history of CT scans she has had. That is how they noticed that the spot had not grown or shrunk over the entire time of treatment.
 
Dr. Ho, her oncologist, has told me it is not a major cause of concern, but all of the things I read suggest it is more likely bad news than good. I am, once again, learning more about cancer than I ever intended. The questions to ask are:
  1. Is the nodule calcified. Not a firm indicate it is benign, but a good sign
  2. Is the nodule smooth edged or not? Smooth is generally better.
  3. If it was your daughter, would you wait a month to find out if it grew?

We are now in the position of decided whether to operate or wait until the next scan. The fact the other nodule was not cancerous is good news. The fact this one is growing is not. If it is benign, you don’t want to make your daughter go through an operation. If it is not, you want to know as quickly as possible, to avoid further metastasis. At 4mm, it is very small (a bee bee, if you will), so it is not that dangerous, but cancer is a tricky fellow and we really know so little about it.

Doctors generally give you odds, when they are being honest. They know that x out of 100 patients make it five years without incident. But, for each child it is not a percentage. It is either 0% (the child dies) or 100% (the child lives). The question I have to ask is whether the odds are better with surgery (some risk) or with waiting (some risk).

It is my sincere hope we are just a bit too paranoid and everything is going according to schedule.

BTW, there is some hope on the horizon. President Bush signed the Conquer Childhood Cancer Act, which pumps $150 million into childhood cancer research. And, there is a targeted drug for Ewings … unfortunately, it will not be used in humans for years to come.

Peace and Grace,
Greg

Code Generating the Repository Pattern with Substage (Subsonic 2.1)


First, I was wrong in my last post and want to correct my statement (will correct in the post soon). You do not have to specify a web root to generate with SubStage. The tool is just not as well documented as I had hoped at first blush. Considering it is an open source project and still in development, that is not a big deal. When you play with projects as they are being developed, you have a few times where you get a bit burned. Wink
 
Now to the fun. In order to generate Repository code, here are the instructions.
 

Add a New Project

Click on the box with the plus in the upper left hand cornder (just below the menu):

Substage - create new project

The only property you have to change here is the name, which I set to UnitWarehouse. Other potential properties to hit are:

TemplateDirectory – if you have your own custom SubSonic templates
EnableTrace – for projects you wish to trace

Add a Connection String

Right click on the connection string node and choose Add Connection String

Substage - Add connection string 1

Now alter the properties to fit your database and name it something other than New Connection String. For mine, I am working with a work project called UnitWarehouse, so my properties are as follows:

ConnectionString: Data Source=(local); Database=UnitWarehouse; Integrated Security=true;
Name: UnitWarehouse

The name is important for a future step.

Add a Provider

Right click on the project and choose Add Provider.

Subsonic - Add new provider

There are tons of properties here, but there are only two that are mandtory for my first pass:
 
Name: UnitWarehouse
TableBaseClass: RepositoryRecord (default is ActiveRecord)
 

Invoke Providers

The next step is to query the provider(s) and find out what they contain. Click on the Invoke Providers button at the upper left (below the menu).

Subsonic - Invoke Providers

This will take a few seconds, but you end up being able to expand the UnitWarehouse provider now, yielding the following:

Subsonic - After Invoke

Generate Code

Before doing this, I would click on the SubStage configuration and place your code where you want. This is done by changing the CodeGenerationPath property. You can go witht he default, which is under the Program Files where Subsonic is installed. I am not fond of this location, however.

Click on the Provider again and then click the Generate Code button. It is the middle above the left pane.

SubSonic - Generate code

This happens fairly quickly.

First Blush

It looks like this will save me some time. I am not seeing the actual repositories created here, but that is not a huge deal, as I can code this out. I also notice there is a stub for foreign keys (and it recognizes them), but it is not there. I will have to look at the roadmap. Overall, however, I am interested in learning more, so I think I may have found a solution. I do not want to return to .NET tiers, as it painted us into a box last time. It may have improved now, but it is Active Record, not repository, so I don’t want to go there.

Before settling in, I will take a look at CodeSmith, just to be sure there are not any good Repository templates. But, thus far, I am willing to continue to implement SubSonic and see where it leads me.

Peace and Grace,
Greg

Is Cuil really cool?


I was reading a few sites today and found that Anna Patterson, former Google employee (architect?) and her husband Tom Costello are launching a search engine called Cuil (pronounced "cool"). My first impression is not that I am not that impressed.
 

Relevance

 
I ran a few searches on Cuil and Google. The Cuil search consistently got more pages, when it did not give me a "search produced no results" page. On the other side, many of the top results were pure SPAM. Here is an example. Run McDonalds Disney in both engines. In Google. the top link is "McDonalds Dumps Disney".
 
google.
 
In fact, ther results are as follows (all relevant):
 
  • 6 on Disney dumping McDonalds
  • 2 on Disney, McDonalds and China (labor issues)
  • 1 on Disney and McDonalds, pre split
  • 1 on the Downtown Disney McDonalds

ALL 10 relevant to my search.

The same thing in Cuil results in the following:

cuil

Some impressions here. Wow, that looks great. I like the additional features. If I were an eye candy man, I would be sold. But, I spend too much time preaching proper functionality to let a bit of eye candy steer me. Here are the results

  1. Redirects to http://www.adxtn.com/, a "search" page.
  2. Redirects redirects to http://www.adxtn.com/, a "search" page.
  3. Redirects redirects to http://www.walmart.com/rollbacks?dest=120807&sourceid=03237428460627726046 – an affiliate redirect
  4. Site selling disney toys
  5. Redirects to http://www.adxtn.com/, a "search" page.
  6. Same sales site as #4
  7. Same sales site as #4
  8. Same sales site as #4
  9. directs to http://www.adxtn.com/, a "search" page.
  10. Same sales site as #4
  11. Site selling trains (http://kit.pchot1079.com/anniversary.html)

At least 5 of the links do go to something Disney (no McDonalds), but all are on the same site. Seems like it would be easy to figure out how to fake things to be the top site on this search engine.

Thus far nothing useful to me. So, let’s refine our search and type in Disney Dumps McDonalds, the story I am looking for. Result:

We didn’t find any results for “Disney Dumps McDonalds”
 
As you add words, Cuil begins to find less and less, primary not relevant, so it looks worse. I tried a few searches I know return results and was pleased by most of what Google returned.
 
Result: Google wins hands down.

Kid Friendly?

Ouch! How about we try something a bit more risky and type in "redhead". Google seach finds "red hair" (wikipedia), "redhead", a you tube video, redhead brass, realm of redheads, flickr: "redhead", redhead blog and blonde redhead. Of these, all are probably safe to visit for children. Cuil returns 3 sites on Blonde Redhead, 2 redhead blogs, the redhead public school a redhead lovers site and 5 porn sites. Okay, so Cuil is far less kid friendly than Google.

What about Blonde Redhead (a group)? All links on both google and cuil are relevant.

Lesson learned: Teach kids to search properly.

To be fair, I hit on words like plump and foot, as both are innocent words that can be fetishy, and google faired worse on plump, so there is no 100% kid friendly search amongst the "big boys".

Result: Tie, although on hair color, Cuil serves up far more … (pun intended) 

Speed

Google returns results in a fraction of a second no matter what I am seeking. Cuil can be nearly immediate to agonizingly slow. Perhaps once the slashdotters are finished on it, it will return more quickly?

Result: Google wins hands down

Other

I did some Undernet type of searches and nearly the entire front page goes to sites trying to install software on my computer. The same searches on Google yield safer sites, although many are still garbage. When I say Undernet , I am talking about security searches when you are looking for hacks. I am sure searches for warez and other illegal activities would reveal the same type of pop ups and redirects. Google fared far better in these types of searches, as the first page in Cuil was generally a lot of pages trying to install something on my computer.

Summary

The site is impressive looking. I like the layout of the searches. Finding a relevant search is nearly impossible from my tests. Yours might return something better and perhaps this will improve in the future. Currently I see no reason to use Cuil, unless you want your search results to be a bit prettier than Google. Considering SPAM is very often the first page of links, I would find it exasperating to find a good hit. I will not switch.

There is also the bug of not getting results one minute and tons of results the next. I think this might be related to traffic and I really hope they are getting slammed, as if this is a light day, the scalability sucks.

My suggestions for Cuil (if they care)

  • Focus less on hype (we catalog more pages than Google) and more on quality searches
  • Determine why the engine fails on searches that later work
  • Cull out the SPAM (any search phrase with is before it is probably SPAM hsdfas is Disney McDonalds seems to always lead to garbage)
  • Find out why the search engine falters for a long time

I would also recommend not press releasing the world until you get slammed up against the wall a few times by us geeks. We will probably give you a second chance. The world probably won’t.

Peace and Grace,
Greg

Going Subsonic


I am in the midst of retooling one of our tools at work and I need to clean up the data access (done with .NET tiers). As I was working on how to work out thedata access, I thought about Rob Conery’s MVP ECommerce series. In this project, Rob uses the Repository pattern for data access, so let’s talk about that for a second.
 

Repository Pattern

The norm you see in many ASP.NET applications, or even Windows applications for that matter, is a straight-forward progression from UI to database. It looks something like this:

Typical Flow

The problem with this model is it is hard to test your code without actually performing data access. Now, you probably know I am not against testing your data access with a unit test framework, as long as you realize these are integration tests and not unit tests. I also advocate keeping these in different classes and ultimately different assemblies, as they serve very different purposes.

The repository pattern turns this method on its ear a bit, as you feed the Repository to your code rather than have your code call the Repository. By flipping, you can easily create a mock repository, either by creating one in your test assembly (as Rob has done in his project) or by using a Mock framework (I am currently fond of Rhino Mocks, but this could just be my flavor of the month, as I am still trying to determine what exactly I really want in a mock framework). Off the tangent about mocks, the pattern looks more like this:

Repository Flow

Now, this is actually a pretty poor diagram, as you are not necessarily feeding the Repository at the business level, but you can see a bit of the inversion from "normal" flow. This is testable as you can code more like so (from Rob Conery’s ECommerce source). Note that this is only a small portion of the test repository and test code.

 

namespace Commerce.MVC.Data {

    public class TestCatalogRepository : ICatalogRepository {

 

        public IQueryable<ProductReview> GetReviews() {

            List<ProductReview> result = new List<ProductReview>();

            for (int i = 1; i <= 30; i++) {

 

                ProductReview review = new ProductReview();

                review.Author = "TestAuthor";

                review.Body = "lorem ipsum";

                review.Email = "email@nowhere.com";

                review.ProductID = i;

                review.ID = i;

                result.Add(review);

            }

            return result.AsQueryable();

        }

 

    }

}

namespace Commerce.MVC.Tests {

        [TestMethod]

        public void Review_Repository_Can_Return_Reviews() {

 

            ICatalogRepository rep=new TestCatalogRepository();

            Assert.IsNotNull(rep.GetReviews());

 

        }

    }

}

If you need a bit deeper of a look, here is the class diagram of the pattern:

Repository UML

What this tells us is we can have some generic functions in our base class and specific functions in our derived classes, so SpecificRepository is the class we are actually running (with whatever the Repository is called).

Subsonic

One thing always leads to another and I found Rob’s other project: Subsonic. Subsonic is a code free ORM, or at least that is the main vision. There is also a tool called Substage that can generate the classes for you so you can incorporate them in your own project, as Subsonic requires full trust (something you WILL NOT get with many ISPs).

I have not gotten deep enough into Subsonic to state it is definitely my ORM on this project, but I like what I see thus far. My only issue is I am not writing a web application. This leads me to two ways to create my classes:

  1. Create the project as a web project and "steal" the generated assemblies
  2. Use Substage

The later is my choice. The problem is you have to point it to a web application anyway. Thus, I am offering the suggestion that you should be able to use any valid application configuration to feed Substage to fire off the Subsonic engine to create your classes. As it stands right now, you point to the web directory and it reads web.config, so I am kludging this a bit to make it work, but I believe it would be better if you could point at a config rather than a folder.

Update: I was wrong. While the tool is focused on web output, you do not have to kludge it at all. In fact, you need not specify the web application in the configuration to get it working. I will accept my beating now. Open-mouthed

As with MVC Membership, I will eventually look at the Source code and figure out how to make this ORM easily accessible from all types of projects, without kludging. This is not a criticism of the work Rob and others have done thus far. I am not one to shoot a gift horse. I just see a bigger box than the one originally drawn and I want to see it filled.

I will blog about the kludge at some point and detail my journey through this project if I decide this is the ORM for me. My primary focus is testability and the Subsonic project utilizes the Repository pattern, so it already has one major thing in its favor. More forthcoming.

Peace and Grace,
Greg

Boycotting McDonalds


The American Family Association has decided to boycott McDonalds over their joining the National Gay and Lesbian Chamber of Commerce and a $20,000 donation to the organization. It is there position that McDonalds is promoting the gay and lesbian agenda and trying to undermine traditional marriage. What has particularly gotten their rancor is a comment by Bill Whitman, McDonald’s spokesman, who stated “Hatred has no place in our culture. That includes McDonald’s, and we stand by and support our people to live and work in a society free of discrimination and harassment.”
 
Boycotting McDonalds? Now that is something I can get behind. Nah, it’s not because of the gay and lesbian thing. I could really care less. Of course, I support the AFA’s right to protest, but I believe there are far too many more important things to focus on. I also support McDonalds right to support whom they desire. It is all part of the American way.
 
I see McDonalds as a company that can make up its mind what groups to join. And, if they feel this can increase their profits, good for them. If they are doing this for ideological reasons, more power. And what about that $20,000? $20,000 from a multi-billion dollar company is chump change and nothing to get overly excited about.
 
My reason for boycotting McDonalds would be the garbage they sell. But, then I am also up for boycotting the fast food industry completely. Think about it, here is an industry that sells food that often has more than 50% of its calories from fat. In one meal you can get 2.3rds of the daily recommended fat intake for the day (that is max fat intake, by the way). And much of the fat is saturated. Kiss those arteries goodbye.
 
I have been in a mostly informal boycott of fast food for quite some time. There are a few occasions where I have broken down and ordered fast food and I am finding I really do not like it any more. Once you wean yourself away from fast food, you find that it is not really as tasty as you previously thought it was. And, if it really not that tasty, why destroy your body on it?
 
There is one thing I do agree with the AFA on. Disagreeing with others is not the same as hatred. It is commonly regarded as such in this society, where our "news" shows are more about different talking idiots screaming at their guests. And, it is common on the Internet, where logical fallacy has been elevated to an artform. But, it should not enter into our news media, who is supposed to be reporting the news, not forcing us into adopting a certain viewpoint.
 
The UPI went so far as to headline their article "Anti-gay group to boycott McDonalds". I find this to be irresponsible for a news organization, as they are supposed to report the news, not present commentary disguised as a news article. I am not sure I can name  group whose primary charter is anti-anything. There are many times when groups get more bang for the buck out of negatives, of course, but it is hard, if not impossible, to sustain indefinitely. And, if that is your major focus, you either have to morph or die.
 
Peace and Grace,
Greg

Editing Visual Studio Templates


I have seen a couple of posts lately with people asking how to customize what VS spits out when you create a new item in a project, so I decided to post. This post applies to Visual Studio 2008 only. For 2005 and earlier, the templates are still in roughly the same place, but they are not zipped up (actually making it easier to customize them?).

Editing the files

First, we need to find the templates. There are a variety of templates, and it depends on what you want to edit. The general location is:

C:Program FilesMicrosoft Visual Studio 9.0Common7IDE

Or for 64 bit

C:Program Files (x86)Microsoft Visual Studio 9.0Common7IDE

There are numerous directories under this folder, but we are only interested in two for most of our work: ItemTemplates and ProjectTemplates. These folders contain (big surprise coming ;->) item templates and project templates.

Folder

Let’s look at a web project template first. The zip file is located at:

C:Program FilesMicrosoft Visual Studio 9.0Common7IDEProjectTemplates{language}Web1033

We will open up the WebApplicationProject.zip file. Inside we find the following (pictured below):

WebProject

Opening the default.cs file yields the following:

using System;

using System.Collections;

using System.Configuration;

using System.Data;

$if$ ($targetframeworkversion$ == 3.5)using System.Linq;

$endif$using System.Web;

using System.Web.Security;

using System.Web.UI;

using System.Web.UI.HtmlControls;

using System.Web.UI.WebControls;

using System.Web.UI.WebControls.WebParts;

$if$ ($targetframeworkversion$ == 3.5)using System.Xml.Linq;

$endif$

namespace $safeprojectname$

{

      public partial class _Default : System.Web.UI.Page

      {

            protected void Page_Load(object sender, EventArgs e) 

            {

 

            }

      }

}

Now lets move up to a new class template for a web project. You find this at:

C:Program FilesMicrosoft Visual Studio 9.0Common7IDEItemTemplatesWebCSharp1033

The file is webform.zip and we will want to edit codebeside.cs. The only real difference here is highlighted below:

public partial class $codebesideclassname$ : System.Web.UI.Page

{

    protected void Page_Load(object sender, EventArgs e)

    {

 

    }

}

Tags available

Here are a list of some of the tags you might want to use to customize your templates.

·         $clrversion$ – current version of the CLR

·         $guid#$ (where # is a number between 1 and 10) – GUID that replaces the GUID in the project file

·         $itemname$ – Name provided by user (note that the web team has stepped out of the box here

·         $machinename$ – machine created on

·         $projectname$ – Name given in the new project dialog

·         $registeredorganization$ – Registry key value for HKLMSoftwareMicrosoftWindows NTCurrentVersionRegisteredOrganization

·         $safeitemname$ – Name provided by user with all unsafe characters replaced

·         $safeprojectname$ – ditto, but for a project

·         $time$ – Current date time in the format DD/MM/YYYY HH:MM:SS

·         $userdomain$ – Current user domain

·         $username$ – Current user name

·         $year$ –Current year in the format YYYY

Please note that the $time$ variable is in the format DD/MM/YYYY HH:MM:SS and is not localized for culture. This is probably not a big deal, but it should be noted. Here are a few “undocumented” tokens:

·         $fileinputname$ – unsafe file name, sans extensions

·         $fileinputextension$ – extension of above

·         $runsilent$ – In template, no clue what this does (Boolean value)

·         $itemname$ – unsafe item name

·         $itemrootname$ – Seems to be the same as above?

·         $rootname$ – ditto

·         $safeitemrootname$ – Safe file name without extension

·         $rootnamespace$ – Namespace at level you are creating your class.

Some people have stated that $rootnamespace$ is the namespace for the root of the project, indicating there is a different tag when you are going deeper. I have not found this to be true, which indicates this is the namespace at the root of the folder you are creating in. Not as big a deal for web projects, but it does have implications for windows programs, where you can add a folder and have the folder name automagically appended to your namespace.

Note that you can also create your own custom templates and pass custom parameters to them:

http://msdn.microsoft.com/en-us/library/ms247063(VS.80).aspx

The launch page for templates is here:

http://msdn.microsoft.com/en-us/library/6db0hwky(VS.80).aspx

Hope this helps!

Peace and Grace,
Greg

Abstracting databases


I read one of Jeff Atwood’s latest posts called Maybe Normalization Isn’t Normal. It takes a bit to get his point, as it rambles, but I am in agreement with his post, the gist of which is "normalization does not scale". I say this with a grain of salt, however, as it takes a lot of data to really worry about this. So, today, I discuss when and why of normalization. But more importantly, why we abstract.

Normalization

First, I believe you should design everything with a bit of sensibility. Normalize to the nth? Yes, it is a bit of a waste some times, but it helps you identify your objects. Note that you do not  have to actually write out the tables as you identify objects. Also note, this is not going to be your production database. Let’s start with a small example. We want to make a person table, as a person is an object we will deal with.
 
Person
—–
PersonId
FirstName
MiddleName    NULL
LastName
 
Now, we need the user to log in. The question to ask here is whether or not you will ever have people in your system that do not log in. If yes, then the user table is separate from the person table.
 
User
—–
UserId
PersonId
LoginName
Password
 
In the case where everyone either logs in or has the potential to log in, you might wish to stick all of this into a single table. NOTE: User is a keyword in SQL Server, and some other databases, so WebsiteUser, or similar, may be better. What about phone numbers or addresses?
  1. Can they be shared by multiple people?
  2. Can a user have more than one phone number of a particular type?

If you answer either yes, you will end up duping data iff you stick it in the person table. That is generally not good, as data is easy to get out of sync.

The idea here is identify the object and then figure out the context.

Abstraction

Now suppose we opted for radical normalization (phone numbers, addresses, etc. in their own database tables), ala Jeff’s example.

social network database example, normalized 

(c) Jeff Atwood, CodingHorror.com

As we grow, we find this does not scale. Per Jeff, here is what happens:

Normalization certainly delivers in terms of limiting duplication. Every entity is represented once, and only once — so there’s almost no risk of inconsistencies in the data. But this design also requires a whopping six joins to retrieve a single user’s information.

select * from Users u
inner join UserPhoneNumbers upn
on u.user_id = upn.user_id
inner join UserScreenNames usn
on u.user_id = usn.user_id
inner join UserAffiliations ua
on u.user_id = ua.user_id
inner join Affiliations a
on a.affiliation_id = ua.affiliation_id
inner join UserWorkHistory uwh
on u.user_id = uwh.user_id
inner join Affiliations wa
on uwh.affiliation_id = wa.affiliation_id

Those six joins aren’t doing anything to help your system’s performance, either. Full-blown normalization isn’t merely difficult to understand and hard to work with — it can also be quite slow.

So, the solution is to not normalize? Perhaps, but that is not really the point. If you placed the query in a stored procedure, or at least separated out the query object, you can recode a small bit and refactor the database to your hearts content.

This gets to my point. It really should not matter what your schema is. I mean this on a high level, of course, as changing schemas is a royal pain in the rear end. You should look for the schema that gets you to scale, but separation of data means a lot.

In addition, you do not have to grab all of the data with six joins, as the end user is not editing all the user and all of his history at the same time. At least not in a SANE application. I put SANE in caps as far TOO MANY applications are INSANE.

How do you abstract. Well, if you go with code generation, you are going to have to regen some objects. If you are changing schemas, I hope this means you have set up your ORM so it maps to business objects rather than set it on autogen.  But, you can also abstract via stored procedures.

Now, let me take a moment to explain the stored procedure. It has gotten a lot of bad press lately, largely by ORM tool manufacturers, who often embed dynamic code in their objects. While it is true, as they state, that sprocs can perform much worse than generated code, this is not a normal case in most well designed stored procedures. And, you get two benefits: a declarative security model (you can easily state who uses what without coding) and abstraction of the database schema.

I am not suggesting that you have to go with sprocs to abstract, but it is an easy way to get a newbie crew into abstraction. I am suggesting, however, that you require some form of abstraction to separate the schema from the business objects. If editing the DAL is your method, then go for it. If you autogen, and have a means of mapping (NHibernate, for example), then go for it. In fact, there is nothing to say you can’t use multiple solutions to further abstract.

Back to Jeff’s article. He then states that a single table is more efficient:
denomalized schema
(c) Jeff Atwood

This works — queries are now blindingly simple (select * from users), and probably blindingly fast, as well. But you’ll have a bunch of gaping blank holes in your data, along with a slew of awkwardly named field arrays. And all those pesky data integrity problems the database used to enforce for you? Those are all your job now. Congratulations on your demotion!

Now here is the big question? Should this really matter? i am not asking whether performance matters? it does. I am not asking whether schema choices should be made sensibly? they should. What I am asking is why this is such a big deal.

If you properly separate concerns and abstract your database schema (by whatever means you are comfortable with), you can change out the bits rather quickly. With a stored procedure, I start with this:

CREATE PROCEDURE Proc1
AS
select * from Users u
inner join UserPhoneNumbers upn
on u.user_id = upn.user_id
inner join UserScreenNames usn
on u.user_id = usn.user_id
inner join UserAffiliations ua
on u.user_id = ua.user_id
inner join Affiliations a
on a.affiliation_id = ua.affiliation_id
inner join UserWorkHistory uwh
on u.user_id = uwh.user_id
inner join Affiliations wa
on uwh.affiliation_id = wa.affiliation_id

And when perf starts to suck, I simply run the following:

ALTER PROCEDURE Proc1
AS
select * from users

This at least works in theory, since the first query grabs tons of rows, but you get the idea.

Personal Feelings

I feel it is "sh*t all stupid" (from Vacation) to denormalize with multiple rows of history in a user table. I realize this was not Jeff’s point. There are some concepts you simply DO NOT blend. I also believe that moving addresses into the user table makes sense, even if you end up with some dupes in information. I say this with a caveat, of course. If the address change requires the same address change in other places, you should have it in a separate table. Trying to be supercoder to catch all dupes is also stupid.

By the way, if you like to play supercoder, and have not adopted TDD, try it. Proper user tests will convince you that you are not coder god, no matter how good you are. I don’t know how many times I thought I had quickly coded something to find I missed one line or fudged another. Tests illuminate that skill level may reduce mistakes, but it never eliminates them. Stepping off the TDD box now.

My point here is that normalization or denormalization need not ruin your day if you follow the following practices:

  1. Determine business objects first
  2. Use the objects to map tables, but allow the tables to flow to have them make sense
  3. Create contracts between layers. This can simply be creating interfaces on your objects, so new object implementations do not change calls
    As an unwritten rule, this means DO NOT BREAK CONTRACTS
  4. Abstract your database schema (your choice here cowboy)

Now you find that you are the next google and perf is suffering. Cool. denormalize for speed and rewrite your abstraction code only. Much quicker.

AH, but some of you probably realize that abstraction slows things down a bit too? Great. If you are really the next google, you have a success problem on your hands, which means you have beaucoup d’argent (money) to work with to solve the problem. Hope you have lots of stock options.

Peace and Grace,
Greg

MVC 4 released


I started off tonight telling a friend he sucked. Not that he really sucked, but he beat me to MVC Preview 4. Microsoft guys, in their wisdom(?) decided to not release on www.asp.net and stuck to CodePlex (my bad!) until they had content.
 

First Impressions

Finally something you can cling to. Both AJAX and Membership are included. I am still in the process of seeing how far I can get with security trimmings, but my work on the ASP.NET Membeship codeplex may be over. This project is shaping up to something I can work with. I am in the process of working up specs for a project to release this fall for my church and I am now set to do it all in MVC.

Non MVC news

I am sitting out on my porch at 12:45 AM, smoking a cigar and sipping a Gentleman Jack. It is cool, but muggy. My daughter, Miranda, got her Micket button out today and it is a major milestone for me. So much has happened since her diagnosis with Ewing’s Sarcoma and now I feel we are getting there. We still have five years before we can celebrate completely, but life is good.

I sit here thinking about Cody, who passed earlier this year. And Kayla. And Julian.

This week, the Senate passed the Conquer Childhood Cancer Act. It is probably not enough, but I am thankful that more parents, like Cody’s, Kayla’s, and Julian’s (and others I know of that I cannot think of right now), may see some light. It is sad that our scientific community cannot get proper treatments to bear, but it is a step. Hopefully, this will make a difference. I am sure there is some parent out there tonight who has just lost a child, and for whom this is too late. And I feel for them. But I am thankful I am not them.

I am not sure where I am going next. My job is secure now, but I am not sure what I will be doing in the next few months or years. For now, I am just basking in the fact that I have reached a milestone that 30% of the parents who hear "your child has cancer" will never reach. My wife reminded me today of the day the doctor said "your child is cancer free". She still has a waiting game, five years, but the tumor is gone. The chemo is gone. And now, the feeding tube is gone.

I am about to go and bid adieu to a group that I have spent a lot of time in. Not because they are not good people, but because I do not feel the need to debate with them any more. Perhaps I will come back once this election is over.

There are people on both sides who are scared that "that guy" will be elected, whether it is Obama or McCain. In all truth, it really does not matter most. The political system in this country is broken, but at least the checks and balances stop anyone from screwing it up too bad. I am still not sure how I will vote. Either way, it is the less of two evils.

Well, to bed now and tomorrow to more of MVC and furthering my career. I still have a few more sips and I will sit and watch the full mooon. It is beautiful out tonight.

Peace and Grace,
Greg

Remote Assistance


I was recently asked to help someone with Remote Assistance in Windows XP. The person involved sent me the invite, but I was unable to initially connect. If you have had that issue. Let me show you how to solve this, with a bit of editing.
 
First, you have to make sure Remote Assitance is turned on. It is on by default, so if you have not dinked with Windows XP, you should be fine. If not, click on your start menu, then right click My Computer. Click on the remote tab and you can find the checkbox.
 
2RD
 
You may also want to click on the Advanced button and ensure you have not restricted rights too much to actually get help. Creating the Remote Assistance request is simple. I have no clue where to find it "normally", but I do this:
 
  1. Click on the Start button
  2. Find Help & Support and click on it
  3. Type Remote Assistance in the search box and click the button
  4. When it finds Remote Assistance, click on the link
  5. Click on  the Invite Somebody to Help you link
  6. Click on the bottom option: Save Invitation as File (Advanced)
  7. Set the number of hours for the invitation to expire – 1 is default – and click the Continue button
  8. DO type in a password on the next screen – this is a safety measure – and click the Save Invitation button
It is important that you put in a password and tell this to the person, but not in the same email as the invite. Do not send this yet. You will have to edit the file and possibly some other settings on your side to get this working. Here is what the file looks like:
 
<?xml version="1.0" encoding="Unicode" ?>
<UPLOADINFO TYPE="Escalated">
    <UPLOADDATA USERNAME="Gregory A Beamer" RCTICKET="65538,1,192.168.1.8:3389;windowsxp:3389,*,QLzAUI8vpYzdZRgxQj50v2ntEQyjFODglBgW7GeGJcQ=,*,*,cs/oOkosIWmDMyfff7eDAmH0D7o=" RCTICKETENCRYPTED="1" DtStart="1216230419" DtLength="5940" PassStub="Kf*9Ltnk1WtIY3" L="0" />
</UPLOADINFO>
 
Note the highlighted section. This is the IP address the person will try to help you at. Notice anything that will not work? Yes, the address is 192.168.1.#, which means you have a router on your broadband connection. That is not going to work.
 
So, open an Internet browser and type the URL http://whatsmyip.org/ in the browser bar. This will ping back your IP address. Open the file RAInvitation.msrcincident and change the highlighted portion to your IP. Let’s suppose your IP is 68.52.140.22, which would be a Comcast IP (no that is not my IP, but thank you for asking). So, you edit the file to reflect this change.
 
<?xml version="1.0" encoding="Unicode" ?>
<UPLOADINFO TYPE="Escalated">
    <UPLOADDATA USERNAME="Gregory A Beamer" RCTICKET="65538,1,68.52.140.22:3389;windowsxp:3389,*,QLzAUI8vpYzdZRgxQj50v2ntEQyjFODglBgW7GeGJcQ=,*,*,cs/oOkosIWmDMyfff7eDAmH0D7o=" RCTICKETENCRYPTED="1" DtStart="1216230419" DtLength="5940" PassStub="Kf*9Ltnk1WtIY3" L="0" />
</UPLOADINFO>
 
You can also change the port if you desire, it is the number after the colon. Before you do that, pay attention to the rest of the instructions, however, as you will need to focus if you change this.
 
The next part is fairly simple. Log into your router. In most cases, the router IP is 192.168.1.1. If you do not find one there, consult the manual for your router. Log into the router and find the port forwarding section. This can be called a variety of things, like gaming exclusions, or server exclusions, or whatever. You will have to open the port in question (3389 by default) and point it to your IP address. You can find this by opening a command prompt (windows key + r and then type cmd) and typing ipconfig. The IP address will be something like 192.168.1.#, with the last number changing for different machines on your network.
 
Now, zip up the file and send it to your buddy. He can double click, type in the password, and you should be able to get some help. If you do not have a router, your IP address should be correct, so there is no need to jump through these hoops.
 
Hope this helps!
 
Peace and Grace,
Greg