Visual Studio Projects: A Tutorial in Setting Up Solutions to Succeed

As a consultant, I have spent time in numerous environments at various companies. And while I find differences in each of the environments, there are some elements I find in multiple companies that impede there development efforts. One of these is setting up their projects.

In this post, I am going to show a general method of setting up a Visual Studio project that works well on projects that encompass numerous teams, or even program level development that spans multiple applications. The goals of the method are:

• Aid in creating testable software
• Increase team velocity by allowing teams to work in parallel
• Develop with lower risk

In this post, I am going to cover the following topics, from a high level view.

• Domain Driven Design
• Contract First Development
• Loose Coupling and Separation of Concerns
• Working with multiple teams

The items above are covered rather loosely, so I will call them out as we approach them.

Contract First

Let’s jump in feet first.

Rather than beat around the subject, one of the core elements of my set up is focusing on contract first development. This is a paradigm shift for many people in two respects.

1. Contract First means you spend time planning the contracts up front, rather than developing via a stream of consciousness, determining contracts later
2. Contract first dictates setting up interfaces for your software, which many developers are not used to, until they need to add an interface

The core of contract first is I determine anywhere there is a boundary and figure out its public interface, or contract, up front. Some of the boundaries are obvious, as they end up in some form of “user” interface (“user” in quotes here, as the user may be a system, not a human being). A good example of this type of boundary, from a systems perspective, is a “web service”. Note that “web service” here means any service that can be served via a network, so this could be ASMX, Remoting or WCF. When many people talk about web services, they generally only focus on services that reside in IIS and use SOAP, but I don’t see the server or message protocol as a determinant factor, especially when we look at WCF which allows you to switch protocols quite easily. If you want to simply the definition, you can call this a service boundary.

The other main boundary we routinely work with is an assembly boundary, which is exposed the public methods on our classes. In reality, this is the same boundary as a web service, except how the interface is expressed (no WSDL for an assembly). That and we can have an assembly boundary in a single process.

I could mention separation in boundaries, like in-process, out-of-process on same machine, different machine, etc., but the main difference in type of contract  lies in how the contract is expressed.

Contract Expression

On of the simplest expressions of contract is an interface. In .NET languages, you will add an interface to your project, like so:

public interface IRepository<T, in U>
    where T : class 
    where U : struct
{
    IQueryable<T> GetAll();
    T GetById(U id);
    void Create(T obj);
    void Save(T obj);
    void Delete(T obj);
}

public interface IQuoteRepository : IRepository<Quote, int>
{
    IQueryable<Quote> GetAllByAuthorName(string authorName);
}

 

To create a concrete example, you will have the class adhere to the interface, like so:

public class QuoteRepository : IQuoteRepository
{
    public IQueryable<Quote> GetAllByAuthorName(string authorName)
    {
        throw new NotImplementedException();
    }

    public IQueryable<Quote> GetAll()
    {
        throw new NotImplementedException();
    }

    public Quote GetById(int id)
    {
        throw new NotImplementedException();
    }

    public void Create(Quote quote)
    {
        throw new NotImplementedException();
    }

    public void Save(Quote quote)
    {
        throw new NotImplementedException();
    }

    public void Delete(Quote quote)
    {
        throw new NotImplementedException();
    }
}

Fairly simple and I believe we all get this. If we need a service boundary, we will add some attributes to the class and the interface. I then can create a Web Service Description Language (WSDL) file that describes the interface in a way it is easily consumable from a variety of environments.

The point here is the interface is a good code description of a contract, while WSDL is a good way to express the same contract for a “web service”.

Setting Up for Contract First

In Visual Studio, this is an easy exercise. The rule is every time I have a boundary, I need to have interfaces defined. The most common way I see this done is like the following screen capture:

If we run this setup against our goals, we find that we have aided the creation of testable software, as we have interfaces. And, with the interfaces, we have some ability to run teams in parallel, provided the team implementing the library does not change the library too radically. But we still have a high risk of halting development if someone from the library building team checks in broken code, as we cannot utilize the latest library for our interfaces.

There is a way to alter the solution setup to mitigate this risk: separate the interfaces. The following screen shot shows this method:

We can then isolate our teams by creating two solutions, as seen below:

Setup for Library Building team is shown in the previous example, while the setup for the team consuming the library to build the business tier library(s) is set up as follows:

The important takeaway here is both the data and business teams use the domain models and the contracts. The data team is setting up concrete implementations, while the business team is working with mocks and stubs that adhere to the contracts (interfaces). Note also that the business team can set up contracts (highlighted in the screenshot above) so the UI team can work from mocks and stubs.

Very early in the process, we should have the interfaces completed and then compile the .Contracts projects and include the compiled libraries. This should be done as soon as the contracts are firmed up and completed. The reason for doing this is it reduces the temptation to change contracts without thinking them through. This reduces the risk of having teams develop in parallel. It can also foster communication between teams, as it is quite clear when you change contract you are working with “publicly released” software, even if it is only internal to the company.

Working on the Domain

The domain is another area we have to spend some time on. In my projects, I will divide the entire solution into the following “layers” or “tiers”.

• UI – contains one or more user interface projects
• Business – contains the application logic
• Data – Data Access bits
• Framework – Components shared across multiple applications
• Domain – Domain (data) models shared across layers
• Test – Unit and Integration tests coded with a particular unit test framework

I will cover these in a bit.

Nearly every project will consume the Domain project, as it contains the domain models. The data layer will be responsible for filling the models and persisting them back. The actual data access technology should not matter (DataSets, EF, LINQ to SQL), as the models are independent of access. The business layer will perform business rules validation, etc, on the models. And the UI will bind the models in many cases. I say “many cases” as certain UI technologies may be more suited to creating presentation models that may or may not have direct parity with the domain models. A prime example of this is MVVM (Model – View – ViewModel), which is commonly used with Silverlight. The idea of separating domain models from presentation models is also present in ASP.NET MVC, especially when strongly typed models are used on your views.

Step-By-Step

Before going through the steps, let’s make up a silly story of what we are building.

Greetings, Inc. is a company that creates hello messages that plug into other company’s application. They are interested in a web application version of their old COM based application that delivers a personalized Hello message with a random quote. The application is to be developed using .NET technologies and has to be flexible enough to support various user interfaces.

I am in the process of researching setting this up as a template and sharing it. I am not there yet. So, here is a brief step-by-step on how I set up a project. This explanation goes through the ideal order of setting things up.

1. Create an Empty Solution with the following naming convention <CompanyName>.<ProjectName>. In this case, the application is GreetingsInc.HelloWorld
   
2. Create Solution folders in Visual Studio 2010
   

   

3. Create the same folders in Windows Explorer – this is an optional step, but I find it easier to develop this way, as my logical folders correspond to a physical folder. To do this, I run the following script (makeDirectories.bat):

   mkdir Business
   mkdir Data
   mkdir Domain
   mkdir Framework
   mkdir Test
   mkdir UI

   cd Data
   mkdir SqlServer

   This creates the following structure:
   

4. The next step is to create a domain model project: GreetingsInc.HelloWorld.Domain
   

   After creating the project, I create my two domain models: HelloMessage (the full message) and Quote (quote returned from database):

   

5. Add a project to the Data folder called GreetingsInc.HelloWorld.Data.Contracts
   
6. Reference the Domain project
7. Add a Generic Repository and then HelloMessage and Quote repositories:

   

We can go on here, but at this point, I am ready to start Test Driven Development of my business layer and allow the developers of the data layer to go off and start developing their work on an independent schedule. Here is how the interfaces appear:

Here are two tasks that can now be done in parallel:

• Data tier team begins setting up Repositories for data access, adhering to the interfaces
• Business tier team sets up mocks for the Repository to deliver a known answer and use them to create the business functionality

Summary

One of the most oft missed portions of development schedules is starting with decoupling as a first step. By focusing on the domain models first, then contracts, you force the project to a very loosely coupled state. In addition, you set up the work so the contracts (interfaces here) can be compiled and consumed in a “locked” state, reducing risk of the project not being able to integrate at a later date. This is an extremely valuable exercise.

Peace and Grace,
Greg

Twitter: @gbworld

Application Architecture: The Aspects of Development

This is the first in what is likely to be a long series of entries, as there are many introductory topics that people are missing when it comes to developing quality software. This particular post comes from an idea I have for a book on getting people focused on application architecture (or designing applications, if you like to think in those terms) and development.

At the Nashville .NET User’s Group Christmas party, I got into a conversation with John Kellar about application architecture. He stated that the important things in application architecture can go by the acronym ARP, which stands for Availability, Reliability and Performance. I don’t disagree with John on these, as these are 3 noticeable points of “failure” in applications. By noticeable, I mean these points are areas of concern for end users as they manifest themselves very clearly in the user interface.

I understand what John was saying, and we agree on application architecture to a very large extent. But many may see the P in ARP as a call to ensure every routine runs in the fewest number of milliseconds, no matter how complex the code gets. Or the A as making sure every web app has a web farm of servers supporting it. And this takes us to a very myopic view of the architecture, or design, world.

I have yet to come up with an acronym for the entire picture. When I look at development, I see a wheel, like this:

I like using a wheel as pictured above, as it makes for a nice analogy. Like balancing the tires for a smooth ride, you will have to weight the different aspects for  smooth application development, deployment and maintenance. Improper weighting leads to a bumpy ride. In worst case scenarios, it leads to a flat tire.

Here is a breakdown of what the aspects (of development) on the wheel mean (in alphabetical order, not order of importance – more about order of importance later):

• Availability: The ability to get to the application to use it.
• Extensibility: The ability to add new features.
• Maintainability: The ability to change the code base easily.
• Performance: The ability to run quickly.
• Reliability: The ability to reliably serve data and persist alterations to data back to the data store.
• Scalability: The ability to add additional users without breaking the application
• Security: Ensuring only properly authenticated users can use the application and only see data they are authorized to see.

There is no one size fits all answer to what you put the greatest focus on. Just like a wheel on a car, you add weight to different parts of the wheel to balance it. Why not focus on everything? The problem is you can’t. Examples:

• Scalability often requires moving application bits across multiple servers, but server boundaries introduce network lag and decrease performance.
• Performance often requires very tight algorithms that only advanced developers understand, but complex algorithms make a solution less maintainable as you have to keep advanced developers on staff for maintenance (where they generally DON’T wish to be).

Performance is Overrated

I am possibly going to get in trouble for this statement, but stick with me until you finish this section before firing off comments. First a disqualifier. When I talk about performance being overrated, I am talking two things:

• In most apps (Google excluded?) you have more than enough horsepower in your system to perform the application with a simple algorithm
• In choosing performance, if both algorithms are equal in complexity, then performance IS NOT overrrated

The statement is a generalization from my years answering questions in forums. Many, if not most, of the questions in forums relate to one of two types of questions:

• Can you solve my problem? This generally means “fix the problem the way I am trying to solve it” rather than “fix the problem”
• Which of these runs faster?

The focus on performance is so heavy that people are often creating very complex algorithms to compare against simple constructs. When someone asks “should I run a binary compare or convert to strings first” I will ask “do the people maintaining the application understand binary comparison?” If yes, then go for the faster algorithm. If no, then you have to weigh if using a binary compare is necessary to achieve your performance goals (in one application I worked on many years ago, the answer was yes, despite the learning curve associated).

I should also state that on a personal level, you should learn complex algorithms. They can set you apart from the competition when you try to get jobs, are going for a raise, etc. I just don’t buy the “we have to use the fastest algorithm” argument in all applications. In fact, I would say I am generally against complexity except when needed.

Here are a couple of things I know to be true:

• Processors will get faster in the future
• Memory will get faster in the future
• The price of  new technology goes down over time, after the R&D expenses are recouped
• Memory, processors and even servers cost less than rock star developers

There are times when you will have to use complex algorithms, or even go to a C++ (or C) DLL to get maximum performance for you application. When you do, I suggest encapsulating the routine in its own library, where it can be maintained separately from the solution. This is not always possible, but it is a good rule of thumb. And,m if you program for Google, you might be writing complex algorithms most of the time. For the rest of you, weigh out your performance needs.

A quick story. In 1998, I worked for a company as a web developer (title: webmaster). The company was trying to determine the direction to go with development and I was asked to put together a brief presentation on the route I would suggest. I chose Visual Basic as the direction. My reasoning:

• There were tons of VB developers in Nashville (resource intensive)
• The cost of VB developers was lower than other languages (except perhaps mainframe programmers)
• The VB skillset translated well to ASP, which was our preferred web framework

When I presented the findings, one developer fought for C++. His reasoning? Performance. The stalemate was solved by paying a consulting company about $100,000 to determine the best course of action. They recommended VB. While I did not think of it at the time, here was our situation:

NOTES

• The red line is the slowest the app could run.
• The green line was the max we felt the app would need to scale over the next year (very aggressive number)
• The C++ figures were extrapolated from an app we migrated from C++ to ASP, so they are not necessarily accurate

As you can see, C++ performed much better. From a sheer performance aspect, it is a logical choice. But when we look at requirements of the application, C++ was not worth the cost, as the VB application performed well under the bar for the next year. Realize also, that we could have chosen a beefier application server or a web farm before choosing C++ and incurring the extra expense for developers.

Now, there are times when an algorithm choice is sane, even if you are not focused primarily on performance. Consider the following:

Code Snippet

1. public int Multiply(int a, int b)
2. {
3.     int returnValue = 0;
5.     for (int i = 0; i < b; i++)
6.     {
7.         returnValue += a;
8.     }
10.     return returnValue;
11. }
13. public int Multiply(int a, int b)
14. {
15.     return a * b;
16. }

The second algorithm is superior to the first, both on a performance standpoint and a maintainability standpoint. I would not choose the first, even though it solves the problem. But if I had to choose between lambda expressions and LINQ, I might default to LINQ, despite a slight performance penalty, as it is easier to teach LINQ than lambda expressions. Perhaps not the best example, as I love lambdas, but I think you get the point.

DEVELOPER NOTE: The difference between the two algorithms is largely based on the size of the number. If you are dealing with small numbers, there is absolutely no difference between the two algorithms, so the choice is largely a maintainability choice (and perhaps reliability, as complex algorithms are more error prone).

The Most Important Aspect

As I mention above, the application determines where you put the weight(s) to balance the wheel for a smooth ride. But, if I had to choose one aspect to focus on, it would generally be maintainability. Maintainability costs business more than any of the other aspects. This goes back to “programmers cost more than servers”, but also deals with the fact that an application spends more of its lifecycle in maintenance than development.

To understand the mindset of the majority, let’s look at a statement I have heard many times in my career (this is a paraphrase, the words change):

A simple problem has a simple solution
A complex problem has a complex solution

I see this a bit differently, as I tend to start with maintainability and then adjust based on the application needs:

A simple problem generally has a simple solution
A complex problem generally has one or more simple solution(s)

Does this mean I never write complex code that requires a rock star to maintain it? Certainly not. There are times that require very complex algorithms to fulfill the requirements of the application. In general, however, highly complex algorithms are not necessary.

Back to The Aspects

When I mentioned John’s concept of ARP, I said these were external concerns, or those related to the user interface. Here is a brief breakdown of all of the aspects when viewed from different people. I am using the typical concerns in the order I see them and not necessarily the order I feel they should be placed in.

The main takeaway here is ARP is very important in holding and keeping customers. A site that is unavailable, unreliable or slow must be extremely compelling to keep users using it. If it is an internal application, you have a bit of a fudge factor, as users HAVE to use the application, but these concerns should be weighed in. On the other hand, you need to worry about other aspects as well. As the architect, you have to think of all of these four groups and weigh which concerns are the most important. You never neglect any of the aspects, but like truing a tire, you weight them properly for a smooth ride.

Takeaways

Here are a few things you should get from this blog entry:

• There are many aspects to development
• Each application is different, so you have to weight the aspects of development for each application.
• Keep the skill set of the maintenance crew in mind as part of your weighing effort
• Your personal focus will often depend on which role you are playing at the time, but be mindful of other roles (esp. as an architect)
• Rule of thumb: Only go as complex as necessary to solve the problem projected out for the lifetime of the servers invovled
• Rule of thumb: When in doubt err on the side of maintainability – this will buy you more from management in the long run, although you often have to summarize the ROI to get the kudos
• Key point: Improper weighting of development aspects leads to a bumpy ride.

Hope this helps.

Peace and Grace,
Greg

Twitter: @gbworld

Future Topics Planned (will add links to topics when done):

• What is an Application?
• What are the Best Requirements Documents?
• The Software Development Lifecycle
• What Development Methodology Should I Use?