Best Practices: Organizing Development Projects

Today I am sorting through some problems with a check in (not mine, but helping determine root cause). I now have things down to a single error:

So, I know my problem is some source file is missing from the drive, but not from the project. This means an errant check in, so I am on track. The problem is I cannot find the source in Visual Studio very easily because the project has been organized into folders and this particular project does not fit the organization scheme.

So, this led me to write this entry about organization of Visual Studio projects. I have a few rules I follow:

1. Be consistent
2. Have the file system match the project organization (optional)
3. When you rename something in a file (class, project name), be sure to rename the files
   This includes class files, names of projects, etc.

You can live with just one of these, but doing both gives you a great leg up when it comes to maintenance. Now, I know we don’t generally think much about maintenance while we are building, but maintenance ultimately costs an business more time and money than spending a bit of time to make sure your file organized in a uniform and consistent manner.

NOTE: As an aside, I did find the test by searching for {COMPANY}.AppFramework.Services.Providers.ESTests. I found that the project was actually named {COMPANY}.AppFramework.Services.Providers.IntegrationTests. I will cover this in point #3. Below is the first line of the AssemblyInfo.cs file in the project {COMPANY}.AppFramework.Services.Providers.IntegrationTests:

[assembly: AssemblyTitle("{COMPANY}.AppFramework.Services.Providers.ESTests")]

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

1. Be Consistent

This one is a bit difficult to give a concrete set of rules for, as the nature of consistency really depends on the way your organize your projects. When I am building a project, I tend to organize into buckets based on functionality. The most common buckets (which correspond to folders in my Visual Studio solution) are:

• Core – The actual business logic that performs the behavior for the solution
• Domain – Models and exceptions used across many/all projects in the solution
• Experience – Projects to gain access to the functionality in the solution (User interface projects, service projects, etc.)
• Framework – Common functionality
• Persist – Functionality to persist state between sessions
• Test – Tests for the various projects

But this type of organization can be less effective when you further break down a very large project, as I am working on now. So, to the problem at hand. I think I am looking for an Enterprise Settings test project, which is part of the service model for the project I am working on, but it is not found in the solution at the correct spot. In section #3, I cover this a bit more, as was already mentioned. Here is the Solution Explorer:

2. Have the file System Match the Solution Structure (optional)

This particular advice need not be applied if you keep the file structure flat, meaning all of the projects are folders underneath the solution folder (default behavior). As soon as you start to organize the solution physically, you have to make sure the structure in the solution explorer matches, however. If you don’t it makes files very hard to find.

Since I organize by “type” of project (Core, Domain, Experience, Framework, Persist, Test, etc.), I find it just as easy to create the projects in the proper location. Now, I am quite religious about this when I am creating projects for user’s groups and conferences, as it makes it easier for the person downloading the code to follow along. I am not as religious in the workplace, especially if there is no standard for organization in place. This is primarily to avoid restructuring.

Why avoid restructuring? There are a couple of reasons, but they primarily relate to Team Foundation Server and Team System. I imagine the same is true for any source control system, but reorgs in TFS are easier done when you rebuild a completely new project with the new organization. The problem there is you lose history. So, if you are using a source control system, you are back to leaving everything flat or having a firm, largely immutable, standard that is followed.

3. Be consistent in naming

This one is dear to heart, as I currently have to do a refactoring of class names, as some of the names we have right now are confusing or way to similar to other names.

What do I mean by “be consistent in naming”? If you are going to change the name of a class, you need to change the name of the file. It makes things much easier when searching for a file when there is a problem in Visual Studio.

As an slightly off topic subject, you should also not have multiple classes in a single file. I say “slightly” as multiple classes in a single file presents the same problem as having the file named one thing and the class named another.

I am a bit lax on project names, which was the core of the problem I was having today. BUT … and this is a big but … you need to make sure any disparity makes sense and a standard is written. For example, the project MyCompany.MyProject.SubArea.Influence.CoreTests.MSTest.Integration.dll can be shortened without much problem, as long as you follow the same scheme in all of the projects.

Summary

The main takeaways for this post are as follows:

1. You have to take the time to refactor when you change things. This is not just refactoring of code to remove smells, but refactoring the project to ensure the naming is correct (to standard) and consistent (which the standard should cover).
2. You need to consider both the structure on the physical drive and the structure in your solutions when you develop the standard to help with takeaway #1.

Peace and Grace,

Greg

Twitter: @gbworld

SQL Server 2012 Released

Microsoft announced earlier today that SQL Server 2012 has been released to manufacturing, so you can now go and download copies on MSDN (developer edition here). If you do not have an MSDN subscription, the evaluation is available here. As mentioned earlier this week in my first Visual Studio 11 blog post, you will also want to download the SQL Server Data Tools from here (especially true if you are working with database projects currently and plan on moving up to Visual Studio 11 soon).

Peace and Grace,
Greg

Twitter: @gbworld

Performance Improvements in Visual Studio 11

I have been playing with Visual Studio 11 in my work lately and I am fairly impressed with the product. In line with Microsoft’s push to move most of the Framework bits to asynchrony, the VS 11 IDE now uses async throughout its stack. This greatly improves the apparent performance for many common tasks.

The greatest performance improvement seems to be when working with Team Foundation Server (TFS). In our environment, we have large projects stored in TFS that often take more than a minute to load before you can do any work. In VS, the initial load appears very fast. But, the intitial load and being able to work on files is a different story, so the performance increase is not quite what it initially appears. Please understand that I am not complaining, as the UI reacts much faster, giving me more time to be productive … but I do say you should take the apparent stellar performance with a grain of salt, as asynchronous does not mean you can start coding while the tasks run in the background.

Another nice feature is a smaller footprint. On startup, devenv consumes 80 MB instead of 220 MB. With our solution loaded, the memory footprint is also smaller, but only by about 15%.

When I get some time to actually instrument some of this, I will post some more information.

Peace and Grace,
Greg

Twitter: @gbworld

Visual Studio 11: Project Backwards Compatibility

As you are probably aware, Visual Studio 11 is currently available in beta (find it here). The release is at a point where Microsoft is confident enough it is solid they are providing a “Go Live” license, meaning there will be a supported update path from the beta to the RTM version and your code today will be supported so it works once the.NET 4.5 Framework is released as RTM. Over the next few weeks, I am going to dig into Visual Studio 11 and the .NET 4.5 Framework. Today, I want to talk about project compatibility.

Solution and Project Compatibility

If you have been in .NET for a while, you probably remember moving from one version of Visual Studio to another and having both your solution and project files converted. In cases where the project file was updated via a wizard, it was a one way street. For the solution file, there were some instances where simply changing the number in the solution file solved the problem.

I remember this most vividly moving from VS 2005 to VS 2008, as the core framework team migrated, but the UI team was unable to migrate due to a time crunch. Hindsight being 20-20, we should have delayed the entire organization updating. The solution was to have two sets of project and solution files.

I don’t remember the same pain for solutions in VS 2010. While the solution file number did increment, I don’t remember the solution breaking things (perhaps a slip in memory?). I do, however, remember the project files requiring update.

Visual Studio 11 to the Rescue

The problem is solved in Visual Studio 11, as both the solution file and the project files are compatible. You still cannot introduce .NET 4.5 features into a project and expect users of Visual Studio 2010 to be able to compile. But, if you keep the framework version set to 4.0 or earlier, you can use VS 11 while the rest of your team uses VS 2010, with no issues.

Project backward compatibility (VS 11 to VS 2010) is a highly touted featured for VS 11. This makes it easier for larger IT organizations/departments to upgrade to VS 11 in groups.

But there is a Gotcha

There is one issue with the upgrade: Database projects still require updates. For many of you, this is probably a non-issue, as you have no .dbproj files to contend with. If you use database projects, however, this is a big issue. Here is a shot of Visual Studio asking if you would like to upgrade; If you choose yes, you will end up with a .sqlproj file (more in a bit).

Fortunately, there is a workaround to the problem, and you can do something about it today, months before your teams start upgrading. The new database project uses a .sqlproj file for the project. This may not fire off any recogniztion, but the current tooling that uses this project type is SSDT, aka the Microsoft SQL Server Data Tools. You can see information on the project here.

If your database projects can be converted to use SSDT, the .sqlproj project file will roundtrip between the VS 11 beta and VS 2010 SP1 (SP1 is a necessity for SSDT). The download is through the web platform installer, found here, but you should read the brief release notes here prior to installation, especially if you have VS 2010 Pro or Ultimate and have not installed SP1 yet.

NOTE: If you attempt to open a data project in VS 11, it will prompt you to upgrade the project. You will have the option of ignoring, but this will exclude it from the solution. If you install SSDT prior to the attempted opening, you will be prompted to convert in VS 2010, as shown in the following screenshot (i.e. Same Result):

This request is to ensure you consider moving to the newer format so you can take advantage of the latest tooling surrounding data projects.

Summary

The good news is VS 11 is backward compatible to standard project types. The bad news is database projects are not compatible. But you should download the SSDT and upgrade your database projects anyway, so doing that now will help you avoid this snafu.

Peace and Grace,
Greg

Twitter: @gbworld

Config And Code and Bears! Oh My!

Where I am working right now, there is a need for framework configurations that are different from application configurations. I should alter that to say, it is not a need, but a desire. The main endpoint here is there are certain configuration elements only people who are very well trained in framework should edit, so a separate set of configs hides this from the end user developer. I am not sure I personally agree with this, but it is what it is. This comprises the first part of the “equation” that prompts this entry.

The second part of the “equation” is the use of IoC containers, in particular StructureMap, so different implementations of an interface can be spun up at run time. In the latest versions of many IoC Container projects, the container is populated using registries coded in a .NET language. You  then discover the many “registries” and spin them up at run time.

Let’s look at the two issues separately and I will try not to go ADHD and find too many shiny things to add to the entry.

Configuration

In .NET today, you have the ability to set up multiple configurations for an web application without any work on the part of the developer, other than adding the specifics to configuration. Out of the box, you have a configuration for debug and release, as in this web application:

It is a bit too bad you don’t automagically get this for all project types, but it is a foundation that can be built on. And, you can add additional specific configurations by simply adding new configuration definitions. Unfortunately, this has to be done at the global level, prior to creating the project, as there is no automagic “oh you have a new configuration and I need a specific config file” type of functionality. If you have just started a new project, you can delete the web config, create new configurations (Build >> ConfigurationManager) and adding:

If I then add a web configuration file, I see the new configurations:

Beyond the small bits above, the how of setting up new types, extending to other project types, etc., is beyond the scope of what I am talking about in this post. Just know that there is a system in place, at least for web application, and you can manually add to this system.

NOTE: The bits behind the multiple build configurations has more bells and whistles than this simple example. If you need this type of functionality, Google and you will find many examples.

On the current project I am working on, there are two concepts this does not solve.

1. Hiding internal (framework) configurations from the end user developer
2. Allowing customization of configuration at more than 2 levels

The end game looks something like this.

For the record, I am not fond of hiding configuration bits from end user developers. Sure, someone whacking configuration can cause problems that fall back on the team, but when the config is in the open, it is easy enough to say “didn’t I tell you NOT to do this?” … as long as you have it documented, that is. I don’t believe in using obfuscation through obscurity or hiding as a means of controlling “bad” behavior (was: technology does not solve process problems). As an analogy, is it better to hide a gun from your user, knowing a clever user might still find it, or send the user to a gun safety course? I vote for the later (neither will save you from someone “going postal” in their code, of course, but that is another story).

I am also not convinced that customization at multiple levels is a need. It is a nice thought exercise, but the actual use cases where it is NEEDED are very small, perhaps even non-existent. I am separating wants from needs, as there are probably others who WANT a multiple level “configuration conglomerator” piece. Assuming you actually NEED something like this, I can offer a few suggestions:

1. Consider moving the configuration processor to build time, so the combined configuration file appears in the /bin folder. If you want to obfuscate through obscurity, to hide elements from the developer, you can’t use this, but I have stated my feeling about hiding the gun, where he can still find it, over sending the user to a gun safety course.
2. Write out the end configuration, even if the configuration is completely pulled from memory at run time. if you have the file dumped, you have the ability to read through it and discover where the problem lies without searching or debugging. In fact, if you make a tool that creates config for all enviroments you can check before you deploy (defensive deployment – what a freaking concept!).

Coded Configuration

On the project, IoC containers are also used. For those not familiar with IoC containers, they are essentially a property bag of concrete implementations for an interface, with a factory to produce the concrete implementation. The general scenario is configure a variety of concrete implementations for your interfaces as the application bootstraps which are then used at runtime. Every IoC Container I know allows the developer to either explicitly implement a concrete implementation (by name, for example) or have it automatically get the correct instance based on configuration and rules.

We are using StructureMap currently, and it has the idea of a coded “registry” to set up the IoC Container “property bag”. Essentially, this means the bits you would normally have in a configuration file are now coded and included in your library.

The positive side of coded registries is you can offer multiple implementations out of the box, so the developer simply calls the correct one (based on how you have set up, so perhaps by name, but perhaps by another means – not important for now).

The negative side is when they have to be overwritten at the application level. It is not a big deal if you have one framework coded implementation that is overridden by a single application, but it can be overly complex once you start stacking different layers, as the config files were in an earlier picture. If anything is loaded in the wrong order, you can end up with a disaster. And catching these types of mistakes involves setting up your own instrumentation.

I am not fond of complex systems. I am less fond of adding multiple complex systems on top of each other. That leads to …

Code Versus Configuration Files

I was not going to jump on this when I initially jumped into this entry, but code versus configuration files is an age old question. In UNIX, files are the norm, even beyond configuration. In MS-DOS and early windows, you had the .ini file. As we started into the NT age, the idea of registry came about for configuration and in the COM age, the registry helped facilitate DLL hell. When we moved to .NET, we went back to configuration files. Now, many of the projects are going to coded configurations (like the registries in StructureMap).

ADHD moment: I personally can see an argument for both types of methodologies, but you have to set some rules around their use. In my world, a good delineation would be to use the registry to offer instances and then configure with a file in your application. Or perhaps, you are releasing a library as a product (internally or for sale) that contains multiple implementations. This is especially true if you don’t see the average user extending the system. If so, I would likely argue for configuration files again. NOTE: The specific examples above are unimportant to the post beyond giving a mental picture.

Back to our regularly scheduled show: What is dangerous is when you end up with part of your configuration in a configuration file and the rest in some coded construct. If you have to add a lot of flexibility to your configuration, then it is probably best to leave everything in configuration, with the shipped library as product being a possible exception. A single multiple file/step process is better than 2, especially if the processes are complex.

Disagree? I accept that may be true where you are, but I see this in practice where I am today. An application is working fine in staging and it goes to production and something goes to hell in a hand basket. If I have a domain expert, he might be able to say “that is config” or “that is IoC”, but a non-expert will have to step through things. And, when you have multiple complex systems you either force rollback or increase downtime. Ouch!

Actually, I will punch the baby for a bit. I am not sure I agree that going back to coded configuration is a good idea, except perhaps as packaged software. But, even with the packaged bits, I see Microsoft stating “if you add this, you have to add this to configuration”, so the precedence is “allow the user to customize configuration and guide them”. And, you can create VS tools or templates to add the bits, so it is not a huge deal.

I also find that hiding all of the bits to solve the package problem, but creating a huge extensibility and perhaps maintainability problem is a big mistake. And, I find many who move away from this direction do it simply because they don’t like to write documentation and guide users/developers. Is this really wise?

Note I am not against the registry concept completely, I just don’t see value in most of the scenarios I see it being used in. I see it adding more complexity for very little benefit, at best.

Summary

I probably sidelined a bit and this ends up more as a rant, but let me see if I can make some points that you might be able to take home.

1. If you find you are using multiple systems to configure an application, consider refactoring to one. In fact, seriously consider it.
2. If you are packaging a utility library with multiple implementations (perhaps a SQL versus Oracle versus MySQL), you can considering moving this out of the registry. Consider the extensibility likelihood in the mix, however, as you may make your software hard to extend in the process of making “use it out of the box” an easy scenario.
3. In everything in software, consider all of the needs before firmly committing to a direction, especially a direction that adds complexity.

Feel free to comment, as I would love other points of view. This is a random post and I am going into more of the “Core as Application” direction in the next series I will start shortly.

Peace and Grace,
Greg

Twitter: @gbworld

The Future is HTML5

Just recently, a list I belong to had a discussion on Windows 8 applications. At one point, the thread grew a bit philosophical and the possibility (or fact?) that HTML5 was the future was presented. This spurred quite a bit of discussion.

If asked, I would state that HTML5 is the future, but disagree with the statement “HTML5 is THE future”. As a developer, especially were I strictly a UI developer, I think it would be stupid to not learn HTML5 now and stay somewhat ahead of the curve. Why stupid? Multiple reasons:

1. HTML5 is the next rich web technology
2. With tools like PhoneGap, HTML5 is a decent mobile UI technology
3. With Microsoft using HTML5 for Windows 8, it is a UI technology that takes you to the most popular desktop

So, yes, HTML5 is the future. But this future is not necessarily an exclusive future. In fact, if it is, then the advanced developers of the world, especially the teachers, trainers and speakers, have done a piss poor job of mentoring up the next wave of development professionals.

Silverlight and WPF

When you have a discussion with Microsoft developers about HTML5, comparisons with Silverlight are inevitable. And, with so much overlap, I think Silverlight is dead, at least in its current incarnation. Unless Microsoft is willing to go head to head with HTML5 and make Silverlight a standout (a move they have not made to date), why would someone choose XAML over the markup in HTML5 when HTML5 is so much more portable? Rhetorical, as I realize those who have already invested heavily in learning Silverlight will hang on.

But, Silverlight could live on as a mini .NET library that the browser can access. In my opinion, this is the strength of Silverlight, even if it is not as sexy as XAML. For this reason, I think Silverlight is on a decline, as far as UI goes, although there is still a potential for life. Perhaps marrying the Silverlight underpinnings with HTML5 will keep Microsoft on the “smart client” side.

On to WPF. With Silverlight representing most of what WPF can do (it at least hits the 80/20 mark), I don’t see much of a future for WPF. If HTML5 had not already invaded the desktop, I might see a greater need to keep WPF around, but if the desktop UI, the mobile UI and the web UI can be expressed with one technology, why do we need WPF.

.NET and the rest of the “world”

Unless JavaScript can become a viable middle tier language, .NET languages are safe, which means it is safe to continue to learn C# or VB. Java is also safe. Both of these are “doubly safe”, at least for the near future, as so much time has been invested in creating applications and training developers. In the long run, it will depend on whether we continue to push SOLID principles, etc., for quality software.

If we focus on .NET alone, the web form paradigm is in danger if Microsoft does not introduce more HTML5 goodness for web forms. ASP.NET MVC has already embraced HTML5. I see WinForms holding on by a thread, at least until you can create a formed application as easily in HTML5 (Microsoft did a poor job with WPF in Visual Studio, so many did not take the plunge). Once we have tooling, even WinForms will have a difficult time staying afloat.

Long term, I see .NET, as a framework, safe, provided Microsoft properly positions it and JavaScript is not lifted as an Enterprise paradigm for all “tiers” (in quotes due to my assertion the application is the business logic and UI is a veneer on top – more in other blog posts). There is too much functionality to rewrite and JavaScript, at least in the near term, is a weak language for Enterprise application development beyond the UI. I think Microsoft has to figure out how to make .NET viable on the UI side (HTML5 specs give some options), so the next crop of developers don’t see markup, CSS and JavaScript as the only way to develop.

Final Words and Advice

As a developer, I think you need to learn HTML5. If you are a UI developer, this is doubly true, as HTML5 is for the UI. If JavaScript moves into the middle “layer”, all the better for newbies that start their learning with HTML5 as a UI technology, although the idea of JavaScript being the only language makes me cringe as badly as having vanilla as the only ice cream choice or apple as the only pie. Regardless of feelings, HTML5 is going to be around as a web UI technology. With Adobe’s help (PhoneGap), it will comprise a decent chunk of mobile UI development. With Microsoft’s help (Windows 8), it will comprise a decent chunk of Windows UI development.

Peace and Grace,
Greg

Twitter: @gbworld

Organization is Key to Development

I started out this morning looking at a the Test List Editor on a project I am working on. At the top were a bunch of broken tests with a few live tests. At the bottom were hundreds of live tests, but none were organized. There were 538 live tests and at least twice that many that no longer existed. Ouch!

Unit Test Organization

I am thankful someone pushed the idea of unit testing to the organization, but the lack of organization of the tests gives me pause. What if a developer only wants to run tests for a particular assembly? In the current setup, you have to organize the tests by project and then manually click each of the tests. This is a time consuming effort that has to be repeated each time you have to run the tests.

Fortunately, this nightmare can be solved in Visual Studio by simply placing tests into lists. You can then run the tests in isolation rather than running every single test in the entire suite. At minimum, I would focus the tests on area and break down so unit tests and integration tests (if any) are separated. Further separation may be needed to test specific functionality. Another possibility is starting with tests that run for a build and those that don’t. The important takeaway is you should organize your tests. Below is an extremely minimalist approach I took in a recent project.

In case the above seems a bit strange, I use a modified BDD style when testing. I say modified as I use the default MSTest bits and organize by behaviors. Why is a topic for another blog entry.

In short, if you are going to create tests (and you are, right?), you need to keep the tests organized. If you need an analogy, think of the test list editor like a library. Unless the books are ordered so you can find them, they are worthless. Every library needs a librarian, which brings me to the next topic.

Other Organization Needs

Another clear organization need in every development shop is a clean way of organizing documentation and other artifacts. One of the most popular tools for organizing content is SharePoint. And nearly everywhere I go I find something like this:

Pretty standard search and I find numerous items that have absolutely nothing to do with the topic I am searching for. In addition, I find multiple versions of the same document. The second issue can be cured, at least partially, by informing users how to open a document for editing and save back rather than save a new document with the same name each time, but you still need someone to keep up with the organization.

Summary

I guess the point of this post is anytime you have a library, you need a librarian. Without one, the “books” get moved all over the place until it is very hard, if not impossible, to find them.

Peace and Grace,
Greg

Twitter: @gbworld

Using Flags Instead of Arrays of Enum

This is a short post based on a problem I had today where I have to either loop through an array of enum multiple times to ensure proper ordering or actually tear the array apart. The problem, however, is very common.

Here is the issue, so you understand the context of this article. I have a service method that requires an array of a particular enum. The call is something like this:

var client = new ServiceClient();
var request = new TreeRequest( { more here} );
var purposesArray = { PurposeEnum.Systems, PurposeEnum.Accessories };
var response = client.GetTree(request, purposesArray);

This seems fine, if you examine it directly, as I can iterate through the enum values and run the service to product the hierarchy needed. But, there are two issues.

1. The enum values must run in a certain order, as the root of the tree comes out of a particular value
2. The current service call accepts multiple values, but only runs the first one

The second item above is not problematic from the enumeration array standpoint. I would have created a method with only one purpose and then added an overload when I was ready, but I am not completely opposed to preparing for the future (not quite true, as this particular future may never be, but that is a topic for a rant post).

The top item is the more problematic item. let’s illustrate. Suppose someone sets up the enumeration array like so:

PurposeEnum[] enumValues = { PurposeEnum.LastValueToRun, PurposeEnum.MiddleValue, PurposeEnum.ShouldBeFirstValue };

Now imagine this order is important (as it is in the service I have) and I have code like this:

foreach(var purpose in enumValues)
{
AddToTree(context, purpose);
}

Even if we assume an array is guaranteed to keep objects in order rather than take a place on the heap, I now have a useless answer returned to the client consuming my service, as I need to ensure the Systems enum value is first.

Worse than that, I have tightly coupled my service to the client, as the client has to understand my array is not really an array, but an array with a set of business rules dictating order. Ouch!

To decouple, if I want to guarantee proper ordering, I have to employ something that ensures the value with primacy is first in the array, like the following code.

foreach(var purpose in purposesArray)
{
if(purpose == PurposeArray.Systems)
{
newArrayList.Add(purpose);
}
}

foreach(var purpose in purposesArray)
{
if(purpose != PurposeArray.Systems)
{
newArrayList.Add(purpose);
}
}

PurposeEnum[] newArray = newArrayList.ToArray();

The problem here is I end up with a lot of code just to keep ordering. Nasty! Nasty! Nasty!

There is an easy solution when ordering is important … (fanfare please) … Bitwise operations. In the case of an enum, this means [Flags]. If the PurposeEnum was set up with flags, it would appear like so:

[Flags]
public enum PurposeEnum
{
Systems = 1,
Accessories = 2,
Warranties = 34
}

You can actually simplify the above a bit, but the way I wrote it shows that the flags are binary in nature (each item is the value of the previous times 2). This is important, as using binary allows me to specify order internally without regard to whether the programmer consuming my code knows the proper order to get results.

Now, I can have a person simply send me a single value, and I can order based on business rules:

if((purpose && PurposeEnum.Systems) == PurposeEnum.Systems)
{
//Run the first item to set the tree up properly
}

//ETC

Either of these is fine from the client side, as they yield the same integer value:

var purposeArray = PurposeEnum.Systems || PurposeEnum.Accessories;
var purposeArray = PurposeEnum.Accessories || PurposeEnum.Systems;

The order is dicated by how I consume the enumeration, not the order in which the client places items on the stack.

This is not the only use of flags, nor the best use of flags. It is merely something I noticed today where I have to think about ordering of an array.

Peace and Grace,
Greg

Twitter: @gbworld

Requirements and Acceptance Criteria Are Critical

Over the past few months, I have worked on a project with precious few requirements and no acceptance criteria. Essentially the dictate is done means “keep coding until I say it is fine”. The problem is this insanity never ends, as changes are placed into queue before the coding is “done”.

There are a few myths I have seen in action in various businesses. Here they are with the reality following:

• Agile means loose requirements and very little documentation – This is false. Actually Agile needs requirements and documentation as much as any other methodology, the requirements are just stated in the form of a user story and some of the documentation is in the form of unit tests.
• Business needs to be fluid, so we should not tie down “done” with a firm set of criteria – In reality, you need to define done as explicitly as you can. You may change (request) the definition over time, but a developer without a clear path is as likely to code away from the solution as code towards it.
• Hire a bunch of really smart developers and you can get around firm specifications and acceptance criteria – While very smart developers are faster at coding their way out of a hole, and less likely to code inescapable holes, there was only one real miracle worker that ever walked the face of the earth … and it is not your developer.

  NOTE that really smart developers often over-engineer or over-complexify solutions when left to their own designs.

We keep trying to view planning (with its outcome of requirements and a definition of done) as an optional exercise. This is generally done in the name of time, but how much time is blown having a developer rewrite code? In my experience, not defining what is being built has always turned out more expensive than spending time up front to plan. If you are truly overloaded, rather than avoid planning, you should force planning. Otherwise, you will spin at the end of the project, trying to “plan” after development, which means figure out how to get the code to do what I want with the least amount of hours. This is an exercise is low quality and relies on a lot of luck.

A couple of things I have noticed in my career:

• “Do everything this {website} does in a {mobile application} is not a requirement (you can change the {} types for your situation, if you like). As much as you might think it is a requirement, what is being done is up for interpretation, even if the basic business rules are the same, and the developer’s interpretation and yours are as likely to be different than they are to be the same. This is especially true when moving from one form factor (ex: web to mobile) or coding paradigm to another (ex: webforms to MVC).
• “Code this until I am happy with it” is a recipe for disaster. While it leaves a lot of flexibility for the person who wants to be “happy” to massage the outcome, it causes badly bolted on code as new rules are manufactured. In the lucky cases, the code can be refactored to some level of quality, but this is more of an exception than a rule. Ouch!

Let’s take a quick example I can think of lately. The requirement, as loosely stated, was put together a tree of categories for eCommerce and append products to the tree. Then create a detailed file for the products. The acceptance was “I will let you know when it is right”. Here is part of the history:

1. Add categories from another source
2. Exclude some categories
3. Replace images found in data source with a dictionary of categories and images
4. Use a template to create a tree where none existed and append to a specific spot in the tree
5. Create a featured products file
6. Append that file to a specific node in the main tree file
7. Exclude categories with no children or products
8. Scrub HTML for certain nodes in the XML document, replacing many of the characters with HTML escape sequences
9. Search for certain “illegal XML characters” and replace them (if acceptance criteria were built, it might have been as simple as convert from UTF-8 to UTF-7 – live an learn?)
10. Do a binary scrub of characters not caught with the loose ruleset in #9

All in all, numerous hours (weeks?) were burned due to the lack of planning on many levels, equating to tens of thousands of dollars. If this is the rule in the organization, and you add all of the projects up, how many hundreds of thousands, perhaps millions, of dollars is the company burning through that could be avoided by setting down requirements and a definition of done?

What if requirements change during development? If this is not the best reason to use Agile methodologies, I don’t know what is.But, even if you use waterfall, you stop development on the feature(s) in question and refine the requirements before moving forward. To either a) let the development continue or b) continue altering output until “happy” is like dining in a really crappy fine dining establishment … a lot of money for a really bad meal.

The point here is you can’t plan a trip without knowing where you are going. By the same token, you can’t build an application without knowing what done looks like.

Peace and Grace,
Greg

Twitter: @gbworld

Why Planning is Important

I have been working on a project for quite some time that is considered time sensitive. Due to the time sensitive nature, actually creating full requirements and acceptance criteria is considered frivolous, so the concept is code and verify, based on some unknown standard. It is not quite that bad, but when someone (generally me) argues for planning, I am told the following are equally valid ways of approaching development for this project.

1. Creating requirements and acceptance criteria (i.e. planning) and then coding to match the requirement and acceptance criteria.
2. Coding based on a loose, one sentence, “specification” and then fixing issues based on User Acceptance testing.

In short,  the group has opted for a “show me what you have and I will then tell what is wrong with it” versus “tell me what you want and I will show you what you have told me you desire”. While we ultimately end up with the same product, I disagree these are “equally valid ways” of developing a new application.

To illustrate this, think back to your last vacation. If you have not had one in a long time, you should go on one soon, but if you can’t imagine your last business trip. It went something like this:

You woke up at some time during the day (or night) and packed enough clothes for a week’s worth of time. Your spouse and children did the same thing. You then got in the car and drove to the airport and stated to the ticket agent “I have $1500 to fly and 4 people, so I need whatever ticket is the closest to $375 per person for a round trip”. Once you had your tickets, you got on the plane and flew.

When you landed, you went down to the car rental agencies and got a car, using the same logic. “I have $500 to spend on a car, so give me whatever car I can get for a week with that much money.”

You then drove off out of the airport and got on the road whichever direction fit your whim and drove until you found a hotel that seemed to be within your budget. You went up and got a room.

Once in the room, you looked at the local guidebook and figured out what types of things you could do in this town with the amount of day you had left and went out and had fun.

Does this sound even remotely like your last vacation (or trip)? No? Why not? Well, if you are a normal person, you can see all types of problems with this. Here are a few:

1. What if $250 only buys a ticket to Idaho, in the middle of winter? Did you pack for snow or the beach?
2. What if you can’t find a flight with 4 tickets available?
3. What if the car rental agency does not have a car that fits your budget?
4. What if the hotel does not have rooms available?
5. What if the town you landed in has nothing you are interested in seeing?

I am sure I can think of more, but the point is that you would not schedule a vacation out of town without some idea of where you were going to go and where you were going to stay, right? But you would be this careless with a “mission critical” project?

There is a belief in some circles that not knowing everything about the final product means you should not nail anything down. To me, this is about as stupid as stating “I can’t book a room at the Hilton for our vacation because I am not sure what I am going to order for breakfast, lunch and dinner every day we are on vacation”. Yes, there are some parts of the project you may not be able to figure out right now, but not planning anything as a result means you are going to, at best, spend a lot more time (i.e. money) to develop the product.

What has happened on my current assignment is I am now getting back pieces to fix. They are logged as defects, but without a criteria of what is right, nothing, outside of failure to work, is really a defect. And, since there is no criteria, “defects” are being found in serial. Take one step forward, find a “bug”, fix that one bug. Rinse. Repeat until all the “bugs” are washed out. The biggest issue here is we can, with any confidence, state when the final product is going to be ready for release, as there is no firm definition of what done looks like.

This topic has been covered as nauseum, of course. Steve McConnell has some interesting insights in his book “The Project Software Survival Guide”, which I think should be mandatory reader for every manager that needs IT work done.

From McConnells’ company site, I have found a couple of neat gems from his book. The first talks about when a defect is created and how much it costs to correct. This is shown below.

If you do not have time to write requirements and work out the architecture and design, every defect is a requirements defect. Pretty expensive proposition, yet this is precisely what is being done.

Here are a set of images from Steve McConnell’s own site that describes perception versus reality. Management often believes this is the optimal development path:

They accept that there is a certain amount of unproductive time (meaning time not moving forward towards the end of the project, whether reworking bad code or going to get coffee), but they don’t want the burden of planning, which is seen like the picture below:

If I get rid of the planning (process), I end up with more productive time. The problem is the reality looks more like this:

Without proper planning, what started as a small amount of thrashing, ends up consuming a huge amount of time, as more and more “defects” are discovered and more and more code has to be reworked based on differing understandings of what is acceptable and what done look like. In addition, more and more time gets consumed in process, including planning how to get from “what we now have” to “what is acceptable. One thing missing from this diagram that is present in the book is the dot just before process and thrashing consume 100% of daily efforts. That dot indicates the luck projects that get released before all of the participants drown in the sea of non productivity.

The ideal situation is enough planning to understand both what done is and how to determine each feature is acceptable to fit that picture of done. This is shown in the final graphic linked from McConnell’s site.

In short: planning works. If you want to read McConnell’s entire treatise on this topic, click here.

My next topic, which I will post in a few days, will cover why “hiring rock stars” is not an adequate alternative to proper planning.

Peace and Grace,
Greg

Twitter: @gbworld