Architecture: The CASA Model- Getting to the Core


About 9 years ago, I was looking at code from a client and dealing with how hard it was to maintain compared to new code I was writing. In the process, I started seeing patterns in the code that I saw in numerous code bases I had worked in over the years. In particular:

  1. Most code was not organized well for maintainability
  2. Best practices were known by all developers, but few were practiced
  3. Very little automated testing
  4. The systems were not domain focused
  5. Switching from one user interface (say windows forms application) to another (web) required a lot of work, as the developers created a new application.

One issue that led to this mass of spaghetti code was the n-tier architecture idea, which gave people the principle that an application is made up of a user interface, business logic and data. Thus, a windows application and a web application were two different things. Naturally, since they were different things, you had to build them separately. In reality web application and a windows applications, as we know them, are not applications. Web and windows are two means of presenting applications. 

That simple paradigm shift lead me to start considering methodologies used for development, so I created what I thought was a new, or at least varied, methodology. I called it Core as Application. The original premise looked like this.

 

Core As Application, circa 2010

In this model, the Core IS the Application. This means an application is made up of domain objects (State) and actions on the domain objects (Behavior). In short, software is a bunch of BS? 

To facilitate the application, I need a means of presenting information to an interacting with users (May be human users or machine users). And, I need a means of persisting state between sessions. I should be able to switch presentations and persistence mechanisms without writing a NEW application, as the Core IS the application. 

NOTE: Core AS Application was a progression of Core Is Application so I could have a nice acronym.

There were a couple of things I did not realize at the time.

  1. Core as Application is NOT a methodology. It is a model. 
  2. Other people were working the same model at the same time, although focused very much on the architecture. You know them as Onion Architecture, Hexagonal Architecture and Clean Architecture

On point #1, I refined the concept to the Core AS Application model (CASA for short, since it needed a cute name). On point #2, all of the “architectures” mentioned deal with the same concept: domain models are the center, followed by business logic. I can can cover these in another post in the future.

Before moving forward, let’s cover a few terms as I envision them right now in CASA. The definitions are oversimplifications to an extent, but help keep you, the reader, on track.

Capability: A business capability is a function or service a business provides to its clients, internal and/or external.  Applications should be centered around capabilities and systems around 

Application: The software solution providing centered around a business capability.  In CASA, the application will be a single core with contracts separating it from presentation and persistence mechanisms.

System: A program or set of programs to solve a business problem that can span multiple capabilities. System can be the same as an application, in simple business problems, but can also required integration with other applications. As an example, an application that connects to multiple services to accomplish its work is a system. In CASA, if you see more than one core, it is an integrated system.

Solution: Solution is used here to indicate a packaging of separate projects to create an application. In this regard, it is synonymous to a Visual Studio Solution. A project will be a single Visual Studio project. 

In this post, I want to cover CASA as it exists today and why it works well in development.

The CASA Model 2018

The general principle of CASA is the application is the business logic at the center of the model. This means I can switch presentation and persistence without changing the application. I sit this core (behavior) on top of domain models (state), as the domain is the foundation of the application. I place contracts on either side of the Core, as I want to facilitate testing, which sits on top (the Contracts serve more than just aiding tests – to this in a moment). And testing, primarily automated, sits at the top. The model looks like this today.

There are a couple of differences between this model and the original.

  1. The domain is no longer part of the core, but the foundation. 
  2. Contracts have been added between the core and the persistence mechanism (in general, this will be a database)
  3. Contracts have been added between the core and the presentation mechanism (normally a user interface, but could be a service of some other machine to machine mechanism).
  4. Tests have been added on top.
  5. Adapters have been placed next to the contracts. 

In general, each of the various boxes will be contained in separate projects in a solution (assuming .NET, but the concept can be used in other programming paradigms). Here is a sample solution showing how this is used. 

For a better understanding of the value of the model, let me tie it back to some other industry concepts you might be familiar with, some of which were included in the design of the CASA model: Domain Driven Design, Contract-Fist Development, SOLID Design Principles, DRY, YAGNI and KISS, although some of the concepts are implicit rather than explicit.

  • Domain Driven Design – I don’t use all of Eric Evan’s domain driven principles, as I am rather stringent on separating my BS (behavior and state). I have 2 common domain projects in every solution: one for models and another for exceptions and I do not include behavior in a domain object, unless it is related to the object. As an example, an address may have a ToString() method that delivers a concatenated address – this is behavior, but specific to the object.
  • Contract First Development – The concept of writing concepts prior to developing is not new. In C++, you create interfaces first, in the form of .h (header) files. Delphi, which may, in some ways, be considered the inspiration for C#, also had a contract first approach. With REST APIs, the contract is a bit more implicit and found in the URL. For CASA, the idea is write the contracts first to ensure you understand the seams in your application.
  • Test Driven Development – I am definitely one that believes writing tests first helps you ensure your code is focused on what your users desire. I use code coverage metrics with my teams, but it is just a measure not a goal. 100% code coverage with bad tests is as unacceptable as very low coverage. I should also note acceptability is the real goal, not writing tests. 
  • SOLID Design Principles – Organizing code into specific areas of concern cover the (S Single Responsibility), while contracts handle the I (Interface Segregation) and aid the D (Dependency Inversion). The O (Open/Closed Principle) and L (Liskov Substitution) are not directly addressed by CASA, although using interfaces will help you think twice about breaking contracts, especially when you end up with multiple classes utilizing the same interface.
  • DRY – Do Not Repeat Yourself. This is not inherent in the model, so you have to create a process to handle this, as well as governance. For the process, when you find repeat code in multiple classes (or worse, projects), you should start refactoring ASAP. A couple of normal methods to employ would be to move the common code into a base class (two classes on the same interface, for example). Or you move the code into a separate class and/or project and reference it. As for the governance, if you are not using code reviews as part of your development process, you should. 
  • YAGNI and KISS – Also not directly covered in CASA, but you find the separation of the code makes it much easier to solve problems through testing and focusing on one problem at a time. Once you grasp the concept, you will find it easier to keep things simple and avoid bloating your code. If you do try to think ahead, you will find the model fights against you. In the future, I plan on talking a bit about testing with CASA and will add the link here.

Expanding CASA

As I started examining the model, I started thinking about solutions I had seen in various Enterprises. The area CASA seemed to fail is when you added a services layer to the application. First, some developers had an issue with the paradigm shift involved with thinking of services as a presentation concept. A bigger issue is what happens when you add a tier of services, as shown below:

I adjusted this by moving services to its own folder in the solution. This helped with the paradigm shift. Recently, I have found this also works with mobile applications, as you can use the same persistence library for the mobile application and the server side application. 

I should note, the service holds one application and the user interface, which used the service, holds another (at least potentially). So I had two cores, which means another solution. From an organization standpoint, you have two applications. When I add a second service, it becomes more evident. If you focus on microservices, which are mini applications with a service front end (yes, a simplification), it is absolutely clear an application with a service, layer or not, is actually multiple applications. I am not one for having too many projects in a solution, but as long as you can develop without massive compile and test cycles, it is a place to start. Solve then refactor.

From the standpoint of the model, it became clear I simply needed to chain two cores together with a service, which looks like this:

Moving to Mobile

In the past 6 months, I have gotten heavily focused on mobile presentation in React and Xamarin. As React cannot be directly compiled using Visual Studio, I won’t cover it here (I actually think I might be able to solve that problem, but that is another story). Xamarin is a bit easier, but changes a few things in organization.

In particular, with Xamarin, you end up having to compile at least parts of your core for each platform, especially in an offline application. You will be persisting to both the service (server side) and the mobile database from the mobile device. And to a server side database when the mobile application contacts the server to persist.

  1. Create a subfolder called Mobile under the Present folder. This is where the Xamarin.Forms project and the iOS and Android specific libraries go (I will likely rework this in the future, as these are not technically presentation libraries, but utilities).
  2. Make sure any libraries that are used on the server and the mobile device are .NET standard. NOTE: I think Core can work, but standard provides a few other options. 
  3. If you require any functionality that uses the full .NET Framework, ensure it is behind a service, as Xamarin cannot compile the full framework to native code.
  4. To facilitate the mobile application saving locally and on the server, configure it to use 2 implementations of the same contract: one to save locally and one to save via the service. The service will use one implementation of the contrract, of course.

I currently have the service in the same solution as the mobile application. When I separate it out as a separate product, I will have two solutions pointing to the same contracts for persistence. This may necessitate placing the contracts as their own product to avoid development in one solution breaking the other. I think the team is small enough to not jump either of these hoops right now (NOTE: I envision an entry on product mindset this week and will link it here). 

So, here is the solution at present. (NOTE: This is a work in progress, as I am still developing this. Thus far it is working)

This seems awfully hard?

Any time you switch paradigms, it is hard. For years after .NET was released I saw developers coding standard VB in VB.NET projects, as they had not made the paradigm shift. Microsoft made it easy to avoid making the shift by adding training wheels to VB in the form of the library Microsoft.VisualBasic.Compatibility. During the early .NET years, I used to recommend switching to C# as the paradigm shift would become part of learning the new language syntax. 

Once you get the paradigm shift, however, you will notice it becomes very easy to find code when you discover a bug. Part of this is in the organization the model provides, but a good deal of it is in learning the concept of a domain (which is outside the scope of this post).

Does it Work?

I have yet to see a case it does not. But I am also not arrogant enough to state you must use CASA to get the benefits. Below are the design principles I am using. If you already adhere to all of these principles, you might think twice before adopting what I have written here. Some of these principles go beyond the concepts in CASA and will be covered on this blog later.

  • Domain Driven Design – Focus on the business problem you are solving. Domain Driven Design will aid you in this principle. Your state objects should be named in terms familiar to your business.
  • Contract First Development – Understand the seams in your application before coding anything around the seams. To simplify, as you deal with different technical concerns, consider it a seam. In CASA, there is a contract for persistence, one for the core and one for any dependency that can be swapped out, or capability. If you ever feel another team could write a separate implementation, you write a contract. Why first? Because, like Test Driven Development, you learn about your code before you code it.
  • Requirements – Write requirements for acceptance. This is a pre-step for testing, as understanding what is acceptable will make it much easier to write a full enough set of tests.
  • Test Early – I am not a 100% purist in test driven, but I am a stickler for writing a test first on any problem you are not 100% sure of (which is almost everything) and adding tests on top of those that you are. (NOTE: Over time you will find some things you were 100% sure of at first, you should have written the test first, as you were cocky in your confidence). One more point: Acceptability should be a goal for all testing, not just acceptance testing. In standard TDD style, this means writing more test methods. In behavior driven, it means acceptability in your specifications.
  • Coverage – Code for acceptability, not percent. Percent is a good metric to find areas that might be lacking, but never incent on code coverage, as humans are very good at being creative to get incentives. Good coverage includes the following: 1) covering business use cases completely to ensure user acceptance 2) covering unacceptable cases 3) covering the fringe. I will focus on this in another post (and link here).
  • Testing errors – If a bug is found, or a lack of acceptability is discovered, you immediately write a failing test. If you cannot write a test that covers the condition that FAILS, you do not understand the bug well enough to fix it, so you SHOULD NOT try to fix it. (NOTE: lack of acceptability is not the same thing as a bug. If a user discovers something does not work how he or she intended, it is not a bug, but a change in understanding of what is acceptable – same process, but don’t have your team take the blame when someone realizes their vision does not work as they would like).
  • Build the tests up from the unit – If you are focused on covering for acceptability, you will find the exact same tests you run as unit tests are often run as integration testing. With contracts, you will mock the contract for your unit test and use a concrete implemenation of the contract for integration tests. If you use behavior focused tests, you will find you are running the same tests, just changing dependencies (did that blow your mind? If not, you are already in the Zen garden). Because I see the persistence store (database generally) as the castle that protects the king (data is king for business), I write an integration test on this early on. I might write it first, especially on non-user interfacing persistence applications. But I sometimes unit test the presentation first to ensure I understand what the user intended and then go to the database. Knowing how the user will interact with the system and how I store (optimized data store, not just a CRUD mock of the business objects), I can mock the persistence layer and unit test all core business functionality. If what I just wrote sounds a bit confusing, I will write a blog entry about this soon.
  • Product Mindset – This is the most complex topic (and deserving of its own blog entry – link here in the future). A few things I recommend. If you find you are no longer actively developing on part of your code, consider making it into a product. If you started with CASA at the start, these will likely end up being utility classes that you want to reuse, but this technique can also serve well if you have not use CASA, or similar, in the past. The main point is once you find a capability, add a contract and move the code to its own project, then to its own solution, and finally put it in your own NuGet repository. (I guess I also need to talk capability thinking in the future? – link here in the future).
  • Agility – This really has nothing to do with CASA, at least not directly. But Agile gets you thinking in small bits of functionality at a time, which works very well with CASA.

In Summary

In this entry, I focused on the Core AS Application, or CASA, model. CASA is a model that focuses on a visual representation of code organization with a focus on best practices, like Domain Driven Design, Contract First Development and SOLID Design Principles.

In the future, I will break this down further, focusing on how this works with various concerns, such as Requirements, Testing, Product Focus and Agile Development.

Peace and Grace,
Greg

Twitter: @gbworld

Automated Testing is NOT optional


Once upon a time there was a young apprentice who did not understand why automated testing was so critical. As he was under pressure to churn out magic for an evil wizard, he felt he did not have time to test his spells. In addition, he did not know how to tell if the spells worked correctly. Thusly, he randomly Harry Pottered the incantations and hoped for the best. As he had learned not to fry off his own appendages, he felt this fire and observe method was adequate and did nothing to change.

A medieval tale? Nay, I say. This is today. And I am finding despite years of talking testing, and it’s variants, at conferences, on blogs and even in video courses, so many people still don’t get it. I will distill why automated testing is important and then see if I can drill into some details and get you back on the right course. And, if you are already doing everything I state in this post, then find a place with two for one spells and that second beer is on me. 😀

A cautionary tale

In this story, our hero has inherited an app. Well, not exactly, as the app has to be stabilized and released prior to takeover. The only thing standing in the way is testing. And, since there is precious little automated, the testing is manual. And slow. And error ridden. And … okay, I will stop there, as I think you get the picture.

In each testing cycle, as things are fixed, the testing gets deeper and deeper, and finds more bugs. These bugs were not caught in the original runs, as the manual tests were running into surface bugs. As the surface bugs disappear, the testers has time to start hitting fringe cases. And, as these are wiped out, the testers start testing at the same time and run into cross account bugs, meaning one person sees someone else’s data. Along the way, you see circular bugs, where fixing one brings up the other and vice versa.

Sound familiar? Every single one of these problems can be solved by automated testing. Every single one.

But, Greg, automated tests do not solve this problem. You still have to go after surface bugs (those on the happy path) and drill deeper.

True, but once I write an automated test, it is part of my library of tests which I can run over and over again. Yes, I can do this by creating test plans and checklists and manually testing, but – and here is the key – I have to rely on discipline to ensure all tests are completed each time. Plus, the automated test session is a few minutes in length (seconds, or less, in the case of unit testing) whereas my manual testing session may take hours. I have been playing this game for the past week and I have spent more than 8 hours, plus 8 hours of another employee’s time, to manually test a solution lacking automated tests.

Yes, there is the other side of the coin, which is the time necessary to write the tests. Initially, you take a productivity hit writing tests, as it takes time. They pay back dividends when you get to maintaining your solution, however.

Here is what I have found in the past week (all of which could have been found through automated tests):

  • Buttons not wired up – UI testing
  • Items that do not delete – Unit testing
  • Items that appear to delete, but stay – UI testing
  • Validation failures – Unit tests (backend) and UI testing (front end)
  • Items that cross accounts – Unit testing and/or UI testing
  • Slow reports – performance testing
  • Autosave issues – JavaScript unit testing

Every single test above could have been automated. Let’s start there and I will talk about maintainability and acceptability first.

Lions and Tigers and Test Frameworks … Oh My!

One of the first questions I get asked is “which test framework is best?” My answer: YES.

Honestly, I don’t care what test framework you use. I most often use MSTest, which is included with Visual Studio. I sometimes augment with a BDD framework like SpecFlow. But, I have worked with NUnit, XUnit, JUnit and LMNOP (which stands for “insert your favorite here”). I can tell you strengths and weaknesses of some of them, but I have yet to find a case where one of the ones I have been using for some time just sucked. Start somewhere.

In your toolbelt, you should consider, at minimum:

  • A unit test framework – in .NET, this will likely be MSTest, NUnit or XUnit – as mentioned, I generally use MSTest. Wikipedia has a nice listing here which covers a plethora of languages.
  • A test runner – Visual Studio has one built in. Resharper adds its own, which has some nice features (you use Resharper right?). NUnit comes with one, etc. In other words, don’t sweat the small stuff.
  • A mocking framework – I bounce between RhinoMocks and MOQ, but this has been a customer driven decision.
  • A JavaScript unit test framework – Unless you never do web or hybrid mobile, I would definitely add this. I happen to gravitate to Jasmine, but have played a bit with JSUnit, Karma and Mocha.
  • A user interface focused test tool – Selenium is the main open source tool for this type of testing. I have picked up Katalon recently to test, as we are on Mac and PC now. Katalon sits on top of Selenium and works in Chrome, which was a requirement for us.

A few other things you might be interested in

  • A BDD Framework – I use SpecFlow in .NET, but there is a bit of a learning curve. Plus, you can write behavior focused tests without making the BDD paradigm shift (see later)
  • An acceptance test tool – I happen to be fond of Fitnesse, as you can set it up with either Excel or a Wiki and let business add test conditions. It gets a bit strange when you tests magically fail because they have edited the wiki, but it sure shortens the feedback loop. Please note you can test for acceptance without a separate tool by making sure you identify all of the conditions
  • Some type of static code analysis – Visual Studio contains some tooling, which you can accentuate with extensions like Microsoft Code Analysis. I also recommend having static code analysis as part of your delivery pipeline if you use Continuous Delivery disciplines.

It is also nice to have a means of getting to fringe conditions in your tests – Intellitest in Visual Studio Enterprise (the evolution of Pex) is a great tool for this (Pex and Moles were tools from Microsoft research that were included in earlier versions of Visual Studio).

Well, I don’t accept that

What are you trying to get from automated testing? When I ask this question, the general answer focuses on quality code. But what is quality code? Certainly bug free, or at least critical bug free, is a must. But is that enough? I say no.

When we look at development and maintenance, maintenance is the long pole. The better your tests are, the easier your solution is to maintain. This should be obvious, as the tests provide guides to ensure any maintenance fixes avoid creating additional bugs you now have to conquer (think about those circular, fixing A causes B and fixing B causes A, types of bugs).

But I think we have to go beyond quality alone and think in terms of acceptability. This means you have to ask questions to determine what behaviors are acceptable. More important, you have to ask the RIGHT questions. Below is a sample Q&A with a business users.

User: A person can enter a number between one and one hundred in this box.

Dev: What if they put in zero?

User: It should warn them.

Dev: Same with negative numbers?

User: Yes, and everything over one hundred as well.

Dev: Can we make it a drop down so they can only pick one to one hundred?

User: We find drop downs get too unwieldy for our users when there are 100 numbers.

Dev: And no letters, right?

User: Exactly.

Dev: Can we put in something that ignores when someone types in letters in that field and text like the user name and password fields saying 1 to 100.

User: Oh, both would be great.

From this conversation, we understand that acceptance includes a user interface that stops a user from entering anything other than the integers 1 to 100. For many developers, adding constraints to the user interface would be enough to ensure this. But one should understand the business code must also contain checks, because code should never trust user input.

As you go through this exercise with your end users, you will find acceptability is beyond quality alone. It deals with perception, as well. And users will help you create a more comprehensive test solution. In summary, think both about maintainability and acceptability as you.

Software is a Bunch of BS

Some days I think of male bovine output when I see code, as some of it really smells. But when I talk about software being BS in conference talks, I am thinking about the dual aspects of code, which are behavior and state.

  • Your users are most concerned with state. When they pull out a record and change it, they want to be confident the new data is placed in the database, so they can find it in that state during their next session.
  • Your dev team is most concerned with behavior, or what they do to state (domain objects) as a user interacts with it. The behavior should result in the correct state changes.

This is why I recommend testing that focuses on behavior. I have, in the past, used a tool called Specflow when coding in .NET languages (SpecFlow can also be loaded directly into Visual Studio through extensions – I have the search here, as there are different extensions for different versions of Visual Studio. 2017 is here). SpecFlow requires a unit testing framework to work and works with many (MSTest, nUnit and xUnit are all included, although xUnit requires a separate NuGet package).

The upside of SpecFlow, for me, was when I pushed code over to teams in India. I could write the cucumber and generate the scaffolding and have the use BDD to create the code. For the first week or two, I would correct their tests, but once they got it down, I knew I could create the test scaffolds and get the desired results. The downside was the learning curve, as writing tests in cucumber takes some practice. There is also a paradigm shift in thinking from standard unit tests.

If SpecFlow is a bit too much of a curve right now, you can write behavior tests in a unit test framework by moving away from a single “UserTests” class with multiple tests to a single test per behavior. I will write a complete entry on this later, but the following pattern works well for behavior (I have the template outline in bullets and then a very simple example).

  • Class is named for the behavior being tested. In this case the class is named “WhenMultiplyingFiveTimesFive”
  • The code that runs the behavior is in the Class Initialization method
  • The variables accessed by the tests are kept in private static variables
  • The tests contain expected values and assertions

And here is the class.

[TestClass]
public class WhenMultiplyingFiveTimesFive {
    private static _inputA = 5;
    private static _inputB = 5;
    private static int _actual;

    [ClassInitialize]
    public static ClassInitialize(TestContext context) {
        _actual = MathLib.Multiply(_inputA, _inputB);
    }

    [TestMethod]
    public void ShouldReturn25() //This can also be ThenIGet25 or similar (cucumber)
    {
        int expected = 25;
        Assert.AreEqual(expected, _actual, string.Format("Returned {0} instead of {1}.", _actual, expected);
    }
}

The downside here is you end up with a lot of individual test classes, many with a single test. As you start dealing with more complex objects, your _actual will be the object and you can test multiple values. On the positive side, I can  generate a good portion of my tests by creating a simple program to read my domain models and auto generate stub test classes. It can’t easily figure out the When condition, but I can stub in all of the Should results (i.e. the tests). If you opt for the SpecFlow direction, you will generate your scaffold test classes from cucumber statements, like:

    Given I am Logged In
    When I Call the Multiply method with 5 as A and 5 as B
    Then it returns 25

Or similar. This will write out annotated methods for each of the conditions above, which is much like a coding paint by numbers set.

Process This

I will have to draw this subject out in another blog entry, as testing discipline is very important. For the time being, I will cover the basics you should include the DOs and DO NOTs of a good testing strategy.

  • DO write tests for all behavior in your system
  • DO NOT write tests for domain objects. If the language vendor has screwed up getters and setters, you have a bigger problem than testing. One exception would be domain objects with behavior (although in my shop you would have to have a really good argument for including behavior in state objects)
  • DO ensure you have proper test coverage to match acceptability
  • DO NOT use code coverage metrics as a metric of success. 100% coverage with crap tests still yields crap software.
  • DO ensure every bug has a test. DO write the test BEFORE you fix the bug. If you cannot write a test that goes red for the condition, you do not understand the bug. If you do not understand the bug, you should not be attempting to fix it.
  • DO ask questions of your users to better understand the user interactions. This will help you create tests focused on acceptability.
  • DO look at fringe conditions and conditions outside of the box.
  • DO add tests prior to refactoring (assuming there are not adequate tests on the code at the time).

None of this is automagic. You will have to instill this discipline in yourself and your team. There are no free lunches.

With legacy code, prioritize what needs to be tested. If you are replacing certain sections of the application completely, writing tests is probably a waste of time. Also focus on the things that need to be updated first, as a priority, so you have the tests completed. I should note you may have to reorganize a bit prior to adding tests. This is dangerous. But, using refactoring tools (Visual Studio or Rehsharper), you can lessen the risk. You need only pull things out far enough you can add unit tests, so extract method is your best friend here.

Summary

If you have never read the book The Checklist Manifesto, I would advice considering it for your bookshelf. The book focuses on how checklists have been used to save lives. Literally! Once you see the importance of a checklist, you can start thinking about automating the checks. Then, you will understand why automated testing is so important.

In this post, I focused on the very high level, mostly aiming at what the developer can do up front to greatly increase quality AND acceptability of the solutions coded. I strongly adhere to the idea of testing as much as is possible. Once I see a pattern in the tests, I generate them (often with Excel, which is a whole story in and of itself). But I focus on ensuring every condition I can think of is put in a test. I will use code coverage as a means of discovering areas I don’t have tests in. This does not mean I will create a test for every piece of code (domain objects being a prime example), but that I use the tools to help me help myself … and my team.

Peace and Grace,
Greg

Twitter: @gbworld

Why EF Code First Sucks … and how to fix it if you must use it


NOTE: EF is a tool. As with all tools, it has pros and cons. The headline is designed to get you to examine your useage.

I am a contrarian in many cases. I look at what is in front of me and ask one of the following questions:

  • When would I use this thing I don’t like
  • When would I not use this thing I do like

By constantly questioning, I find myself being a good devil’s advocate. It also keeps me focused on solving problems the “right way”, which means adding in the variables as I find them and validating my knee jerk reaction over and over again. Sometimes this means not adopting the latest FUD, no matter how awesome it is, as it will cause too much disruption (read: too much risk).

As a developer, I think you should explore everything you find and constantly be curious. As an architect, and now a manager, I think you have to take in account how many moving parts you are introducing before making a decision to add something new. I also am very strong at examining pros and cons before striking out. This does not mean I don’t make very quick decisions at times, but that I strive to always do at least some research before firing up the engine and traveling away from home.

For me, Entity Framework makes things easy and much faster to develop. But, it also puts you in a box, like all software designed to develop (even Visual Studio puts you in a box, just larger). There are some projects that fit in the box, but some that do not. As I work on mission critical applications, in an Enterprise setting, I find the maintainability and acceptability of the solution to outweigh time to market in most cases. This makes EF less attractive to me, especially since most EF developers I have met understand the basics and not much more. When you build software any idiot can use, I can guarantee you some of the people using it are idiots. The easier it is, the more idiots.

In particular, this article is focused on issues I am having with a current project I am testing. The problems I have found are not unique, however, as I have seen them in other instances. In this case, there are a few additional issues that exacerbate the problem. And that takes us to “why this post?”

To put it plainly, if you are going to use EF Code First, don’t use the defaults. Let’s dive in.

The problem with EF Code First

Let me be fair. This is not really a problem with EF Code First as much as understanding how to use it correctly. The issue I see is so many people DON’T know how to use it correctly. In fact, it seems to be a one-size fits all tool for so many people. And a great number of them are ignorant about the pros and cons of the product and the cause and effect nature of the issues it creates when used in default. And, as mentioned in the last section, some are idiots. 😉

Auto-generation is attractive. I utilize my skill in coding domain objects, push build and deploy and voila (wah-lah for those who hate French?). Now I have a site. If I just went with defaults, however, I have all of the following problems.

  • My database looks just like my business objects.
  • My interfaces to my data are CRUDdy (simply create, read, update and delete on tables that simply reconstitute my business objects)
  • My database is hard to optimize if looking just like my business objects is not the optimal way to solve it.
  • If I use GUIDs, which are great for certain scenarios (multiple live servers, creation of IDs on disconnected devices, etc.) the default clustered index can create issues down the line (mostly fragmentation)
  • If I use strings in my business objects, they are NVARCHAR(MAX) in my database
  • By default, I have no constraints (other than the auto-generated primary keys). This means no foreign keys in my database.
  • Filtering is done on the user interface.

Once again, I fully realize much of this is ignorance on how you SHOULD design your application with EF Code First, but with no warnings or guidance in Visual Studio to stop the developer from doing this, I am finding this type of bad EF Code First design is more the rule than the exception.

NOTE: I use the word business objects rather than domain models in this post, as I think the lack of domain focus exacerbates this problem. I will write about domain models in a future post and why domaincentric design (I use the made up domaincentric word to distinguish from a purist Domain Driven Design approach – see: Eric Evans).

Annotating with Attributes

I can cure most of the ills with attributes on my business objects. This includes putting keys of the correct type, adding max lengths and avoiding naked strings on objects. I will go through each of the issues I have discovered (not a complete list).

  • Clustered keys on GUIDs (uniqueidentifier data type in SQL Server)
  • NVARCHAR(MAX) fields
  • Foreign keys

Clustered Keys on Guids

When you use a clustered key, the items in a table are stored in the order of the key. With numeric keys, this means 1 comes before 2, etc. With a common key type being an IDENTITY field, or autoincrementing integer, this type of field works nicely. As each key is created, the new data is placed after previous data. You do not end up with table fragmentation very often, which makes maintenance of your database easier.

As long as you continue to add to the end of the table, the clustered key is very efficient. But, when you stick something in the middle of existing records, it causes some extra work, as it fragments the data table. Without getting too much into the internals of RDBMS, or SQL Server in particular, the underlying pages get fragmented. if the data completely fills a page, the fragmentation is cured by reorganizing pages. This is not trivial, but it is far less intrusive than when multiple rows exist in a single page and the internal data of the page has to be reorganized. Yes, there are things you can do to lessen the blow, like altering the fill percentage of the page, but there is a risk of fragmentation.

Why is fragmentation bad? It reduces the efficiency of the database, which can impact performance.

When you use a Globally Unique IDentified (GUID, which is represented as a uniqueidentifier in SQL Server), the GUIDs may be created sequentially, but this is not guaranteed, especially over time. This has to do with the algorithm to create GUIDs to “guarantee” they are unique (there is still a minuscule risk of creating 2 GUIDs that are alike, but it would be extremely rare – 2^128 or 1 in 340,282,366,920,938,000,000,000,000,000,000,000,000 – a very minuscule risk). When a new GUID is created, it can be sequential for quite some time. Then it is not. And over time, you end up with a lot out of sequence. And you start fragmenting. And performance suffers. And little children start crying … really loud.

Fortunately, this is easy to fix.

Version 1: Fragmentation city

[Key()]
public GUID Id { get;set;}

This is the same as the following in SQL Server.

tbd

Version 2: Non-fragmented bliss

Please read this entire section before attempting (Why? Because I am a smart a**). Let’s solve this problem.

[Key(NonClustered)]
public GUID Id { get;set;}

Bliss!

Yes, I a screwing with you. Entity Framework leaves NO WAY to add a non-clustered primary key in Code First with data annotations. So, you have to sling some code. This will require at least EF 6.2 (released a year ago, so you are on this if you are new to EF). In the OnModelCreating event, add the following:

modelbuilder.Entity().HasKey(k => k.Id, config => config.IsClustered(false));

I have also seen it done like this (very slight variation).

p.HasKey(k => k.ColumnId).ForSqlServerIsClustered(false);

The important take away is you dink with Is Clustered and setting it to false. Both create a migration that sets up a non-clustered index.

Cat’s and dogs sleeping together … and world peace.

If you are prior to EF 6.2, I feel really sorry for you. You can create non-clustered indexes, but not non-clustered primary keys. In that case, i would stop using Code First and cry. 🙂

NVARCHAR(MAX)

Before getting into the problem, I want to rant a bit. NVARCHAR(MAX) was created to solve a problem that, for the most part, did not exist. The problem is this:

What if I have a field that normally contains a manageable amount of data, but occasionally contains the entire context of War and Peace?

I see this as a possibility, but a very rare one. In most cases, when you have a s**t ton of data in a field, that is the norm, and when you don’t, that is the norm. Usually you can either constrain or not. And, in cases where you have small amounts of data, it is more often from bad test plans on BLOB (Binary Large OBject) fields where the tester does not want to copy and paste and wait.

NOTE: I am willing to listen to cases where NVARCHAR(MAX) is a proper design choice, as I realize I do not have all of the answers. Just ping me and let’s argue over it. 😉

Okay, I am being a bit too snarky. Let me regain my composure.

NVARCHAR(MAX) is designed to add some performance gains for the rare “I can either type in my name or enter War & Peace” scenarios. Anything small enough to fit into the SQL Server page is put in the page. Anything not, is placed in a type of BLOB storage. This means it can be much faster on records that fit in the page and only slow down on those that overflow.

Sounds Great, right? Yes, but it has some implications. You cannot index an NVARCHAR(MAX) field in SQL Server. I think this should be obvious for anyone who things about it, but let me describe the scenario and see if you can spot the problem.

  • You have a table with NVARCHAR(MAX)
  • There are 1,000,000 records in the table
  • You add the contents of War & Peace into 1/2 the records
  • You create an index on that field

Do you see the problem. Hint: How big is the index.

To solve the fact a person could create a multiple terrabyte index on a field, you cannot index using any BLOB field, even one, like NVARCHAR(MAX), that is pseudo-blob.

But, guess what EF does on strings? If you guessed “creates an NVARCHAR(MAX) field” give yourself a golden star.

Is this a problem? Probably not in most cases, but if you know you are creating a field like city, will you ever have a city that contains 1,000,000 characters? Not until we explore Zeekbart in the Omega quadrant (they are very verbose aliens) or start coding in Entish (LOTR’s creatures for the non-Tolkein fans).

Fortunately, you can solve this one very, very easily. Just add a data annotation.

[MaxLength(100)]
public string City { get; set; }

 

Wala Wala Washington. Now it will no longer create a MAX field and you can index city. And, yes, this is a common use case, although I would consider extracting the city information into it’s own table if you wish to use it as part of your queries (one primary reason for indexing this field is ad hoc searching on demographic fields, like “how many users in New York City had a problem”).

Foreign Keys

This is not an issue, as attributing for foreign keys is extremely easy to do. The problem is I find many people using EF Code first FAIL to do it. Your code is auto-generating your database folks. And constraints are a necessary element of ensuring the database catching bad data by throwing an exception when you try to put bad data in. Never rely on the brilliance of your developers to keep you company in business. Instead constrain the database so they can’t mess things up.

I ran into this problem (not EF, but lack of constraints) with a client in the early 2000s. I was asked to look at a problem with their data. When I logged on an examined the schema, I found no foreign keys. Asking the dev manager, he stated “the foreign keys were causing errors, so we removed them”. I said, “of course they were causing errors, because the code was set up to create data problems.” I got paid for identifying the problem, but they were unhappy I could not fix the missing data. Needless to say, they are out of business now. All because of a lack of foreign keys.

Fortunately, you can avoid this type of embarassment by adding … dun, dun, duh … foreign keys. Guess how? Yes, data annotations.

[ForeignKey(“KeyName”)] public int OtherTableId { get; set; }

I am not going to cover it in this post, but you can do composite (multiple field) keys, as well. I try not to use composite keys as a rule. Not because I am against the idea, per se, but because I find it harder on coders in many instances. If you start with the idea each table has a derived key as its primary key, it is very clean. You can then refactor the idea where a composite key would be better.

The Data Looks the Same

This is more of a design issue, but EF makes it very easy to simply pass objects back and forth between layers. If you adhere to principles of Domain Driven Design, you understand each domain is unique in how it treats models. Let’s take an eCommerce company as an example. In particular, let’s look at the customer object.

  • When an order is taken, a customer ID is added. For the sales department, when the order is retrieved, the customer information, including addresses, is needed to track down any issues.
  • For the warehouse, there is no need to have a customer object. You only need an order number with the items to pull.
  • For the shipping department, you need an address and phone number and nothing else.

I could go on, but if you use EF in default mode, you will be tempted to have a single domain and filter on the user interface.

This is what was done in an application I recently tested. Grab everything for all roles and then filter out what is not needed. But, since the application was not designed out enough, the following problems surfaced.

  • Some smaller tables were retrieved in toto and not filtered for the location or the user. Thus, if I added something, someone in another location saw it, even though they should not have.
  • As binding was not constrained, the tables had all of the information. If I grabbed a user object, I got all of the data if it was not filtered correctly.

Sure, the above have some other design issues that contribute to the problem, but EF in default forces you to filter after retrieval. When you do not, you are likely adding extra work, which may not be justified over other methods of data access (a topic for another day).

The bigger issue is the database storage looks just like the domain model. And, if you are using the EF defaults and auto-generation, you end up stuck in this paradigm. Data can be optimized with a few tools, like indexes, but once you figure it out, you need to make sure the changes are reflected in the EF model. In some cases, this is hard, and that which is hard is rarely tried (until someone threatens to fire you – then again, they threaten and you find a new job, so it is still not done).

In Summary

I am not a huge fan of Entity Framework in the Enterprise (important caveat?). Not because it does not work in many cases, but because it has a lot of moving parts and some that are missing. In most cases, you end up having to use custom migrations to get around the fact it is so easy to step outside of the box created for you. Once you start slinging code with every migration, you start asking whether going to scripted database changes is a better approach than data migrations in EF (I will argue that it IS in a future post, but your mileage may vary and I accept that each tool has its pros and cons).

A hill I will die on is “defaults are bad”, at least when it comes to Entity Framework in an Enterprise setting. I will also state you have to annotate your data if you are going to use your model to generate it. I would prefer don’t design your RELATIONAL database with OOP objects, but your mileage may vary.

Peace and Grace,
Greg

Twitter: @gbworld

Windows 10: Getting around “Buy Wi-Fi” on membership networks


So I found a nice “feature” in Windows 10 that really annoys the crap out of me. It is the buy Wi-Fi feature. It seems like it might be a nice feature at times, like when you are somewhere, cannot find free Wi-Fi (rare) and need to connect to get some business done.

The problem with the feature is networks you have membership on, that Microsoft has contracted with, default to “buy wi-fi”, as in the follow capture

image

That’s right, with a Marriott Platinum status, sitting in a room in the hotel, the default is to buy Internet. This is really irritating.

Getting On Without Paying (This case “paying twice”)

So this really bugged me until I got to playing with it. If the Network is one you should be able to get on, you can click on the network and the following pops up. Click on the link circled below:

image

Once you click “other options from the provider”, the logon screen for the hotel appears. In this case, it shows I am still connected from last night. The network will boot me off in a bit.

image

If you select view services, you will then see your purchase options for the network.

image

And there is one more caveat. To get on the network, if you don’t know this already, you have to download the Windows 10 Wi-Fi app from the Microsoft Store just to see the options. Until you download the app, you don’t even have the option to explore other options with the provider.

Personally, I think this is a fail and Microsoft needs to rethink. It would be better, in my opinion to have “other options” be the buy option, as this irritates the crap out of users. Oh, wait, Windows is the only option when you have certain careers?

Peace and Grace,
Greg

Microservices in .NET part 2: Silver Bullets and Free Lunches?


I have spent the better part of this week digging into micro services and I love the idea. Here are some benefits I see that can be realized by using a microservices approach:

  1. The granularity level allows developers to stay on a single context while solving a problem. This singularity of focus makes it much easier to dig into the details of a specific object or set of objects. In many cases, it will quickly expose poor planning in an organization and provide rationale for fixing the process. As an example, a service that has a lot of churn is probably one that was not planned out well (not talking finding additional uses, but rather having to rebuild contracts and re-architect the service on a regular basis)
  2. The services are simple,making it easy to maintain the individual components.
  3. The methodology forces a good separation of concerns.
  4. You can use the best tool for the job rather than stick to a single platform, programming language or paradigm, etc. This is a dual edged sword, as I will uncover a bit later.
  5. Isolated solution problems can easily be fixed without much impact. If you find your employee microservice has an issue, you can fix it without deploying the entire solution.
  6. Working with multiple services enables the use of concepts like Continuous Integration (CI) and Continuous Delivery (CD). This is also dual edged, as you almost have to go to a full blown CD implementation to use microservices architectures. I will hit this later, as well.
  7. You can get multiple teams working independently of each other. This was always possible, of course, as I have pointed out in my Core As Application blog entries (one here), if you will take the time to plan out you contracts and domain models first. (NOTE: In 2010, I was told “you cannot design contracts first, as you don’t know all of the requirements up front”. By 2011, I proved this wrong by delivering using a contract first approach, both ahead of time and under budget – a bit of planning goes a long way).
  8. Systems are loosely coupled and highly cohesive.

This is just a short list of the benefits. The problem I see is everyone is focusing on the benefits as if we have finally found the silver bullet (do you have werewolves in your organization?) and gained a free lunch. This article focuses on some of the downsides to microservices.

DevOps

As you move to smaller and smaller services, there are many more parts that have to be deployed. In order to keep the solutions using the microservices up and running, you have to be able to push the services out to the correct location (URI?) so they can be contacted properly by the solutions using them. If you go to the nth degree, you could conceivably have tens, if not hundreds, of small services running in an Enterprise.

As each service is meant to be autonomous, this means you have to come up with a strategy for deployment for each. You also have to plan for high availability and failover. And there has to be a solid monitoring and instrumentation strategy in place. In short, you need all of the pieces of a good API Management strategy in place, and you need to do this BEFORE you start implementing microservices. And I have not even started focusing on how everything is wired together and load balancing of your solutions. On the plus side, once you solve this problem, you can tune each service independently.

There is a burden on the Dev side that needs to be tackled up front, as well. You need to start thinking about the requirements for monitoring, tracing and instrumenting the code base and ensure it is part of the template for every service. And you have to plan for failure, which is another topic.

As a final point on this topic, your dev and ops team(s) must be proficient in the combined concept of DevOps to have this be a success. Developers can no longer pitch items over to Ops with a deployment document. They have to be involved in joint discussions and help come up with plans for the individual services, as well as the bigger solutions.

Planning for Failure and Avoiding Failure

Services will fail. Looking at the plethora of articles on microservices, it is suggested you use patterns like circuit breakers (avoid hitting a failed service after a few attempts) and bulkheads (when enough “compartments” are “under water”, seal the rest of the solution from the failure point). This is a fine avoidance strategy, but what if the service failing is a critical component to the solution working?

Not mentioned in the articles I have read is a means of managing services after failure. Re-deploying is an option, and you can make redeployment easier using quickly set up virtual environments and/or containers, but what if reaching that portion of the network is the point of failure. I would love to hear comments on this next idea: Why not look at some form of registry for the services (part of API Management, similar to UDDI, etc.) or a master finder service that exists in various locations and that all applications are aware of. Another idea would be to include as part of the hyper-media specification backup service locations. But either of these solution further exacerbates the reliance on DevOps, creating even more need for planning solutions and monitoring released solutions.

I don’t see microservices working well without a good CI/CD strategy in place and some form of API management. The more I look into microservices the more I see the need for a system that can discover its various points on the fly (which leads me back to the ideas of using a finder service or utilizing hypermedia to inform the solutions utilizing microservices where other nodes exist).

Contracts and Versioning

When you develop applications as a single Visual Studio solution (thinking in terms of projects and not products?), you have the ability to change contracts as needed. After all, you have all of the code sitting in front of you, right? When you switch to an internal focus on services as released products, you can’t switch out contracts as easily. You have to come up with a versioning strategy.

I was in a conversation a few weeks ago where we discussed versioning. It was easy to see how URI changes for REST services required versioning, but one person disagreed when I stated changes to the objects you expose should be a reason for versioning in many instances. The answer was “we are using JSON, so it will not break the clients if you change the objects”. I think this topic deserves a side bar.

While it is true JSON allows a lot of leeway in reorganizing objects without physically breaking the client(s) using the service, there is also a concept of logical breakage. Adding a new property is generally less of a problem, unless that new element is critical for the microservice. Changing a property may also not cause breakage up front. As an example, you change from an int to a long as planning for the future. As long as the values do not exceed the greatest value for an int, there is no breakage on a client using an int on their version of the object. The issue here is it may be months or even years before a client breaks. And finding and fixing this particular breakage could be extremely difficult and lead to long down times.

There are going to be times when contract changes are necessary. In these cases, you will have to plan out the final end game, which will include both the client(s) and service(s) utilizing the new contract, as well as transitional architectures to get to the end game without introducing a “big bang” approach (which microservices are said to help us avoid). In short, you have to treat microservice changes the same way you approach changes on an external API (as a product). Here is a simple path for a minor change.

  1. Add a second version of the contract to the microservice and deploy (do not remove the earlier version at this time)
  2. Inform all service users the old contract is set to deprecate and create a reasonable schedule in conjunction with the consumers of the microservice
  3. Update the clients to use the new contract
  4. When all clients have updated, retire the old version of  the contract

This is standard operating procedure in external APIs, but not something most people think about a huge deal when the APIs are internal.

I am going to go back to a point I have made before. Planning is critical when working with small products like microservices. To avoid regular contract breakages, your architects and Subject Matter Experts (SMEs) need to make sure the big picture is outlined before heading down this path. And the plan has to be conveyed to the development teams, especially the leads. Development should be focused on their service when building, but there has to be someone minding the shop to ensure the contracts developed are not too restrictive based on the business needs for the solutions created from the services.

Duplication of Efforts

In theory, this will not happen in microservices, as we have the individual services focusing on single concerns. And, if we can imaging a world where every single class had a service (microservices to the extreme?) we can envision this, at least in theory. But should we break down to that granlular a level? I want to answer that question first.

In Martin Fowlers article, he talks about the Domain Driven Design (DDD) concept of a bounded context. A bounded context is a grouping of required state and behavior for a particular domain. Martin Fowler uses the following diagram to show two bounded contexts.

In the diagram above, you see some duplication in the bounded contexts in the form of duplication of customer and product. In a microservices architecture, you could conceivably move customer and product to its own service and avoid the duplication, but moving a concept out simply to avoid duplication is not the best motivation in all cases. If you can also make a customer or product business capability, I would wholeheartedly support this approach, but it is not always the case (another side bar).

When would you not want to separate out Customer and Product? In short, when the domain concept of these objects is different. In the sales context, a customer contains sales specific information, including terms of sale (net 60 days?) and other items that may not exist in a support context. If we are talking a company that ships products (as opposed to a service only company), we can add other contexts, like shipping and warehousing, that have radically different customer views. In the warehouse, a customer is completely unimportant, as they are focused on pulling orders. From a shipping standpoint, a customer is a name, a shipping address and a phone number. No need for any additional information. A customer microservice either spits out a complete object, allowing the services to filter (not a great idea from a security standpoint) or it provides multiple interfaces for each of the clients (duplication of efforts, but in a single service rather than multiple consumers and/or services, so it does not avoid duplication). A product can also be radically different in each of these contexts.

My advice to starting out is starting with bigger contexts and then decomposing as needed. The original “microservice” can act as an aggregator as you move to more granular approaches. Example of transitional states from the contexts above.

  1. Discover of duplication in the sales and support microservices leads to a decision the customer and product should be separate services
  2. New customer and product services created
  3. Sales and support services altered to use the new product and customer services
  4. new version of sales and support services created to avoid serving product and customer information
  5. Clients altered to use the new services as well as the sales and support services

This is one idea of migration, as we will discover in the next section.

Where do we Aggregate?

If we go back to the bounded context discussion in the last section, we see the need to aggregate. The question is where to we aggregate? You need to come up with a strategy for handling aggregation of information. I am still groking this, so I am not offering a solution at this time. Here are some options I can see.

Option 1 – Client: In a full microservices architecture, the client may be responsible for all aggregation. But what if the user’s client is a mobile application. The chattiness of a microservice architecture is hard enough to control across your internal multi-GB network infrastructure. Moving this out onto the Internet and cell networks expounds the latency. I am not saying this is a bad option in all cases, but if you opt for this approach, more focus on the latency issue is required from your mobile development team. On a positive note, if the client application can handle single service failures gracefully, you reduce the likelihood of a single point of failure.

Option 2 – Final service boundary: In this approach, the outermost service contacts the many microservices it requires to get work done and aggregates for the client. I find this more appealing, in general, for mobile clients. And it reduces the number of “proxies” required for web, simplifying the user interface client. As a negative, it creates a single point of failure that has to be handled.

Option 3 – Aggregation of dependencies: In this approach, the higher level service (closer to the client) aggregates what it requires to work for the client. At first, I liked this option the best, as it fits a SOA approach, but the more and more I read about the microservices idea, the more I see this as a potential combination of the bad points of the first two options, as you introduce numerous points of failure on the aggregate level while still potentially creating multiple points for latency in your client applications. I still think this might be something we can think through, so I am providing it.

If you can think of other options, feel free to add them in the comments.

Testing One, Two, Three

I won’t spend a lot of time on testability, but the more moving parts you have to test, the harder it is. To understand why this is so, create an application fully covered in unit tests at every level, but developed by different teams, and then integrate. The need for integration testing becomes very clear at this moment. And what if you are not only integrating multiple libraries, but multiple discrete, and very small, services. There is a lot of discipline.

I find the only reasonable answer is to have a full suite of unit tests and integration tests, as well as other forms of testing. To keep with the idea of Continuous Integration, only the smaller tests (unit tests) will be fired off with each CI build, but there will be a step in the CD cycle that exercises the full suite.

There is also a discipline change that has to occur (perhaps you do this already, but I find most people DON’T): You must now treat every defect as something that requires a test. You write the test before the fix to verify the bug. If you can’t verify the bug, you need to keep writing before you solve it. Solving something that is not verified is really “not solving” the problem. You may luck out … but then again, you may not.

Summary

There are no werewolves, so there are no silver bullets. There is no such concept as a free lunch. Don’t run around with hammers looking for nails.The point here is microservices is one approach, but don’t assume it comes without any costs.

As a person who has focused on external APIs for various companies (start ups all the way to Fortune 50 companies), I love the idea of taking the same concepts inside the Enterprise. I am also intrigued by the idea of introducing more granularity into solutions, as it “forces” the separation of concerns (something I find so many development shops are bad at). But I also see some potential gotchas when you go to Microservices.

Here are a few suggestions I would have at this point in time:

  1. Plan out your microservices strategy and architecture as if you were exposing every service to the public. Thinking this way pushes you to figure out deployment and versioning as a product rather than a component in a system.
  2. Think about solving issues up front. Figure out how you are going to monitor your plethora of services to find problems before they become huge issues (downtime outside of SLAs, etc). Put together a disaster recovery plan, as well as a plan to failover when you can’t bring a service back up on a particular node.
  3. In like mind, plan out your deployment strategy and API management up front. If you are not into CI and CD, plan to get there, as manually pushing out microservices is a recipe for disaster.
  4. Create a template for your microservices that includes any pieces needed for logging, monitoring, tracing, etc. Get every developer in the organization to use the template when creating new microservices. These plumbing issues should not requiring solving again and again.

Peace and Grace,
Greg

Twitter: @gbworld

A foray into micro services in .NET


Last month, I was given an assignment to attend a workshop with one of our clients. The workshop, as it was called, turned into something more like a hackathon, with various groups attempting to POC some concepts. The big talk of the workshop was micro services.

If you are not familiar with micro services, you should take a look at Martin Fowler’s article. The idea behind micro services is simple. You take an application (which might be a service) and then break it down into smaller, very focused services.

As an example, consider eCommerce. You have online store, which uses a service to handle the various actions required for a customer to order items and have them shipped and tracked. If you examine this from a micro-services perspective, you will separate out the customer shopping cart, order processing, fulfillment and tracking into different services. As you dig deeper you might even go further.

In this entry, I want to dig into micro services a bit deeper, from a Microsoft .NET perspective, but I first want to give some foundation to the micro services concept by understanding how we got here in the first place. That is the main focus of this entry, although I will talk about a high level implementation and some pros and cons.

NOTE: For this article, as with most of my posts, I will define the word “application” as a program that solves a business problem. From this definition, it should not matter how the application interacts with users, and you should not have to rewrite the entire application to create a different presentation (for example, windows to web). I use define the word “solution” as a particular implementation of an application or applications.

Foundations

When we first learn to create applications, the main focus is on solving the problem. This is most easily accomplished by creating a single executable that contains all of the code, which is commonly called a monolith (or monolithic application).

TBD

Monoliths are most often considered bad these days, but there are pluses and minuses to every approach. With the monolith, you have the highest performance of any application type, as everything runs in a single process. This means there is no overhead marshalling across application processes spaces. There is also no dealing with network latency. As developers often focus on performance first, the monolith seems to get an unfair amount of flack. But there are tradeoffs for this level of performance.

  1. The application does not scale well, as the only option for scale is vertical. You can only add so many processors and so much memory. And hardware only runs so fast. Once you have the most and fastest, you are at maximum scale.
  2. There is a reusability issue. To reuse the code, you end up copying and pasting to another monolith. If you find a bug, you now have to fix it in multiple applications, which leads to a maintainability issue.

To solve these issues, there is a push to componentizing software and separating applications into multiple services. Let’s look at these two topics for a bit.

Componentizing for Scale (and Reusability)

While the monolith may work fine in some small businesses who will never reach a scale that will max out their monolith, it is disastrous for the Enterprise, especially the large Enterprise. To solve this “monolith problem”, there have been various attempts at componentizing software. Here are a list of a few in the Microsoft world (and beyond):

COM: COM, or the Component Object Model is Microsoft’s method for solving the monolith problem. COM was introduced in 1993 for Windows 3.1, which was a graphical shell on top of DOS. COM was a means of taking some early concepts, like Dynamic Data Exchange (DDE), which allowed applications to converse with each other, and Object Linking and Embedding (OLE), which was built on top of DDE to allow more complex documents by embedding types from one document type into another. Another common word in the COM arena is ActiveX, which is largely known as an embedding technology for web applications that competed with Java applets.

DCOM and COM+: DCOM is a bit of a useless word, as COM was able to be distributed rather early on. It was just hard to do. DCOM, at its base level, was a set of components and interfaces that made communication over Remote Procedure Call (RCP) easier to accomplish. DCOM was largely in response to the popularity of CORBA as a means of distributing components. COM+ was a further extension of DCOM which allowed for distributed transactions and queueing. COM+ is a merger of COM, DCOM libraries and Microsoft Transaction Server (MTS) and the Microsoft Message Queue service (MSMQ).

.NET: If you examine Mary Kirtland’s early articles on COM+ (one example here), you will read about something that sounds a lot like .NET is today. It is designed to be easily distributed and componentized. One problem with COM+ was DLL Hell (having to clean out component GUIDs (globally unique identifiers) to avoid non-working applications). .NET solved this by not registering components and returning to the idea of using configuration over convention (a long fight that never ends?).

Competing technologies in this space are EJB and RMI in the Java world and CORBA and the CCM as a more neutral implementation. They are outside of the scope of this document.

The main benefit of technologies are they make it easier to reuse code, reducing maintainability, and allow you to more easily distribute applications, providing greater availability and scalability. You can still choose to build monolithic applications, when they make sense, but you are not tied to the limitations of the monolith.

Services

One issue with many of the component technologies was they tightly coupled the consumer to the application, or the “client” to the “server”. To get away from this, a group of developers/architects at Microsoft came up with SOAP (Yes, I know Don Box was actually working for DevelopMentor as a contractor at Microsoft and Dave Winer was heading UserLand software and only partnering with Microsoft on XML and RPC communication, but the majority of the work was done there). With the creation of SOAP, we now had a means of creating applications as services, or discrete applications, which could focus on one thing only. That is sounding a lot like micro services … hmmmm.

In the Microsoft world, SOAP was used to create “web services”. The initial release of .NET in 2002 allowed one to create services using HTTP and SOAP as ASMX services (a type of document in ASP.NET) as well as create faster RPC type services with Remoting (generally these were internal only, as it was hard to play with them outside of the Enterprise, due primarily to more tight coupling to technologies, much less play with them outside of the Microsoft world).

By 2006, with the release of .NET 3.0, Microsoft had merged the concepts of Remoting and ASMX web services in the Windows Communication Foundation (WCF). You could now develop the service and add different endpoints with ease, allowing for an RCP implementation and a web implementation off the same service. WCF really came to fruition about a year later with the release of .NET 3.5.

The latest service technology to enter the fray is Representational State Transfer (REST). In the Microsoft world, REST was first introduced in REST toolkit, an open source project. From the standpoint of an official Microsoft release, it was released as the WCF Web API. It was a bit kludgy, as WCF works in a completely different paradigm than REST, so the project was moved over the web group and is now implemented on top of ASP.NET MVC as the ASP.NET Web API.

Methodologies, Technologies and Tools

One more area we should look at before moving to micro services are methodologies, technologies and tools used to solve the monolith “problem”.

Methodologies

The first methodology that gained a lot of acceptance in the Microsoft world was the n-tier development methodology. The application was divided into UI, Business and Data tiers (note: today Microsoft calls this Presentation, Middle and Data tiers), with the push towards separating out the functionality into discrete, purposed pieces. Below is a typical n-tier diagram for an application.

Around 2010, I realized there was a problem with n-tier development. Not so much with the methodology, as it was sound, but with the way people were viewing the models. Below is an n-tier model of an application:

The issue here is people would see an application as the merger of presentation, business logic and data. Is this true? Let’s ask a couple of questions.

1. If you create a web application with the exact same functionality as a windows application is it 2 applications or one? In implementation, it was often treated as two, but if it was logically and physically two applications, you were duplicating code.

2. If you want to add a web service to expose the functionality of the application, do you rebuild all n tiers? If so, should you?

My view is the application is the part that solves business problems, or the core of the functionality. You should be able to change out where you persist state without changing this functionality. I think most people understand this as far as switching out one database server for another, like SQL Server to Oracle. When we get to the logical level, like changing out schemas, a few get lost here, but physical switches with the same schema is well known and most find it easy to implement. Switching out presentation is what most people find more difficult, and this is generally due to introducing logic other than presentation logic in the presentation portion of the solution.

NOTE: The naming of the tiers in 3-tier and n-tier architecture has changed over time. Originally, it was common to see UI, Business and Data. The illustration above calls the tiers Client, Middle and Data. I have also seen Presentation, Application and Persistence, which is closer to the naming I would use in my models.

To better illustrate this concept, in 2010 I came up with a methodology called Core as Application (seen in this article on the importance of domain models). In this model the core libraries ARE the application. The libraries for presentation can easily be switched out, and have responsibility for shaping the data for presentation.

image

The “Core as Application” model requires you start with your domain models (how data is represented in the application) and your contracts, both for presentation and persistence. Some of the benefits of this model are:

  1. Focusing on domain models and contracts first pushes the team to plan before developing (no, there is no surefire way to force people other than tasers ;->). This is a good representation no matter what methodology or model you use, but it is critical if you are going to have multiple teams working on different parts of a solution.
  2. You can have multiple teams working in parallel rather than relying on completing one layer prior to working on another. You will have to resynchronize if any team determines the contract needs to be changed, but the amount of rework should be minimal.

When you look at “Core as Application” from a SOA standpoint, each service has its own core, with a service presentation layer. The persistence for higher level applications is the individual services. This will be shown a bit later.

Technologies

We have already covered some technologies used to solve the monolith problem. COM and .NET are good examples. But as we move even deeper, we find technologies like the ASP.NET Web API is useful. The technologies do not force us to not create monoliths, as even Microsoft pushes out some examples with data access, as LINQ to SQL, in a controller in an MVC application. But they do get us thinking about creating cohesive libraries that serve one purpose and classes that give us even more fine grained functionality.

Tools

We also have tools at our service. Visual Studio helps us organize our code into solutions and projects. The projects focus on a more fine grained set of functionality, helping us break the monolith up. If we follow best practices, our solutions end up with more projects, which can easily be broken down into individual micro services. Speaking of which, this a good time to segue.

Onward to Micro Services

Micro services are being presented as something new, but in reality, they are nothing more than Service Oriented Architecture (SOA) taking to the nth degree. The idea behind micro services is you are going extremely fine grained in your services. It is also stated you should use REST, but I don’t see REST as an absolute requirement. Personally, I would not aim for SOAP as there is a lot of extra overhead, but it is possible to use SOAP in micro services, if needed. But I digress … the first question to answer is “what is a micro service?”

What is a Micro Service?

I am going to start with a more “purist” answer. A micro service is a small service focused on solving one thing. If we want to be purist about it, the micro service will also have its own dedicated database. If we were to illustrate the order system example we talked about earlier, using the “Core as Application” model, the micro-services implementation would be something like the picture below.

image

If you want to take it to an extreme, you can view micro services the way Juval Lowy viewed in 2007 (video gone, but you can read the description). His idea was every single class should have a WCF service on top of it. Doing so would create a highly decoupled system possible, while maintaining a high level of cohesiveness. Micro services does not dictate this type of extreme, but it does recommend you find the smallest bounded context possible. I will suggest a practical method of doing this a bit later.

image

One difference between Juval Lowy’s suggested “one service per class” and micro services is the suggested service methodology has changed a bit. In 2007, WCF was focused on SOAP based services. Today, REST is the suggested method. Technically, you can develop a micro service with SOAP, but you will be adding a lot of unnecessary overhead.

Below are Martin Fowler’s characteristics of a micro service:

· Componentization via services – This has already been discussed a bit in the previous two paragraphs. Componentization is something we have done for quite some time, as we build DLLs (COM) or Assemblies (.NET – an assembly will still end in .DLL, but there is no Dynamic Link Library capability for COM inherently built in without adding the Interop interfaces). The main difference between an assembly (class library as a .dll file) and a micro service is assembly/dll is kept in process of the application that utilizes it while the micro-service is out of process. For maximum reuse, you will build the class library and then add a RESTful service on top using the ASP.NET Web API. In a full micro services architecture, this is done more for maintainability than need.

· Organized around business capabilities – This means you are looking for bounded contexts within a capability, rather than simply trying to figure out the smallest service you can make. There are two reasons you may wish to go even smaller in micro services. The first, and most reasonable, is finding a new business capability based on a subset of functionality. For example, if your business can fulfill orders for your clients, even if they are not using your eCommerce application, it is a separate business capability. A second reason is you have discovered a piece of functionality can be utilized by more than one business capability. In these cases, the “business capability” is internal, but it is still a capability that has more than one client. Think a bit more about internal capabilities, as it may make sense to duplicate functionality if the models surrounding the functionality are different enough a “one size fits both (or all)” service would be difficult to maintain.

  • Product Not Projects – This means every service is seen as a product, even if the only consumer is internal. When you think of services as products, you start to see the need for a good versioning strategy to ensure you are not breaking your clients.
  • Smart Endpoints and dumb pipes – I am not sure I agree completely with the way this one is worded, but the concept is the endpoint has the smarts to deliver the right answer and the underlying delivery mechanism is a dumb asynchronous message pump.
  • Decentralized Governance – Each product in a micro services architecture has its own governance. While this sounds like it goes against Enterprise concepts like Master Data Management (MDM), they are really not opposites at all, as you will see in the next point
  • Decentralized Data Management – This goes hand in hand with the decentralized governance, and further illustrates the need for Master Data Management (MDM) in the organization. In MDM, you focus on where the golden record is and make sure it is updated properly if a change is needed. From this point on, the golden record is consulted whenever there is a conflict. In micro services, each micro service is responsible for the upkeep of its own data. In most simple implementations, this would mean the micro service contains the golden record. If there are integrated data views, as in reporting and analytics, you will have to have a solution in place to keep the data up to date in the integrated environment.
  • Infrastructure Automation – I don’t see this a mandatory step in implementing micro services, but it will be much harder if you do not have automation. This topic will often start with Continuous Integration and Continuous Delivery, but it gets a bit deeper, as you have to have a means of deploying the infrastructure to support the micro service. One option bandied about on many sites is a cloud infrastructure. I agree this is a great way to push out microservices, especially when using cloud IaaS or PaaS implementations. Both VMware and Microsoft’s Hyper-V solutions provide capabilities to easily push out the infrastructure as part of the build. In the Microsoft world, the combination of the build server and release management are a very good start for this type of infrastructure automation. In the Linux world, there is a tool called Docker that allows you to push out containers for deployment. This capability also finds its way to the Microsoft world in Windows Server 2016.
  • Design For Failure – Services can and will fail. You need to have a method of detecting failures and restoring services as quickly as possible. A good application should have monitoring built in, so the concept is nothing new. When your applications are more monolithic, you more easily determine where you problem is. In micro services, monitoring becomes even more critical.
  • Evolutionary Design – I find this to be one of the most important concepts and one that might be overlooked. You can always decompose your micro services further at a later date, so you don’t have to determine the final granularity up front. As a micro service today can easily become an aggregator of multiple micro services tomorrow, There are a couple of concepts that will help you create your microservices we will discuss now: Domain Driven Design and Contract First Development.
Domain Driven Design

Domain Driven Design (DDD) is a concept formulated by Eric Evans in 2003. One of the concepts of DDD that is featured in Fowler’s article on micro services is the Bounded Context. A bounded context is the minimum size a service can be broken down into and still make sense. Below is a picture from Fowler’s article on Bounded Contexts.

When you start using DDD, you will sit down with your domain experts (subject matter experts (SMEs) on the domain) and find the language they use. You will then create objects with the same names in your application. If you have not read Eric Evans Domain Driven Design book, you should learn a bit about modeling a domain, as it is a process to get it right.

NOTE: you are not trying to make your data storage match the domain (ie, table names matching domain object names); let your database experts figure out how to persist state and focus on how the application uses state to create your domain objects. This is where Contract First comes into play.

Contract First Development

Once you understand how your objects are represented in your domain, and preferably after you have a good idea of how the objects look in your presentation projects and your data store, you can start figuring out the contracts between the application, the presentation “mechanism” and the persistent stores.

In general, the application serves its objects rather than maps them to presentation objects, so the contract focuses on exposing the domain. The presentation project is then responsible for mapping the data for its own use. The reason for this is presenting for one type of presentation interface will force unnecessary mapping for other types. As an example, I have seen n-tier applications where the business layer projects formatted the data as HTML, which forced writing an HTML stripping library to reuse the functionality in a service. Ouch!

How about the persistence “layer”? The answer to this question really depends on how many different applications use the data. If you are truly embracing micro services, the data store is only used by your service. In these cases, if storage is different from the domain objects, you would still desire spitting out the data shapes that fit the domain objects.

How to Implement Microsoft Services, High Level

Let’s look at a problem that is similar to something we focused on for a client and use it to determine how to implement a micro services architecture. We are going to picture an application for an Human Resources (HR) services company that helps onboard employees We will call the company OnboardingPeople.com (and hope it is not a real company?).

Any time a new employee comes on board there is a variety of paperwork that needs to be completed. In the HR world, some of the forms have a section that is filled out by the employee and another that is filled out by a company representative. In the micros services application architecture, we might look at creating a micro service for the employee portion and another for the employer portion. We now have a user facing application that has two roles and uses the two services to complete its work. Each service surrounds a bounded context that focuses on a single business capability.

image

Over time, we start working with other forms. We find the concept of an employee is present in both of the forms and realize employee can now represent a bounded context. The capability may only be internal at this time, so there is a question whether it should be separated out. But we are going to assume the reuse of this functionality is a strong enough reason to have a separate capability. There is also a possibility the employer agent can be separated out (yes, this is more contrived, but we are envisioning where the solution(s) might go).

image

If we take this even further, there is a possibility we have to deal with employees from other country, which necessitates an immigration micro service. There are also multiple electronic signatures needed and addresses for different people, so these could be services.

image

In all likelihood, we would NEVER break the solution(s) into this many micro services. As an example, addresses likely have a slightly different context in each solution and are better tied to services like the employee service and employer services rather than a separate service that is able to keep context

Pros and Cons

While micro services are being touted in many articles as THE solution for all applications, silver bullets don’t exist, as there are no werewolves to kill in your organization. Micro services can solve a lot of pain points in the average Enterprise, but there is some preparation necessary to get there and you need to map it out and complete planning before implementing (I will focus on another article to go into more detail on implementation).

Pros

One of the main pros I see mentioned is the software is the right size with micro services. The fact you are implementing in terms of the smallest unit of business capability you can find means you have to separate the functionality out so it is very focused. This focus makes the application easier to maintain. In addition, a micro services architecture naturally enforces tight cohesion and loose coupling. Another benefit is you naturally have to develop the contract up front as you are releasing each service as a discreet product.

You also have the flexibility to choose the correct platform and language on a product by product (service by service) basis. The contract has to be implemented via standards for interoperability, but you are not tied into single technology. (NOTE: I would still consider limiting the number of technologies in use, even if there is some compromise, as it gets expensive in manpower having to maintain multiple technologies).

Micro services will each be in control of their own data and maintain their own domains. The small nature of the services will mean each domain is easy to maintain. It is also easier to get a business person focused on one capability at a time and “perfect” that capability. It goes without saying micro services work well in an Agile environment.

Micro services architecture also allows you the ability to scale each service independently. If your organization has adopted some form of cloud or virtual infrastructure, you will find it much easier to scale your services, as you simply add additional resources to the service.

Cons

Micro services is a paradigm shift. While the concepts are not radically different, you will have to force your development staff to finally implement some of the items they “knew” as theory but had not implemented. SOLID principles become extremely important when implementing a micro services architecture. If your development methodologies and staff are not mature enough, it will be a rather major paradigm shift. Even if they are, a shift of thinking is in order, as most shops I have encountered have a hard time viewing each class library as a product (Yes, even those who have adopted a package manager technology like NuGet).

There is a lot of research that is required to successfully implement a micro services architecture.

· Versioning – You can no longer simply change interfaces. You, instead, have to add new functionality and deprecate the old. This is something you should have been doing all along, but pretty much every developer I have met fudges this a good amount of the time. It is internal, so I can fix all of the compiler errors, no problem. This is why so many shops have multiple solutions with the same framework libraries referenced as code. You should determine your versioning strategy up front.

· URI Formatting – I am assuming a REST approach for your micro services when I include URI formatting, but

· API Management – When there are a few services, this need will not be as evident. As you start to decompose your services in to smaller services, it will become more critical. I would consider some type of API management solution, like Layer 7 (CA), Apigee, or others, as opposed to building the API management or relying on an Excel spreadsheet or internal app to remind you to set up the application correctly.

· Instrumentation and Monitoring – Face it, most of us are bad at setting the plumbing, but it becomes critical to determine where an error occurs in a micro services architecture. In theory, you will know where the problem is, because it is the last service deployed, but relying on this idea is dangerous.

· Deployment – As with the rest of the topics in this section, there is a lot of planning required up front when it comes to deployment. Deployment should be automated in the micro services world. But deployment is more than just pushing applications out, you need to have a rollback plan if something goes wrong. In micro services, each service has its own deployment pipeline, which makes things really interesting. Fortunately, there are tools to help you with build and release management, including parts of the Microsoft ALM server family, namely build server and release management.

In short, micro services are simple to create, but much harder to manage. If you do not plan out the shift, you will miss something.

Summary

This is the first article on micro services and focuses on some of the information I have learned thus far. As we go forward, I will explore the subject deeper from a Microsoft standpoint and include both development and deployment of services.

Peace and Grace,
Greg

Twitter: @gbworld

Solving the Microsoft Mahjong Classic Puzzle–January 9th


This particular puzzle was a bit of a b*tch. Despite being flagged easy, it takes quite a bit of time, largely due to the placement of a few tiles that requires a strategy that is a bit out of the norm. Here is my guide to solving the puzzle using the tranquility tile set. (If you are using another tile set, you can still use this post, but will have to go by position alone.

First, here is how it looks as you start.

Starting-Point

To explain the solution, I need to do 2 things:

  1. Identify the tiles in shorthand
  2. Grid off the board

Identifying the Tiles

In Mahjong, there are a group of tiles that can only match an identical tile; these are 3 suits (wheels (or dots), bamboo (or sticks) and numbers (or cracks)). the four winds (north, south, each and west) and a set of three dragons (red, green and white).

There are also two suits in which you can match any of the tiles: Seasons (spring, summer, winter, fall) and flowers (plum, orchid, bamboo and mum).

Wheels

The wheels are easy to identify. They have a dot or wheel. You count the number of wheels to determine the number of the tile. I label these with a lower case w after the number: 1w – 9w.

Bamboo

Bamboo is also easy, as you count the number of bamboo fronds. The exception is the 1 of bamboo, which looks like a bird. I label these with a lower b after the number: 1b – 9b.

Numbers

Numbers are a bit different unless you read Chinese. Here is how to identify them. 1, 2 and 3 of numbers have 1, 2 and 3 vertical lines across the middle of the top of the tile. The 4 looks like an arabic W. 5 is the most complex symbol. It has an up and down stroke followed by what appears to be a lower case h with a cross at the top and underline. 6 is a stick figure without the body segment. 7 appears a bit like a t, 8 like a broken upside down v and 9 is like a cursive r. I label numbers with a lower case n: 1n – 9n. Some of the numbers are shown below.

numbers

Winds

Winds are easy to identify, as they have the initial of the wind direction in black in the upper left corner of the tile: N, S, E, W.

Dragons

There are three dragons: red, green and white. The red dragon is a stick with a circle colored red. The green dragon appears like a green bush or a green flame. And the white dragon is a blue square. I have a red and blue dragon shown below, with the numbers all ready on the graphic:

dragons

Seasons

Seasons are found on green tiles. You can match any season with any other season.

seasons

Flowers

 

Gridding off the board

On this board, there are 5 rows from top to bottom, starting from the 2 of wheels (or dots) in the middle top. and ending with row5 which has a south wind on the far left side. I am going to label these rows 1 through 5.

Rows

There are 13 rows, which I label A through M. Leaving the rows in place, here are the columns.

Rows-&-Columns-&-Half

Notice that some of the tiles, like the 2 of bamboo (2b) on row 4 (partially hidden by the West wind (Wwind). These are half columns, which I label by adding an X. In particular, this is column Ax. Here is a shot showing the rows and the half columns

Rows-&-Halfs

And here is rows, columns and half columns.

Rows-&-Columns-&-Half

Designating Position

To designate a position, I also need to indicate the level of the tile. On this board, the 4 tiles on row 4 (starting with a flower and ennding with the West wind are at level 4. The North wind located at 5-M is at level 2 (there is one tile underneath it.

My location designator consists of row, column and level, with hyphens in between. The North wind at the lower left, for example, is 5-M-2. I will add the tile name (or type in flowers or seasons) in front of the location. Here is a map of a few positions, so you can understand the system.

Locations

Solution

As pointed out on the Mahjong FaceBook page, you have to focus on the bottom row. This is normal when you are solving a Mahjong puzzle, as you should focus on three areas.

  1. long lines
  2. Tall stacks
  3. tiles that cover more than one tile

Not focusing on these areas is bad. This particular puzzle is a bit insidious as there are some tiles that only have one pair and are located under other tiles that have only one pair. This makes it hard to clear off the board. Here are the areas to look out for:

North on top of north
North-on-North

2 6 of bamboo (only pair of them in game) on top of row 5
2-6-of-bamboo

Two white dragons on row 5 at level 2
2-white-dragons

Two red dragons on the bottom level of row 5:
2-red-dragons

The general strategy here (as mentioned on the Facebook page) is to start row 5, level 3 from the left side, then level 2 from the right side and level 1 from the left side. This is not 100% true, as you will see.

The first step is to start clearing row 5, level 3, from the left. There is a 5n here at 5-B-3. You can see another 5n under one of the flowers. The designation for clearing this tile, fitting the scheme is 5n 2-J-2 5-B-3. To get to this tile, you have to clear flowers, which are designated flowers 2-Ix-3 3-G-4.

First-2-moves

You then continue on the left side of row 3 for 2 more moves. Here are the first four moves. Counter intuitive moves are marked with a starStar.

Move Tile Location 1 Location 2
1 flowers 2-Ix-3 4-G-3
2 5n 2-J-2 5-B-3
3 Star 1n 2-C-2 5-C-3
4 Wd 1-J-1 5-D-3

The 3rd move, above, is counter intuitive. You would think you should clear off 5-B-3 with 3-K-2, but I found this move makes the puzzle unsolvable. At the end of the three moves, you have a board that looks like this:
board-after-4-moves

It should be clear enough how to use the moves table now. Here are the moves to clear off the rest of the top of row 5. Note that we will start from the left side now (purple), as we will with row 2 (blue). Also note this table has some rows that say ALL, followed by a number of pairs. This means clear everything of that type off the board.

Move Tile Location 1 Location 2
5 Nwind 4-H-4 5-M-2
6 Wwind ALL (3 pairs)  
9 Rd 4-F-3 4-I-4
10 4n 2-Hx-4 4-E-2
11 Swind ALL (2 pairs)  
13 EWind 3-I-4 5-k-3
14 9n 1-I-2 4-Bx-2
15 8n 3-B-3 4-Bx-1
16 4b 1-I-1 4-Ax-2
17 2w 2-Gx-3 4-G-3
18 3w ALL (2 pairs)  
20 7w 3-Ax-1 4-L-1
21 5b 1-H-1 3-I-3
22 Mug Nwind 2-G-2 5-I-3
23 4w 3-Ex-1 5-H-3
24 6w 2-F-1 2-K-1
25 5w 2-J-1 4-J-3
26 3b 4-I-3 5-G-3
27 1n 3-K-2 5-F-3
28 Wd 3-J-2 5-E-3

Move 22 (beer icon) is also critical, as there is a north on a north in position 2G (at level 2 and 1). Exchanging either of these north winds for the north wind at 4-G-2 At this point in time, the top of row 5 should be cleared, as shown below:
After-28-moves

The next step is to clear off both row 4 and 5. Because of the 6 of bamboo (6b), you have to start row 5, level 2 from the right. Here are the steps to clear row 5, level 2 (purple) and much of row 4, level 2 (blue). 

Move Tile Location 1 Location 2
29 1b ALL (2 pairs)  
31 9w 3-G-3 4-G-3
32 Nwind 1-G-1 4-G-2
33 7b 5-K-1 5-L-2
34 Ewind 3-Kx-1 4-Ax-1
35 6n 2-A-2 3-G-2
36 2b 2-Hx-2 3-Jx-1
37 8b 3-Bx-1 3-I-2
38 Swind 3-H-2 4-H-2
39 1w 2-I-2 5-J-2
40 Seasons ALL (2 pairs)  
42 3n 4-J-2 5-H-2
43 9b 4-K-2 5-G-2
44 4n 3-G-2 5-F-2
45 Gd 2-H-2 5-E-2
46 6b 5-B-2 5-D-2
47 1w 3-Ix-1 5-C-2

Here is the board after these moves:
Near-Solving

You should now be able to solve this without my help, but here are the moves.

Move Tile Location 1 Location 2
48 flower 2-H-1 5-B-1
49 2w 1-G-1 4-K-1
50 Gd 4-J-1 5-C-1
51 2n ALL (2 pairs)  
53 8w 4-H-1 5-E-1
54 3n 5-F-1 5-K-1
55 Rd 5-G-1 5-J-1
56 7n 5-H-1 5-I-1
57 3b 3-Hx-1 4-G-1
58 2b 2-A-1 3-Gx-1

Hope this helps.

Peace and Grace,
Greg

Twitter: @gbworld