SOA Lessons: Insulate Your Client

In my last post, I lamented on the idea of creating an external layer where all of the services sit in a single project. In this post, I would like to talk about the idea of passing data directly from internal services through an External API without mapping the data. This topic was covered a bit differently in a previous post if you would like to look at the issue in greater detail.

As a disclaimer, this post focuses on a particular set of problems I have seen with immature External APIs that may not manifest themselves in more mature SOAs.

External API consumers are valuable to business. If they aren’t, there is no reason to have an External API. Initially, the external clients may not serve much value to your business, but overtime they do. This is especially true in an ECommerce situation, where you clients are using your APIs to help you sell. But this “truism” applies to all sorts of businesses.

From the standpoint of the development team, the external consumer is often “out of site out of mind”. Unless you are attempting a coordinated effort (a topic for another blog post?), your support and infrastructure teams will often have more of an idea of what your clients are doing than the development team.

The primary solution would seem to be “make the external client a first class citizen” and this is a dynamite idea. The problem is external clients are often not easily monetized, making it hard to place a priority on them.

Standard SOA

The title of this section is a bit misleading, as it suggests I am going to guide you through the creation of a “standard” SOA. Instead, I am going to cover a generic SOA setup I see in many organizations.

At the bottom layer of the SOA, services are created to expose persisted state (data). The services at this level may present objects that represent rows in database tables, especially in less mature systems, but more often tryt o distill the data a bit to serve up objects needed by various business applications.

On top of this layer, you find services that aggregate multiple services. This layer or layers of the SOA, models are being created that can be consumed by user interfaces. And it is at this layer many SOAs create the objects consumed by both internal and external clients.

The topmost layer comes in two flavors. The first is what my current client calls a “domain services layer”, which serves to map the aggregate service model to a domain model for the external world to consume. The other is a pass through layer in which the objects of the aggregate layer pass through the external service, which is not much more than an endpoint on an external service.

Insulating the Client Through Process

When the external layer is a pass though layer, breaking changes to objects on the aggregate layer will not always break the pass through. In cases where there are no changes to the external layer, service references are updated and the service is deployed. But the client can’t consume them.

To insulate the client from these changes requires process. A proper versioning strategy must be put in place and older versions of the internal services must be kept up until all clients have moved to the latest version. This also means external clients must be included in the testing process, even if their inclusion is only during the User Acceptance cycle.

In general, this works best if the External API team has some measure of governance over the internal services it consumes. At minimum, the external team needs the ability to issue a go – no go order when breaking changes occur in the pass through models or force the internal service to leave an older version alive while clients migrate to the new version.

If process and a versioning strategy are not in place, it is dangerous to maintain your external API as a pass through layer.

Insulation through Domain Models

Another means of insulation is to set up a Domain Model for the external API and “mapping” the returns from the internal service(s) to the External Domain Model. The main negatives to this approach are a bit of overhead (performance) and a maintenance requirement, especially when the maps are hand coded.

The benefit is breaking changes are instantly seen when the service reference is updated, as the changes now break the mapping. This insulates the client from change, as this breaking change has to be fixed. As the exception can often be fixed through mapping changes alone, the external client need not know the software was even changed, so the external API service can undergo a minor, rather than a major, revision.

Process or Domain Modeling?

The question comes to which should you use … process? domain modeling? both?

In my experience, attempting to solve a technological problem with process alone (or, expecting people to follow standards and rules) only works in an organization where software is reviewed completely prior to deployment. I have certainly worked in organizations with this commitment, but for every company I have consulted that had governance over process, there are 10 that do not.

This leads me to the domain modeling approach and insulation through adherence to contract. The entire underlying system may introduce breaking changes, but as long as the changes can be mapped to a consistent model, the outside world has no clue these breaking changes occurred.

The Cost of Change

As a kind of summary, I want to talk about how much change costs, or more specifically, how breaking changes to external APIs can be very costly.

Initially, an External API is a cost center. Until there is adoption, there is no financial benefit to having clients consume your data. But, even at this stage, failures in the API cause clients to shy away from using them. Enough issues and the stability of your API becomes public and most sensible people avoid it. This ensures the API will never be anything other than a cost center. Worse, it can be the proverbial albatross around your neck.

Over time, the stability of the API and its contracts, becomes more critical, as more and more clients begin to use the service. At this point, a breaking change is generally caught sooner, but rollbacks become extremely expensive.

While setting up mapping to an external domain model is not a cure all, it certainly adds a measure of safety for your clients to use the service.

The Point

The point here is changes that break software internally may be annoying, but they can most often be fixed without the outside world having any clue there is turmoil. Changes that break external software are more dangerous, as they cannot be hidden from view. By enforcing proper versioning strategies and treating the external API as a first class citizen, you can protect yourself from these public breakages. In cases where these processes cannot be put into place, breaking changes can be hidden from the outside world by insulating the clients by mapping to an external model.

From my experience, adding the small bit of overhead to protect the client from internal changes is the safest path, especially when you are in the early stages of creating processes to properly version change. If your organization is mature enough to have proper governance on releases and properly tests both internal AND external responses, a process driven approach is an option. And, if true performance is a showstopper, it may be your better option. Otherwise, adding a bit of insulation for clients is a very wise idea.

Peace and Grace,

Twitter: @gbworld

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: