Troubleshooting: The Lost Skill?

This blog entry comes from an email I received this morning asking me to “check in files”, as my failure to do so was causing a work stoppage. After a very short examination, I found the following:

  1. I had no files checked out (absolutely none)
  2. The problem was a permissions problem, not a check out problem, or the person who could not check in their files was not being stopped by my failure to act, but by someone else’s incorrect granting of permissions
  3. I had no permissions to solve the problem (i.e. grant the correct permissions

Further investigation of the problem would have revealed it was a permissions issue. In this case, the only consequence is another day lost of productivity, and a wonderful opportunity to learn. In some cases, the consequences are more dire. Consider, for example, Jack Welsh, the CEO of GE.

Jack made an assumption and ended up destroying a manufacturing plant. In one telling of Jack Welsh’s story, the dialog goes something like this:

Jack: Aren’t you going to fire me now?
Manager: Fire you? I just spent 2 million dollars training you.

Considering Jack Welch is now one of the most successful executives of all time, it is good his manager was able to troubleshoot the aftermath to a problem Jack had worked through on assumption. The point is plain: When we don’t troubleshoot a problem, we go on assumptions. In the email I received this morning, there was an assumption I had files checked out. Rather than test the assumption, work stopped.

Tony Robbins tells a story in his Personal Power program about a suit of armor. As he is walking on stage, every time he moves close to a suit of armor there is feedback. The audience eventually starts screaming at him “it’s the armor”. But he continues to stand near the armor and the feedback eventually goes away. He then moves away and the feedback comes back. Turns out there was a fire on a very close frequency and the messages were interfering with the microphone.

Personally, I think the above story is a myth, as I know the FCC is very careful on doling out bands and it is unlikely a microphone has the same band as emergency services. But this is also an assumption, and proper troubleshooting would have me examining the issue.

The path of least resistance

On Facebook … nay, on the Internet as a whole, a large majority of items are written out of assumptions or biases, and not an examination of the facts. For most people, whether you agree with Obamacare or not is not an exercise in examining the facts completely and then drawing conclusions. Instead, a quick sniff test is done to determine if you feel something smells, and then action is taken.

Let’s take an example. In 2006, the news media reported on the Duke Lacrosse team raping an African American stripper. The case seemed open and shut, as the evidence piled up. Duke University suspended the team and when the lacrosse coach refused to resign, Duke’s president cancelled the rest of the season. The case seemed so open and shut, Nancy Grace (CNN) was suggesting the team should be castrated.

When the assumptions were removed, a completely different story was told. Not only was the evidence thin, much of it was manufactured. The District Attorney, Ray Nifong, was disbarred and thrown in jail for contempt of court.

We can also look at the George Zimmerman case, where the initial wave of “evidence” painted another “open and shut” case. But the “open and shut” case, based on assumptions, began to crumble when it was discovered ABC edited the 911 tape to paint Zimmerman as a racist and carefully choose the video and picture evidence to paint a picture of a man that had no wounds and was the obvious aggressor.

The point here is not to rehash these cases, but to point out that assumptions can lead to incorrect conclusions. Some of these assumptions may lead to dire consequences, while most just result in a less than optimal solution.

Now to the title of the section: The path of least resistance.

When we look at the natural world, things take the path of least resistance. Water tends to travel downhill, eroding the softest soil. Plants find the most optimal path to the sunlight, even if it makes them crooked. Buffalos would rather roar at each other to establish dominance than fight, as the fighting takes precious energy. And humans look for the least amount of effort to produce a result.

Let’s pop back to Obamacare, or the PPACA (Patient Protection and Affordable Care Act), as it illustrates this point. Pretty much everyone I encounter has an opinion on the subject. In fact, you probably have an opinion. But is the opinion based on assumption? You might be inclined to say no, but have you actually read the bill? If not, then you are working on distillations of the bill, most likely filter through the sites you like to visit on a regular basis. And, more than likely, you have chosen these sites as they tend to fit your own biases.

I am not deriding you on this choice. I only want you to realize this choice is based more on assumptions than troubleshooting. Troubleshooting takes some effort. In most cases, not as much as reading a 900+ page bill (boring) or many more thousands of pages of DHS rules (even more boring). But, by not doing this, your opinion is likely based on incomplete, and perhaps improper, facts.

Answering Questions

I see questions all the time. Inside our organization, I see questions for the Microsoft Center of Excellence (or MSCOE). I have also spent years answering online questions in forums. The general path is:

  1. Person encounters problem
  2. Person assumes solution
  3. Person asks, on the MSCOE list, to help with the assumed solution – In general, the question is “How do I wash a walrus” type of question rather than one with proper background of the actual business problem and any steps (including code) taken to attempt to solve it
  4. Respondent answers how to solve the problem, based on their own assumptions, rather than using troubleshooting skills and asking questions to ensure they understand the problem
  5. Assumed: Person implements solution – While the solution may be inferior, this is also “path of least resistance” and safe. If the solution fails, they have the “expert” to blame for the problem (job security?). If it succeeds, they appear to have the proper troubleshooting skills. And very little effort expended.

What is interesting is how many times I have found the answer to be wrong when the actual business problem is examined. Here are some observations.

  • The original poster, not taking time to troubleshoot, makes an assumption on the solution (path of least resistance)
  • Respondent, taking path of least resistance, answers the question with links to people solving the problem posted
  • If the original poster had used troubleshooting skills, rather than assumptions, he would have thrown out other possibilities, included all relevant information to help others help him troubleshoot, and would have expressed the actual business problem
  • If the respondent had use troubleshooting skills, rather than assumptions (primarily the assumption the poster had used troubleshooting skills), he would have asked questions before giving answers.

To illustrate this, I once saw a post similar to the following on a Microsoft forum (meaning this is paraphrased from memory).

Can anybody help me. We have a site that has been working for years in IIS 4. We recently upgraded to Windows Server 2008 and the site is no longer serving up files located at C:\MyFiles. I just hate Microsoft right now, as I am sure it is something they changed in windows. I need to get this site up today, and f-ing Microsoft wants to charge for technical support.

The first answers dealt with how to solve the problem by turning off the feature in IIS that stops the web server from serving files outside of the web directory structure. While this was a solution, troubleshooting the problem would have shown it was a bad solution.

Imagine the user had written this instead.

We recently upgraded to Windows Server 2008 and the site is no longer serving up files located at C:\Windows\System32.

Turn off the feature in IIS would have still solve the problem, but there is now an open path directly to the system for hackers. And, if this is the way the person implements the solution, there is likely some other problems in the code base that will allow the exploit.

The proper troubleshooting would have been to first determine why ASP.NET files were being served from C:\MyFiles instead of IIS directories. Long story, as the reason had to do with an assumption developing on a developer box generally, perhaps always, led to development of sites that did not work in production. So every developer was working on a production server directly. The C:\MyFiles was created from an improper assumption about security, that is was more secure to have developers working from a share than an IIS directory. This led to kludges to make the files work, which failed once the site was moved to a server with a version of IIS that stopped file and folder traversing. This was done as a security provision, as hackers had learned to put in a URL like:;

Or similar. I don’t have the actual syntax above, but it was similar to above and it worked. So IIS stopped you from using files outside of IIS folders. Problem solved.

Now, there are multiple “solutions” to the posters problem:

  • Turn off the IIS feature and allow traversing of directories. This makes the site work again, but also leaves a security hole.
  • Go into IIS and add the folder C:\MyFiles folder as a virtual folder. This is a better short term solution than the one above. I say short term, as there is some administrative overhead to this solution that is not needed in this particular case.
  • Educate the organization on the proper way to set up development. This is not the path of least resistance, but a necessary step to get the organization on the right path. This will more than likely involve solving the original problem that created the string of kludges that ended with a post blaming Microsoft for bringing a site down.

Troubleshooting The Original Problem

I am going to use the original “Check in your files” problem to illustrate troubleshooting. The formula is general enough you can tailor it to your use, but I am using specifics.

First, create a hypothesis.

I cannot check in the files, so I hypothesize Greg has them checked out.

Next, try to disprove the hypothesis. This is done by attempting to find checked out files. In this case, the hypothesis would have easily been destroyed by examining the files and find out none were checked out.


So the next step would be to set up another hypothesis. But let’s assume we found this file as “checked out”. The next step is to look at the person who has the file checked out to ensure the problem is “Greg has the file checked out” and not “someone has the file checked out”.


Since the name Greg Beamer is not here, even if the file were checked out, he cannot solve the problem.

Next, even if you have a possible solution, make sure you eliminate other potential issues. In this case, let’s assume only some of the files were checked out when examined, but the user was still having problems uploading. What else can cause this issue.

Here is what I did.

  1. Assume I do have things checked out first, as it is a possible reason for the problem. When that failed, look at the user’s permissions on the files in question. I found this:
  2. Hypothesis: User does not have proper permissions. Attempted solution: Grant permissions
  3. Found out permissions were inherited, so it was not a good idea to grant at the page level. Move up to the site level required opening in SharePoint online, where I find the same permissions.
  4. Now, my inclination is to grant permissions myself, but I noticed something else.

    which leads to this

    which further led to this (looking at Site Collection Images):

The permissions here are completely different. The user is not restricted, so he can access these.

I did try to give permissions to solve the issue:

But end up with incomplete results:


I further got rid of the special permissions on some folders, as they were not needed. More than likely added to give the developer rights to those folders. I still have the above, however, which means someone more skilled needs to solve the problem.

The point here is numerous issues were found, none of which were the original hypothesis, which was reached via an assumption. The assumption was

I cannot check in, therefore I assume someone has the files checked out. Since Greg is the only other person I know working on the files, I assume he has them checked out.

Both assumptions were incorrect. But that is not the main point. The main point is even if they were correct, are there any other issues. As illustrated, there were numerous issues that needed to be solved.


Troubleshooting is a scientific endeavor. Like any experiment, you have to state the problem first. If you don’t understand a problem, you can’t solve it..

You then have to form a hypothesis. If it fails, you have to do over, perhaps even redefining the problem. You do this until you find a hypothesis that works.

After you solve the problem, you should look at other causes. Why? Because you either a) may not have the best solution and b) you may still have other issues. This is a step that is missed more often than not, especially by junior IT staff.

Let me end with a story on the importance of troubleshooting:

Almost everyone I know that has the CCIE certification took two tries to get it. If you don’t know what CCIE is, it is the Cisco Certified Internetwork Engineer certification. It is considered one of the most coveted certifications and one of the hardest to attain. The reason is you have to troubleshoot rather than simply solve the problem.

The certification is in two parts. A written exam, which most people pass the first time, and a practical exercise, which most fail. The practical exercise takes place over a day and has two parts:

  1. You have to set up a network in a lab according to specifications given at the beginning of the day.
  2. After lunch, you come back to find something not working and have to troubleshoot the problem

Many of the people I know that failed the first time solved the problem and got the network working.So why did they fail? They went on assumptions based on problems they had solved in the past rather than worked through a checklist of troubleshooting steps. Talking to one of my CCIE friends, he explained it this way (paraphrased, of course):

When you simply solve the problem, you may get things working, but you may also end up taking shortcuts that cause other problems down the line. Sometimes these further problems are more expensive than the original problem, especially if they deal with security.

Sound similar to a problem described in this blog entry? Just turn off IIS directory traversing and the site works. Both the poster and the hacker say thanks.

Please note there are times when the short solution is required, even if it is not the best. There are time constraints that dictate a less than optimal approach. Realize, however, this is technical debt, which will eventually have to be paid. When you do not take the time to troubleshoot, and run on assumption, build in time to go back through the problem later, when the crisis is over. That, of course, is a topic for another day.

Peace and Grace,

Twitter: @gbworld

Obamacare: Rising Prices for Subpar Insurance?

I have avoided writing a blog entry on this subject lately, despite all of the news, both positive and negative. I have avoided it because of the mass ignorance on both sides. Talking to my Conservative friends, I hear horror stories of premiums rising 300%. Talking to my Liberal friends, I hear how the rising prices are due to getting off subpar insurance and how much better off people are.

My opting in on writing something came from an article posted by James Shore (@jamesshore on Twitter) that essentially blames the hysteria over rising prices on the Tea Party. There is no more balance in the newer article (blaming the Tea Party) than the original only stating what the letter stated and not examining other options. The question is which is closer to the truth:

  1. People are paying more for insurance
  2. People are getting better insurance

In reality, it is a bit of both.

Equivalent Insurance for the same/higher/lower price?

The first thing you have to understand about the ACA is there are no equivalent plans. When I see someone stating “people are paying less for equivalent insurance” or “people are paying more for equivalent insurance” I say “bullshit”. There are no equivalent insurance plans.

This is due to the new standards for policies. Example? Find a plan in 2009-2013 that had a deductible or out of pocket max of $6350. You can’t. Why? Because the insurance  companies uses rounded numbers. You can find a plan with a $1000 out of pocket max. You can also find $2500, $5000, $7500 or even $10,000 or more. But you will not find one ending with $350. There was no good reason to do it.

I am personally under the view the government purposefully chose off numbers for minimum standards so you could not compare. It is not in any party’s best interest to be able to compare, and politicians are human. Human beings like to work out the rules to benefit themselves. So this is not a big conspiracy, just an “it is what it is”

The end result is it is hard to compare like plans. Now, part of this is the insurance companies. They may have had a $6000 out of pocket max plan that is now $6350. So the plan is actually worse than before and if the price went up, it is true you are getting less for more.

That leads us to the topic of subpar insurance.

Subpar Insurance?

Here is an nice little clip of Nancy Pelosi stating essentially “the prices are not going up; people are, instead, getting proper insurance”. In other words, the government came in and saved us from “subpar”. You can see the mantra in the video below.

And, Pelosi is correct … according to the government. According to the government, having a policy with higher than a $6350 out of pocket max or deductible is bad. This was not the ACA directly, but the HHS, who got to determine the minimum standards of care. Do you agree with the minimum standards? Just understand, it is not an individual choice what minimum coverage looks like any more. And, from that perspective (government should decide what is par), Nancy Pelosi is right that everyone whose coverage is cancelled had “subpar” insurance. If a $6350 out of pocket max or deductible are subpar, then high deductible plans are all subpar.

If we wanted to lower healthcare costs, a stated goal of the ACA, we should have more people on high deductible plans, paying their day to day care. This would foster more competition. The government’s role should have been to ensure pricing was transparent to level the field, not ensure everyone an insurance card. Take care of those who cannot afford care? Fine. But translate that to insure everyone? This goes contrary to the stated objectives of the law that was passed.

So, high deductible plans are bad insurance for individuals buying  their own insurance, according to the government. But who is the government to decide what is acceptable for individuals? What if the government told you the minimum acceptable size of a television was 44”. You can no longer buy a $100 7” television, as it is subpar. You now have to spend $400 to get acceptable minimum television. But the law also states you have to have a minimum of 90 Hz refresh rate and 1000p. Now it is $500.

And since there are currently no 44” televisions, you have to get a 45”. Since there is no 90 Hz refresh rate, you have to get 120 Hz and there is no 1000p,so you need to get 1080p. Now, the television manufacturers turn around and create 44”,1000p, 90 Hz television and call them bronze televisions.

The Conservatives bitch that the price went up for equivalent televisions and the Liberals bitch the televisions were subpar. But if people were happy with a 7” television, who is the government to state they can’t have it?

I realize healthcare and televisions are completely different in execution, but the concept is similar. And it is more the individual that determines what is good for them. If the ACA had created bronze through platinum plans, but also allowed individuals to determine if some plan fit the, even if it did not fit the minimums,then the bitching would be less justified, as there would be a choice for those who felt they should be able to decide what works for them. But that would not have been financially viable.

Some Americans truly did have insurance that was subpar. Ignoring this fact does not allow for an honest debate. They were given few choices due to their health or wealth, or lack thereof, and the new law has given them better choices. But the choices are more expensive and some Americans did not need the new “par” standards, or want them.

On the other side, others had good or even great insurance, but it missed one or more items of what is now considered par. When I examined 2013 plans, I found some insurance plans that were actually better for some segments of society that are now subpar. For example, under the ACA exchange plans, you can get a plan that has $65 copays for the first three visits, with no copay once you meet deductible. But if you are relatively healthy, the plan that had 20% coinsurance and no copay might have been less expensive. In fact, with standard doctor’s visit costs, it would have been cheaper than the new ACA plan for most Americans. Not allowed anymore.

But weren’t some plans grandfathered in? Sure, but very, very few. Why? The HHS set rules that substantive changes voided the grandfathered plans and then called a rise in premium substantive at a very low threshold. Since the cost of healthcare went up 15% in 2010, largely due to the new fees charged the medical manufacturers, health insurers and big pharma. Back to this shortly.

The points that are important are

  1. The HHS rules were adopted after the law passed, and voided insurance plans that might have been fine if the standards were decided differently. These are the plans we can firmly blame on the ACA.
  2. The minimum standards included standards in pricing and healthcare, so we are not just talking minimums for healthcare.
  3. The minimum standards are an “all or nothing” proposition. It does not matter if you have better insurance on all points but 1, you still have a cancelled policy.
  4. The government is deciding for you what minimum coverage looks like. This may be fine with you and may not.

Rising Prices

The prices are rising, at least in most states. Since you cannot compare ACA plans to 2013 plans directly, as , it is difficult, if not nearly impossible, to determine exactly how much. Are they rising 2-3 times the amount? Yes, in some cases keeping the same insurance with change provisions is 2-3 times the amount. But, to be fair, people can find less expensive options on the exchanges.

But the promise of Obamacare does not really come in unless you make somewhere between 138% and 400%of the poverty rate, as you get all, or a portion, of your premiums paid for you. And if you are at the lower end up the range, you might get your deductible and out of pocket max lowered for you, as well. In fact, if you are low enough, you can get plans with no deductibles and an extremely low Out of Pocket max.

The sad part here is 400% of the poverty level is well into the middle class, making the middle class a new entitlement class. I don’t think most of the middle class wants to be on programs like welfare, but they are now on the dole if they get the subsidies. But the exchanges are set up to make this less obvious, as you never see the full price of the insurance and may not even see the subsidy amount.

NOTE: Connecticut’s site shows “you are eligible for a subsidy up to $X” but does not show the subsidy amount on the page where you pick policies. I use this site, as you could shop, from the beginning, without putting in any of your information until you checked out.

The media currently heralds 2013, through October, as a banner year for healthcare, with the lowest rise in the cost of healthcare in decades. They also state 2012 was on track with other years, down from 2010 and 2011. This is stated to be proof the ACA works. But does it? In 2010 premium prices went up in the double digits and in 2011 it was up almost 10%, the highest rises in decades. Is 2013 proof the ACA works, or an overly pessimistic market responding. When the new ACA fees went into effect, perhaps the insurance industry panicked?

I predicted the fees would drive up the cost of healthcare when I read the bill(unlike some of our Congress Critters?). The fees, in the billions, had to have an impact. I now see a big rise in 2014, as well. Why? Because we have pushed so much of the law into 2015.

Here is something to consider. Insurers set 2014 prices in 2013, with the idea 7 million would sign up on the rolls by the end of March. But the provision they sign up or be fined has been pushed to 2015. And the provision the insurance companies must fit the framework is also pushed to 2015. The insurance industry has a lot of leeway in 2014 and you may even see mid-year hikes in premiums, ala this year.

Better Insurance?

We then come to the question of whether moving up to par gives you better insurance. If you truly had subpar insurance, as determined by you, not the government, then you may have a better written policy. But it may not truly be a better policy.

One of the unintended consequences of this law was the lengths both the medical providers and insurance companies would go to make money. Many medical providers have opted out of ACA exchange plans, deciding to only take employer plans (and other group plans) at this time. They are willing to forgo less than 5% of the populace as their plans pay less for medical services.

What this means to exchange plan holders is they may not be able to see their doctor, as he is not taking the plans. This does not sound too bad until you look at places where the best hospital for their type of care has opted out of exchange plans. Or worse, as in the case of Seattle Children’s Hospital, which was dropped from exchange plans, they have to travel hundreds of miles to get cancer care for their children, or choose care in an adult hospital, which is not skilled in pediatric cancer, thus lessening care. This is not critical in all areas, of course.

Since I mentioned Seattle Children’s Hospital, I should note that not all limitations in choice are being made by the provider choosing non-exchange plans. In the case of Seattle Children’s Hospital it was the insurance company. This means some of us may get our care lessened even if we are still covered by group plans, as a  reaction to the exchange plans.

Either way, the policy is better on paper, but not necessarily better in the real world. And tightening networks is not the only way the insurance might have gotten worse.

Under the ACA the playing field is leveled quite a bit. On the positive side, it means those unable to get insurance now can get insurance. But the price has to go up for the healthy to pay for the sick adding to the rolls, sometimes significantly. There is a push to get young people on the rolls, as they are healthy and paying a large portion of the bill. In fact, a 40 year old will have better rates than a 26 year old. Don’t believe me? Run the same plan for two males, one 26 and one 40, both with zero dependents. Nice, huh?

Affordable Insurance?

The ACA has made insurance more affordable for some.Those, for example, who have their entire premiums paid and deductibles and out of pocket max lowered, have more affordable insurance. If they also have a pre-exsiting condition, they have a major win.

But since the average American, even with our horrible diet, does not fit these cases, are we not setting the rules for the exception rather than the rule?

I know a great many cancer moms (see note) that saw the ACA as a godsend. Many are now seeing it is going to bankrupt them more unless they lower their income significantly. Under the ACA, unless you have a good portion of your insurance paid, you are going to pay more per year in 2014 for your serious illness than in 2013. I am not stating this is true for all, just a large number of people. With the minimum standards, it is easy to see why.

NOTE: My youngest daughter is a 5, almost 6, year cancer survivor.

The Bottom Line

Some of the people that are getting notices truly had what all of us would agree was subpar insurance. But some did not. Some just had insurance that missed one of the provisions the government states makes insurance par. They  actually had better, more affordable insurance.

The picture is not all doom and gloom, but there were other ways to solve this. And here are a few facts that may shock some of you, as you may be hearing things to the contrary. First, the Republicans did have alternatives to parts of the plan. You may wish to debate on whether or not they were good alternatives, but to say there were none is patently false. It takes a bit of a Google search to find other voices, but they are there if you search hard enough. Second, the Republicans are not balking against the ACA now that it is being put into action, as not one voted for it. You can look at the rollcall and find this out (and please don’t comment back with the action in the house to suspend debate, which some Republicans voted for, as that was not a vote for the bill – in fact, the original house bill had NOTHING to do with healthcare – the Senate stripped 100% of the wording from the bill and started over, something that, by the spirit of the law, is illegal, but apparently not by the letter of the law).

The ACA was voted in as soon as the Democrats had a filibuster proof majority between them and Independents. This would not matter today as Reid changed the rules of the Senate so a filibuster can be broken by a simple majority. It was not a compromised bill; it was one that fit one parties view of what is good and bad, without debate from the contrary side.This does not mean the law is wrong, only that there is a greater chance at least some of the provisions are flawed, if not downright bad, as no ideology is right 100% of the time. There is both good and bad in the law, and, from my reading of the law and HHS provisions, a lot of it is bad, as it does not meet its objectives and serves to drive up prices.

The author of the article, Maggie Mahar, is almost assuredly right that the woman could have gotten cheaper insurance than the $1000 she was quoted on the cancellation notice option(s). But the case is anecdotal. What is factual, from a scientific standpoint, is insurance rates are going up rather significantly in most states for individual policy holders.  Not significantly, as in 300%, but at a rate much higher than rates went up prior to the ACA passing.

But Greg,you might say,I see plenty of examples where the rates are much lower. Do you? Or do you see the total amount lowered? The total amount, in case I have seen that are lowered, is after subsidies. If the American taxpayer is paying a large portion of your premiums,of course the total cost, for you, is lower. But the actual costs, which the rate is based on, is higher. There are only a couple of states where this is currently not true.

To put this in perspective, let me consider my daughters.Suppose one wanted a new toy that costs $200 and I decided to buy it for her, so her out of pocket costs were $0. Did the price go down from $200 to $0? No. The cost remained the same. Let’s say the toy went up in price to$250 and I still decided to pay all of it. In that case, the price did not even remain the same, but from her perspective, the cost was$0, despite a $250 price. Much of the talk I see on the Internet about lower premiums is not lower premiums at all. It is lower out of pocket costs for premiums due the American taxpayer footing part of the bill. What gets me is some people are paying more even with subsidies.

My point here is this:

  1. Naming the Tea Party boogie man is a red herring
  2. While stories out there may be exaggerated (300% increase in premiums), it does not mean higher than average increases are not happening. And, by looking at facts, you can easily find the rates in many states are going up far more than the average.
  3. Insurance under the ACA is not necessarily cheaper. Even if we have cheaper premiums and better policies, cost of deductibles and out of pocket maxes may bankrupt families. Add on shrinking networks and it could be worse.
  4. Anything is less costly to you when someone else is footing the bill.

Peace and Grace,

Twitter: @gbworld