Issue with ASP.NET Temporary Files
March 14, 2008 Leave a comment
Not sure this is a bug, per se, but it certainly is something that can really chomp down on your hind quarters.
Update a site where libraries have shifted. At the point you update and restart, the new files should take over and be JITted, yada yada. The problem is that sometimes the ASP.NET Temporary files are not crushed to death like the worms they are and you end up with errors that you cannot figure out. "But I just deleted all dependencies to that library".
Okay, a story is in order as I can see I am going to confuse the hell out of my readers without context. At work, we have been working with a vendor for a few years. They have a back end product that is essential to our operation. And, while we can see room for improvement, nobody else on the market is doing it much better, if at all. Until we find time to develop our own, we have to go with somebody’s solution.
Forward to the past few days. We have to update the back end. Normally not an issue, but we find implicit coupling between their back end and the sites (which were also, in part, developed by the vendor — I blogged generically about this here and here). The coupling is largely there due to a custom serialization piece, which, as an artifact, requires not only that I send an object with the right properties (in serialization), but that I am on the same libraries. Before anyone thinks to start pointing fingers, I must state that I have seen this particular programming technique on three different Enterprise level projects, with some advanced programmers (as with this vendor), so cockiness is not warranted. Fixing the problem is.
This issue is compounded by the fact the vendor is restructuring libraries. It is partially a refactoring move, as migrating classes into better named assemblies makes things clearer. But, it breaks all public interfaces. Not a problem with the backend, but a serious problem for sites that must adhere to the new libs.
Okay, problem covered.
The Bug and a Solution
I used the evil B word here, but I truly do believe this is a bug. It is a minor bug, overall (although I will show another case where I think it reared its ugly head a bit later), as most of us are in better control of our world and not going through major name and namespace changes in our class libs. This makes the testing of this "bug" a fringe case, which are often missed, so I am not potshotting Microsoft on this one.
The first symptom I had to show the bug was this:
Could not load file or assembly ‘#######.BusinessObjects, Version=1.1.2780.26883, Culture=neutral, PublicKeyToken=null’ or one of its dependencies. The system cannot find the file specified.
This error happens when a DLL is missing from the /bin folder. Normally, the solution is to add the DLL and roll. The problem is this particular library was GONE. It was no longer a part of the site, as it had been refactored out by the vendor.
Solution 1: Rebuild the website definition in IIS. Then hook to www2.sitename.com, where the old site is www.sitename.com. This works, as the site is JITted off of the new directory in IIS, making a whole new Temporary ASP.NET directory for the JITted assemblies. Kewl.
Later on (about 10:30 PM, or so), I decided to short circuit and remove the testing of the new directory before flipping host headers in IIS. This yielded a new surprise. The site was now messing up due to finding the compiled class in two locations … yes, another Temporary ASP.NET fubar. Since I did not test in the directory forcing JIT, solution 1 would not work.
Solution 2: Turn off local IIS and shut down Visual Studio. Clear out Temporary ASP.NET files locally. Open Visual Studio and Publish site locally. The above is just a safety measure and I do not think it actually helped anything. If I get a chance to repro the "bug" to determine exact cause, I will test with and without these "pre-steps". I then moved the files to the new server and set up in a directory. I created a new IIS website definition and pointed to www2, leaving the original pointed to www. I then shut down IIS and cleared temporary ASP.NET files. While still shut down, I switched the host headers and then started IIS back up (actually the WWW publishing service, if you want to be technical).
Hindsight being 20-20, I realize that properly JIT compiling the new site would probably work and explains why solution 1 worked the first time and not the second. I also suspect that I could stop IIS and clear out Temporary ASP.NET files while deploying and end up with the same happy ending.
I have not fully confirmed this, but behavior suggests that ASP.NET JIT does not follow the site naming directly. What I mean here is each website in IIS has a particular log directory, which is tied to the website definition. In ASP.NET Temporary files, it appears that the name of the site and/or host header name enter into the equation. Seeing how the files are organized, I think I am on the right track.
Now, I thought great. I found a "bug", but most of the world will never see it, as most of us are not shifting names and namespaces. Major bug in impact, when it hits, minor in the number of people it will hit (almost nobody). But, it appears I might be wrong. Here is a post in microsoft.public.dotnet.framework.aspnet (subject = .NET Framework 2.0 SP1 causes application to fail on references):
I have a web application deployed in a Production environment, where
it has happily been running for the past 2 years on the .NET 2.0
framework. However, the operations team recently installed the .NET
2.0 Framework SP 1 on the server, and immediately certain sections of
the web site began failing with the errors "Could not load file or
assembly ‘blah’ or one of its dependencies. The system cannot find
the file specified".
I am wondering if others have experienced this issue and/or know why
the installation of the patch has started causing my application to
have problems. When we uninstalled the patch the site began working
as normal again – is my web.config file misconfigured with references
I don’t need and may not exist on the server (I didn’t add these
particular references, the framework did when I compiled), and the
patch now causes the framework to behave differently in validating
This sounds precisely like the problem I was having. I will follow up when the OP gets back (either email or in the newsgroups) and write a KB article if this is, in fact, an issue. His may, in fact, be a completely different issue.
Peace and Grace,