Coding Lifestyle

Caching woes in the garden November 6, 2006

Over the past few weeks I have received numerous bugs on a SharePoint project I’ve been trying to close.  This project had a lot of custom webparts and added functionality.  These various reports all centered on inconsistent behavior resulting from the cache being lost.  The thing was everything worked fine in the development environment (of course).  What was causing the server to give up its cache so quickly?  I tried varying settings in the web.config, monitored application recycles, monitored if the virus scanner was touching web or machine.config.  I tried PartCache, Application State, Page.Cache, and tweaked every setting they had.  I was even reduced to using (shudder) hidden fields in a few cases.

Finally I wrote a webpart for testing the cache.  It allowed you to submit a value or simply refresh (postback).  The webpart stored and displayed the value in every available type of cache: View State, Cache, WP Shared, WP Personal, Session, AppState.

I quickly discovered an odd behavior.  The cache was cycling between values.  I’d submit “one”, “two”, “three” and then simply postback and see the cache cycle between these 3 values.  I shut down every other web site on the server and opened task manager.  Lo and behold I saw three, count em three, w3wp.exe processes running.

Finally the penny dropped and I opened the application pool properties.  On the performance tab there is an innocuous setting at the bottom called Web garden.  While I was away on holiday the customer had been complaining about performance so someone had set this to 3 and forgot to tell anyone.  This simple action, which does not flash any sirens, bells, or whistles, completely changes the environment in which the code runs.  A web garden has multiple processes; a server farm has multiple physical machines with a load balancer.  The effect on cache is the same.  In a garden/farm environment you must persist cache using a custom provider, namely a SQL database which persists a single copy of the cache so that multiple servers/processes all point to a single copy.

You can imagine the impact this had on a code which relied on various types of caching, either to cache data or to maintain state for an individual user.  By changing this single setting back to 1 process, everything was back in order.  If I had a swear jar, I would be destitute.  Here is what MSDN has to say for those who read about the ramifications of their changes before doing:

“Because Web gardens enable the use of multiple processes, each process will have its own copy of application state, in-process session state, caches, and static data. Web gardens should not be used for all applications, especially if they need to maintain state. Be sure to benchmark the performance of the application before deciding whether Web garden mode is appropriate.

When using a Web garden, it is important to understand how session state and round robin works. It is also important to consider of how other application pool settings affect the application.”

So, the lesson learned?  Next time you have a funky caching problem add this to your list of things to check.  Also, swear jars and programmers don’t mix.