Over the past three to four years I have spent extensive time working with customers to improve performance. As part of this endeavor, I have had the pleasure of working with numerous internal, external and third-party development teams for these companies and the experiences have provided a great deal of insight to how people look at performance with software development. This experience transcends platforms as the specific concern with this post has been experienced with ASP.NET MVC Projects, DNN Projects, WPF projects, and even those on mobile devices. Although each platform or process will have different rules and requirements for performance and methods to manage it, there are a few constants. This post is going to be the first of many posts around performance optimization and development. best practices.
What is A Cache?
Most developers that I talk with understand conceptually what a "Cache" system is, however, it is the best practices around the usage that causes that biggest confusion. Simply defined a cache is "a location to temporary store things." These things could be strings, they could be query results, images, or anything else that works with your application. The key is that they are items with a shortened lifetime.
The key to understanding the utility of the cache is that its purpose is to reduce the load from other areas of the system. For example, caching the results of a database query for 10 minutes helps reduce DB Server time and network bandwidth of a frequently used query. It can be also used to store information on a user-by-user basis to help under periods of heavy loads. If information isn't loaded from the cache the item is retrieved/built, put in cache, and returned to the user. The overhead of a cache check operation is minimal and a simple cost of doing business.
The specific cache system used, is not necessarily an item to be too heavily concerned with. Many options exist including MemCache, Redis Caching, and even the built-in Application Cache object inside of the ASP.NET run-time. The key is to find the cache system that matches your infrastructure.
The Common Misconception
As I talk with software vendors I hear one of two common comments. "But my information must be real-time" or "I have too much data.". I'll try to cover these two items as much as I can below.
But My Information Must Be Real-Time
Real-time information is something that is very rare in day to day operations. Most organizations, even those with lots of change can handle a certain level of "staleness" of application data. Additionally, routines can be added to purge cache if the information does change on a regular basis. To put a real-world example to this, I was working with an organization that had a solution that allowed users to interactively manage web-content. Their claim was that due to the nature of users needing to update content on a regular basis that none of their query results or display situations could be cached.
In researching application usage it was determined that regardless of this opinion the changes were happening at a rate of fewer than 2 updates per hour. Nothing at all that would cause huge concern. The concern regarding related data that needed to be refreshed was a valid concern but easily mitigated. We recommended a caching strategy that was aggressive with 20 minute cache times, however, when a new or updated piece of information was saved the cache was enumerated and the old information purged. Yes, the cache traversal was a more expensive operation, but we substantially reduced the load on the application.
I have too much data
The second most common concern is that individuals worry about what they are putting in cache. This is an important item to understand and consider, but it shouldn't scare people away from the cache. Looking at Redis Caching from Azure at the lowest price tier for development/test environments we can have access to a 250 Mb cache for ~$16/month or a 1 Gb cache for ~$41. Increasing to the production tier with replication and a high-availability SLA the pricing goes to ~$41 and ~$103 a month respectively. This is a low-cost operation for most organizations that would "have too much data." Azure offers Redis Cache sizes up to 53 Gb for those with truly too much data.
Simple math calculations will help to determine the best & worse case usage scenarios as it relates to your application and the ability to cache. Should you go past the cache limit, your cache simply will not be as effective. It isn't the end of the world.
A Real-World Look at Cache
I can talk all day about the benefits, but it is a lot easier to take a real-world situation and discuss it in numbers. For a client, I recently developed a charting display module that executed complex database queries and used the resulting data to display a Google Chart to the user with the result. This particular client has multiple instances of this process in place on the homepage for users of the application. A page that is visited on a regular basis, and often revisited multiple times in short succession.
Last year, they were using a third-party solution that did not have any caching in place. As they hit their peak traffic load of close to 7,500 concurrent users the systems started to fail. We disabled the charts and the application instantly recovered. The queries to generate the charts are costly and needed to be cached. We implemented a new model this year where the entire result of the query and the generation of all Google Chart code is stored in a Redis Cache.
Looking at statistics for the past 24 hours, under low traffic with around 750 concurrent users only, more than 55,000 database queries have been prevented by successful cache retrievals. The database server & web servers have been operating at historical lows from this reduction in load.
Conclusion
So what does this mean? My primary focus here is to get people thinking outside of the box at their performance issues and application structures. If you can introduce caching to improve performance it is time well spent. It isn't possible to always make every aspect of an application fast, so use all of the tools at your disposal.