Friday, April 10, 2009

FragmentCache part duex: Cleaning

I realized that I forgot to address an additional, critical, issue with fragment caching: cleaning out expired fragments.  A fragment cache implemented as I described in my last post would update expired fragments when requested, but what about expired fragments that aren't requested?  

If you externalized the cache itself, then your cache will likely support LRU or LFU expiration policies. But, if you're using a in-memory cache in the Java heap space, well, that's a memory leak.  If you cache a lot of fragments, failing to clean the cache is a recipe for maxing out the old generation space and thus causing Full GC's to run back-to-back, collecting just enough space to limp along, but not do any good, until eventually your application becomes non-responsive.  So, an additional requirement for an in-memory fragment cache is that you monitor the usage and have a cleaner task which runs, say, once an hour, to remove fragments that have expired and are not currently locked for update.  Doing this efficiently with a large cache requires using a data structure that supports thread-safe concurrent access, so as to avoid locking the whole cache while cleaning and maintain high throughput.