If you externalized the cache itself, then your cache will likely support LRU or LFU expiration policies. But, if you're using a in-memory cache in the Java heap space, well, that's a memory leak. If you cache a lot of fragments, failing to clean the cache is a recipe for maxing out the old generation space and thus causing Full GC's to run back-to-back, collecting just enough space to limp along, but not do any good, until eventually your application becomes non-responsive. So, an additional requirement for an in-memory fragment cache is that you monitor the usage and have a cleaner task which runs, say, once an hour, to remove fragments that have expired and are not currently locked for update. Doing this efficiently with a large cache requires using a data structure that supports thread-safe concurrent access, so as to avoid locking the whole cache while cleaning and maintain high throughput.
Cash is no longer king
1 hour ago