Sunday, April 5, 2009

The JamLegend Fragment Cache

Abstract
The benefits of fragment caching are pretty well-known.  If you haven't come across it before, the concept is simple:  cache portions of the page to lighten database load by reducing or eliminating redundant queries.  Often, this can be the "most bang for your buck" type of cache. How long a fragment should be cached is a function of the expense of generation, frequency of changes, and the frequency of access.  Ideal fragments for caching are the most accessed and most expensive to generate.

The fragment cache I'm going to discuss is the implementation of the FragmentCacheTag JSP custom tag, in J2Free, the one-day-to-be-open-source framework on which JamLegend is built (you can find more info on J2Free here, and in a few of my previous blogs).  

JamLegend Examples
The "Popular Today" songs lists on JamLegend are a little expensive to generate, shown on all the highest trafficked pages, and don't change too quickly.  As such, we cache that list for 5 minutes, saving us a lot of queries but keeping the list rather dynamic.  On the other extreme, comments change constantly and need to be much more dynamic.  We cache comment threads for 30 seconds.  When you have a high number of concurrent users, even a 30 second cache can save you a lot of queries without sacrificing liveliness.

The Code Samples

<cache:fragment timeout="${30 * 1000}">
Simple cache based on a timeout of 30 seconds.
</cache:fragment>

<cache:fragment timeout="-1" condition="${size(items)}">
Simple cache with no timeout, using instead an additional attribute to refresh the cache when the size of the collection "items" has changed since the cache was last updated. This works well in situations where data changes rarely, and you want it to be available immediately upon change.
</cache:fragment>

<cache:fragment timeout="${30*1000}" disable="${devMode}">
Simple cache based on a timeout of 30 seconds but with an attribute to disable the cache if devMode is flagged. Here, devMode is a application scope attribute set when the server starts up. It's nice to be able to selectively disable certain caches for testing, and have them enabled live without having to change any code.
</cache:fragment>

The JamLegend FragmentCacheTag also supports an additional application scope attribute to globally disable all caching.  One of these days, I'm going to get around to changing the timeout to be specified in seconds, instead of milliseconds; just hasn't happened yet... 

Fragment Caching "Gotchas"
Two of the first problems you'll run into with fragment caching are:
  1. logged in vs. logged out users
  2. concurrent cache refreshing
The first is a problem you're going to encounter simply by using a fragment cache.  The second, on the other hand, you'll only run into if you're building your own fragment cache; if you use the J2Free FragmentCacheTag described here, it will be taken care of for you.

Adding Dynamic Capabilities to Cached HTML 
Sometimes ideal candidates for fragment caching have display differences when users are logged in.  For example, caching comment threads:  logged in users see a "report" link, whereas logged out users do not.  Since a fragment cache holds the generated HTML, as opposed to an object cache or query cache, it isn't easy or efficient to modify the fragment based on logged-in status (though, it's not impossible and I'd appreciate any elegant approaches).  Fortunately, JavaScript comes to the rescue.  

On JamLegend, a "report" link is included in the generated HTML for comment threads whether the user is logged in or not, with style="display:none;" to hide it initially.  Then, if the user is logged in, we use a little JavaScript on page load to show all the report links, otherwise they stay hidden.  This is the same technique used by 37Signals (see JavaScript Makes Relative Times Compatible with Caching).  Since we also check server-side that a user is logged in before allowing them to mark a comment as spam, there isn't a significant downside to having the hidden spam links for users who are not logged in.

Preventing Concurrent Refreshes
If you're caching a fragment because it's expensive to generate, the last thing you want is a few hundred users concurrently refreshing the fragment triggering a few hundred concurrent expensive queries.  So, how do you prevent this?  The first question you have to answer is this:
When user A has triggered a cache refresh, what should happen to other users who hit the same fragment?
Should they:
(a) Head over to user A's thread, wait for user A's thread to complete the query, and then ask for the result?  or,
(b) Should they just see that user A is refreshing the cache and so grab the last cached fragment and return immediately?
Both are acceptable answers, but which you choose depends on how stale you allow your fragment to be.  In case A, you can serve the most recent fragment to an unlimited number of callers while still only generating the fragment once.  Since user A triggered the refresh first, that thread is necessarily ahead of the other threads, so the others will still return is less time than if they were the sole thread refreshing the cache.  In case B, the subsequent threads can return immediately serving the last version of the fragment while assuming that user A's thread will refresh the cache.

For J2Free's FragmentCacheTag, I chose scenario B on the reasoning that the fragment is cached to begin with because it is not crucial that it be 100% up-to-date.  Ergo, what's one more minute with the old fragment?  At some point, I want to add this as an optional attribute, something to the effect of:

<cache:fragment timeout="600000" waitOnRefresh="true">
... where waitOnRefresh enabled would indicate that the cache should not server the last fragment while being refreshed, but rather attach itself to the thread conducting the refresh and get the result when complete.
</cache:fragment>

Obviously, the above only applies to multi-threaded servers.  There is an additional case, though, that applies to single and multi-threaded servers:  what happens in a cluster?  How do you prevent concurrent refreshes when you're running multiple servers?  A few options are: 
  1. Store fragments in a distributed cache, like ehcache, JBoss cache, or memcached, using replication instead of invalidation.  However, there is still the possibility of instance A triggering a refresh then instance B triggering a refresh before instance B received the updated fragment from instance A.  This window could be decreased by replicating a lock-for-update on a fragment, rather than waiting for the complete fragment, but not closed entirely.
  2. Using a central, synchronized cache would remove the race-condition from situation 1, but at the cost of decreased throughput.  Depending on the location and network access to such a cache, it may or may not be worth it.  Ping times intra-ec2-zone are only 1-2ms between instances, so it might be worth it to add a few ms to each request to avoid concurrently executing an expensive query.
  3. Use a distributed cache, as in situation #1, but only allow 1 instance to refresh the cache.  This may or may not work depending on how well you split requests between instances.  You certainly wouldn't want users on instance B to continue to receive stale data because no user on instance A has triggered a cache refresh.  This strategy may work well for pages that are guaranteed to be accessed on each instance.
I welcome comments on any additional solutions to preventing concurrent cache refreshing in a cluster.

85 comments:

Joe Hopkins said...

Thanks Ryan for this great article... however when mentioning distributed caching solutions you forgot to mention one very important player; NCache. They've been in the market for close to 4 years now and with time they've come up with some very nice feature list. Alachisoft also has a Free Distributed Cache called NCache Express.

Ryan said...

Hi Joe,

Thanks for mentioning NCache. I didn't mean to specifically exclude it, rather I haven't actually used it myself (since I haven't worked much with .NET) and so it didn't come to mind when listing options.

Anonymous said...

I do wish there were more people like you around on the interwebs. cream penghilang bekas luka

mba dissertation said...

I talk about with this exceptional redeployment. I in fact do enjoyment in your personal blog though and will keep pending rear for posts.

Meilibahenling said...

Thanks for sharing .Keep posting articles like this. A good example of content presentation.

Lee Shin said...

spot on with this write-up, i like the way you discuss the things. i'm impressed, i must say. i'll probably be back again to read more. thanks for sharing this with us.

Lee Shin
www.trendone.net

Pemutih Emilay said...

Hello, i am glad to read the whole content of this blog and am very excited and happy to say that the webmaster has done a very good job here to put all the information content and information at one place.

Emilay Whitening said...

Hello, i am glad to read the whole content of this blog and am very excited and happy to say that the webmaster has done a very good job here to put all the information content and information at one place.

party organizer said...

This blog was really an awesome site which I had never found it anywhere. Lots of stuff in this site !

mas jangkrik said...

You've written nice post, I am gonna bookmark this page, keep your post jual anion whitening

mas jangkrik said...

Appreciate your sharing this best posting. jual baju anak

conducting research for your paper said...

Great article! it is an eye opening to all. and i'm very much thankful that she wrote her experience for us to let know them respectively

Anonymous said...

Iam very happy for read....Nice Article And Have A Good Luck. Keep Post More About This. visit here

mas jangkrik said...

You've written nice post, I am gonna bookmark this page, thanks for info. Thumbs Up sewa mobil murah jogja

Credit Card Processing said...

I am really enjoying reading your well written articles. I think you spend numerous effort and time updating your blog. I have bookmarked it and I am taking a look ahead to reading new articles. Please keep up the good articles!

mas jangkrik said...

Thanks for sharing this great article! I am always searching for informative information like this! click here

mas jangkrik said...

I love your blogs, Thanks for the nice post. I have already bookmarked it http://www.rental-mobil-semarang.com/ :D

Anonymous said...

guest house in pakistan

Anonymous said...


desi food

Anonymous said...

Rusk Exporters From Asia

mas jangkrik said...

Wow amazing, Nice content I found so many interesting stuff in your blog especially its discussion Thanks to sharing thanks! jasa rental mobil jogja

mas jangkrik said...

Thank you for the great article I did enjoyed reading it, I want to encourage that you continue your great job, have a good day. http://www.hajiku.com

deductible said...

Excellent post. I want to thank you for this informative read, I really appreciate sharing this great post. Keep up your

mas jangkrik said...

Your ideas was a blessing. I was looking for this kind of article for my paper and I got here. Truly educational. click here

Aissa said...

Nice post.Thank you for taking the time to publish this information very informative! So happy to be given a privilege to post a comment here.

aissa
www.joeydavila.net

glutera asli said...

Cool post very informative I just found your site and read through a few posts although this is my first comment, i'll be including it in my favorites and visit again for sure.

mas jangkrik said...

Thanks a lot for such a wonderful post, the stuff posted were really interesting and useful. click here

SEO Manchester said...

I think I am quite anxious about this technical meeting and I want to know the feedback of this meeting.I am sure it will be an extremely informative one and all those will attend it will be quite beneficial.

Timber Merchants Liverpool said...

When virtue and modesty enlighten her charms, the lustre of a beautiful woman is brighter than the stars of heaven, and the influence of her power it is in vain to resist.

pradis tiany said...

hi!
it's a great post. I talk about The benefits of fragment caching are pretty well known.
do you mind if check this one too?
Semberani

http://www.howtoloweryourmortgage.com/ said...

Thanks for making such a cool post which is really very well written.will be referring a lot of friends about this.Keep blogging.

more information said...

Actually I am not habituate to reply a comment on blogs but this blog caught my attention and I am compelled to write a comment here to express my gratitude to you. I would like to appreciate for this awesome work.

Roller Shutters said...

It is a very profitable post for me. I've enjoyed reading the post. It is very informative and useful post. I would like to visit the post once more its valuable content.

AnonymousJeffers said...

The views across the office have made this post an interesting topic of discussion. Some more please. CleverBox Marketing - SEO Manchester

Anonymous said...

Now could be the time to look to increase visitor numbers to this page. It is a great read and will appeal to a wide audience. SEO Agency Manchester

gippy honey said...

Provigil Online Flaming enthusiasm, backed up by horse sense and persistence, is the quality that most frequently makes for success.

paul sam said...

Older people sit down and ask, 'What is it?' but the boy asks, 'What can I do with it?'.
Buy Sleeping Pills

Fashion Shoes said...

We live in a wonderful world that is full of beauty, charm and adventure. There is no end to the adventures that we can have if only we seek them with our eyes open.

Bridal saree said...

This article is very interesting, the writing is very vivid, very pleasant surprise pole; At this point, I do not know what I want to describe. At least the next year or so for his college fund contribution, I get a university degree.

mical clark said...

No one wants to die. Even people who want to go to heaven don't want to die to get there. And yet death is the destination we all share. No one has ever escaped it. And that is as it should be, because Death is very likely the single best invention of Life. It is Life's change agent. It clears out the old to make way for the new.
Sleeping Tablets Online

Anonymous said...

Great post. You should really look at enhancing your social media credibility. If we can be of help feel free to check us out. Buy Facebook Likes

mical clark said...

I've failed over and over and over again in my life and that is why I succeed.
Valium Online

Anonymous said...



If you are looking for a fun game that is in-depth, but not too complicated, Hay Day is perfect for you.
Hay Day Hack Tool

kitty marry said...

The first written records from the historic Olympics being the primary Olympic champ. wifipasswordhacks.com

Jim Lee said...

Laughingstock car will be affiliate open-complete finally-someone view sport where the gamer settings a goat.
Goat Simulator Download

Rehanna Rehanna said...

Clash Of Clans Hack Tool No Survey

http://clashofclanshacktoolnosurveoy.tumblr.com

Anonymous said...


Souplantation Printable Coupons

http://docs.google.com/document/d/1pQTEyWr1N_ny35mSEMsuCEH5Jo2FMX2WZ-jvtzqXjoY/edit

karya ismail said...

This is a great article, Thanks for giving me this information. Keep posting
Mebel Jati :
Mebel Jati :
Kursi Tamu Minimalis :
Mebel Jepara Murah :
Mebel Jati Jepara :
Mebel Jepara Online :
Mebel Jati Jepara :
Mebel Jepara :
Kursi Tamu Minimalis :
Furniture Jati Jepara :
Karya Priboemi :
Kursi Tamu Jati Murah :
Mebel jepara Online :
Mebel jepara Online :
Meja Makan Jati :
Mebel Jepara Murah :
Sofa Tamu Mewah
Meja Makan Jati Minimalis
Tempat Tidur Jati
Mebel Jati Jepara
Mebel Minimalis
Mebel Jepara Murah
Mebel Jepara Murah
Kamar Set Minimalis

Cindy Dy said...

You have a wonderful blog and it is quite unique and interesting. Keep it up and hope to read more updates from your blog.


Jim
www.imarksweb.org

joy said...

I really had a great time stopping by your site, it gives me a lot of information. You can also visit my site, thank you!

n8fan.net

www.n8fan.net

mebel jepara said...

thanks for your information..its very good...please visit my website on kursi tamu minimalis

Slaaptabletten kopen op Internet said...

The real sin against life is to abuse and destroy beauty, even one's own even more, one's own, for that has been put in our care and we are responsible for its well-being.

jhoney said...

valium online purchase
This is what I have been searching in many websites and I finally found it here. Amazing article. I am so impressed. Could never think of such a thing is possible with it...I think you have a great knowledge especially while dealings with such subjects.

Thesis writing service said...

thanks for sharing this wonderful post.very good stuff

Thesis writing service said...

You have wonderful site and good writing

buy diazepam said...

Great info. I love all the posts, I really enjoyed, I would like more information about this, because it is very nice., Thanks for sharing.

jhoney said...

diazepam online pharmacy
where did u come up with the information on this posting?I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work.

Regalos Originales said...

We want to reinvent the phone. What's the killer app? The killer app is making calls! It's amazing how hard it is to make calls on most phones. We want to let you use contacts like never before - sync your iPhone with your PC or mac.

jhoney said...

Regalos
Productos hasta 85% de Descuento. Envío en 24-48h. Productos de Cocina, Electrónica y Electrodomésticos, Hogar, Perfumes y Cosmética, Regalos, Salud y Belleza, Fitness y Deportes

mical3211 said...

Productos hasta 85% de Descuento. Envío en 24-48h. Productos de Cocina, Electrónica y Electrodomésticos, Hogar, Perfumes y Cosmética, Regalos, Salud y Belleza, Fitness y Deportes
Rebajas Online

sarah lee said...

I really enjoyed reading your article. I found this as an informative and interesting post, so i think it is very useful and knowledgeable. I would like to thank you for the effort you have made in writing this article.


edupdf.org

jhoney said...

Buy Facebook Fans
Really you have done great job,There are may person searching about that now they will find enough resources by your post.I like this blog..

IB Maths tutor said...

IB World Academy has been helping students excel in IB Math(HL,SL,Studies tutor in Gurgaon , IB Physics(HL,SL)tutor in Gurgaon , IB Chemistry(HL,SL) tutor in Gurgaon , IB Economics(HL,SL) tutor in Gurgaon and Business Management(HL,SL) by providing the right skills for success. IB World Academy has been helping students excel in IB Math(HL,SL,Studies tutor in Gurgaon , IB Physics(HL,SL)tutor in Gurgaon , IB Chemistry(HL,SL) tutor in Gurgaon , IB Economics(HL,SL) tutor in Gurgaon and Business Management(HL,SL) by providing the right skills for success. IB World Academy has been helping students excel in IB Math(HL,SL,Studies tutor in Gurgaon , IB Physics(HL,SL)tutor in Gurgaon , IB Chemistry(HL,SL) tutor in Gurgaon , IB Economics(HL,SL) tutor in Gurgaon and Business Management(HL,SL) by providing the right skills for success.

Abroad Pharma said...

Wonderful blog. I will keep on coming here again and again. Buy Rivotril (Clonazepam) Online

custom term paper writing service said...

Great website. I am happy to be a part of this site. keep showing your potential.

David Clarke said...

Thank you for sharing this resource! Also you can check out thiss reliable service Perfect Essays Online and buy essays on various topics! Best regards!

Harold Price said...

You have a great insight. Content is pretty good and you have presented in a good manner.I appreciate you for sharing this blog for us. Buy essays online

Holly Miles said...

I was seeking through the web and accidentally visited your website. There is a lot of cool stuff here! You have perfect writing skills! And if you are searching for quality essay writing help, I recommend you to discover more here =)


Reene said...

Thanks the give topic is really a difficult task but your this tutorial made it easy for me to handle it without any problem .
Batman vs Superman Movie Batman Vs Superman 2016 Box Office. Thank you and good luck. Ipl 2016 Live Streaming CPLT20OFFICIAL. Cheers.

sarah lee said...

Love it! Very interesting topics, I hope the incoming comments and suggestion are equally positive. Thank you for sharing this information that is actually helpful.


ufgop.org
ufgop.org

Patricia Tabor said...

Well explained the fragment caching concept. It eradicates my doubts regarding its concept. I would like to create a document based on this topic for best custom essay writing service.

حسام داود said...


level تسليك مجارى بالرياض
افضل شركة تنظيف بالرياض
تنظيف شقق بالرياض
تنظيف منازل بالرياض
شركة غسيل خزنات بالرياض
افضل شركة مكافحة حشرات بالرياض
رش مبيدات بالرياض
شركة تخزين عفش بالرياض
تنظيف مجالس بالرياض
تنظيف فلل بالرياض

andrea chiu said...

Thanks for sharing your article and for giving us the chance to read it. It is very helpful and encouraging. Visit my site too.

triciajoy.com

www.triciajoy.com

Dissertation Writing said...

This is a good post. Cache have an important role in our database. The redundant portion will remove from these.The perfect guide on dissertation revision .This is a good post. I think it is a wonderful idea. I like travel and visit new places. I am a professional writer. Our online writing services helps student for writing their dissertation and essay like writing documents.

xjd7410@gmail.com said...

insanity workout
louis vuitton outlet stores
adidas originals
michael kors outlet
coach factory outlet
replica watches
toms shoes
gucci handbags
louis vuitton handbags
coach outlet store online
timberland outlet
louis vuitton handbags
kate spade outlet
oakley sunglasses
oakley sunglasses
cheap jordans
oakley vault
burberry bags
air jordans
coach outlet store online clearances
beats solo 2
lebron james shoes
coach outlet store online
tory burch outlet
adidas ultra boost
cheap ray ban sunglasses
hollister shirts
ralph lauren polo
coach outlet store online clearances
chenyingying20160806

Cindy Dy said...

So happy to be given a privilege to post a comment here. You have a wonderful site. Thank you for the effort to publish this.

www.gofastek.com

huong duong said...

Today I will send the music lover a gift, I believe you will love it. Try it and send me a comment
baixar musicas gratis , baixar musicas , download musicas , musicas download , baixar musicas de gratis

huong duong said...

You have seen the usual boring chat with friends through social networks yet? If you got bored then try this one, a friend recently introduced me. It was really fun
snapchat , snapchat baixar , baixar snapchat , snapchat download , download snapchat

Abhijit said...

Are you willing to get level of popularity instantly? You require to Buy Facebook Comments to become recognized to this world. buy facebook status comments

Jessica Jeon said...

Are you presently looking to ways to gain global recognition quickly? You just require to Buy Facebook Followers to get well known online. buy real fb followers

tian chun said...

Having continuous ray bans sunglasses
innovation and uncovering of the attitude, the earth's top Oakley sunglasses wholesale
sports brand unveiled 2017 new series Oakley Oakley sunglasses glasses, basic style and fashion features combined with excellent technology, enhanced technology, exquisite materials in addition to unique details, again to build extraordinary sunglasses ray ban sale
quality glasses set, design and style features perfect often the interpretation of the comfort, craze and function of creative combination.
With the accelerated pace connected with life, often in the way of elegant metrosexual oakley sunglasses wholesale
man on several occasions, the demand for healthy glasses become more diversified. Oakley Oakley's designers are keen for capturing the changes in consumer require, with the introduction of new Crossrange type series, respectively, regular modeling and motion functions design, meet the usual have on and sports demand, guide for all occasions with dynamic metrosexual oakley replica
man stylish attire and extraordinary performance.
Oakley Oakley has enjoyed often the world's top players benefit, this time the upgraded EVZERO STRIDE Light Sport shape zero all-weather super dynamics will not live up to expectations. For a final can be used for a variety of sports sunglasses, zero instructions all-weather large frameless style and design to minimize the glasses enduring the, perfectly fit the nasal and face, no matter what concern, can help the wearer to do the best performance, bring a blinding aura for every fashion mobility.
Take the light route HOLBROOK METAL Pioneer - ultra-thin metal is another style, they have rectangular lens and steel frame will be the perfect combination connected with life aesthetics and performance properties. What is more, the sunglasses was obviously a detail hundred-percent control, consuming high purity materials give 100% UV protection at the same time, a wonderful show color change is definitely sensitive to the wearer, get incomparable vision to enjoy.
As you might think pursuit of a positive life style of folks, Fin Box - is often a rectangular rogue you don't need to miss. The design of the classic sq . lens sunglass ray ban cheap
is full of masculinity, but not only the material lightweight fashion, but with four sets diverse sizes of the nose, considerably improving the comfort of wearing.

aaa kitty20101122 said...

michael kors outlet store
harden shoes
louboutin shoes uk
yeezy boost 350
michael kors outlet
adidas online shop
nike football boots
led shoes
adidas nmd
adidas stan smith shoes

aaa kitty20101122 said...

michael kors outlet store
true religion jeans
air jordan
michael kors outlet
mlb jerseys
fitflops clearance
adidas superstar shoes
tory burch shoes
tory burch shoes
michael kors handbags

aaa kitty20101122 said...

adidas nmd
air jordans
http://www.kobeshoes.uk
adidas tubular
golden goose
air max 2016
true religion outlet
kobe shoes
nike polo
jordan 11 retro

Love Kpop said...

I'm puzzled with lots of exercises. I was afraid I could not do the right time despite my hard work. I need a support person.
http://run3play.com