Dec 092009
It seems current design cannot bypass the bottleneck from relationship, I think I need to re-think about the design.
It was said doing denormalization at the write time is a good way, as these nosql data stores are really fast on write. However, it may be too much to do it. My current scenario is every user has 10~100 buddies, if I do the denormalization during the write time, it still not clear to me how to set up a schema to fit it.
I found I’m still sort of in RDBMS world, need to jump into this nosql universe as soon as possible :P.
3 Responses to “Thinking of redesign”
Sorry, the comment form is closed at this time.
I found one problem … which is pretty obvious but I didn’t notice – my test data set is too big to fit into memory, so it start swapping which significantly slow down the system.
I’m tuning my data set now, while changing codes as well (for thread safe), will see.
I tried svn version of cassandra (so-called 5.0-beta2), which … signaficantly improved key distribution, as well as the performance. These two improvement gave me much better result, though I need to figure out if there is anything that I can improved or not.
I finally gave up the idea of letting Cassandra determines the token for partition by itself, at least for those initial nodes. Obviously it is not doing that great and doing nodeprobe loadbalance later is painful.