Plan for NoSQL

I’ve read too many articles talking about this NoSQL stuffs, now I have to have a plan to proceed (with what? :-W).

First of all, I’m going to remove MySQL and OpenLDAP installation in my testing environment :P. MySQL is kind of slow to me though I have pretty much experience in setting it up, include replication, etc, and I will check to see if all applications can be done in a key-value based data store (check below). OpenLDAP is another story, I still haven’t figure out how to set it up with replication – last time I tried was 2.2/2.3, but 2.4 introduces a whole new approach to do replication and I think I’m going to leave it untouched for now. Note that I still need to come back to LDAP later on since it is still perfect solution for running corporate-like application, such as what I did couple of times before – integrate mail, IM, wiki, blog, etc together with single user-id.

OK, back to NoSQL, couple of things to do:

1. Consistent hashing, I still need to read all those articles and try out different implementation, I don’t think I will compose my own, but I need something can work on Linux and Windows (OSX? Don’t think so), and support some major programming languages (C/C++, PHP/Python, Java). I also need to do test similar to what I’ve done and understand how it affects deployment.
2. Try out different engines, most likely I won’t try out things too fancy (read it “complicated”), for example, I will prefer Redis over memcachedb just because memcachedb’s replication is not that simple to me, I believe anything complicate in setup will be a headache in maintenance. I will also skip those so-called document store/graph store unless they can support simple key-value store in same performance (then those features will be a nice add-on). I don’t have the list so far, but I will get one in the coming weekend. Things to be tested include installation, replication, fail-over, backup and recovery, monitoring, etc. Also programming language supported will be another important factor, I wish a similar list to item #1.
3. Application … I’m going to conclude “traditional” web features that involves data store, and check to see how to implement them in distributed key-value data store. For example, user registration, login, edit preference/profile is one of the fundamental features, and buddy related operation (add as buddy, blacklist, check online status, notify buddy for event/be notified by buddy) is another one. Things current in my mind include message feature (internal/external IM/mail), post features (threaded post like forum, vote/survey may be in this category as well), and maybe some search features. I don’t think I can come up with a full list in the coming days, but I will keep posting here.

This is pretty much what’s in my mind. All these stuffs are seems to be new and almost none of them are well packaged, so after 4~5 years of using yum/apt, now I need to do what I used to do – build everything from scratch, if I have time, I will compose some packages so to ease my deployment.