Dec 302009

I was thinking of uninstalling QQ on my machine as for months (at least weeks) there was nobody pinged me.

Then … someone pinged me, asking me if I can help getting a part-time job, this is interesting :).

I will leave QQ running there for sometime, and will uninstall it if nobody pings me for, say 3 months.

 Posted by at 02:39  Tagged with: ,
Dec 302009

Brand new 😀

Playing with Sabayon now, an “out-of-box” gentoo, seems fun, but I have to write down something as I never touched this before:

  • rc-update/rc-status to add/list current enabled services
  • “emerge –sync” to sync package meta data, and “emerge –search” to search, then “emerge pkg” to install package “pkg” – it takes some time as everything will be built from source
  • change /boot/grub/menu.lst, vga=normal to avoid loading system with graphics

Could be more, will post then.

Dec 282009

I was running everything on Fedora (7 VMs), but steadily, I changed some of them to different distros … just for fun.

Now I have two Fedora 12,  one CentOS 5.4, one openSuSE 11.2, one Debian 5.03, and two Ubuntu 9.10. I will think about converting one of the Fedora machine and one of the Ubuntu machine to something else, but haven’t decided yet.

Dec 252009

Got hair cut to near bald.

Short hair makes me feel good, I no longer have to worry about losing hair as it seems nothing to lose :D.

I will keep this length for sometime.

 Posted by at 18:55  Tagged with:
Dec 172009

Jan 17 will be my 10 years here, yesterday I got the gift for 10 years service – an espresso machine.

Not much to say, just cannot believe I’ve been working for this company for 10 years, and I’m also happy to be here in 10 years even in different group, city, even country. It was a great company, but it’s having some problems now, and I wish it can take off to be great again.

Not much to say, happy 10 years to myself.

 Posted by at 10:55  Tagged with:
Dec 162009

I don’t want to say it’s good, as I feel performance is not that great compare with the old version. However, the look-n-feel is neat, makes it a cute product and makes me thinking – the old version is too “formal”.

It seems in now a day, people care UI more than functionality, if I had the choice I would not upgrade to the new version so that I can keep my productivity, but it’s a pity it was set to upgrade automatically (I made the setting? not sure).

Will (have to) keep using it till I get familiar, anyway I don’t have any other alternative, yet, or maybe it’s the time to look for alternative.

Dec 152009

2009 is a tough year to everybody, I just wish world becomes better in year 2010.

My plan for year 2010 – let’s make this a leisure talk, I don’t want to promise anything since this is a way too dynamic time that everything can be changed in days. However, if things moving smoothly, I wish I can:

Try to find something interesting to do as my job … no detail yet. I wish I can still be here with my current company, but just in case it becomes impossible, I may think about some different industry.

Next comes to my mind is re-arrange development environment at home, whenever I said “re-arrange” I actually meant … more :D. In a ideal world (though the world can never be ideal) I will retire the P4 box, use a iMac replace it, and get a 8 cores machines as my dedicated testing environment which is currently running on a dual core desktop, and then I will upgrade the 4-core desktop to Windows 7, and retires the PowerBook and the Asus laptop (1.1G …) as well. In this way I will have only 5 machines at home, but with 8 more cores.

If I can get a 8 core box, I will run 10~15 VMs on the machine, so that I can testing all sort of cluster ideas. Also, I’m going to use different distro (again), include CentOS/Fedora, Debian/Ubuntu.

I wish I can get familiar with Python on web development, I should evaluate a framework though I don’t think I’m going to use it anyway, but if there is any chance, I should build up a site with the framework, to make it my RAD solution for web development :). As for Python, I also need to try out doing extension with C/C++, so that in case there is any performance bottleneck comes from Python I can get over it.

I don’t know if I will have interest do desktop development again, if I have it could be the time to try Delphi again, but I can imagine after couple of days excitement, I will move to Visual C++ because of Delphi’s buggy system (please, Delphi, please make yourself better, or just workable). I don’t have any specific project so far, but if I can at least re-write the P2P webcam thing. Java can also be an alternative for desktop development, I will definitely NOT use Java on Web as it is just horrible, but it may fit into desktop, who knows.

Hobby study – get to try Ada and Lisp (yea, again). I will check to see if I can make Ada a server side programming language, and try to do some small client side applications. I don’t have confidence on working on these two, but whenever I have time, I will always do something.

There are something purely for learning, as I don’t want to put money on trying them out – all those cloud services like Amazon S3, Ec2, Google App Engine, and maybe something from Microsoft, etc. I need to at least understand how these things are working (i.e. operation related stuffs), so that once I need them, I won’t run into blind. I may also try out Ubuntu’s private cloud (is this its name?) and see how it can fit into an enterprise environment.

I don’t want to put any personal/family/friend plan here, it’s too hard to plan and also involves privacy issue :D, but roughly speaking, I wish I can spend more time with people around me, make them happy. I want to get some amount of money to pay off part of my mortgage, or get a new car, but I never had a good plan of how to getting money other than salary, so it could be just a wish.

Dec 112009

Got a new domain (transferred from previous Windows Live free domain registry), but not sure what I’m going to do with it.

Could be another blog and some testing – after I started doing this English blogging, I found that less Chinese came to read. There are still some good technology that I want to discuss in Chinese, or let’s say, get them to be known in China, so running a Chinese blog could be a good idea.

Anything else than that does not make too much sense at this moment. Godaddy’s Python is still at 2.4, I don’t think I can play with Python there, and I don’t have any other hosting plan yet, maybe I should shop around.

Dec 092009

It seems current design cannot bypass the bottleneck from relationship, I think I need to re-think about the design.

It was said doing denormalization at the write time is a good way, as these nosql data stores are really fast on write. However, it may be too much to do it. My current scenario is every user has 10~100 buddies, if I do the denormalization during the write time, it still not clear to me how to set up a schema to fit it.

I found I’m still sort of in RDBMS world, need to jump into this nosql universe as soon as possible :P.

Dec 082009

Testing a prototype that uses Cassandra as the back end storage, the simple application is doing user authentication stuffs, it logs in and then get user’s profile and then show details on the web page.

I hit performance problem with buddy related operation – every user may have 0~20 buddies, I want to show each buddy’s last login time on the result page, and actually I’ve retrieve everything for those buddies. The most direct implementation as I did first, is using user object to get data back, obviously this is not good as for every user object, client needs to access Cassandra cluster to get data back, the TCP round trip would be a pain.

Then I added something to user object’s constructor, which load all buddies info in one shot (Cassandra’s multiget_slice API), things are getting better but this doesn’t seems reasonable to me as for most time, we don’t need buddy info (such as authentication), and getting buddies info back is just a waste of time.

So I added a new method to the user class, called load_buddies, this will load buddies info on-demand. This makes authentication pretty fast, but still keep the ability of loading buddies info in batch mode.

After all these the performance is … still not good, my test case is one login failure every ten requests, and for successfully logged user, I should buddy id and last access time, and also change the user’s last login time. The performance, with my current setting, the worst response time is about a second, while 90% request were done in less than 600ms.

There must be something can be tuned, though VM could be the reason of slowness. I will check following stuffs:

  • Apache HTTPd configuration, it seems prefork is performing better than worker, there may be more can be tuned include both HTTPd and wsgi
  • Python class optimization, I will review the implementation of user class, as I don’t want to make user class too complicated to be used
  • Cassandra performance, actually this is what I’m worrying about, as during the tests, Cassandra boxes’ CPU utilization is about 80% – 70% on user, 10% on sys, roughly, it could be the bottleneck

Without the buddy operation everything’s fine – the worst response time is about 600 ms while 90% requests are below 400ms. Relationship is a pain, it’s the bottleneck, but in this social era, there is no web application can live without relationship …

BTW, my testing environment:

  • Test client running on PowerBook, using ab – I will check if there is anything else can be useful
  • Servers are all running on same physical box controlled by proxmox, this includes a web server, a LVS director (to load balance Cassandra nodes), and 3 Cassandra nodes
  • The server box uses Ethernet, PowerBook is on wireless. I don’t think there is any issue for this as connect time is pretty low.