Python vs. Perl

 Triviality  1 Response »
Aug 222013
 

I’m working on a set of data refreshment scripts, which get data from file, do some transform, then send to a HTTP interface. Since the HTTP interface is kind of slow compare with reading and transforming data, I have several forked children processes to handle HTTP part.

Everything was done by Perl about 6 months ago, and everything seems good … until I started picking up Python. First sight is that Pyhon program is about 50% of Perl in term of LOC, which makes it easier to read, but seriously I don’t care about this too much as the logic is quite simple, however, when I tested Python programs and found that they are at least 50% faster than Perl’s, I felt nervous.

Two examples – Perl takes 13 seconds, Python takes 5, Perl takes 34 minutes, Python takes 10. Actually I’m really nervous at this moment thinking of my poor Python skills, I always worry if I made anything wrong with the translation (from Perl to Python), even I have verified result data for quite some times.

Will dig in after converting all scripts to Python.

Should I move to GDChart?

 Triviality  No Responses »
Aug 022013
 

I think I’m going to play with Perl, PHP, and Python at almost the same time for some time, and thinking of the fact that dynamic graph is one of the most important feature for web pages, I think I need to find a cross-language solution.

I’m happy with PHPlot and decide not to touch GD::Graph, and I came across GDChart which seems to be good in cross-platform/cross-language area. I will give it a try and will post the result here.

How to solve problems

 Triviality  1 Response »
Jul 192013
 

I was assigned to a Web project which is to present data analysis result to users. Original data came from Web log, plus some extra information, then headed to Hive, then populated statistics files after scientist’s analysis There are several interesting topics: geo graph, rendering another web page, and metrics graph, roughly speaking, I have no idea of any of these at the beginning of the project.

I think the best decision I made is to use GD based solution, actually that could be the only solution I can think about, I decided to use PHP for Web, plus Perl for batch processing, this actually seems not to be quite right as I’m migrating everything to PHP now as there is not much “real” batch processing, and everything could be done in shell. Also I decided to use server-side DOM model (read: PHP DOM) so not to slow down the project by my poor JS skills, actually my PHP skill is not that good but JS is definitely *poor*. Continue reading »

Trying PHPlot

 Triviality  No Responses »
Jul 182013
 

PHPlot looks better than Perl’s GD::Graph … I don’t know how to exactly describe the difference, but feel like: neat, easy to understand, easy to control, and better default values.

I’m migrating some BI sites to PHPlot so to get rid of Perl stuffs, so to make things purely PHP based, thus other PHP guys can take over it easily.