Which is the most popular social network site in your country?

Monday, December 17, 2007

The numbers speak for themselves – Social Networking Sites are popular all over the world.

  • In 90 (79%) countries a major social networking site features in the top 10 sites of that country.
  • In 19 of these countries, the social networking site is the highest ranking site in the country – ranking higher than any search engine.
  • From a sample of 116 countries only 2 (Taiwan and Vietnam) didn’t include a popular social networking site in the list of the top 100 websites.

The popularity of social networking sites is no surprise, and several statistics (1, 2, 3) have been published about the major social networking sites like facebook and myspace. There are however few reports on the use of these sites by geographic region. The only geographic distributions I came across were from Comscore, the Social Network Sites paper published in JCMC, and ValleyWag .

Using the Many Eyes platform I created three different visualizations of the most popular SN sites used in each country. The data used for determining the country popularity was collected from Alexa ratings. For more information on how the data was extracted see – how to collect geographic website rankings from the internet.

The world map is a colour coded map with each social networking site represented in a different colour. Where data wasn’t available, the country border is not displayed. If you click on a site from the list on the right, the countries that use that site are highlighted.

This second display shows a rectangular table display (treemap) of the data divided either by social networking platform, or by country. To alter between the displays reorder the treemap hierarchy by dragging the ordering on top of the visualization display.

The third visualization shows the ranking of the social network sites, and the number of internet users in each region. In the darker coloured regions, social networking sites ranked higher than other websites. The size of each rectangle is proportional to the number of internet users in the country, the bigger the rectangle, the more users there are.

Which visual representation of the data set do you prefer, and why? Do you think that one of the displays is superior to the others? Can you think of other different ways to present this data graphically? The aim of this exercise is to display some interesting data using Many Eyes and stimulate discussions on the different visualizations and data presented.

If you’d like to voice your comments, comment on the specific visualization by clicking the comment link in the respective visualization. The data used to generate the results is freely accessible on the Many Eyes site. You can use the uploaded data to create other visualizations in Many Eyes. After all, if you reply with a picture it’s like you’re writing a thousand words, isn’t it.

How to collect geographic website rankings from the internet?

Monday, December 17, 2007

The data for the social network sites by country visualisations was collected from the Alexa country rankings site. This is the only known available free source which provides this information for a reasonable number of countries. I wrote some perl scripts to automate the process and allow the same data to be collected in the future for comparison. (I can send the scripts to anyone who’s interested, just drop me an email). This is a brief description on how the data was collected:-

1. Extract the top websites in each country
The main ranking page for each country was first saved. The site ranking, site name and site description were extracted from the source html.

2. Identify the social networking sites
Most websites in Alexa have a brief description about the website. All the sites that had the word social in the text were saved in the social network definition file.

This process proved to be a good first guess at generating a list of social network sites. Some of the identified social networking sites (like badoo, yonja and perfspot) weren’t in the wikipedia list of social networking sites. As expected, the process also identified sites with the word social that aren’t SN sites. These sites were manually deleted.

After running the SN extract process and the results were analysed, some countries, especially non English speaking countries, had very low scores for the social networking sites that were originally identified. This was because these countries were in fact using social networking sites in their own language. This was most evident in Russian speaking countries using the website Vkontakte, which is very popular in Russian speaking countries.

Finally a few popular sites that weren’t identified from the above process were manually inserted from the Wikipedia list of social networking websites.

Why are QQ.com and Flickr not there?
The list of selected SN sites is by no means exhaustive and there are some popular sites that have social networking features embedded in them which make them almost SN sites. Qq.com the popular Chinese site, and Flikr are two prime examples. These sites were excluded from the list because they are not purely Social Networking sites, but they evolved from another website form (instant messaging and photo sharing in the case of Qq and flickr) into social networking sites.

This is the final list of sites that were used in the section process.

3.Retrieve the highest ranking website from the country rankings
The final extraction process selected the highest ranking SN site from the list along with the ranking and site name. The website address was used to match the websites to avoid any language problems with different character sets. The original idea was to include only SN sites that have a ranking between 1-20, however in order to populate the world map visualisation better this ranking restriction was omitted. In some cases there’s more than 1 SN site listed in the top 10 list which suggests that the distinction between the SN sites used in the country is not clear cut. Unfortunately in the world map visualisation it was difficult to split the country in 2 to show two different SN sites, therefore only the first ranking site in each case was considered.

4. Adding data from other sources
To help with the visualisation of the data collected data some additional data that wasn’t available in the original data source was required. The additional data was continent data, and the number of internet users in each country. At first I tried to look for this data from the new data search engine graphwise however the results returned were far from satisfactory. A google search later pointed me to the data sources that were finally used to get this information.
Continent Data
Internet Users Data

Mysql Login / Run script / Export CSV

Wednesday, March 8, 2006
  • Login: mysql [database name] -u [user] -p
  • E.g. mysql baseball -u mike -p
  • Run Script: mysql [database name] -f -u [user] -p < file_name.sql
  • E.g. mysql baseball -f -u mike -p < insert_teams.sql (use -f to ignore errors)
  • Export data to CSV: SELECT * INTO OUTFILE '/somefile.csv' FIELDS TERMINATED BY ',' LINES TERMINATED BY 'n' FROM table WHERE column='value'";

Perl Search and Replace One Liner

Wednesday, March 8, 2006

perl -p -e s/think/tank/g < input_file.txt > output_file.txt

Software Quality Assurance Testing Articles

Sunday, February 12, 2006

Software Quality Assurance Testing Articles - Articles, white papers, and more on important software testing topics.

Good Reads

Friday, December 9, 2005

I'm not posting too much nowadays and the excuse I'm using with my writing soul is that I'm focusing on doing things rather than writing about doing them. Today I've managed to read longish posts and it was time well spent, so I'll share:-

On broken windows

Tuesday, November 8, 2005

One of the many beneficial side effects of the conference was that I learned about the Pragmatic Set of Books. I managed to get "The Pragmatic Programmer" and "Pragmatic Unit Testing" and I thought that anybody who reads this blog should know about the broken window concept.

"In inner cities, some buildings are beautiful and clean, while others are rotting hulks. Why? Researchers in the field of crime and urban decay discovered a fascinating trigger mechanism, one that very quickly turns a clean, intact, inhabited building into a smashed and abandoned derelict.

A broken window.

One broken window, left unrepaired for any substantial length of time, instills in the inhabitants of the building a sense of abandonment—a sense that the powers that be don't care about the building. So another window gets broken. People start littering. Graffiti appears. Serious structural damage begins. In a relatively short space of time, the building becomes damaged beyond the owner's desire to fix it, and the sense of abandonment becomes reality."

Source: The pragmatic programmer - A Hunt, D Thomas - Chapter 1 - Software Entropy

I think that the concept of having a broken window ruining a whole project and setting an open path for disaster is very true and applicable to both software and real life. I can recognize that I'm very susceptible to fall in the trap. Apart from code where I've seen it happen in my projects it also happens in daily life - I try to diet and keep it up until I break the first window by eating a piece of cake, I try to stop swearing until the first time somebody pisses me of and sets me on my usual route, and the list goes on and on...

I can also identify with the cleaning up the glass. If say I tidy up a project by refactoring, updating documentation etc. I have a feeling that the project is back up to scratch once again and is alive. The same happens with my room when I tidy up and everything is in order I start liking to live in the room again, until I break the first window again by leave it uncleaned for a day.

Word: pragmatic
Meaning:

  1. archaic - Busy, Officious, Opinionated
  2. relating to matters of fact or practical affairs often to the exclusion of intellectual or artistic matters : practical as opposed to idealistic
  3. relating to or being in accordance with philosophical pragmatism
Usage: pragmatic men of power have had no time or inclination to deal with... social morality -- K. B. Clark

Report from New York

Sunday, November 6, 2005

Ah, New York. Visiting the big apple was one of my 10 things to do before I die and I'm glad I made it. Getting here wasn't easy by any standard because all the flights out of my country were full and I even I got offloaded from the plane, after getting physically on it, because the pilot didn't want me in the cabin jump seat even though I had all the necessary authorizations. The alternatives to get to London were very bleak because all the British were returning back home after the midterm holidays. At last I got lucky because some passengers no showed on the midday BA flight to Gatwick and I got on instead. From here onwards everything was plain sailing - got the National Express from Gatwick to Heathrow (circa 1 hour 15 minutes) and then the 6pm flight to JFK. The flight landed at around 930pm and by 1030 I was in the Roosevelt Hotel. I need not say that I missed the ice hockey game between the NY Rangers and Montreal Canadians that I had already purchased the tickets for; $53 down the drain cause of a bastard pilot.

The hotel room wasn't as sophisticated as I expected it to be for $280 a day and there wasn't a playstation in the room as they claim on their website. The rest of the hotel though was very glamorous with high ceilings full of chandeliers and expensive looking décor. The Software Test and Performance Conference was held on the Mezzanine Level or for us non Americans 'The First Floor'. This floor is a dedicated conference area with about 10 different suites to host more than 450 visitors that attended the conference. This said, there wasn't space to host a buffet on the last day of the conference as they usually do so I got no free food at all.

As they say you compare things to others you're familiar with, and the closest anything comes to New York for me is London. The difference is that NY has got all those huge skyscrapers and characteristic doses of America piped into it. Anyone who's ever played battleships should easily be able to work out how to navigate the streets of Manhattan because the road network is a pure grid. To make things easier both the avenues and the streets are sequentially ordered (at least where I was walking) so getting around is easy.

The reputation New York has for shopping is well deserved because there're lots and lots of shops around here. I didn't find a huge number of department stores of the type where they sell anything, in fact the two only department stores I visited where Virgin and Best Buy (which is an electronics shop). Talking of department stores I might tend to put the Toys R' Us store in Times Square with the list also. Grandiose and deserves a visit even if you're too old for toys (who is?)

In the end after a three day conference with only two effective shopping evenings including a 1 hour midday break I brought home:-

  • Levis Jeans ($38)
  • Sports Jacket ($20)
  • Brand new swatch (the first one the Times Square shop sold) for my missus.
  • Grey Hooded Sweater
  • Another sport Jacket ($57)
  • $200 worth of books from B&N (only 3 of them were computer books)
  • 3 packets 3x5 index cards and a 15 pack colored sticky notes
  • Hiking boots ($130)
  • Ralph Lauren Eau de Parfum from Saks (another present)

Now if I tell you that my dad looked pretty amazed when I returned back with 4 bags instead of 2 you're likely to believe me. And also if I tell you that I'm broke to the bones you'll not laugh at me.

With the unbelievable amount of restaurants (see this view from Google Earth) and delis around I couldn't resist the temptation to spoil myself in an American indulgence no matter that I wasn't hungry. In the afternoon I went to a deli/self service and got a couple of Chinese food and noodles in a plastic container which was then weighed and cost $10.21 with a berry juice drink rather cheap considering that there were at least two full plates in my choice. For dinner which I didn't plan to take, I was lured into a sports grill diner with a sign for lobster at $12.75 but then I ended up taking a Times Square burger and a pint of Guinness to feel more American (total cost with tips $25) while watching ice hockey on the big screen. The following day I resisted the temptation to eat to spend more time shopping and on the last day I tried to redeem the conference voucher at a snack place called Cosi, but the $15 voucher didn't validate and I had to pay for a sandwich, bitter iced tea and a chocolate cake.

On a more serious note - the conference was fantastic. On the first day we had a full day tutorial from BJ Robinson, a Microsoft Technical Trainer, about Testing Techniques. Apart from bashing Microsoft we learned the functional testing techniques of boundary values testing, equivalence class partitioning and pairwise testing, and the structural techniques behind code coverage. I think the single most important think that sank down in my brain was that testing is not just randomly bashing at the keyboard trying to find bugs. There actually exist techniques that help you execute exercise the code in a way to potential uncover errors without spending eternity and a half testing. The lecture was very insightful, beginner friendly, informal and very practical - just the thing I was looking for. I worked with my bench mate a lady working at Boeing and we went along quite fine, I wasn't a social beast.

By far the best lecturer of the conference was Robert Sabourin. This hyperactive human has his own very particular way of teaching by shouting out loud, showing us fun videos, spending ½ the lesson telling us a story seemingly unrelated to the subject but very relevant to the whole point of the lesson, pacing wildly in the room and being very informal. The first class was about bug prioritization and the central concept was that - The priority and severity of bugs is CONTEXT DEPENDENT. Priority specifies when a bug is going to be solved and severity specifies how dangerous it is to keep the bug in the system.

I ended up choosing an additional two classes of Rob after being enthralled by his first premier. In this class as we had the opportunity to do some Unit Testing in a team using a Test Driven Development (TDD) approach. Using a post office scenario we had to first create the test cases for the different package types then with the help of some black felt, Velcro and flowchart symbols design the system flowchart from the test cases. Amongst other things I realized that it's not trivial to move to a TDD mindset after using traditional approaches to development for so long.

The rest of the classes I attended to where:-

Recruiting, Hiring and Motivating top testing talent (Feldstein) - Excellent lecturer who shared some great tips on the role of testing in an organization and how testing should be looked at from employers. If everybody thought like this I would already have a job in testing because I met most criteria he specified. Btw, testers must be developers .
Exploiting Web Application Code - SQL Injection (Bodkin) - Informative and enlightening lecture on the importance of application security and the real threats found in web applications. Easy and effective.
Automate Database Testing - Testing and using stored procedures (Sweeney) - I already knew most stuff about stored procedures but the idea of using stored procedures for automated testing was worth the 90 minutes.
How to optimize your web testing strategy (Nguyen) - Nothing to do with actual web strategy and more focused toward management.
Software Endgames - How to finish what you've started (Galen) - Again too much focused on management and the importance of defining goals to attain them.

I will hopefully post my personal incoherent course notes either tomorrow or the day after.

Update:The course notes are here: Testing Conference Notes

Looking for company

Monday, October 10, 2005

If anyone is going to attend the software test and performance conference in NY next month send me a mail. My company is poor enough to send me alone and I wouldn't mind knowing somebody before the conference.

In the mean time these links are going to be useful for the rest of the week.
Pavel's Online Presence - Pavel's software engineering oriented blog
Testing Reflections - The title says it all

Oracle Database Shutdown

Monday, September 5, 2005
Did you ever forget to write "immediate" after shutdown when trying to stop a database? When this happens Oracle waits for all connections to terminate before shutting down which usually takes ages. Oracle obviously disallows the creation of new processes so if you try to shutdown the database again using the immediate keyword you will receive the "Not connected to Oracle" message. To recover from this situation you have to open another session and startup force the database and then shut it down again using shutdown immediate.

Linux Linux Everywhere

Thursday, June 16, 2005

At the moment I'm having Linux for breakfast, lunch and dinner in a desperate attempt to learn anything about Linux in preparation for the RHCE course next week. The more I learn the more I realise how little I know, and how unlikely I'm going to pass this exam. At least though its fun learning, and I'm gradually uncovering the huge community of Linux Users which seems bigger and bigger every day. These are some interesting sites I came across these few days.

FrozenTech
Has got loads of Linux live CD's. Live CD's are Linux distributions that run only from a single CD without touching anything of your current OS. I've downloaded a Kill Bill edition of SLAX and Kanotix both of which were very easy to install and use.

Distrowatch
With all these distribution you need something to help you choose. Distrowatch is the answer.

LUG Radio
The site of the most popular Linux Radio show around. Worth downloading some podcasts there's much to learn from hear (including some swear words).

The Linux User Show
Home of the best podcast I've listened to till now. This was the first show but I'm eager to hear what Jon has to offer. Highly recommended

linuxiso.org
Download any Linux Distribution from here

Really Linux
For those who're still new to the Linux world this is a friendly introductory site to this new world.

Re-engineering stale old processes

Wednesday, May 11, 2005

Do you want to find flaws in a process in it in your organisation? The solution to this problem is simple – try to computerise the system. When you are computerising a system you have to study the current system in such detail and depth that very few other studies can hope to achieve. This week we started the rollout of the Leave Management Information system we've implemented in the last couple of months and we're facing the baffled users of the system who don't know what hit them.

At face value the system serves as a mere paper reducer because it replaces the old leave application forms, however at a deeper level it is being used to enforce the streamlining of a leave process in the organisation. Throughout all these years the leave application and authorisation process has been moulded into dozens of different formats with different rules applied to different departments without any control and rational. Now the system is being used to find out and eliminate the exceptions and loopholes in the process which have now become the norm.

There are several benefits you can gain from introducing an information system, but one which is not commonly acknowledged and adequately quantified is; how much savings does a system bring with by reducing abuses to the old system. Despite the fact that the system was implemented for all the wrong reasons, we're now recognizing some of the more subtle yet more beneficial advantages the system will bring. This is also the first information system that is going to be used throughout the whole company for about 2000 personnel. I'm pleased with the achievement we've obtained as it was no easy task for only two people to undertake in such a short time.

Microfeeds

Tuesday, April 26, 2005

I tried to post this comment on Eric Lunt’s blog in response to this post, but for some strange reason it was deemed to have questionable content.

Microfeeds can solve the problem of many users who cannot keep track of their replies to different posts on the web. Their full potential in this respect will materialise itself if they are adopted by discussion/bulletin board and blogging systems. When discussion groups serve individual posts using microfeeds we can go back to the good old newsgroup reader days, where you can keep track of all the interesting discussions in different discussion through a single program.

Weekend collection

Monday, December 6, 2004

I am dead tired but I feel that I must write something at the end of the week. I'm not feeling very coherent so I'm going to make some inconsistent sparse annotations.

Problem of the week - Leave application design.
Major Problem of the week - Christmas presents
Currently reading - Deception Point by Dan Brown
Best J2EE book out of a dozen - J2EE Bible by James McGovern. This is the first book of the bible series I ever liked.
Film watched today - Twisted (Review) nice suspense thriller starring Samuel L Jackson, Andy Garcia and a great Ashley Judd. 3 stars.
Freshly installed game - Need for speed 2
Winning Java IDE editor - IntelliJ for the great J2EE support it offers. Eclipse also has some good plug-ins for J2EE but mostly generate Xdoclet code which I'm not familiar with at the moment.
UML editor – Tried Poseidon v3.0 and after 2 hours editing and saving the program crashed and I lost all my work. Ditched it in favour of Visio 2003
Achievement of the week – Finished video edit for RAI program
Tragic event of the week - My mother's aunty died yesterday

Must visit links for the week
Our holiday in Malaysia - Nice pictures, travelogue and tips
Del.icio.us - Excellent links site but I think there's more to it.

This week I reflected about the effect of having two bosses at work. It takes some time getting used to the idea but after a while you start realising that you can try getting the best of both worlds. As the English say "Too many cooks spoil the broth", and in cases like this the broth is spoilt with their obvious conflict. As soon as conflict settles in then the way to go is to choose the right person for the right task to get what you want.

In line with this thought I was thinking about the way we think about people with whom we interact everyday. In a stable relationship the general impression is pretty static, however in tremulous relationships the love-hate level tends to vary depending on the waters. One time you agree with someone and think he's in the right and the next time he make something to completely set you off. I am now trying to identify the point at which the impression steadies itself around a stable point and to what extent that point is influenced by the person's actions.

Things that were missing from our video

  • Beaches taken from the sea - All beach views are taken from inside the beach which has a lesser effect on the picture.
  • Local people, faces and expressions – Don’t be shy to take these shots after all most locals are amused at having their pictures taken
  • Shots from planes / buses / cars. It seems that we’re teleporting from one place to the other
  • Airport shots - These make good introductions
  • Think about how the scene will appear before you shoot
  • Pay attention to the lighting and colour, they are the most important elements of the video and the most difficult to arrange.
  • Lessen zoom because the camera shake is incredible at high zooms
  • Learn how to use the tripod and use it.

Programming stuff

Monday, November 29, 2004

The next couple of weeks will be spent coding at work but unlike other programming assignments this one was not handed over to me but to my only remaining colleague, as the other one is due to leave next Tuesday. I will try to help her get to grips again because after University her programming skills were deeply buried underground. As usual at the beginning of new projects I go through the technology frenzy period which is an endeavour to learn new technologies for the sheer joy of learning them. This leads to overwhelming and also creates previously unforeseen decisions which need to be taken.

The big dilemma in implementing the online leave application system was whether to use an EJB logic tier or not. We have a very strict time-frame for the task because the first phase needs to be implemented by the beginning of January, so the simpler architecture was a viable option. However, after considering that this application is going to be used by the whole company, has complex business rules, and will most likely require heavy maintenance, we decided to opt for an EJB tier architecture.

This is the first time that I'm going to work with my mate on a 'real' programming project, so I decided to document and standardise my coding conventions. I remembered that I once found a document by Sun on the subject so I tried to look for it by incorrectly typing "java blueprint" in google, but I stumbled across something different yet equally useful. The Java Blueprint project is a sort of best practices tutorial that encourages good practices in J2EE programming, something I was in need of. I got immersed in the documentation especially in the sample project (Pet Store Application) and have spent the last of these four days reading and analysing the system, with the hope of learning something.

The pet store application is a demo of an online shopping pet store. I first download the J2EE 1.3.1 versions and while going through a whole lot of trouble with the classpath configuration, I decided to download a newer version. Browsing through the chaotic section of the java downloads I was forced to download version 1.4.1 which comes at a hefty 104MB, bundled with Sun's Application Server. This sluggish new version comes with a very friendly web server administration tool which is excellent to configure the server quickly not having to edit text files. This version uses a pointbase database engine instead of the traditional cloudscape engine that was usually shipped with pervious j2ee's. Pointbase is a 100% pure java database engine especially marketed for the Java Platform. For a database that comes packaged with J2EE it looks quite good and it comes with its own graphical editor obviously written in Java which again helps in getting to grips with the product quickly.

The two most prominent aspects about the blueprint examples were the usage of a web application framework (WAF) and the use of JSTL. Until now most of my Java web programming was simple, one man band programming with strict timeframes so I never had the chance to learn how to use a web application framework. I'm completely convinced about the advantages that it offers but I'm still undecided whether to employ one for this new project given the strict timeframe that we have. What makes the choice more difficult is that there are a million different WAF's out there, to the extent that there is a project called WAFER that tries to compare between the different implementations. I'm positive that the most renown one is Struts however it seems that Cocoon is another good competitor in the area. From the little code I saw which uses these frameworks, tag libraries are used in all JSP pages, instead of code scriptlets. Given the task they do it's obvious that taglibs are the right choice however until know I was comfortably implementing JSP with scriptlets. This practice is now risking extinction because the separation between logic and presentation that JSTL offers is brilliant.

Getting your hands dirty programming rekindles the battle of the IDE's. Since I adore installing and trying out software I installed two editors at the same time Eclipse at home and IntelliJ 4.5 at work. I've had more time evaluating IntelliJ and I was very impressed with the improvements of the latest version. This version supports most application servers featuring automatic deployment and plug-ins that make the program very extensible. I installed Eclipse out of curiosity about all the hype it generates. At first I didn't have any clue how to use the program so it looked pretty basic, providing the usual editor functionality. Today though I had some free time to fiddle around with it and I was awed. I went to two eclipse plug-in sites MyEclipse and Eclipse Plug-in Resource Centre and all of a sudden the program boosted itself thousand fold. Does anybody think there's an IDE better than these?

At the moment I'm still trying to figure out how to make best use of the program, and to help me out I found a series of 9 videos about Eclipse (Eclipse Video Courseware). Lately I've realised that videos have a great teaching potential. In fact I've learned most of Adobe Premier by the support site videos and last week I got 2 CD's of Linux videos that seem great. I think that comparatively videos are a faster way to learn the intricacies of software programs than books, because you're seeing the actual thing happening before you and you're more likely to remember it.

Ganis is back

Saturday, November 27, 2004

After two weeks wandering around in the cold and rain my black cat returned back home. I'm overjoyed! At the moment he's sleeping in my lap.

In the last weeks I've put my developer's hat back on and I'm eating and breathing Java. After a long dispute we were instructed to write an online leave application program where all employees must be able to apply for their leave online. This application was triggered after some very nice guy from another department wrote an Access application for his secretary and claimed that it can be extended for the whole company. As usual all previous priorities got mixed up badly.

In an effort to streamline my coding practices I wrote a brief document on coding standards and programming conventions that I should follow. The document is mainly a cut and paste of the relevant parts from the referenced sources.

Open Water and co.

Monday, November 15, 2004

What a waste! We spent $20 to watch a horror home video on a cinema screen. The film was Open Water (Review) which guess what, is yet another shark horror movie. We meant to watch "Hero" but because of our obsession with worms we were late for the film and so we missed it. To give the devout credit to the film, it was much more realistic than all the best parts of jaws put together. The problem was that it is a very low budget bordering the amateur which is a waste watching it at the cinema. The worst part of the film was by far the completely out of context nude scene at the beginning which does an excellent job of priming you towards the banal ending. 2 stars for the suspense.

This week was document writing week. I had the Revenue Management document at work and my brother's SRS assignment at home. Ironically enough, both of the documents were not my job but as usual I ended up doing all the corrections and most of the document especially at work. My brother made a big improvement in his approach toward school work because he spent 3 whole days and nights working on the document. Until 5 days ago he couldn't bear spending more than an hour doing something. What's even more surprising is that he actually is paying attention to formatting and presentation, which was something he really had to work on. Again I cannot say the same for my work colleagues who were more interested in presenting my work than doing it themselves.

For the recommendation document I needed to compare two different systems with each other. I soon realised that in all the lessons I've been through in Computer Science nobody ever thought us how to evaluate systems or conduct feasibility studies. Go figure, in all my feasibility sections of the assignments it was always feasible to implement the project. Now I'm trying to research some metrics and diagramming techniques to help with system evaluation and feasibility study. I found this good paper entitled A framework for systematic evaluation of software technologiesby A. W. Brown and K. C. Wallnau which compares two different technologies and uses two great visual techniques; an onthology diagram and a spiral diagram which I don't know what's called. In my research I also found a good document which describes Revenue Management for those interested.

Unfortunately today we had a funeral of our friend's father who passed away yesterday at the age of 71. It was very sad watching him cry knowing that he's the most funny person and clown of our group. In the afternoon it was too windy to go anywhere so we remained at home watching a travel program called Alle falde del Kilimangiaro. At a certain point there were two features of holiday videos sent by the public, and my love got the brilliant idea of sending our video. I was very pleased with the idea knowing that she wouldn't dream of doing so if she thought the video was crap. The problem now is translating the video to Italian because the program airs on the Italian channel Rai Tre.

Who has the right to choose software - boss, user or IT admin?

Tuesday, October 26, 2004

Two weeks ago I had the opportunity to get philosophical at work. We’re in the last stages of purchasing a very important and revolutionary system for the company, which will change most of the way we do business, and I’m part of the team responsible for the purchase.

The team is divided into 3, a super user and eventually the system administrator (not IT related), his boss and the IT department (where I come in). We evaluated three systems in total, one of which is by far the best but too expensive to consider.

From the remaining two systems one is basic and lacks most of the advanced functionality while the other one is more powerful, complex and expensive. From a personal perspective I would defiantly go for the second option since I was too unimpressed by the first, but the super user is much more in favour of the more basic system. His boss is undecided because he likes the better system yet at the same time he thinks that our company is so small and inexperienced that we should start with a basic product then upgrade in the future. I don’t agree with this approach and reasoning, because I’m sure that a basic system will bring along basic work practices which will not result in anything significant enough to warrant the purchase in the first place.

In this context who has the right to choose the software; the super user representing the final user, the boss who is supposed to know which product will return the best benefit or the IT man who knows which is the best technical system?

Software development life cycle in low maturity organisations

Tuesday, October 26, 2004

Phase 1 – Project Initiation
Phase 2 – Wild Enthusiasm
Phase 3 – Disillusionment
Phase 4 – Chaos
Phase 5 – Search for the guilty
Phase 6 – Punishment of the innocent
Phase 7 – Promotion of non-participants
Phase 8 – Definition of requirements

Microsoft Lesson

Saturday, October 23, 2004

Today on the last day of the course, at last the training got interesting mainly because it became more applicable to day to day use. The lessons were about disk management and data storage. These are some general important principles:-

  • Always use NTFS file system for OS's that support it. NTFS has got security, compression and several other desirable features (ex. remembering previously flagged bad sector locations after a format) that are not present on FAT and FAT32.
  • You can ALWAYS use quick format when formatting disk. Full format is only 'required' when you want to delete sensitive data from disk and even then most probably data can still be retrievable. For super sensitive data deletion you have to do a low level disk format.

In line with what I was thinking a month ago, there are millions of features in software packages that nobody knows they exist, which in some cases might be excellent time and effort savers. Windows itself is full of these unadvertised components. If for nothing else, this course highlighted the importance of MMC console; a tool which encompasses all administration requirements of a windows system, both server and desktop machine.

The MMC console can be used for almost all management tasks both within a local machine and also on remote computers. This tool makes remote administration really easy just by simply selecting the machine name you require to administer once you select the appropriate snap-in.

Using MMC console

  1. To use the MMC console you just click select run from the start menu [windows key + R] and type MMC (on computers with multiple privileges you might have to use the runas command).
  2. You can now add several components to the MMC console depending on the task you want to do. Go to file -> add remove snap/in -> add.
  3. Try adding the computer management snap in which is a collection of other snap ins to allows you to do pretty much everything to manage your computer. The disk management facility allows formatting new disks (there is no need for partition magic), changing drive letters and work with dynamic disks to span and strip disks. Other options include monitor event logs, services, disk defragmentation, user and group management etc.
  4. Also add the group policy snap in. This allows you to implement a computer policy based on how you want groups of users to use the computer. You can restrict access to most resources by creating policies. For example you can remove items like my computer and recycle bin from desktop, standardise desktop images, hide the control panel, etc.

The last part of the lesson was about disaster recovery management which is a fancy word for backup and restoring. Like all backup classes the golden rule in backup is: Your backup is only as good as your restore procedure.

Now, have you ever wondered what the following items mean in windows?

achieve file attribute
This file attribute is used by windows backup software to indicate that a file requires backup (if the attribute is set).

load last known good configuration
You're likely to see this option after a crash. This option loads the registry settings as of the last log on, so if you logged on installed a new driver and the computer crashed then the last known good configuration loads the registry before the driver got installed. If on the other hand you manage to log in the system after you install the driver than the computer malfunctions this option becomes pretty useless because the cocked up registry is saved.

Creating a startup disk in windows XP / 2000 / 2003

  1. Select the show hidden files and folders and uncheck the hide protected operating system files, from the folder options under tools in my computer.
  2. Copy
    • boot.ini
    • ntdetect.com
    • ntldr
    • if they exist copy bootsect.dos or ntbootdd.sys
  3. From command line type attrib -h -s -r a:*.*

For more details check out the microsoft support site

Why Work?

Thursday, August 12, 2004

Why am I working at work instead of writing my blog? Why am I searching work at home instead of dedicating more time to my blog? Why did I go to work yesterday on my birthday? Am I'm programming?

Boss is full of shit

Dry Links:-

Portland Pattern Repository

Test Infect your Java applications

Extreme Programming

Failure Notification

Thursday, August 5, 2004

Work is too busy to blog at the moment so that's why my posts are less frequent than usual. At home I'm either very tired after work, or else I'll have Malaysia homework to do which leaves less time for words.

This week I decided to refactor my automatic download / extraction programs to make them more robust and less susceptible to un-notified errors. This need came from the fact that I realised the stupidity of users, most of which don't bother to notify you when there is a download error. People do not realise that if they don't notify me I would not know what is going on with my automatic programs. At last I decided that I had to make the programs themselves notify me if they decide to take a day off.

Now I've filled batch files with ERRORLEVEL trapping and added job failure notification in my scheduling program Automize. From experience it's much better to get e-mailed when the program is not working rather than when it is, because as time passes you start to unconsciously disregard success emails. The problem with this approach is that if the job is not run in the first place then you're not notified. It would be a great idea if a daily report can be sent at the end of the day so that at a glance you can identify the status of each job.

Hand in hand with this issue, this weekend I got interested in testing programs and frameworks. This whole testing thing is an integral part of extreme programming. It seems that the most common Java testing framework is Junit. JUnit is an open source unit testing framework, but it is extended to different variants of Java programming (like Cactus for JSP and Servlets and JUnitPerf for testing performance and scalability)

As a last note on computing, there is a good article about passwords or their lack of, in this new blog.

Yesterday we watched Gothika (review) starring Halle Berry and Penelope Cruz. At least the film proved slightly better than the newspaper articles. The first part could have been much more interesting but at the end of the second part the story gets interesting. 2.5 Stars.

Database problems and solutions

Friday, July 16, 2004

This week was a hard programming week at work and I had to reface the hardships of database programming in Java with JDBC. On Monday morning I decided to solve the quotes problem when inserting and updating into the database. As in most programming languages, SQL gets quite upset when certain special characters are used incorrectly. In SQL's case the single quote (') is the trouble maker because it has a special use to enclose a character / string value. If you've got a field and a user types in a single quote the single quote must first be "escaped" by inserting another quote, before adding it to the insert statement. So for example insert into remarks value ('I don't want to drive'); must become insert into remarks value ('I don''t want to drive');.

My first attempt at solving this problem was to use the replaceAll() function in Java to replace all ' by '', for each field. This method is cumbersome and not very elegant, so I had to look for a better solution. The answer was to use the PreparedStatement object instead of the Statement object to update values in the database. Apart from solving the quotes problem the PreparedStatement pre-compiles the SQL statement passed as a parameter, thus making it faster than the normal statement. For more information of the usage of the PreparedStatement object see Sun's Java JDBC basic tutorial.

While I was learning how to use the PreparedStatement object I stumbled across a new connection leakage error. The actual error was: ORA-01000 maximum open cursors exceeded. To help me analyse the fault better I found two SQL statements to document the number of open cursors in the database:-

SQL Statement 1: List of al open cursors, users and the SQL executed.

select user_name, status, osuser, machine, a.sql_text
from v$session b, v$open_cursor a
where a.sid = b.sid;

SQL Statement 2: The number of truly open cursors.

select a.value, b.name
from v$mystat a, v$statname b
where a.statistic# = b.statistic# and
a.statistic# = 3;

The culprit for the error was that I was had an unclosed PreparedStatement well hidden inside a loop. Although it was not so difficult locating the error, the problem was not fixed immediately by changing the code, so I started going around in circles looking for problems that didn't exist. The reason for this was that Oracle frees the inactive connections every 15 minutes (this might be dependent on a database parameter [INITSID.ORA]), so the open cursors had to be freed before the code could work.

This week I had very little time to experiment with common applications but I found a quick way to load Adobe Acrobat faster i.e. without loading the plug-ins. To do this all you've got to do is press [Shift] while the program is loading. Apparently this feature also works with other programs that load plug-ins.

Some links:-

Calciomercato.com
This is the most complete source for summer football transfers, covering most of the European clubs. (Calciomercato means Soccer Transfers in Italian)

Perry-Castaneda Map Collection
Need a map? Go here.

Awstats
The best open source web log analyser around. I compared it to analog and webalizer but they are too featureless against awstats.

Ofofo
Photo storage and sharing by Kodak