12 May 2007

Why Alexa Traffic Ranking May Be Misleading

It has to do with how the sample is collected and the sample size.

According to Alexa, only browsers with the Alexa toolbar are taken as part the sample. Trained statisticians will know that this constitutes a selection bias. To make matters worse, the number of Alexa users compared to the total web surfing population is mediocre, at best. When the sample size is too small, it may lead to inaccurate results.

These deficiencies could very well mean a great difference between Alexa's representation and actuality.

To illustrate, suppose U have a website with ten thousand visitors but only one of them has the Alexa toolbar installed. Alexa will take it as though U had only one visitor. Now, suppose U have a second website with only a thousand visitors but all of them have the Alexa toolbar installed - Alexa will account for one thousand visitors!

Clearly, if we rely on Alexa's representation, the second site will outrank the first site, even though we know this is the exact opposite of reality.

Learn more about how Alexa's traffic ranking works.

Pay particular attention to the points listed under "Important Disclaimers".

1 comment:

Wei-Yee Chan said...

Precisely.  And here's the best example of Alexa's inaccuracy:

What Have I Done To Deserve This?

This morning, I did a check with Google Analytics and the creeps were nowhere on my top 20 countries list. Yet, on my Alexa page, they were showing up as number 1.