Technorati Top 100 blogs analysis methodology problem - different data sets!
Theron Parlin pointed out a severe problem in the data comparison of Technorati 100 Here Today Gone Tomorrow, which discusses shifts in the Technorati list of top 100 bloggers. It turns out that it compares data generated by two different algorithms! He's right (and I shouldn't have let the sociology distract me from investigating it, since I knew of the algorithm change). The Top-100 algorithm was changed in September 2005, per
http://www.technorati.com/weblog/2005/09/50.html
"Technorati now displays the total number of links from blogs over the last 6 months. Up until now, we displayed a count of all links from blog homepages, which tended to weight more highly blogs that have been around for a long time, even if they have not been posting recently."
The two sets of data being compared (from different time periods) are apples vs. oranges. They use two different means of making the list.
Sigh. This is a sad illustration of what's wrong with blog evangelism. First, the initial analysis was echoed, because it was viewed as saying something blog evangelists wanted to hear (one quote: "Unlike the old world, in this new world, quality is for all to see. Mr. Louis proves that." - which wasn't proved at all, except in a trivial sense). Then the politics distracted from an underlying technical flaw. Finally, 100% like "old media", the correction is never going to be heard to a comparable audience, since it's just not as good a story, even though it's right. Popular wins over accuracy.
And the irony is that further, it'll likely be said that the bogosphere is better, because of an obscure comment pointing out the problem, and a page somewhere that might potentially conceivably imaginably be found by a search engine. Don't you just feel the thrilling power of the new media?
Moreover, Tristan Louis does good work, and anyone can make a mistake. So I'm not motivated to run around to the echoings and make a futile attempt at once again battling A-lister's snake-oil. Again, why bother? Contrary to the promise of "having a voice", nobody much will hear. :-(.
By Seth Finkelstein | posted in cyberblather | on February 27, 2006 11:59 PM (Infothought permalink)
Over-modesty ill becomes you, Seth! - someone heard, so the tree DID fall ...
I thought long and hard about this one as people emailed me about the change after I posted the entry. At the end of the day, however, I decided that it does not matter much. The reason being that, whether the algorithm had changed or not has little matter in that the top 100 should be the same all the time if the theory of gatekeeping was right. few algorithm could break that kind of power. However, it appears that the list is much more active. I'll revisit the list in another six months and, with an algorithm that won't change, I expect to see as much movement then as I did now (but I could be wrong as I was surprised to see as much movement as I did on this one. I thought that it would show little to no move when I set out to do my analysis and discovered that my own thinking was wrong)
I think people are hearing. For all the drums are beating about the power of weblogging, it doesn't take an overly sharp person to see something has drastically changed in the last month.
I think we have done peaked.
Shelley, people may be hearing some of the skeptical media columnists - but they sure aren't hearing me! :-(