A Squeeling [sic] Lunatic and the PageRank Algorithm
An item (Delusions of Community) in the otherwise apparently unattributed Pensieri di un lunatico minore ("Thoughts of a minor lunatic") blog attacks my blog entry Publish the PageRank Algorithm Now! for things I said, things I didn't say, and also contributes ad hominem personal attacks on me to this discussion.
My original entry was reposted in my O'Reilly blog.
Here's the opening salvo from the minor lunatic: "squeels Harold Davis on an otherwise reputable O’Reilly site." [Thanks - at least I know how to spell "squeals."] Before I get to some substantive issues, here's the concluding attack: Also, for someone who so admires the “feedback” from open source, he [Harold] doesn’t have comments or trackbacks turned on on his own blog.
Trackbacks and comments are something I've thought about a great deal. I have comments and trackbacks turned off on my blog for two reasons. One is administrative: I simply don't want to deal with the spam that results from keeping them open. (It's my experience with the other blogs that I administer that do allow comments and trackbacks that this is a real problem.) More important, from a conceptual viewpoint, my blog is my blog: I use it to express my opinions. I am not a public utility, and I have no delusions of grandeur that I am comparable in any way to Google (alas!). My blog is not a community forum, it is a bully pulpit, and anyone who wants to comment can do so in their own blog, as the minor lunatic did so trenchantly. (And, at least, I sign my own name to my opinions.)
In my original post I clearly noted "It's probably unreasonable to expect Google to publish how PageRank really works in light of competition from other search engines, and the efforts of SEO Webmasters to game the system." I'm in fact deeply disturbed by the gaming of the system that is going on. I think it is leading to increasingly bad search results. In other words, the search spammers are winning. This is too bad. Is the answer more of the same old same old -- which isn't working? (And keep it a deep, dark secret at the same time?) I think not. In the long run, it is effectively possible to reverse engineer PageRank empirically if in no other way because the results are so self-evident.
I mention all this because the minor lunatic notes (supposedly from friends inside Google) that keeping everything secret is the only way to stay ahead of the search spammers. Since this isn't cryptography, he says, where secrecy means bad design, it is "instead a situation where secrecy is the only option..."
Rhetorically, he asks "Does Mr. Davis really believe everything benefits from a million unexpert eyes? Google has managed to hire just about everyone in the field, so the likelihood of someone solving some huge problem in their equation is pretty small. How many of the big math problems get solved by random people, and they’ve been published for hundreds of years?
...
Google doesn’t stand to benefit one whit from Joe Random Programmer looking at a 100 variable equation. Most programmers suck at understanding equations to start with, otherwise you wouldn’t see so many hair-brained bad ideas that would have been solved with the opening of a volume of Knuth.
Google has become a magnet for the best and brightest in many many fields." [I've omitted some of the argument.]
Well, no, I don't believe that everything benefits from openness. (I never said I did.) I just believe that the mechanisms behind forces that have a huge impact on our lives should be transparent. We should be able to verify the results of elections, and we should understand (at least roughly) how Internet search orders its results. (If you get the idea that I'd dearly love to review the real PageRank algorithm, you are right.) Keeping things secret will not foil the spamsters. I distrust authority even when it is as benign as Google, and I am always mindful of Lord Acton's dictum about absolute power corrupting absolutely.
Yes, I agree with the minor lunatic that I'd rather have smart people who are naturally interested in a field working on it, than everybody under the sun regardless of proclivity or talent. That said, it is my absolute conviction that more transparency, and more community involvement, would benefit the formulation of Google's search algorithm.
Posted by Harold Davis at April 19, 2005 03:37 PM
