The Googleplex Blog: Harold Davis's Blog

Google AdSense in RSS and Atom

In a contextual sea change, Google has announced a beta program that allows publishers to embed AdSense contextual ads in RSS and Atom syndication feeds.

"This is gonna be huge… like HUGE HUGE HUGE!!!" gloats Jason Calacanis, the President of Weblogs, Inc., which - as you might expect - publishes a bunch of syndication feeds. As a publisher of syndication feeds in a small way, I suppose I ought also to be glad. But actually, I think monetizing RSS and Atom feeds in this way in part defeats the purpose of having a feed. Feeds are simply an information stream that point to further information. If they get cluttered, they cease to be useful, and subscribers will cancel. In some sense, the RSS or Atom feed is an advertisement for the full content in and of itself.

Case in point: it is against the policies of Hot Feeds and Syndication Viewer to display feeds that carry ads.

Here's the way Weblog's unofficial Apple feed, which Calacanis is using to test the syndication AdSense program, looks (with ads) in Syndication Viewer. Each Google AdSense ad is simply an HTML table embedded in feed items like this [identifying numbers and actual link omitted]:

... <item> ... <description> <![CDATA[ ...actual item... <table border="0" cellspacing="0" cellpadding="0"> <tr> <td><span style="font-size: 11px;"> <br/></span></td> </tr> <tr> <td><a href="http://imageads.googleadservices.com/pagead/imgclick/XXXXXX pos=0"> .... </a></td> </tr> <tr> <td><div align="right"><a href="http://www.google.com/ads_by_google.html" style="font-size: 9px; color: #999999;">Ads by Google</a></div></td> </tr> </table> ]]> </description> </item>

The good news: it ought to be trivial to parse these ads out of incoming feeds, simply by eliminating table tags and their contents from item entries if in no other way. I will certainly do so in Syndication Viewer.

Tautologies: Logical, Linguistic, and George W. Bush

A logical tautology is a statement that is true by definition, or, as the dictionary puts it, "a statement true by virtue of its logical form alone." For example, in the tautologous cliche department:

Wherever you go, there you are.
What will be, will be.

The dictionary has another definition for tautology, sometimes called a linguistic tautology: "Needless repetition of an idea, statement, or word." This kind of tautology can be used rhetorically as a figure of speech to emphasize a concept (it is related to a pleonasm). More often, linguistic tautologies are basically pathetic and silly (or at least unconcise and poor use of language). For example:

Past history
Planning ahead
Main crux
True fact
HTML Language
Sierra Nevada Mountains (snowy mountains mountains)

Here are some of President George W. Bush's tautologic utterances:

"It's very important for folks to understand that when there's more trade, there's more commerce." —George W. Bush, at the Summit of the Americas in Quebec City, April 21, 2001

"If affirmative action means what I just described, what I'm for, then I'm for it." —George W. Bush, during the third presidential debate, St. Louis, Mo., October 18, 2000

". . . the past is over." —George W. Bush, after making up with John McCain, Dallas Morning News, May 10, 2000

(Thanks to Language Log for the Bush tautologies.)

Microsoft is IBM and Google is Microsoft

When I opined over on my O'Reilly blog that Microsoft was the bloated (but rich) IBM of our day, and that smart, nimble Google would overtake Microsoft, I got a lot of flak. For example:

"Is this a troll? IBM remains a much larger company than Microsoft, with 96 billion USD in revenue last year compared to MS's 38 billion. Google had *3* billion in revenue in the same period, but with a price-to-earnings ratio of 137(!), compared to IBM's 15 and Microsoft's 26, whose stock would you buy?
IBM and MS create software for sale. What exactly does Google sell, besides advertising?..."

Now Fortune Magazine comes along with a cover article ("Search and Destroy: Why Google Scares Bill Gates") that more or less says what I said...

Changes to Google's AdSense

The changes to Google's offerings to advertisers that I wrote about yesterday are of course interesting to publishers who make money with AdSense.

The impact is that Google is offering to advertisers (besides the ability to target specific sites) a new way to pay: by impression (CPM) rather than per-click-through (CPC). The AdWords program will somehow balance the two payment schemes to find the one that pays the most.

From a publisher's viewpoint, it's nice to receive payment every time an ad is displayed on one's site (rather than when the ad is clicked). This is what CPM means. How much these CPM ads will filter down to smaller publishers remains to be seen: to the extent that they do, it becomes another monetization option for publishers with worthwhile content. (And also, one doesn't have to worry about click fraud with CPM!)

Google Announces Site Ad Placement

In a further departure from its roots in searching, Google has announced a new program that will allow advertisers to choose sites for target ads.

I've written in the past about Google's transformation (at least looking at revenue) from a search company to an advertising broker. But contextual advertising - Google's other-than-search bread-and-butter - still involves technology that automatically caluclates relevancy, just like a searching algorithm, and produces a marketplace for words. Whether the context is evaluated correctly or not by the automated mechanism is another story.

In the new Google order of things, advertisers interested in branding can pick their sites without regard for contextual relevancy. The New York Times bills the changes as a move away from search for Google, and Brad Hill in his blog calls the move "industry shaking."

Advertisers will pay for the new-style ads on a CPM basis, or per ad impression (not per ad click as with contextual ads), although the process of purchasing these ads will be blended with the traditional Google CPC (pay per click) word auction process.

These ads are intended to appeal to big advertisers who are looking for general branding (for example, all kinds of advertisers of luxury goods would probably like to appear on BMW's site, even if the ads were not contextually relevant to cars).

Context-free ads may also work for advertisers who are better able to determine relevance than the automated algorithms - it makes sense to put ads for cheese on a oenophile site, but AdSense probably doesn't think so. Google's revenue stream will be a winner, as will big advertisers and owners of desirable Web content. Possible losers: anybody but Google in the business of brokering ads.

Stop Tiptoeing Around the Radical Right!

Microsoft recently bowed to pressure from a local church (the Antioch Bible Church) and its pastor Ken ("Hutch") Hutcherson to stop supporting local legislation that would have put a statute on the books forbidding anti-gay discrimination in Washington State. ("Hutch" is an ex pro-football player and powerful figure in politics in the Redmond, Washington area.)

In a company-wide email, CEO Steve Ballmer explained that he and Bill Gates supported the legislation personally. However, as one blogger quoted in the NY Times coverage put it, the company didn't want to offend the religious right.

Here's New York Times coverage of the affair. Microsoft's change of heart on the anti-gay discrimination legislation is not surprising considering the bloating and beaurocratization of the arterioslerotic Microsoft I blogged recently. This company wants to keep out of trouble.

Another somewhat humorous marker of the changing times was the "resignation" (or firing) of the head of PBS over the creation of an episode of the children's show Postcards from Buster. In the episode, Buster the bunny rabbit visits a lesbian couple. Here's an interview with the new PBS director, who would like to appeal to more conservative viewers.

It's high time to stop tiptoeing around the religious right. For starters, let's drop calling the Bush-Cheney gang "conservatives." They're not (a conservative doesn't want fast change). They're radicals, with an agenda of theocracy and more than a whiff of facism. Stop them before it is too late!

Why Our Health Care System Is Broken

Paul Krugman has a nice op-ed in today's New York Times which provides a simple explanation for why the United States has the most expensive and least successful healthcare system of all the developed nations.

It is that too many health care resources are wasted trying to shift costs onto other parties. This matches my experience, and the "other guy" is often you and me. As Krugman also points out, a side-effect of the system is that doctors have to spend lots of money on hiring staff to fight the insurance companies.

Let's stop this insanity and fix this broken, crazy system starting now!

Statistically Improbable Phrases (SIP)

Statistically Improbable Phrases (a/k/a "SIP") is the improbable term Amazon.com uses as a search ranking technique. Here's Amazon's explanation.

In more-or-less plain English, here's how this works. Amazon indexes the "Search Inside" content of the books in its catalog (that is, the books in which publishers provide this content). In many cases, Amazon provides a list of SIPs on the main listing page for the title. For example, Starting an Online Business for Dummies by Greg Holden has a number of linked SIPs listed, including "your online business." These SIPs are phrases that appear with anomalous frequency in the inside content of the cataloged book compared with the entire the rate of occurence of the SIP in the universe of books in general. This statistic over-occurence implies that the SIP is a significant representation of the content of the book.

By clicking one of the SIP links, you get other books in which the SIP occurs, sorted from most to least by the number of SIP references. For example, "Web Analytics" and "E-Commerce for Dummies" have the next highest occurences of the SIP "your online business" after "Starting an Online Business for Dummies."

This is a different and somewhat appealing way to use Amazon's search facilities to find books in which the author uses distinctive phrases. Longer run, the concept has an elegant simplicity (as did the original PageRank algorithm), and may be useful for automated tagging and ranking of content.

Click here for a lively discussion of SIPs in the context of author as phrase maker, and here's a fun discussion and list of adult SIPs on Amazon (over 18 only please click this link).

French Military Victories

The top result for the Google search French Military Victories is a parody page (also called a "Google bomb"). You can get directly to this Google-look-alike page that comically ignores the Battles of Hastings, Agincourt, Castillon, Hohenlinden (and so on) by entering the "French Military Victories" phrase in the Google search box and clicking the I Feel Lucky button. (The parody page on the Albinoblacksheep domain that looks like it is part of Google is, of course, the most popular search result for the term.)

Erica Sadun, in an O'Reilly Weblog, has a round-up of Google Easter Eggs and other Google anomolies.

Earth Day Google

Happy Earth Day!

Tracking Your Search History with Google

Google has a new feature that tracks your search history. (Click the link to open the sign-up page for the application, which otherwise can be accessed through Google Labs.) This is another one of Google's wonderful tools that is a "beta" that is not really a beta.

So far, the functionality is pretty straightforward and (at least for me) very useful. When you are logged in, and you can log in of course from any computer, Google keeps track of your searches. You can click on any of the links that represent a saved search to see the full text of a search. You can also retrieve searches by date using the calendar that the Search History Tool provides.

Once you sign up for the Search History Tool, your Google home page changes. Up on the right-hand top, you'll see your sign-in email, a link that takes you to your account history (which is where to find the calendar and search links, and also the ability to remove any or all search items), a link that takes you to your Google account settings, and a link to sign out. If you do sign out, Google's home page will show you a sign-in link.

Keeping track of my search history is a very useful feature for me. I can't tell you how many Google searches I do a day (probably in the three or four digits), although the Seach History Tool will in fact tell me this. Many times, I've "lost" information from a search that I thought I didn't need (but actually did!) The Search History Tool will pretty much solve this problem for me, I think.

Down the road, the Search History Tool will probably let Google refine searches for me based on my search history (it remains to be seen how helpful this is).

The Search History Tool may allow customization that is an important weapon in the battle against search spam, because I may be able to "train" my future searches by deploying a "Junk" setting against my Search History results. Other forms of search customization, once I'm logged in to search, are also possible of course.

I also see the Search History Tool as a Trojan horse for the introduction of more Yahoo-like services. Google needs to know its users better to create these services: and what better way to know someone than to keep track of their searches?

Pope Benedict Rewrites History

Joseph Ratzinger, the new Pope Benedict XVI, recalls (according to the New York Times) that the Roman Catholic Church was his bulwark against the Nazi regime, "a citadel of truth and righteousness against the realm of atheism and deceit."

Huh!?! In fact, the Roman Catholic Church, with its ancient tradition of anti-semitism, was more than a little collaborative with the Nazis, and did less than nothing to save the Jews.

For a good book on this topic, see the National Book Award winning Constantine's Sword (by James Carroll, a devout Catholic, at least before he wrote the book).

If there is a God up there concerned with us mortals, God must be looking down, bemused by the hypocrisy of Pope Benedict's recasting of history.

Responding to the Response to My Response

A blogger took exception to my post about opening the PageRank algorithm in my O'Reilly blog. I responded to the response here. The blogger responded to my response, and I made the following response to the response to the response (confused yet?). (This was originally posted as a comment to the other blog, but I thought it worth adding here as well.)

As far as I am concerned, this is a much more reasoned comment than your first one (although I still wish you would sign your name as I don't want to spend the time looking you up, and I don't want to refer to someone I am having a dialog with as "il minore" whatever).

The primary thing you said I said that I didn't (and that in fact I don't believe) is that everything should be open. I do not believe this, and never said I did. Some things should, and some things shouldn't -- although I think Linux is a case in point of something that has clearly benefited from being open.

It's both a blessing and a curse to see both sides of an issue. The reason for the "polarity" of my position is, of course, I see the problems with any kind of disclosure of PageRank. Bearing in mind these problems, and the unlikeliness of it ever being disclosed, here are the reasons I think at least some more community discourse regarding the precise nature of PageRank would be helpful:

(1) PageRank, based on my searches, is not working as well as it used to. My impression is that the rate of deterioration is increasing. So it is not the case of "if it isn't broken, don't fix it." Rather, it is this isn't working, and Google is playing catchup to try to make it work, kludging together something with 100 variables (!). The elegant simplicity of the PageRank concept has clearly been lost.

(2) The time delay built into newer iterations of the Google model really bugs me. I like my information fresh! And as someone who is frequently putting up web sites, I like to be able to get them picked up fast without resorting to chicanery myself.

(3) In fact, Google is the predominant way people find information on the web. Anyone who thinks this is not very important to people, politics, and life is naive. And, Google itself is more of a community effort than may be apparent. Case in point: Google uses the community-run Open Directory Project for major taxonomic information.

(4) It's bad when Microsoft is heavy-handed and secretive, but OK when Google is? Come on, Googlers may be the good guys, but let's hold them to the same standards as everyone else.

(5) No, I do not believe Google has hired all the smart people with something to contribute to search. What baloney! Sometimes the best ideas in fact do come from outside the box.

A Squeeling [sic] Lunatic and the PageRank Algorithm

An item (Delusions of Community) in the otherwise apparently unattributed Pensieri di un lunatico minore ("Thoughts of a minor lunatic") blog attacks my blog entry Publish the PageRank Algorithm Now! for things I said, things I didn't say, and also contributes ad hominem personal attacks on me to this discussion.

My original entry was reposted in my O'Reilly blog.

Here's the opening salvo from the minor lunatic: "squeels Harold Davis on an otherwise reputable O’Reilly site." [Thanks - at least I know how to spell "squeals."] Before I get to some substantive issues, here's the concluding attack: Also, for someone who so admires the “feedback” from open source, he [Harold] doesn’t have comments or trackbacks turned on on his own blog.

Trackbacks and comments are something I've thought about a great deal. I have comments and trackbacks turned off on my blog for two reasons. One is administrative: I simply don't want to deal with the spam that results from keeping them open. (It's my experience with the other blogs that I administer that do allow comments and trackbacks that this is a real problem.) More important, from a conceptual viewpoint, my blog is my blog: I use it to express my opinions. I am not a public utility, and I have no delusions of grandeur that I am comparable in any way to Google (alas!). My blog is not a community forum, it is a bully pulpit, and anyone who wants to comment can do so in their own blog, as the minor lunatic did so trenchantly. (And, at least, I sign my own name to my opinions.)

In my original post I clearly noted "It's probably unreasonable to expect Google to publish how PageRank really works in light of competition from other search engines, and the efforts of SEO Webmasters to game the system." I'm in fact deeply disturbed by the gaming of the system that is going on. I think it is leading to increasingly bad search results. In other words, the search spammers are winning. This is too bad. Is the answer more of the same old same old -- which isn't working? (And keep it a deep, dark secret at the same time?) I think not. In the long run, it is effectively possible to reverse engineer PageRank empirically if in no other way because the results are so self-evident.

I mention all this because the minor lunatic notes (supposedly from friends inside Google) that keeping everything secret is the only way to stay ahead of the search spammers. Since this isn't cryptography, he says, where secrecy means bad design, it is "instead a situation where secrecy is the only option..."

Rhetorically, he asks "Does Mr. Davis really believe everything benefits from a million unexpert eyes? Google has managed to hire just about everyone in the field, so the likelihood of someone solving some huge problem in their equation is pretty small. How many of the big math problems get solved by random people, and they’ve been published for hundreds of years?
...
Google doesn’t stand to benefit one whit from Joe Random Programmer looking at a 100 variable equation. Most programmers suck at understanding equations to start with, otherwise you wouldn’t see so many hair-brained bad ideas that would have been solved with the opening of a volume of Knuth.

Google has become a magnet for the best and brightest in many many fields." [I've omitted some of the argument.]

Well, no, I don't believe that everything benefits from openness. (I never said I did.) I just believe that the mechanisms behind forces that have a huge impact on our lives should be transparent. We should be able to verify the results of elections, and we should understand (at least roughly) how Internet search orders its results. (If you get the idea that I'd dearly love to review the real PageRank algorithm, you are right.) Keeping things secret will not foil the spamsters. I distrust authority even when it is as benign as Google, and I am always mindful of Lord Acton's dictum about absolute power corrupting absolutely.

Yes, I agree with the minor lunatic that I'd rather have smart people who are naturally interested in a field working on it, than everybody under the sun regardless of proclivity or talent. That said, it is my absolute conviction that more transparency, and more community involvement, would benefit the formulation of Google's search algorithm.

Maps and Satellite Photos @ Google

If you haven't tried it, the relatively new mapping capabilities at Google are very cool. I like the maps. You enter an address (or portion of one). The user interface is very sparse, with a widget in the upper left to control zooming in and out and panning across a map. Like Mapquest, you can get driving directions to or from an address. Unlike Mapquest, there are no annoying ads, pop-ups, and other distractions. You can use the Google maps to find businesses or services of a specific type in a given locale.

For reasons I can't quite put my finger on, I think the Mapquest maps may actually be a little better for navigating by car than the Google maps. But one feature of the Google mapping application is, in fact, cool beyond belief. If you click the Satellite button on the upper right hand corner of the screen, you can see the aerial, satellite photographic view of any map. The zooming and panning tools work with these satellite pictures.

Start with where you live from above and pinpoint your block and rooftop. You can zoom in and out, see your whole city or state. Kids love this.

Some fine print: Google maps and sat photos are limited to the United States and Canada (more world coverage is promised soon). Coverage in rural areas can be spotty. This, however, corresponds rather well to the areas that are not much sought after (click here for a Google engineer's visualization of frequency of search by locale).

More fine print: the photos seem somewhat dated (for example, big elm trees can be seen in the aerial view of my house, they came down in winter storms over two years ago). There are the usual sporadic reported glitches in the maps (this is not unique to Google's maps).

Here's a neat application that combines Google Maps and Craig's List so you can view the location of Craig's List real estate listings.

Regan Moves to California

No, not President Reagan (he's gone to a far, far better place!). Judith Regan, the publisher, is moving her company to Los Angeles.

Judith Regan's ReganBooks is an imprint of HarperCollins. HarperCollins is part of Rupert Murdoch's News Corporation empire.

In his Fresh Books blog, agent Matt Wagner bemoans the dearth of publishers in California. Well, as of now, Judith Regan is helping redress this issue. Regan is wildly successful, highbrow and lowbrow (sometimes both at the same time) and not afraid of controversy. What you may not realize is that she also peddles sex books. Her sex books use the same formula as her books that don't concern sex: a cocktail of vulgar and highbrow, heavy on the celebrity, with just a dash of controversy thrown in. Shaken, not stirred.

Case in point: Many of the books featured in the Hot Reads section of Hot Feeds are published by ReganBooks, such as the Jenna Jameson bio, Toni Bentley's Surrendur, and so on.

Leonardo Google

Leonardo Da Vinci Google

Mister Softee in His Big Blue Period

This is a tale of two software companies (or maybe three).

They were the best of companies, and they were the worst of companies. And they've yet to go to a far, far better place to peddle their software.

To get back to my story, corporations, like people, have a lifecycle with a beginning, middle, and end.

Once upon a time, a long time ago, in a land far far away, Big Blue - IBM - was the be-all and end-all of everything to do with the computer industry. Beaurocratic, ponderous, rich, and powerful, IBM watched the nimble Mister Softee - Microsoft - steal the software side of the computer business out from under it.

Mister Softee was everything Big Blue wasn't: a teenage rebel, improvising like crazy, able to turn on a dime, handing out stock options like candy in a bowl near the cash register of a restaurant where the food isn't so good.

Now Mister Softy has grown as soft as its nickname and frumpy, with middle-aged love handles to match. Stock options are long gone. Microsoft pays a dividend!

Microsoft wants to climb the enterprise.

This company is no longer nimble, and takes literally years to pass software through its beaurocratic process before release.

We didn't love Mister Softee when he was young and agile, but he always impressed us with his vigor and chutzpah (though we always wished he built better, less buggy, software).

Now Mister Softee is as rich as Croesus and out-IBMs IBM. He's dangerous! He's fat! He's rich! His speed of innovation is falling way behind compared to younger rivals like Google. More than ever, he's fun to despise.

Maybe, just, maybe, Mister Softee is also starting to become irrelevant (thanks to Open Source, Linux, and the Web).

Press Release: Building Research Tools with Google

Building Research Tools with Google for Dummies by Harold Davis is now available from Wiley Publishing and your favorite bookseller. Here's the full press release.

Delay in Ranking a Feature, Not Bug

According to a recent article in SitePro News, an online publication aimed at Webmasters who want to optimize their sites, Google's delay in ranking sites, and the delay in according credit to inbound links, is a feature, not a bug.

I've written critically about the longer and longer wait times for sites to get indexed as a problem (see Is Google Painting Itself into a Corner?). Now, according Lawrence Deon, an SEO (Search Engine Optimization) expert and the author of the article titled "Surviving Google's Aging Delay" in SitePro News, it turns out that Google does it on purpose as part of the arms race with those gaming the system. The "probationary period" makes sure that there are no instant returns for "manufacturing" tons of links, but it also makes it harder for newcomers to break into the system. In addition (this may or may not be an unintended side effect) it makes it more worthwhile than it used to be to purchase AdWords slots from Google to draw traffic, at least during the probationary period.

The idea that Google probably intends to slow indexing and ranking down, and that this is (in some ways) to Google's financial benefit, makes me call again for publication of the details of the PageRank algorithm. (See Publish the PageRank Algorithm.) Let the antiseptic of open scrutiny and discussion work its magic on this matter that is so important to the Web!

Trashing Movable Type

If anybody was spending their Sunday reading my blog, they'd have noticed that it was mostly hosed (meaning trashed). They might also have noticed that links to individual blog entries still worked, more or less. What a mess!

As they used to say (whoever they are) when I worked at an enterprise database company, "Live by a database, die by your database!" The problem was that my Web hosting company had hosed the instance of the MySQL database used to store the Movable Type entries, and were taking their precious time about restoring the database from backups. Compounding the problem, yesterday before I released the database was corrupted, and after it had been partially restored, I rebuilt the site, writing over the then-current state of the database.

To be absolutely precise, Movable Type doesn't actually store the content entries in the database, at least the way I have it configured (each entry is a file). But it does store pointers to all the information in the database tables, so that without this information one has nothing.

Movable Type, and other server-side blogging software, works with a database and a Web server to form a Content Management Server (CMS). Yes, this is complex software. One should back up content regularly, and not just rely on a remote host to do this with enough care. (Mea Culpa!)

Blogging software started out as something to meet an apparently simple purpose - to facilitate posting of an online diary in reverse chronological order. As time as gone by, this functionality has expanded. Blogs automatically put out syndication feeds, for one example. And blogging has become a widespread communication channel, used for corporate communications, technical support, and so on, as well as personal communications. The complexity of installing, configuring, and maintaining server-side blogging software now rivals that of "grown-up" enterprise content management servers (products from Documentum, Filenet, Microsoft, Vignette, and others).

If you're not a happy propeller head, you probably shouldn't get involved in running your own blogging software like I do (it is a time sink if nothing else).

There are a number of services that will host your blog: Google's Blogger, LiveJournal, or Typepad (both Live Journal and TypePad are associated with Movable Type.)

For the blog you want, hosting costs something. If you are going to run your own copy of Movable Type, you can get Six Apart, the company that publishes Movable Type, to install it for you (again for a fee). And if you want anything fancy in your weblog design, you probably should find someone who is good at creating customized sites using the software and work with them.

Needless to say, I breathed a big sigh of relief when I got my database restored (rolled back to Saturday). I rebuilt my blog, and it was pretty much as good as new. I'm now resolved to be conscientious about personal content backups (in addition to the ISP), ah, after I finish this one more entry...

Google National Library Week

It's Time to Scour the Shire!

In the final set piece in J.R.R. Tolkien's epic The Lord of the Rings, the four hobbits return to their beloved Shire to find it overrun by ruffians. (For some reason, the scouring of the shire was omitted from Peter Jackson's otherwise wonderful movie version of LOTR, which proceeds more or less directly from the defeat of ultimate evil and the coronation of the new king to Frodo's departure overseas with the Elves.)

The four hobbits are the protagonists of the epic novel. They are Frodo Baggins, Samwise ("Sam") Gamgee, Meriadoc ("Merry") Brandybuck, and Peregrin ("Pippin") Took. (I won't go into the question of class relationships between Sam and the other three, or the possibility of an erotic attachment between Sam and Frodo, as these are topics for another day.)

Frodo, Sam, Merry, and Pippin return home to the once bucolic Shire. The Shire is the home to slothful, boring, stuck in their ways, but kind hobbits with surnames like Grubb, Chubb, Burrows, Hornblower, Bolger, Bracegirdle, Goodbody, Brockhouse, and Proudfoot. Hobbits like to eat, sleep, gently party, and smoke "pipe weed." Inside most hobbits, however fun loving and lazy the exterior, is an inner core of strength.

The four heros have returned home following their date with consumate evil. In the process, Frodo and Sam have destroyed the one ring, the ultimate embodiment of the ability of evil to corrupt (in the sense of Lord Acton's dictum, that power tends to corrupt and absolute power corrupts absolutely.) The hobbits are not the same as when they left home on their great adventures; for better and for worse they have been changed by their experiences.

The Shire has changed, too, while they were gone. In some sense, the Shire of their carefree days is gone forever (at least for Frodo). The new Shire has been touched by evil. Trees and gardens are gone. Rivers are polluted and dying. Great red-brick factories belch stink and smoke all day long even though there is nothing for the factories to make. There are new ugly buildings, and lots of rules.

The ruffians who run the Shire are mean, petty, greedy, and self-righteous. These thugs have taken power in collaboration with a few rich hobbits. But the ruffians have forgotten the courage and core of steel that lies within ordinary hobbits. Frodo, Sam, Merry, and Pippin raise the Shire, and assemble an army of the people. The thugs are easily thrown out of their country, and the natural order is restored.

J.R.R. Tolkien knew evil. He served in the trenches in the First World War (most of his friends died). He wrote the LOTR during the Second World War. The question for us: where in 1930s Germany timeline are we? Have we passed the point of no return?

It's time for us to scour the Shire! Let's get rid of these greedy, self-righteous, mean, theocratic, anti-constitutional thugs who are running our country. Awake! Awake! Fear, Fire, Foes! Awake!

Related link: House majority leader Tom DeLay says the judiciary has run amok (New York Times account)

Do You Need an AP to Network via Wireless?

A question I've been getting a lot lately from readers is, "Do you need an Access Point (AP) to network via Wi-Fi Wireless?"

Well, the short answer is you don't need an AP to connect (ad-hoc mode works fine for this), but the connection in and of itself doesn't buy you networking.

Here's a typical reader question and and a longer answer.

"I have a laptop and a PC, both with wireless cards. Now what I interpreted out of your book is that these clients can talk to each other, intuitively I did not think this was so, as I was under the understanding that it took an active AP (Access Point) in order for a wireless connection to be made.

"I followed your process in chapter 15 to configure the ad hoc network for both the laptop and the PC and yup, my luck it did not work. Well luck has nothing to do with it, it either has the capability or does not!!

"Help!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"

Signed -- Without Wireless Networking in Seattle

Dear Without Wireless:

First, you may be interested in an updated version of my chapter with the Wi-Fi configuration instructions (including ad-hoc mode). It is available for free download on the Wi-Fi area of Braintique.com.

Second, in theory your two computers with wireless connection should be able to "talk" with each other in ad-hoc mode, although (based on response from my readers) sometimes this seems very hard to set up. The most obvious gotchas here (and there may be others):

- The cards need to speak the same flavor (e.g., 802.11a may not talk to 802.11g)

- The cards need to be set to the same station

- Station settings terms of encryption need to be the same on both cards

Next, realize the following limitation: you won't be able to network two computers by virtue of wirelessly connecting them anymore than if you simply connected them by USB or Ethernet ports.

It takes more than a connection to make for networking, you need a router, or a computer set up as a server, or P2P networking software on both machines, and you need to make sure the computers are set to share with each other, are part of the same workgroup, etc.

Generally, if they don't network by wire, there's no reason to expect them to network wirelessly, even if they connect.

Just to reiterate, you don't need an Access Point to connect two machines wirelessly (using ad-hoc), but you do need networking hardware and software (most Access Points provide built-in routers, which takes care of this).

A practical use for ad-hoc connectivity: Suppose you have a wire line network that uses a router to interface with the Internet (the so-called WAN connection). Let's say you have a laptop with wireless capability, and a wireless USB connector for a computer on your network. You could then set up ad-hoc networking via the wireless connection to allow you to connect to the Internet from the wireless laptop.

Blogging Makes History

Captain's Quarters, a blog written by Edward Morrissey, a Minnesotan and self-described "libertarian conservative" has helped blogging change history.

Morrissey's blog has been carrying an eye-witness account of the public corruption hearing in Toronto centering on the highest levels of the Canadian Liberal Party. A Canadian federal judge instituted a publishing ban covering the hearing in Canada with a flimsy rationale. Canadian news media posted the URL for Captian's Quarters for all Canadians to read; it received 400,000 hits yesterday.

Morrissey, a staunch conservative and Republican, says, "These information bans are self-defeating for free societies. The politicans know, the media knows, but the Canadian voters are left in the dark..."

As of today, the judge had at least partially lifted the ban, almost without doubt due to the information getting out through Captian's Quarters.

You can't keep important information secret in a world filled with bloggers, and this is truly a change for the better.

Related link: New York Times coverage

Some Obvious Truths about Click Fraud

If you don't know what click fraud is, here's a quick definition I just coined: clicking on contextual ads with no interest in purchasing the goods or services advertised and the intention of defrauding the advertiser or enriching the contextual publisher.

Click fraud is a major problem for contextual advertising vendors on the Internet like Google and Yahoo. An article about click fraud made the front of today's Wall Street Journal (link not supplied because WSJ is a pay-only site).

In addition, a somewhat under-reported lawsuit has been filed by Arkansas retailer Lanes Gifts (and others) against Google (and others) alleging systematic overpayment due to click fraud.

So I think it's time to state some - for me - obvious truths about click fraud. To wit, obviously:

- There is some click fraud (nobody really knows how much right now)
- Advertising brokers (Google, Yahoo, etc.) are taking counter measures (they can probably detect click fraud on a statistically large scale)
- Getting an adequate refund and explanation if you've been a victim of click fraud is not very likely (but then as a publisher on the Web it's unrealistic to expect good customer service from any of the sources of advertising revenue, at least in my experience)
- In the aggregate, contextual advertising does work and delivers targeted prospects much more effectively than any other method (if contextual ads didn't work, advertisiers wouldn't be paying the big bucks for them)

So where does this leave us? Advertisers and others need to accept that click fraud will always be with us. It should be understood that the Googles of this world will do their best to keep click fraud to reasonable levels, but that there will always be a fudge factor added to the number of "real" clicks. Get used to it and get over it!

Related links:
Contextual Advertising: Not
Foiling the Click Fraudsters
Words for Sale
Click Detective

Published: Building Research Tools with Google for Dummies

A real thrill: to step out on the porch and bring inside the carton with newly printed copies of my book, Building Research Tools with Google for Dummies. It's finally in print! Yippeee! (I think it looks great, but then again I would! Go out and get a copy and see for yourself if it doesn't help with your Google research conundrums...)

Related Links:
Building Research Tools with Google for Dummies companion Web site
Building Research Tools with Google for Dummies on Amazon

Fired Blogger Formulates Blogging Policy

In an exciting example of the Peter principle in action - rising to the level of one's own incompetence - Mark Jen a/k/a ninetyninezeros, who was fired for his blogging by Google, is now helping to formulate the blogging policy at his new employer, Plaxo.

Here's the syndication feed for his new blog on Syndication Viewer, and (for those of you interested in the back story) the feed for the blog that got him fired. He's no longer updating the old blog, and the new blog is a decidedly more corporate affair, including proper use of lower and upper cases.

An RSS blast from the past and an interesting blog feed

I've added a feed which is a blast from the past to Feedly.com. In this feed, each RSS item is a day from the famous 17th Century diary of Londoner Samuel Pepys. The diary entries are very thoroughly cross-linked, so you can quite quickly get up to speed with what is (was?) going on. Here's the feed displayed as HTML on Feedly.com.

I've also added the feed from Dave Taylor's Intuitive Life Business Blog to the Syndication Viewer. Here's the feed displayed as HTML on Syndication Viewer. Dave's a nice guy and seems to share a lot of my interests. A recent item on his blog asked "How important are publishers?" Dave's blog item includes a reference, made by Wiley publisher Joe Wikert, to an item in this blog ("The Times are Changing in Computer Book Publishing").

The Search of Tomorrow Meets the Chef of Today

A recent article in Information Week in a series about the future of software is called Search For Tomorrow.

As you'd probably expect, the article is about the state of searching software, but more about searching software today than in the future. I'm reminded that science fiction, whenever it is nominally set, is actually about the time it is written in. I'm also vaguely reminded of the classic Ralph Kramden routine from the Honeymooners in which as part of his get rich scheme of the moment he plays the Chef of the Future.

So what's the word from the Chef, er, Search of the Future? According to the Information Week article, it's big business: not only for contextual search-related advertising (who would have thunk it?), but also because of the enterprise need to search "unstructured" information. Federal Homeland Security is pumping r&d funding into the field. Microsoft is investing in basic search research.

According to the article, current search innovations tend to fall into the following areas:

- Tweaking search-results ranking algorithms (e.g., Google with more than 100 variables to calculate PageRank)

- Combining a web-style search with complementary, simultaneous other search sources, such as an encylopedia (Microsoft)

- Using current in-progress open documents as contextual hints when a search is engaged (Autonomy)

- Tagging unstructured data (text, email, audio, video) with meta-data (Autonomy, IBM)

- Better understanding the semantics of search requests (IBM)

- Creating a unified architecture for managing unstructured and structured data (UIMA from IBM)

The Information Week piece concludes: "The [search] tools we use now work pretty well. But more esoteric ones employed by just a handful of people today [the Chefs of the Future?!] could portend better approaches to come."

I find myself perfectly OK with complex query syntax (one of the supposed bugaboos of users in the article) but quite disappointed with search today. As a researcher, I have a reputation of being able to find anything, and usually I can although it may take a while. But I'd like to see search tools much more along the lines of the Star Trek computer: "Computer, show me an overlay grid of this, that and the other..." and whambo, the information appears correlated and easy to appreciate, so I can see where the enemy vessel is hiding.

To get there, obviously we need searching software that is much more intelligent, and capable of better contextual and syntactical analysis. But until we get something close to the Star Trek model, it is ridiculous to speak of the future of search being here now.

Have a Drink of Google Gulp!

Have you had a drink of the Kool-aid, er, Google Gulp, yet?