The Googleplex Blog

More on ads in syndication

A little while back I complained about Google's new beta program adding AdSense ads into syndication feeds. I got some flack for this. I also shared a PHP regular expression that will filter out the Google ads if you happen to be rolling your own reader (or using mine).

The issue continues to be controversial. I think Dave Taylor gets it exactly right in a piece subtitled "Corrupting the idea of information syndication." I'm with Dave.

Here's a somewhat opposing viewpoint in an industry wrap-up from Webpronews: basically, ads in syndication are coming, so get over it! (And you people run AdSense on your blogs, anyhow!)

One issue that gets lost sight of (at least in my opinion) is why Google gets cut so much slack with these kinds of things. Can you imagine the hue and cry if Microsoft began offering to place ads in RSS syndication? Think about it.

Code for Stripping Google Ads from RSS

In a previous entry in the Googleplex Blog and on O'Reilly I mentioned that Google was beta testing contextual AdSense style ads in RSS and Atom feeds. One company immediately placing these ads in its RSS feeds is Weblogs, Inc., whose Prez Jason Calacanis is annoyed with "folks who take our feeds and republish them without premission to their websites... these folks wrap our content in ads..."

At least in the current implementation, it's easy to remove these ads in a feed display application because they are implemented as HTML tables, which really have no business in syndication feeds. Here's a PHP function that uses regular expression pattern matching to remove all table rows from each item description in a feed:

<?php ... function strip_ad ($in_text){ $gunk = '/<tr>(.|\n)*<\/tr>/'; return preg_replace($gunk, '', $in_text); } ... echo strip_ad($item[ description ]) . "</p>" ; ... ?>

Here's the Weblogs Unofficial Google Weblog RSS feed (note the embedded Google ads in item descriptions), and the same feed displayed using this code with ads stripped on Syndication Viewer.

An RSS blast from the past and an interesting blog feed

I've added a feed which is a blast from the past to Feedly.com. In this feed, each RSS item is a day from the famous 17th Century diary of Londoner Samuel Pepys. The diary entries are very thoroughly cross-linked, so you can quite quickly get up to speed with what is (was?) going on. Here's the feed displayed as HTML on Feedly.com.

I've also added the feed from Dave Taylor's Intuitive Life Business Blog to the Syndication Viewer. Here's the feed displayed as HTML on Syndication Viewer. Dave's a nice guy and seems to share a lot of my interests. A recent item on his blog asked "How important are publishers?" Dave's blog item includes a reference, made by Wiley publisher Joe Wikert, to an item in this blog ("The Times are Changing in Computer Book Publishing").

New at Feedly.com

The featured feed of the week at Feedly.com is the best of Craigs List. This is a fun feed. The links are to posts on the list that run a pretty big gamut, but there's plenty of drama and soap opera, and, yes, romance.

A little less obviously, I've changed the architecture under the hood at Feedly.com. The site itself is now generated on the fly from its own RSS feed.

Related links:

Feedly site
RSS Syndication feed for Feedly (XML)

Roundup at the RSS Corral

The electronic investor in today's Barron's has a roundup of RSS syndication feed applications that I think mostly gets it right.

The piece mentions the following syndication readers: RSS-Reader as a lightweight standalone RSS reader (it lets you paste feeds in or choose from a list, and is free), FeedDemon ($29.95), Newsgator Outlook Edition ($29, integrates with Outlook), and Pluck (free, either standalone or an Internet Explorer plug-in). Also noted, the Firefox browser and Thunderbird email client feature built-in (albeit a bit clunky) RSS reading.

These are all reasonable software choices for reading syndication feeds (whether RSS or Atom). The piece also mentions the issue of finding feeds, suggests several directories of feeds, and notes My Yahoo provides access to a claimed 150,000 feeds as well as a Web viewer for the feeds.

The conclusion: "RSS is still a bit too, well, pushy for our tastes. Do you really need another set of interruptions added to your day...Many investment and news sites already offer more targeted e-mail alerts for must-have news."

I agree that we don't need any more interruptions in our lives, but the Electronic Trader misses the utility of syndication for delivering certain kinds of information. My Syndication Viewer and Feedly are designed to get around the "too pushy" problem of syndication viewers. No special software is required, just a normal Web browser. You visit the sites when you want. You don't have to search through massive feed directories (or search the Web for feeds), because the best feeds (and only the best feeds) are presented. (Feedly presents six top feeds on a rotating basis, Syndication Viewer provides a moderately extensive categorized list of feeds.)

You can bookmark the feed or feeds you are personally interested in for future reference. For example, the SEC EDGAR feed of insider trading provides a great mechansim for getting insider trading information on a timely basis with both a granularity and an ease of scanning that is hard to duplicate elsewhere.

Featured Feeds

Our new site, www.feedly.com, features six editor choice best of the best syndication feeds that are fun, interesting, and useful. The feeds will be rotated regularly as I find new and exciting feeds.

Be the first on the block to know!

I've added a Business and Investing category to the Syndication Viewer including feeds from the New York Times, Wall Street Journal, MarketWatch, and SEC EDGAR filings. I have also added an analysis and update feed from The Economist (in the News category) and Alexa hot search (a feed that tracks hot search terms) in the Internet category.

Here are the details and links for the Business and Investing Feeds:
Aggregation of business and investing feeds
Selection of business and investing feeds
New York Times Business (Breaking business news)
Wall Street Journal (market news)
Wall Street Journal (US business news)
Dow Jones MarketWatch (Stocks to watch)
Smart Money (Stock screener)
Market Wire (Investment opinions)
Forbes.com (Investment news and strategies from Forbes)
EDGAR (SEC EDGAR insider trading filings)
Editorial note: I'm really looking forward to using the EDGAR feed. Syndication seems a really useful and timely way to have this information available. This kind of information can be used to make money trading if you get it early, and syndicating directly from the SEC seems like a great way to be the "first on the block" to know.

Validate those feeds!

It's telling to me that a high proportion (meaning 10-20%) of the feeds I want to add to the Syndication Viewer don't validate. This means that something is technically wrong with these feeds, usually something easy to correct. The consequences in the Syndication Viewer range from fairly benign (for example, the dates associated with items are all wrong) to fatal: the feed won't display.

The moral I draw from this is that RSS and atom feeds are still way under-subscribed. Otherwise, someone else would have reported the problem to the creator of the feed.

Feed syndicators, please validate your feeds and fix any problems that are pointed out in the validation process. My Submit your feed page provides a good feed validation mechanism. It is in your interest to make sure your feed is working.

I've added three new feeds:
eBay General Announcements
Mortgage News Daily
Quicken Loans Mortgage News

Regarding the eBay feed, I want to figure out a good way to generate a syndication feed from ad-hoc eBay queries. If you are puzzled about why I've included two mortgage-related feeds, see my Words for Sale blog entry.

Amazon Feeds added

I've added Amazon feeds which update the bestseller lists for "Computers and Technology," "Literature and Fiction," and "Science Fiction and Fantasy" to the Syndication Viewer.

About today, SlashDot, Wired, and Washington politics

Four new feeds added to the Syndication Viewer:
About today (top daily consumer advice and tips from About.com)
Slashdot (news for Nerds, stuff that matters)
Wired News (how technology affects our lives)
Washington Post (feed oriented to politics)

Feed Topic Links

I've provided the following feed topic links for your convenience:
Google
RSS
Blogging Technology
Research and the Internet
Lifestyle
News
These links open the aggregated feeds that I've collected (because I think they are useful, interesting and/or fun) in these topic areas in your browser. Once you've opened the page combining the feeds in your browser, you can bookmark it (add it to your Favorites). Next time you select the page, it will show updated entries for all the feeds. You can also make your own selection of feeds on the RSS Viewer page.

Please contact me with suggestions for additional RSS or Atom feeds to add.

More RSS Aggregation!

My free RSS Web Viewer is new and improved! This is the third version (in as many days, so what does that tell you about obsessiveness?)

Feature improvements include a bigger number of preset feeds, the ability to instantly view any feed (by clicking the HTML link next to the feed), and the ability to open the RSS or Atom syndication feed itself (by clicking the Feed link next to each feed in the preset list). You can also bookmark an aggregated collection of feeds that you would like to view together again in the future. For example, here's a link that opens five feeds related to Google. Another example: this link opens three lifestyle related feeds (pregnancy, babies and parenting).

Notes: The ability to aggregate feeds in a single bookmarked link is particularly handy because the content of the linked page is dynamic. It changes as the content of the feeds changes. Managable addresses were created for these links using the TinyUrl service.

Thanks for suggestions and help with this project to Peter Scott, Peter Cooper, and Phyllis Davis.

Now I have to get back to my "real" work! But more features and enhancements when I get the chance. Please contact me with suggestions for feeds to include, thoughts about application enhancements, and (of course) flowers and chocolate...

New Feature in RSS Web Viewer

I've done a major upgrade on my RSS Web Viewer. The RSS Web Viewer lets you view any RSS syndication from the Web. You can enter the address of the feed, or choose from a preset compendium of feeds. The new feature allows you to multiselect preset feeds. This means that you can view all the information on the feeds about a particular area of interest (for example, Google or RSS) without having to open feeds one after the other.

About RSS

Here are some questions and answers about RSS from Googleplexblog.com's RSS FAQ.

What is RSS?
RSS - Really Simple Syndication - defines a specification that is used for distributing "feeds" of information. RSS itself is a form of XML (eXtensible Markup Language), and complies with the XML 1.0 specification.

The feed defined by an RSS XML file contains structured information about content items, such as the information title, an HTML link to the full information content, and an excerpt from the content.

Where can I read the RSS specifications?
There is no single RSS specification. Instead, there are different versions and variations of RSS. The most important RSS versions are RSS 1.0 and RSS 2.0, but within these major versions there are minor variations. And even feeds that comply with the RSS 1.0 or RSS 2.0 specification may name specific fields differently from one another (for example, Date or pubDate). You can learn more about the various RSS specifications at Harvard Law RSS Specification information site.

What software do I need to view an RSS feed?
The RSS XML file used for syndication is pretty structurally simple, so you can actually look at one manually and get a pretty good idea about an RSS feed (what it is called, what the first few content items are, and so on). This means that writing software that can "read" and RSS file is an easy and straightforward proposition. It's easy enough that many people and companies have done so, and many people use this kind of software. According to a
recent survey, more than 6 million Americans get news and information using RSS software. Software that you use to display RSS feeds is often referred to as an "RSS aggregator" -- "aggregator" because it is intended to combine more than one RSS syndication feed. You can download software that lets you view RSS feeds in your Web browser, as part of an email
program, or standalone on your desktop. You can also
view RSS feeds using any Web browser with no special software on this site. There's a very complete list of RSS software on the O'Reilly XML.com site. If you are interested
in adding an RSS reader to Internet Explorer, Pluck is a good choice. I enjoy reading RSS feeds along with my email. If this appeals to you, you may be interested to know that the
Mozilla Thunderbird email program is also an RSS reader. Enewsbar is a standalone Windows desktop program that presents RSS feed items on a scrolling ticker. Obviously, there's a great deal of variety in software that can be used to read RSS feeds. You should experiment and find what suits you best!

How can I find RSS feeds that interest me?
That's a very good question. There are tons of great RSS feeds out there. There are certain to be some that interest you. But finding
them is not always easy. It is likely that the RSS software that you use to read syndicated feeds will offer you some suggestions about RSS feeds that are available. You can also find RSS feeds using Feedster, with is an
RSS search engine. Tara Calishain of ResearchBuzz has some interesting ideas about how you might find RSS feeds automatically. I've added some of my favorite RSS feeds as presets to the
Googleplex RSS Viewer page. Here's the link for the RSS feed for
the Googleplex Blog.

Finally, keep your eyes open while you search the Web. More and more RSS feeds are cropping up. If you see a link to a feed while you are browsing pages
that interest you (many RSS feed links are marked with small orange buttons), add it to your RSS viewer before you lose track of it.

How can I add a feed I see on the
Googleplex RSS Viewer page to my own RSS reader?
Here's how:

1. Open the Googleplex RSS Viewer
   page


2. Select an RSS feed that interests you from the list, for example, BBC World News


3. Click Display Feed. The RSS feed items will be displayed on the page


4. At the top of the feed, you'll see the words "Syndication Feed" followed by the feed's title in a hyperlink. Click the hyperlink. This will
   open the home page for the site that sponsors the RSS feed in your browser


5. Look for the RSS feed or feeds offered by the site (you may have to search the site for them)


6. Click the RSS link to open the XML feed in your browser. You can read the full address for the feed in the Address Bar
   of your browser.


7. Copy and paste the address of the feed from your browser's address bar into the RSS Feed managemant system of your RSS reader software

Why do I see XML when I click the link to a feed?
An RSS syndication feed is an XML file. You are seeing the contents of this file when you open it in your Web browser. To view the RSS feed normally,
it is necessary to open the link in software designed for displaying RSS feeds (see "What software do I need to view an RSS feed?").

How can I display an RSS feed on my Web page?
See the Googleplex Blog about page for information about this.

How can create my own RSS feed?
Almost any modern publication or content management system will either optionally or automatically create an RSS syndication feed based on the content you supply. In particular, it's natural to combine blogging with syndication, because by creating blog entires you are also potentially creating syndication feed items. So most software used for creating blogs also publishes RSS syndication feeds of the blogs, almost as a by product. MoveableType is probably the best of these blog creation tools, and it provides a template-driven automatic RSS feed creation facility.

If you look through the list of RSS software on the O'Reilly XML.com site you'll find many tools for standalone creation and management of RSS feeds.

Why should I care about RSS?
What am I? Your mother? Why should I tell you what to care about?
Here are some of the reasons to care about RSS:

• Lots of people use it: according to a
  recent survey, more than 6 million Americans get news and information using RSS software. If you don't syndicate with RSS, you are missing an audience and a market.


• RSS is a great way to find timely information in a manageable form: the information is "pushed" to you, but you can control when and how you view it


• RSS is one of those apparently simple technologies with powerful implications: the combination of RSS and blogging will change the way information is published and read