Solving feed overload

Freshness Warning
This article is over 7 years old. It's possible that the information you read below isn't current.

Ilya Grigorik has some thoughts on how to solve the information overload that often comes with feed reading. If you subscribe to hundreds of feeds, how do you find the interesting stuff without reading everything? How do you separate the signal from the noise.

Keyword filtering is one common approach. But there’s some significant problems.

I use keyword monitoring to let me know on new product announcements and to track public mentions of certain people or products. It does work in that instance. However, the act of me specifying the keyword presupposes a priori knowledge of the topic I would like to track. The reason why I cast my net so wide across so many sources is exactly because I don’t know what I’m looking for - I just want to make sure that once something significant happens, I’ll know about it! Not only does keyword filtering fail here, but the act of choosing the keywords is extremely hard in itself.

To some extent, feed readers appear to try and justify their existence (and boost their usage) by recommending feeds to watch. (Note that we’re sometimes guilty of this at Feed Crier, too) Suggesting additional feeds to read based on what you’re already reading is only contributing to the problem. Ilya rightly takes we feed reader developers to task, asking us why our suggestions are so self-serving, when we should be helping our users.

One approach that Ilya rejects is allowing our friends to decide what I read and what I see. He suggests that doing so places a burden on them and only works if we’re using the same system anyway. I’ve found that this is not the case, but only when looked at from a slightly different angle.

Before we descended into the echo chamber, blogs were a log of interesting places on the web. That’s the root of the term even: web log. The early blogs were human filters for the vast web. People chronicled their daily journey through the intarwebs, leaving breadcrumbs for others to follow. Almost three years ago I suggested that enterprise blogging could perform the same function for intranets by allowing employee blogs to act as human filters for the most important things.

Human filtering can work. I started subscribing to Roland Tanglao’s blog several years ago because he was an effective filter for high-output bloggers like Dave Winer and Robert Scoble. He separated the signal from the noise—if something interesting was said on a vast number of blogs, I could count on Roland to reblog it for me. His blog as since morphed into his personal interests, but there are other bloggers that do the same things.

Ilya suggests two approaches, building smarter feed readers and an interestingness measure that comes from the community.

Shifting the burden on the RSS reader could yield some improvements. Clustering, correlation, prior reading patterns can all be taken into account when information is sifted through the filters.

He recognizes the problem with a smart reader—that it takes a fair amount of training to start doing it’s job. A reader needs to watch your habits and learn what you find interesting, something that takes time. My friends at Touchstone are working on solving the problem in part by passively watching what you do on your computer and using that information for training. This passive approach can ease the training burden. Training in this case doesn’t require you to do anything different than what you are already doing. Their attention engine looks at what you’re already paying attention to and helps you find other things that you’d find interesting.

A community interestingness measure is what memetrackers are working on. Tailrank, Techmeme, and others (disclosure: I’m an advisor to Tailrank) are using the linking patterns of blogs to determine what the community at large finds interesting. Tailrank takes this a step farther by helping you find the interesting things in the feeds you’re already reading. Their My Tail service allows you to tell them what blogs you’re reading and get a customized page and feed of the most interesting items in those blogs. And the clustering behavior of memetrackers can help readers find related topics and other points of view.

I have a nagging concern, however, that these systems designed to reduce information overload could reduce serendipitous findings as well. If all you see is things that you already find interesting, how do you expand your horizons? By reading a vast and eclectic collection of feeds, I’ve been exposed to new and interesting ideas—ideas that I wouldn’t have even known I’d have liked. Will attempts to reduce information overload contribute to creating a monoculture?

Dave Kaufman - Techlife
December 22, 2006 10:31 AM

2 things... 1 - I am still waiting for the FC update I asked for when it opened for biz....the "youtube bar" - most popular, most subscribed, most viewed, newest feeds, staff picks, etc. You get that. 2 - Based on your article, the second half...I think a TiVo suggested viewing/reading is what they seems to be proposing. Makes sense, but only if they have an API. No offense to them, but I want my reader (FC of course!) to see what I am reading, see what others like me are reading, see the keywords from the posts of all of us and suggest single posts to view. Loyalty is a bitch I guess. And yes the new tagline for FeedCrier should be, "You had me at Hello."

Chris Saad
December 22, 2006 5:24 PM

Thanks for the mention my friend :) Just to clarify - Touchstone can actually, if you give it permission, index your hard disk (including documents, IM chats, browser history etc) to create an instant and ongoing snapshot of your interests - no handshake period. And because we scan a broader range of content than just your interactions with OUR application, we are less prone to making the wrong assumptions (we have all heard about the Tivo assuming it's user was gay because he watched Queer Eye for a Straight Guy once). Also I agree that serendipity starts to fade, but at Touchstone we focus on one problem and one problem only - escalating alerts. This means you get alerted about important stuff to your life in proportion to its importance. If it’s important, then it can rout it to FeedCirer for IM Alerting. It's a PERSONAL Attention Manager. If you want serendipity then we suggest you ALSO use a Popularity Engine as well like Digg or Techmeme etc.

Yoz
December 25, 2006 12:50 AM

I'm also a big believer in human filtering; en masse, I think it's a great self-balancing system. Some people are great for finding stuff that (a) I find interesting and (b) I haven't seen. I usually end up finding these people through others' "via" links. While putting more smarts into feed readers could help, the things I'd most like to see are the dumber-but-faster tools like Reblog being built into the more popular feed readers.

Your comments:

Text only, no HTML. URLs will automatically be converted to links. Your email address is required, but it will not be displayed on the site.

Name:

Not your company or your SEO link. Comments without a real name will be deleted as spam.

Email: (not displayed)

If you don't feel comfortable giving me your real email address, don't expect me to feel comfortable publishing your comment.

Website (optional):

Follow me on Twitter

Best Of

Recently Read

Get More

Subscribe | Archives

7

Recently

Aereo: a parable (Jun 26)
Here's why people who understand both technology and copyright law think the Supreme Court's Aereo decision defies common sense.
The birth of cubicle hell (Apr 29)
Where do cubicles come from?
Double Opt-In helps you (Apr 2)
A simple reason why double opt-in is a good thing for marketers.
Make the most of opportunities (Mar 5)
Often, part of success is being in the right place at the right time. What will you do with the opportunity when it happens?
Email as a model for future mobile phone service (Feb 10)
What if your phone wasn't tied to your phone number?
MySQL inadequecies (Feb 9)
This guy seriously hates MySQL and helps you understand how to use it better.
invisible Fence (Mar 22)
The New York Times has a paywall now. Sorta. If you don't choose to ignore it.
Black status icon for Chrometa (Mar 17)
Replacing the status icon of Chrometa

Subscribe to this site's feed.

Elsewhere

Tropo
Voice and communications platforms, including Tropo and Phono. Work.
SacStarts
The Sacramento technology startup community.
Pinewood Freak
Pinewood Derby tips and tricks

Contact

Adam Kalsey

Mobile: 916.600.2497

Email: adam AT kalsey.com

AIM or Skype: akalsey

Resume

PGP Key

©1999-2014 Adam Kalsey.
Content management by Movable Type.