Solving feed overload

Freshness Warning
This blog post is over 15 years old. It's possible that the information you read below isn't current and the links no longer work.

Ilya Grigorik has some thoughts on how to solve the information overload that often comes with feed reading. If you subscribe to hundreds of feeds, how do you find the interesting stuff without reading everything? How do you separate the signal from the noise.

Keyword filtering is one common approach. But there’s some significant problems.

I use keyword monitoring to let me know on new product announcements and to track public mentions of certain people or products. It does work in that instance. However, the act of me specifying the keyword presupposes a priori knowledge of the topic I would like to track. The reason why I cast my net so wide across so many sources is exactly because I don’t know what I’m looking for - I just want to make sure that once something significant happens, I’ll know about it! Not only does keyword filtering fail here, but the act of choosing the keywords is extremely hard in itself.

To some extent, feed readers appear to try and justify their existence (and boost their usage) by recommending feeds to watch. (Note that we’re sometimes guilty of this at Feed Crier, too) Suggesting additional feeds to read based on what you’re already reading is only contributing to the problem. Ilya rightly takes we feed reader developers to task, asking us why our suggestions are so self-serving, when we should be helping our users.

One approach that Ilya rejects is allowing our friends to decide what I read and what I see. He suggests that doing so places a burden on them and only works if we’re using the same system anyway. I’ve found that this is not the case, but only when looked at from a slightly different angle.

Before we descended into the echo chamber, blogs were a log of interesting places on the web. That’s the root of the term even: web log. The early blogs were human filters for the vast web. People chronicled their daily journey through the intarwebs, leaving breadcrumbs for others to follow. Almost three years ago I suggested that enterprise blogging could perform the same function for intranets by allowing employee blogs to act as human filters for the most important things.

Human filtering can work. I started subscribing to Roland Tanglao’s blog several years ago because he was an effective filter for high-output bloggers like Dave Winer and Robert Scoble. He separated the signal from the noise—if something interesting was said on a vast number of blogs, I could count on Roland to reblog it for me. His blog as since morphed into his personal interests, but there are other bloggers that do the same things.

Ilya suggests two approaches, building smarter feed readers and an interestingness measure that comes from the community.

Shifting the burden on the RSS reader could yield some improvements. Clustering, correlation, prior reading patterns can all be taken into account when information is sifted through the filters.

He recognizes the problem with a smart reader—that it takes a fair amount of training to start doing it’s job. A reader needs to watch your habits and learn what you find interesting, something that takes time. My friends at Touchstone are working on solving the problem in part by passively watching what you do on your computer and using that information for training. This passive approach can ease the training burden. Training in this case doesn’t require you to do anything different than what you are already doing. Their attention engine looks at what you’re already paying attention to and helps you find other things that you’d find interesting.

A community interestingness measure is what memetrackers are working on. Tailrank, Techmeme, and others (disclosure: I’m an advisor to Tailrank) are using the linking patterns of blogs to determine what the community at large finds interesting. Tailrank takes this a step farther by helping you find the interesting things in the feeds you’re already reading. Their My Tail service allows you to tell them what blogs you’re reading and get a customized page and feed of the most interesting items in those blogs. And the clustering behavior of memetrackers can help readers find related topics and other points of view.

I have a nagging concern, however, that these systems designed to reduce information overload could reduce serendipitous findings as well. If all you see is things that you already find interesting, how do you expand your horizons? By reading a vast and eclectic collection of feeds, I’ve been exposed to new and interesting ideas—ideas that I wouldn’t have even known I’d have liked. Will attempts to reduce information overload contribute to creating a monoculture?

Dave Kaufman - Techlife
December 22, 2006 10:31 AM

2 things... 1 - I am still waiting for the FC update I asked for when it opened for biz....the "youtube bar" - most popular, most subscribed, most viewed, newest feeds, staff picks, etc. You get that. 2 - Based on your article, the second half...I think a TiVo suggested viewing/reading is what they seems to be proposing. Makes sense, but only if they have an API. No offense to them, but I want my reader (FC of course!) to see what I am reading, see what others like me are reading, see the keywords from the posts of all of us and suggest single posts to view. Loyalty is a bitch I guess. And yes the new tagline for FeedCrier should be, "You had me at Hello."

Chris Saad
December 22, 2006 5:24 PM

Thanks for the mention my friend :) Just to clarify - Touchstone can actually, if you give it permission, index your hard disk (including documents, IM chats, browser history etc) to create an instant and ongoing snapshot of your interests - no handshake period. And because we scan a broader range of content than just your interactions with OUR application, we are less prone to making the wrong assumptions (we have all heard about the Tivo assuming it's user was gay because he watched Queer Eye for a Straight Guy once). Also I agree that serendipity starts to fade, but at Touchstone we focus on one problem and one problem only - escalating alerts. This means you get alerted about important stuff to your life in proportion to its importance. If it’s important, then it can rout it to FeedCirer for IM Alerting. It's a PERSONAL Attention Manager. If you want serendipity then we suggest you ALSO use a Popularity Engine as well like Digg or Techmeme etc.

Yoz
December 25, 2006 12:50 AM

I'm also a big believer in human filtering; en masse, I think it's a great self-balancing system. Some people are great for finding stuff that (a) I find interesting and (b) I haven't seen. I usually end up finding these people through others' "via" links. While putting more smarts into feed readers could help, the things I'd most like to see are the dumber-but-faster tools like Reblog being built into the more popular feed readers.

This discussion has been closed.

Recently Written

The Trap of The Sales-Led Product (Dec 10)
It’s not a winning way to build a product company.
The Hidden Cost of Custom Customer Features (Dec 7)
One-off features will cost you more than you think and make your customers unhappy.
Domain expertise in Product Management (Nov 16)
When you're hiring software product managers, hire for product management skills. Looking for domain experts will reduce the pool of people you can hire and might just be worse for your product.
Strategy Means Saying No (Oct 27)
An oft-overlooked aspect of strategy is to define what you are not doing. There are lots of adjacent problems you can attack. Strategy means defining which ones you will ignore.
Understanding vision, strategy, and execution (Oct 24)
Vision is what you're trying to do. Strategy is broad strokes on how you'll get there. Execution is the tasks you complete to complete the strategy.
How to advance your Product Market Fit KPI (Oct 21)
Finding the gaps in your product that will unlock the next round of growth.
Developer Relations as Developer Success (Oct 19)
Outreach, marketing, and developer evangelism are a part of Developer Relations. But the companies that are most successful with developers spend most of their time on something else.
Developer Experience Principle 6: Easy to Maintain (Oct 17)
Keeping your product Easy to Maintain will improve the lives of your team and your customers. It will help keep your docs up to date. Your SDKs and APIs will be released in sync. Your tooling and overall experience will shine.

Older...

What I'm Reading

Contact

Adam Kalsey

+1 916 600 2497

Resume

Public Key

© 1999-2022 Adam Kalsey.