Need someone to lead product management at your software company? I create software for people that create software and I'm looking for my next opportunity. Check out my resume and get in touch.

Broken aggregator

Freshness Warning
This blog post is over 21 years old. It's possible that the information you read below isn't current and the links no longer work.

I read the contents of several hundred sites each day through my news aggregator. I’ve switched aggregators a few times over the last few years, starting with a home-grown web-based aggregator, followed by Radio, Amphetadesk, then Radio again, back to Amphetadesk, and currently Aggie. If the site doesn’t have a news feed, I either create one through a small scraping tool I built, or don’t bother to read it.

All the aggregators I’ve tried have a serious flaw. They let feed developers break the aggregator display. A single unclosed <em> tag causes all the posts from all the sites from that point on to be italicized. That’s something that I can generally live with, but it’s possible for those unclosed tags to compound (or would that be to aggregate?) and make a mess of my display.

Today, one feed forgot to close an <em>, another forgot to close a <b> and another forgot to close a <small>. What I ended up with was tiny, bold, italicized text that’s next to impossible to read.

The problem could have been worse. If someone forgot to add the > to the end of a tag, the rest of the text on the page might be ignored, at least until a closing angle bracket occured.

But much worse, it’s possible for a feed developer to inject malicious code into their feed, and most aggregators would happily render the HTML to the browser. A bit of clever JavaScript in a feed that exploits a browser’s vulnerability to cross site scripting attacks could do quite a bit of damage.

I suggest that the various aggregator developers take steps to eliminate these problems. Check for unclosed tags and close them. Strip things like script tags from the feed before rendering them.

Meredith
January 18, 2003 3:10 AM

I'd love to hear more about the homegrown web-based aggregator you built. After getting frustrated with those written by others (including some you mentioned above), I'm trying to write my own using MT and mt-rssfeed.

Adam Kalsey
January 18, 2003 8:59 AM

It was built in ASP. All it really did was grabbed the XML files, parsed them and cached the result. The admin interface was quite primitive. You can see a cached copy of the aggregator output on the Wayback Machine at http://web.archive.org/web/20010803122838/http://kalsey.com/news/

Bill Kearney
January 21, 2003 2:57 PM

While it doesn't serve the immediate need, feeds listed on Syndic8 do get marked as bad when they produce invalid XML. You could have an aggregator double-check with Syndic8 (via XMLRPC) as to whether a feed is known to be working or not. That and you can always run it through the validator and send that URL to the feed author.

This discussion has been closed.

Recently Written

Mastery doesn’t come from perfect planning (Dec 21)
In a ceramics class, one group focused on a single perfect dish, while another made many with no quality focus. The result? A lesson in the value of practice over perfection.
The Dark Side of Input Metrics (Nov 27)
Using input metrics in the wrong way can cause unexpected behaviors, stifled creativity, and micromanagement.
Reframe How You Think About Users of your Internal Platform (Nov 13)
Changing from "Customers" to "Partners" will give you a better perspective on internal product development.
Measuring Feature success (Oct 17)
You're building features to solve problems. If you don't know what success looks like, how did you decide on that feature at all?
How I use OKRs (Oct 13)
A description of how I use OKRs to guide a team, written so I can send to future teams.
Build the whole product (Oct 6)
Your code is only part of the product
Input metrics lead to outcomes (Sep 1)
An easy to understand example of using input metrics to track progress toward an outcome.
Lagging Outcomes (Aug 22)
Long-term things often end up off a team's goals because they can't see how to define measurable outcomes for them. Here's how to solve that.

Older...

What I'm Reading

Contact

Adam Kalsey

+1 916 600 2497

Resume

Public Key

© 1999-2024 Adam Kalsey.