XHTML Services?

Freshness Warning
This article is over 15 years old. It's possible that the information you read below isn't current.

Jon Udell waxes nostalgic about the good old days of screen scraping HTML in order to build the first generation of Web services. That’s great and I’ve built my share of screen scraping applications as well. But then Udell goes on to propose that companies should abandon modern Web services technologies in favor of screen scrapes helped along by well-formed XHTML.

Udell’s reasoning is that Web services through SOAP is too complicated. “But if I’d had to register for an API key and locate WSDL documentation for each of the three services whose results I compared, I probably wouldn’t have bothered,” he says. His entire argument is based on his experiences with the Google API and their specific SOAP implementation.

Google requires that anyone using their API register for and use an API key — an identifying token that lets Google track the usage of their API down to a specific user or application. Google requires it, but the SOAP protocol does not. Most SOAP services don’t have any sort of key and if you were building a tool for an intranet, you probably wouldn’t need or want such a scheme. Not only does Udell miss that point, but he also forgets that SOAP isn’t the only Web services technology.

Udell says that a primary threat to your intranet is disuse. If people find it too difficult to create and use information on the intranet, they won’t bother. That’s true; if you create onerous processes that content creators must follow, they’ll find ways around them, publishing their information in ways that you don’t expect. But Udell’s assertion that building data access through Web services will make it too difficult for people to use your data is preposterous. Screen scraping is more difficult and more apt to fail than using stable, published APIs. And with REST, the APIs are just as easy to access as any other Web document.

As an example, let’s use product data for my new camera. What’s easier — scraping product data from Amazon’s Web page or getting it in XML format from their REST interface? For each method I have a unique URL that I request to get the data. There aren’t any complicated steps to follow for either system. But the HTML version, even if it were well-formed XHTML, would be significantly harder to retrieve meaningful data from. And changes to the display of the information would often mean changes to the structure of the HTML, necessitating further changes to my screen scraping application. Amazon does require a developer’s token (an API key, essentially), but again, that’s only so they can control usage. There’s no reason at at all that a REST system like this couldn’t be built without it.

But doesn’t creating a REST interface mean more work for the content producers? Probably not. Presumably your corporate intranet is using some sort of content management system. Otherwise there’d be no way to enforce this XHTML-only rule. Furthermore, that content management system probably stores the content in a database somewhere separate from the presentation of said content. All you need to do is build one REST interface that retrieves the required content from that database and presents it as a pre-determined XML document instead of an HTML document. The content producers could go along creating content as they always have, blissfully unaware that they are also populating a Web service.

Udell’s XHTML scraping suggestion has significant risks as well. Remember that making the process of content creation difficult will ensure that people find other ways to create content — ways that you don’t control. But in advocating screen scraping, Udell says, “it’s true that creating XHTML pages requires more discipline than hacking out HTML, and it may incur some retraining costs.” Not only are you going to make it difficult for people to build systems that automatically consume information, but you also propose making it more difficult to create it?

People will flock to things that are easy. RSS took off because it was easy to create and easy to consume. Sure, it would be possible to create screen scraping applications that would take any well-formed XHTML content source and pull that content into a newsreader. But it’s much easier for everyone concerned to create a simple, easy-to-understand format that contains all of the information in logical chunks and just run with it.

Your comments:

Text only, no HTML. URLs will automatically be converted to links. Your email address is required, but it will not be displayed on the site.


Not your company or your SEO link. Comments without a real name will be deleted as spam.

Email: (not displayed)

If you don't feel comfortable giving me your real email address, don't expect me to feel comfortable publishing your comment.

Website (optional):

Follow me on Twitter

Best Of

  • How not to apply for a job Applying for a job isn't that hard, but it does take some minimal effort and common sense.
  • Movie marketing on a budget Mark Cuban's looking for more cost effective ways to market movies.
  • California State Fair The California State Fair lets you buy tickets in advance from their Web site. That's good. But the site is a horror house of usability problems.
  • Customer reference questions. Sample questions to ask customer references when choosing a software vendor.
  • Comment Spam Manifesto Spammers are hereby put on notice. Your comments are not welcome. If the purpose behind your comment is to advertise yourself, your Web site, or a product that you are affiliated with, that comment is spam and will not be tolerated. We will hit you where it hurts by attacking your source of income.
  • More of the best »

Recently Read

Get More

Subscribe | Archives


Assumptions and project planning (Feb 18)
When your assumptions change, it's reasonable that your project plans and needs change as well. But too many managers are afraid to go back and re-work a plan that they've already agreed to.
Feature voting is harmful to your product (Feb 7)
There's a lot of problems with using feature voting to drive your product.
Encouraging 1:1s from other managers in your organization (Jan 4)
If you’re managing other managers, encourage them to hold their own 1:1s. It’s such an important tool for managing and leading that everyone needs to be holding them.
One on One Meetings - a collection of posts about 1:1s (Jan 2)
A collection of all my writing on 1:1s
Are 1:1s confidential? (Jan 2)
Is the discussion that occurs in a 1:1 confidential, even if no agreed in the meeting to keep it so?
Skip-level 1:1s are your hidden superpower (Jan 1)
Holding 1:1s with peers and with people far below you on the reporting chain will open your eyes up to what’s really going on in your business.
Do you need a 1:1 if you’re regularly communicating with your team? (Dec 28)
You’re simply not having deep meaningful conversation about the process of work in hallway conversations or in your chat apps.
What agenda items should a manager bring to a 1:1? (Dec 23)
At least 80% of a 1:1 agenda should be driven by your report, but if you also to use this time to work on things with them, then you’ll have better meetings.

Subscribe to this site's feed.


Adam Kalsey

Mobile: 916.600.2497

Email: adam AT kalsey.com

Twitter, etc: akalsey



©1999-2019 Adam Kalsey.