Need someone to lead product or development at your software company? I lead product and engineering teams and I'm looking for my next opportunity. Check out my resume and get in touch.

Autodiscovery of hierarchies through tagging

Freshness Warning
This blog post is over 14 years old. It's possible that the information you read below isn't current and the links no longer work.

There’s been a growing amount of thought recently on how to use the data from folksonomies to organize information in other ways. One blogger suggests data mining folksonomies to populate a hierarchical taxonomy. Through David Weinberger I find that Library Thing notes that tags standing alone don’t always provide as much context as a taxonomy.

The ordered structure of subject headings gives added meaning. History > Philosophy is very different from Philosophy > History—a distinction that isn’t necessarily apparent when searching history or philosophy separately as tags.

Expanding on this concept a bit, I’d note that with a detailed taxonomy it will be difficult to rely soly on a document’s tags to determine where within that taxonomy the document should go. Software > Design is a very different concept than Design > Software (the act of designing a piece of software as opposed to a piece of software used for design) and if your document simply includes the tags design and software how would the classification engine decide where it goes? With a larger tag set it should be possible to infer additional meaning. The additional tags could provide clues as to whether you’re talking about a software product or the process of creating software.

It’s possible for instance to look at the tags mustang, ford, horsepower, specs and determine that this is about the Ford Mustang automobile instead of a horse and that mustang, running, animal are tags that don’t describe a car. But building a taxonomy based on these tags will require that a person with knowledge about automobile brands build a dictionary of tags that map to nodes in their taxonomy. Once you have a corpus of tags that do and don’t place a document into your taxonomy you can start using machine learning—perhaps through statistical analysis—to learn what other tags would define a document as either a horse or a car.

While part of this problem can be solved by using sufficiently large and detailed training sets, but it’s going to be a lot of work. You can’t assume that training a learning algorithm where tags belong in one taxonomy will map well to other similar taxonomies.

Taxonomies are structured for a particular knowledge domain by someone with detailed understanding of that domain. An algotrithm isn’t able to understand nuances in how I want my content structured. It is unable to adjust for my personal preferences and biases. Tagyu tends to classify pages about podcasts as Entertainment but you might feel that everything podcast-related is better suited for the Technology category. With this sort of problem in a generic flat taxonomy, imagine how much more complicated it would be to place items into a detailed hierarchy with lots of similar nodes.

With Tagyu, I’m working on some solutions to this, and I’m excited to see what others are coming up with.

ram
November 29, 2007 10:24 PM

hi pretty interesting. we are also trying to work on the same problem. would definitely be interested to see what can be achieved. check us out at www.findnearyou.com . we have these hierarchical DBs and freetagging by users and we need to use these to divine user intent and deliver what they are searching for

This discussion has been closed.

Recently Written

How to advance your Product Market Fit KPI (Oct 21)
Finding the gaps in your product that will unlock the next round of growth.
Developer Relations as Developer Success (Oct 19)
Outreach, marketing, and developer evangelism are a part of Developer Relations. But the companies that are most successful with developers spend most of their time on something else.
Developer Experience Principle 6: Easy to Maintain (Oct 17)
Keeping your product Easy to Maintain will improve the lives of your team and your customers. It will help keep your docs up to date. Your SDKs and APIs will be released in sync. Your tooling and overall experience will shine.
Developer Experience Principle 5: Easy to Trust (Oct 9)
A developer building part of their business on your product needs to believe that you're going to do the right thing for them and their customers.
Developer Experience Principle 4: Easy to Get Help (Oct 8)
The faster you can unblock a stuck developer, the better their experience will be.
Developer Experience Principle 3: Easy to Build (Oct 5)
A product makes it Easy to Build by focusing on productivity for developers building real-world applications.
How to understand your product and your market (Sep 30)
A customer development question you can ask to find out who your product is best for and why they'll love it.
Developer Experience Principle 2: Easy to Use (Sep 28)
Making it Easy to Use means letting the developer do everything without involving you.

Older...

What I'm Reading

Contact

Adam Kalsey

+1 916 600 2497

Resume

Public Key

© 1999-2020 Adam Kalsey.