Is the Open Directory Project in Trouble?

by Harold Davis

Related link: http://dmoz.org



The ODP (Open Directory Project) is in trouble. A leading source of taxonomic information on the Web, the ODP relies on volunteer human editors to vet web sites for inclusion in the directory. Google, Yahoo, and others use ODP information (Google's use of the ODP as the basis for the Google Directory is explained in Chapter 7 of my Building Research Tools book). Inclusion in the ODP essentially means indexing status and traffic.

The ODP is run by Time-Warner's AOL's Netscape division in accordance with the Debian Social Contract. Part of the thought behind ODP is that "humans do it best": automated systems and understaffed editors at commercial search companies cannot keep up with constant change on the web.

Neither, as it turns out, can a volunteer system. Hundreds, or thousands, of categories are currently without editors. For those of us who want to get our sites into the ODP, there's an increasing feeling of lack of dynamism - and delay. When I told an editor that it could take 4-6 months to get into the ODP, and one needed to come back and keep trying if not listed the first time, the response was incredulity and bemusement. Bemusement that so static and clogged an institution should gate-keep the dynamic information model of the Web.

Now it turns out that worse may be afoot. Supposedly, ODP listings are being sold like a commodity - which (in a sense) they are - they beat buying an AdSense ad for effectiveness in getting indexed. And editors supposedly join the ODP to trash their competition.

For more on the troubles at ODP, see this article in SiteProNews, and the (incredibly entertaining if saddening) blog Corrupt Dmoz Editor by an ODP editor operating under the nom de blog "Ana Thema." Ms. Thema makes the ODP sound like the mob in posts like How to Bribe an DMOZ Editor. Some quotes from the blog, which may or may not be totally for real: "Links are a commodity. Links from DMOZ are a hot commodity. Everything in this world is a commodity: everything. If you disbelieve that someone would be so corrupt as to sell submissions into the ODP, then Dorothy, this is your wake up call."

And: "AOL/TimeWarner own DMOZ and they treat it like the dollar chasing b***h it really is. And you should, too. Sabotaging your competitors is not simply about deleting their sites from the categories, but a more subtle and ongoing process of destroying their relevance for important keyword phrases."

Of course, you may wish to read the post just to learn how to go about bribing the editors to get that all-important listing. Just kidding!

Go figure! I thought ODP was sluggish but idealistic. (I've been listed without paying a bribe, so the claim in the blog that a bribe is mandatory is false in at least some cases). It turns out it may be a cesspool. How bad is it? The Web community should find out, and take steps to get this vital institution back on the high road.

4 Comments

wbk
2005-05-30 20:00:55
becoming an editor is cake
when i applied to edit a relatively large regional category, i just wrote something along the lines of: "as a free software developer, the admirable goals of this project really appeal to my sensibilities." i was approved within 20 minutes.
rj_walker
2005-05-31 07:07:01
becoming an editor is cake
Yes I did the exact same thing as you. About once a month. Seven times. Seven automated rejections. I followed their advice to try a smaller category(< 50 sites). No go.


DMOZ (as an organization) have not responded to the need for change. It was clear to me that DMOZ was hurting their credibility and consequently their relevance.


DMOZ have created a great opportunity for others.
For example http://www.world66.com for travel.


I believe in the principle on which DMOZ was founded. But they need to improve the way they organize themselves.


R. Walker

Jack_Krupansky
2005-05-31 16:12:17
ODP definitely a real mess
I agree with your assessment. I have four legitimate web sites and five blogs, but you won't find any of them in the ODP or Google directory, despite my best efforts. You'll only find one of them in Yahoo's directory since I simply don't have the budget to pay for "express" service.


Maybe part of the problem is that the original interest in taxonomy has been diffused as a result of the general success of Google and its anti-taxonomy textmining philosophy, and the interest in manual tagging, folksonomies, etc. Face it, it's easy to place Technorati tags in blog posts that link to web pages, so why beat your head against the ODP "wall".


The real problem: ODP needs leadership, but it's run by a dispersed mob. Communities are great, when they work, but when they're dysfunctional, they really, really suck.


-- Jack Krupansky

kctipton
2005-05-31 18:58:34
ODP definitely a real mess
Your site's been listed in http://dmoz.org/Computers/Programming/Agents/ for about 3 months now. I am surprised you didn't know that.