The Long Tail

by Harold Davis

Reader beware: this is a tale of buzzwords, so if buzzwords bug you, you might want to pass. In fact, it is a long tale of two buzzwords. They are the best of buzzwords, and the worst of buzzwords. And so it begins...

The buzzword, or phrase, that’s my primary topic is “The Long Tail.” “The Long Tail” is a meme—meaning a phrase used to denote a topic of general community discussion. According to various definitions, a meme is (1) an idea that can replicate and evolve, and (2) a basic unit of cultural information subject to mutation, crossover, and adaptation. The use of the term “meme” is credited to evolutionary biologist Richard Dawkins (he used it in his 1976 book The Selfish Gene). Dawkins is the author of many fine books besides The Selfish Gene, including notably The Blind Watchmaker: Why the Evidence of Evolution Reveals a Universe without Design (1986).

“Meme” is, of course, a meme itself, and as such has become a meta-meme. The genetic metaphor is interesting, but from a common sense viewpoint my first, simplest definition (that a meme is a buzzword that is the focus of community discussion) isn’t far off. But I digress. Digression is one of the problems with memes.

To digress further, I like the phrase “the long tail.” It’s evocative. It makes me think of good luck dragons, the greatest story ever told, fur hats, beavers, and particularly Mr. and Mrs. Beaver in C.S. Lewis’s The Lion, the Witch, and the Wardrobe. If you can’t follow this train of associations, don’t worry: it’s my personal weirdness. I also have a racier association, but let’s not even go there.

To statisticians, economists, and econometricians “the long tail” has been a fairly arcane bit of professional slang meaning the long downward slope of many distribution curves, such as the Pareto distribution named after the Italian economist, Marxist, and social theorist Vilfredo Pareto.

Pareto illustrated his distribution curve with his “80-20 rule” regarding wealth in society: 20% of the people own 80% of the wealth. The long tail is (as seen on a graph) the lower and lower probability of a given person having much wealth.

Pareto distribution curves have been found, with the long tail intact, in a wide variety of things including the frequency of words in a text, file size distribution of TCP/IP traffic (few large files, many smaller ones), and the value of oil reserves in oil fields (a few very valuable fields, many less valuable ones), and so on.

Incidentally, I’ve focused on Pareto distribution here because I am so fond of Pareto the historical character who was not only a brilliant thinker, but also stood up for social justice, was an aristocrat (though he disdained the use of his title) who married for love, and a crack practitioner of the arts of dueling. But Pareto distributions are not unique in having long tails. A variety of other probability distributions have long tails, including, for example, exponential distribution.

The concept is pretty intuitively simple: the long tail refers to the long downward slope as probabilities lessen over time. Most distributed phenomena have long tails. For example, a hot product may sell zillions of units for several months. Several years later, it is still selling a few units per month on the long downward slope of its unit sales distribution tail.

Here’s where my tale moves from the long tail (note lower case usage) to the meme “The Long Tail.” The initial capital letters The Long Tail meme was coined by Chris Anderson in a 2004 Wired Magazine article.

As Anderson originally used the phrase, The Long Tail refers to a new reality of the Web in which inventory and distribution costs are low. His article mostly referred to media: “Forget squeezing millions from a few megahits at the top of the charts. The future of entertainment is in the millions of niche markets at the shallow end of the bitstream.”

The implication is that low volume “products,” which sit somewhere down the long tail’s slope, can sell profitably to niche markets. Without The Long Tail, to be successful a product needs high-volume appeal. According to Anderson, and he’s probably right about this, The Long Tail accounts for the success of a variety of Internet business models from Amazon and Netflix (low inventory and distribution costs in stocking even low volume items) to eBay (the ultimate aggregation of niche markets) and beyond.

Anderson is now working on The Long Tail the book (to be published by Hyperion in 2006), and presents his ideas on The Long Tail the Website (“a public diary on the way to a book”).

I think that Anderson and some others are stretching things a bit to apply The Long Tail as a general business principle of our changing times, and I’d hate to see Anderson’s book end up as another vapid business chicken soul for the pocket book.

But what’s quite interesting to me is that the concept of The Long Tail is being applied (I think quite usefully) to the subject of information delivery on the Web. No longer do we have information dissemination via prime time television (the short head). Instead, information is being broken down into thousands of niches, blogs, and feeds (the long tail).

People can reassemble this stuff to find what they need in their own way and on their own time using mechanisms like search, RSS aggregation, and so on. As Eric Schmidt, Google’s CEO, put it at a recent Google shareholder meeting, Google does a good job of serving mid-size businesses who advertise. But Google is looking hard at serving The Long Tail (Schmidt’s words)—individuals and small businesses for which self-help tools like AdWords are ideal. (Google is also working on the Fortune 500 short head 80-20 rule side of things, but that’s another story.)

As The Long Tail meme goes around the Web community it cross-pollinates with another meme, Web 2.0 (for example, see this New Media Musings blog entry). Web 2.0 is shorthand for a set of attitudes, practices, technologies, and design disciplines. Stay tuned to this Long Tail channel for my thoughts on the interaction of Web 2.0 and The Long Tail.