oreilly.comSafari Books Online.Conferences.
Articles Radar Books  

Peer-to-Peer Makes the Internet Interesting Again
Pages: 1, 2, 3

Making galaxies out of dark matter

Clay Shirky, a partner at The Accelerator Group and a writer for Business 2.0, who came up with many of the catchy epigrams of the day, pointed out that peer-to-peer exploits the "dark matter of the

"With peer-to-peer the computer is no longer just a life-support system for your browser."
-- Clay Shirky

Internet." Just as interstellar space contains huge amounts of inert dust that we can't track or measure -- but that may be forming galaxies all the time -- personal computers offer enormous amounts of cycles consumed by busy waits, along with disk space waiting to be filled and communication lines that lie idle in between keystrokes. Peer-to-peer creates constellations of cooperating users out of these wasted resources. As explained by Dave Stutz, a software architect at Microsoft, these three key resources -- CPU power, storage, and communications capacity -- are the prerequisites for each new leap in computer applications. Or, to quote again from Shirky, "With peer-to-peer the computer is no longer just a life-support system for your browser."

Performance concerns many observers. Does peer-to-peer cause unnecessary processing and Internet traffic? Napster is not valid evidence for these concerns, because exchanging large files would stress networks regardless of the technology used to administer the transaction. Many expect peer-to-peer to actually conserve bandwidth when used appropriately.

Intel hopes that they can avoid the crippling effect of users downloading training videos by letting files relocate near groups of systems that request them.

One such observer is Bob Knighten, peer-to-peer evangelist for Intel. His company hopes that they can avoid the crippling effect of users downloading training videos by letting files relocate near groups of systems that request them. Comparable to caching by Internet hubs or multicasting, this technique is distinguished by its use of regular, underutilized computers instead of specialized, expensive file servers. The model of shifting content to where it's wanted is also the basis of Freenet.

But there's no doubt that peer-to-peer will challenge the architecture of current Internet services. Nelson Minar, cofounder of a distributed computing startup named Popular Power, says that peer-to-peer redefines the assumptions behind asymmetric service (like ADSL and cable modems). Michael Tiemann, CTO of Red Hat, adopts a positive attitude and goes so far as to say, "Peer-to-peer may be the critical enabling technology that makes broadband possible."

Flexibility offers a key opportunity

Traditional search services may offer a variety of advanced options, but you can't tell the service how to organize its data. Napster is child's play for finding pop songs, but its rigid classification service doesn't accommodate classical music well.

Peer-to-peer raises the possibility for people interested in a topic to create their own language for talking about it. While different communities may all share an underlying infrastructure, like Jabber's chat service or Gnutella file sharing, the structure of the users' data can emerge directly from the users.

Metadata, which describes each file and the elements within it, holds the key to self-organization. XML is a good foundation -- but only a foundation, because it just offers a syntax. Building on the XML foundation, schemas hold some promise for structuring both content and users' reactions to the content. One slogan we considered was, "Publish my taste, not just my music files."

Schemas can be handed down by standards committees, but they can also be thrown out on the Net by individuals or communities for widespread consideration. Each community will settle on the ones it likes. We called this principle "schema agnosticism."

While we all paid homage to the glories of metadata, we remained skeptical that people would understand the need for it and take the trouble to contribute information. Rael Dornfest, creator of the RSS-based Meerkat service for the O'Reilly Network, said that a little metadata goes a long way, adding logarithmically to the value and uses of the data it describes.

Any service or tool that provides either automated or manual access to reading and writing metadata offers an opening for an intrepid explorer to map the data and thus create an innovative service for other users. (Invisible Worlds is promoting a generalized approach to such APIs through the Simple Exchange Protocol, currently an Internet Draft.) This is a good reason to ask services to expose their data schemas and publish open APIs to their services. It's also a reason to deplore attempts to shut off innovation through bans on deep linking, restrictions on the amount of data one user can retrieve, or forcing developers to resort to "screen-scraping" and other fragile data-retrieval tricks.

The message

Peer-to-peer is so new that we have trouble attaching categorical statements to it. Yet it is also the oldest model in the world of communications. Telephones are peer-to-peer, as are Fidonet and the old-style UUCP implementation of Usenet. IP routing, the basis of the Internet, is peer-to-peer, even now when the largest access points raise themselves above the rest.

Up until the past decade, every Internet-connected system hosted both servers and clients. Aside from dial-up users, the second-class status of today's PC browser set didn't exist. One of our messages, therefore, is:

Peer-to-peer is fundamental to the architecture of the Internet.

Another element of our message is that peer-to-peer brings people together; it builds communities in a way that eludes all the portals that so earnestly try to build them from the top down. Jabber, by providing chatters with an easy way to categorize their content, hopes to help them organize themselves. Weblogs, Wiki, and Meerkat let people follow and comment on the goings-on of other people who interest them. Even a passive sharing of resources, like Popular Power or SETI@Home, makes people feel they're part of something big -- and they like the feeling, according to Minar.

When people collaborate, each has the opportunity -- though not the requirement -- of being a producer as well as a consumer. Those who produce may be relatively few: For instance, a recent study found that only 2% of Gnutella users contribute content, and even on Usenet News the ratio of posters to total readers is only about 7%. But that is enough to keep the system afloat. The important thing is to give everybody the opportunity. Thus our slogans:

Peer-to-peer is the end of the read-only Web.

Peer-to-peer allows you to participate in the Internet again.

Peer-to-peer: steering the Internet away from TV.

Since each peer represents a person (and in fact, a person can appear in many different guises on one or more peer-to-peer systems), these systems can lead to emerging, self-organizing communities.

Some peer-to-peer systems deal in fungible resources, such as the use of idle computing power by SETI@Home or Popular Power. In these systems, resources are valuable because they're interchangeable. But in many systems, where the goal is to share files or metadata, the diversity of the peers is what makes the system valuable. Resources like disk space may start out fungible but then develop differences as users add unique content.

Pages: 1, 2, 3

Next Pagearrow

P2P Weblogs

Richard Koman Richard Koman's Weblog
Supreme Court Decides Unanimously Against Grokster
Updating as we go. Supremes have ruled 9-0 in favor of the studios in MGM v Grokster. But does the decision have wider import? Is it a death knell for tech? It's starting to look like the answer is no. (Jun 27, 2005)

> More from O'Reilly Developer Weblogs

More Weblogs
FolderShare remote computer search: better privacy than Google Desktop? [Sid Steward]

Data Condoms: Solutions for Private, Remote Search Indexes [Sid Steward]

Behold! Google the darknet/p2p search engine! [Sid Steward]

Open Source & The Fallacy Of Composition [Spencer Critchley]