P2P lives on in a worldwide distributed filesystem

by Andy Oram

Related link:

Back in 2001, if not earlier, companies were marketing filesystems
that offered far greater robustness, scalability, and (perhaps)
performance than traditional distributed filesystems such as NFS.
OceanStore is
probably the best known of these systems, although it is a research
project rather than a product.

Now Internet2 is offering such a filesystem across all the dozens of
universities that are part of its network. There's even a Windows
client! That means Logistical Networking could become the first mass
phenomenon in the next generation of distributed filesystems.

The idea of these filesystems is that each file is broken into pieces
when stored, and each piece is sent to multiple systems. Encryption
protects the data from snooping, and digests protect it from
corruption. Never again will you have to stop work because your file
server is down; there will always be another server with your
data. Downloads experience extra overhead because the locations of the
parts have to be resolved, the data could be far away, and the
encryption introduces extra work--but breaking large files into pieces
can help compensate.

There are so many theoretical advantages for these systems over NFS
and CIFS that I am convinced they'll dominate filesystem storage in a
few years. Internet2 has taken an excellent step in that direction. It
may be the most important illustration of the peer-to-peer concept,
far more useful than anonymous distributed systems such as KaZaa.