O'Reilly Network    
 Published on O'Reilly Network (http://www.oreillynet.com/)
 See this if you're having trouble printing code examples

OSCON 2005: Architecting Freedom

by Daniel H. Steinberg

Nick Gall began his keynote address at O'Reilly's Open Source Conference (OSCON) 2005 by reminding the audience of the four freedoms detailed in the Free Software Definition. The two freedoms that Gall stressed were the freedom to adapt a program to your needs and the freedom to improve a program and release these improvements back to the community. "It's great to be able to run a program," he said, "and it's kind of neat to be able to share them and copy them. But the really amazing thing about open source as a community is the power and freedom you have to change the software to make it fit what you want it to do."

In his fifteen minute morning address, Gall pointed to familiar examples from the internet as successes and abstracted lessons from these long-term successes. In particular, he said that the key to the evolvability of these internet applications is the freedom to interoperate, extend, and compose.

Examples of Failures and Successes

Gall asked how you would architect a system for freedom. He said that too much time is spent on ensuring a free or open legal architecture and that we wouldn't really know what will and won't hold up until these licenses are in the hands of litigators. As a second leg, we have considered different models for a free community architecture, including benevolent dictator and grass roots. Gall argued that not enough time has been spent trying to understand what is required for a free system architecture. "How do you modularize a system to enable the freedom to change in a sustainable way? It's easy to hack a system once to do what you want. Then the next person that comes along can't hack it as much because you've screwed things up. You do that ten times and now it's frozen." The system architecture must be innovated on, changed, and enhanced in ways that are both sustainable and decentralized. This bigger idea is based on subsidiary freedoms to interoperate, extend, compose, scale, and link.

Related Reading

Open Source for the Enterprise
Managing Risks, Reaping Rewards
By Dan Woods, Gautam Guliani

In many of the systems developed over the past three and a half decades, either the user can innovate or the vendor can innovate, but not both. There is no framework for decentralized innovation. There tends to be a half-life of five years where systems go from easy-to-change to hard-to-change. Most thirty-year-old software systems that you point to are legacy difficult-to-modify systems. Supporting decentralized innovation requires that we rethink system architecture.

For all of our failures, Gall pointed to the internet for examples of successes. He argued that the "internetwork" architecture has sustained this freedom to change. "IP: the same bit patterns are still streaming over the global network that streamed over thirty years ago. There's a lot more stuffed inside them, but it's the same basic protocols." The internet has been hacked over this long time period and is still thriving. Gall asked what it is about the internet that allows people to add and modify protocols and just hack.

Consider the following four architectures: IP, email, the web, and intermodal containerized shipping. This last one isn't an example of what we think of when we open our laptops to access the internet, but Gall asserted that global shipping is an example of an internet. These each exemplify and support decentralized innovation. If you think back to how you used email twenty years ago, it may seem vastly different than the way in which you use email today. Gall explained that all of the innovations that we now use without thinking are just extensions to the original Internet Message Format.

The Spanning Layer

David Clark coined the phrase the spanning layer in his paper "Interoperation, Open Interfaces, and Protocol Architecture." Clark had said that there is a special form of abstraction that gives you an hourglass shape, which enables extensibility and innovation in how you apply the protocols as well as in how you overlay existing technology with the same protocols.

Gall took Clark's model and generalized it to describe an internetwork architecture and to highlight some of the key principles that make it successful. In particular, he stressed the wide-at-the-top, wide-at-the-bottom shape that you can apply to almost any architecture you see. The backbone of the architecture is interoperability. So, for example, your architecture shouldn't be tied to any platform architecture or language such as J2EE, .Net, PHP, Python, or Perl. Another key is composability, so that you can mix these different subarchitectures on the fly over the internet using intermediaries. In designing protocols, they should be as generic as possible. The wide bottom of the hourglass represents the notion of being federated, so we can overlay a wide range of existing implementations at the endpoints and at the network level to minimize the disruptiveness of moving forward.

The narrow waist of the hourglass represents the simplicity that is required. Gall explained that you get this simplicity by focusing on the three key standards: identifiers, format, and protocol. The identifier might be an address, reference, or name. Similarly, the format might refer to a document, message, or packet, and the protocol might describe the interaction, behavior, or request and reply. When you look at examples from the internet, you can see the power in simplicity. Email, for example, uses an @ address as the identifier, RFC 2822 as the format, and SMTP as the protocol. The web uses URLs as identifiers, HTML/MIME as the format, and HTTP as the protocol. IP itself uses an IP address as the identifier, an IP packet as the format, and IP protocol as the protocol.

Intermodal containerized shipping also fits this model. It uses a UPC-barcode-inspired identifier, a 20-ton packet as the format, and container port protocol as the protocol. Gall explained that containerized shipping was a revolution in logistics. It has enabled interoperation of the world's transportation networks. Note that it doesn't replace anything that existed. Its power is that it overlays the train network, the truck network, and the ship network. A steel container (a packet) can be sent around the world without needing to be opened up.

Web 2.0

Gall said that there are three essential changes to the web that make up what we are beginning to call Web 2.0. Two of them fix compromises made with Web 1.0. Tim Berners-Lee always intended for the browser to natively be read/write. The problem was that the other early implementors wanted to get their demos working quickly and so they left off the write capability. It has taken us ten years to put it back in. One of the standards important to Web 2.0 is how we write to the web.

The second feature of Web 2.0 fixes a problem with interlinking. Hypertext only supports forward links. You can click on a link and go somewhere. But much of the interesting questions have to do with backlinking. Innovations around social software and web search engines are retrofits for backlinks. We can build more interesting networks if we can understand what is linking to a particular page.

Another feature of Web 2.0 is the freedom to scale in the downward direction. We've already scaled upwards to take the web global; Gall suggested that we engage in increasing the granularity. Can we scale it down so that the web works at an interprocess or intraprocess level? IP is already used for system board level buses. "In this [Web 2.0] world," Gall summarized, "links are King, content is Queen, and code is only the Prince, but all three must be free and open."


When you architect for freedom, you are, by definition, enabling unintended consequences. There are the cool and interesting extensions that we've seen with mash-ups, RSS, and podcasting. There are also the perverse effects such as viruses, spam, adware, and phishing. Gall said the idea is not to close it all down out of fear, but to leave it open and try through community, licence, reputation, and trust to edge things back to the serendipity end of the spectrum. He ended his keynote by saying it is not enough to grant users the right to change your software; you must design your system so it is easy or even fun to change.

Daniel H. Steinberg is the editor for the new series of Mac Developer titles for the Pragmatic Programmers. He writes feature articles for Apple's ADC web site and is a regular contributor to Mac Devcenter. He has presented at Apple's Worldwide Developer Conference, MacWorld, MacHack and other Mac developer conferences.

Return to the O'Reilly Network

Copyright © 2009 O'Reilly Media, Inc.