The state of just about everything at OScon

by Andy Oram

Related link:

The second day of the

2003 O'Reilly Open Source Convention

featured a couple dozen tutorials and no less than six keynotes on the
progress of major free software projects: Perl, Python, PHP, MySQL,
Apache, and Linux. I ran down two batteries on my laptop before the
end of the day, and the number of attendees present slid downward
inexorably as well.

I'll say a bit of each project in this weblog, along with summaries of
four tutorials that I attended in part today and a note about Apple at
the end.


Here are some impressions I got during a long day of running from one
tutorial to another.

Who Watches the Watchers: Developing "Translucent" Applications that
Protect Sensitive Data

Delivered by MySQL AB employee Zak Greant, this talk was about
security rather than databases, and as much about policy as
technology. It attracted about 35 attendees.

Zak began with an observation that has been popping up more and more
in recent years: that the major source of sensitive data leaks comes
from trusted people within the system, and that one therefore can not
guard against leaks just using traditional authentication checks
within applications or operating systems.

The solutions he presented included:

Reducing the amount of information stored.

A common example is to verify passwords by storing hashes that can
be used to validate passwords, but not used to reconstruct them.

Break data into chunks that have little value on their own.

For instance, instead of storing all the information about a person in
tables of one database, encrypt it with different keys and store it in
different databases.

Send partners only the data they need.

Reduce the chance that organizations will leak data because their
goals are different from yours, or simply because they don't
appreciate how sensitive it is.

Zak went on to talk about the process for assessing and tightening up
areas of vulnerability.

Building Data Warehouses with MySQL

This talk was a bit more about technology and less about process than
the previous talk on "translucent" applications, but still more about
modeling than coding. I have to admit being somewhat out of my depth
here. The terminology and concerns of data warehousing (assuming that
presenter John Ashenfelter was reflecting standard practice) are
notably different from what I'm used to hearing about in databases. In
addition, Ashenfelter focused on such practical issues as extracting
data from one database to another, cutting down on dump time, and
checking for corruption, The talk was sold out quite early, with 50

Managing Web Services with PHP

Sterling Hughes, whom I heard presenting advanced PHP topics
yesterday, graciously and expertly stepped in at the last minute to
take over this tutorial from a presenter who fell ill.

Sterling's enthusiasm today was driven by two extensions he wrote:
cURL for retrieving URLs, and SimpleXML for parsing XML. Faster for
PHP than a SAX or DOM parser, SimpleXML puts the whole XML file into a
hash that you can access with hash dereferences.

For communications, Sterling recommended XML-RPC in preference to
SOAP, because SOAP support in PHP is not yet mature and XML-RPC is
also more efficient.

Tricks of the [Perl] Wizards

How can I possibly describe this presentation by Mark-Jason Dominus,
one of the most accomplished of Perl's developers and trainers? As he
has done for many years since the Open Source conference started as
the Perl conference, Dominus grabbed the audience by the ears and
whirled us through a virtuoso display of code that takes obscure Perl
features to their limits, or pretty close. It's no use trying to
explain the tricks he showed--I was present for discussions of tied
variables, filters, and autoloading--but the effect was like that of a
circus where an impossibly high pyramid of acrobats keeps growing as
one after another keeps emerging from the bleachers and jumping on

I did wonder about the utility of these techniques (which Dominus
occasionally doubted himself), and this certainly is not a talk to be
attended by Perl's detractors, who already have enough fodder for
their claims that it is tortuous and inscrutable. I have a feeling
that few among the large audience for the talk (about 75 or 80 people)
would use such techniques in their own code. To some extent, the
example code helps to give users a mental model of some of Perl's
basic features. It also helps explain how the minority of developers
who really master these techniques can produce some of the
jaw-dropping extensions that do prove useful.


The long day was followed by quite a long evening, where we heard six
keynotes interspersed with White Camel awards (for Perl), the Frank
Willison award (for Python), Active awards (for several languages),
and pictures of developers' children or favorite mascots.


Larry Wall delivered the seventh of his notorious State of the Onion
talks. Like his others, in my view, this keynote was ultimately about
the miracle of Perl's very existence. He was not shy about pointing
out that he and other developers are making major sacrifices to
continue with the project. He also entertained us with what he called
a universal architecture diagram, which was particularly ironic
because he also invoked the concept of post-modernism, which (as I
understand it) claims that nothing can be universal.

As for hard content, I will leave that up to many other bloggers and
journalists to report.


Inventor Guido van Rossum presented the state of Python. He said
downloads have doubled in two years. I noticed that Python is mature
enough to have backward compatibility issues (an unwillingness to
eliminate old features when offering better replacements). He finished
by announcing that, "Extreme Programming works extremely well with


Shane Caraveo estimated that PHP use has grown 30% in past year, and
cited Netcraft as reporting that it is used on 31% of all domains.
PEAR 1.0 has over 100 modules and extensions. Currently the team is
working on improved OS X support and improved object orientation
through formal enhancements such as exceptions, constructors, and


David Axmark, with some additions by Monty Widenius, summarized the
history and current projects of MySQL AB. Several practical
extensions to SQL include one that makes it easier for Web servers to
show subsets of results as one commonly sees in Web searches. GIS
support has been added. The company is growing quite a bit and
benefiting from investments from SAP and elsewhere

Axmark won two rounds of applause for announcing that MySQL AB is
officially against software patents.


Greg Stein introduced the structure of the Apache Foundation and
offered a brief history. The foundation's goal is to allow innovators
to do what they want--not to impose deadline and market pressures--but
to provide a long-term framework and community that ensures someone
will pick up a task when earlier contributors leave. Apache has
spawned innumerable projects and is generating enormous amounts of new


This presentation by Ted Ts'o--his first attendance at an
OScon--focused heavily on the technical. He pointed out at the end
that the most exciting work on Linux is happening in user space, where
languages and desktop advances are increasing its usefulness. Some of
the details in the 2.5 and 2.6 kernels include:

  • Major thread-related improvements that are particularly good news for
    Java users, including fast userspace mutexes (a kind of locking
    available to applications).

  • IPSEC support.

  • Support for uClinux, which allows embedded system development on
    low-cost chips that lack memory management units.

  • Easier and more intuitive kernel builds.

  • Support for bigger filesystems (16 terabytes on 32-bit systems and 8
    exabytes--count 'em!-- on 64-bit systems).

  • The sysfs filesystem, which will finally remove ancient Unix
    limits on device numbers.

Many 2.5/2.6 changes were put to the test recently when Linus Torvalds
announced his move to the Open Source Development Lab and the OSDL
site was slashdotted. Thanks to improved efficiency in the kernel
(especially the virtual memory management system), the system running
their Web site stayed up through the onslaught.

A correction on
Sunday's weblog

One of the flightier paragraphs in that article, as someone pointed
out to me, could be seen as disparaging toward Apple. Now that my jet
lag has been replaced with conventional conference-engendered
exhaustion, I realize that the criticism is accurate. But read
carefully, the posting should be seen as not disparaging at all. My
concern was to recognize the opinions of free software advocates who
oppose the intrusion of any proprietary software into the free
software space. This is a hotly ranging debate (whose importance to a
large set of software developers is often underestimated) and requires
some mention.

But there is no doubt that Apple computers play a critical role at
this conference. In fact, it's loaded with Apple lovers. I haven't
tried to do an official count, but the silver laptops are gleaming out
all over the place.