Linux Documentation: What's There and What's Needed

by Andy Oram

The following is condensed from a talk I gave at LinuxWorld in
January 2004.




If we look at the types of documentation needed in the Linux
community, it leads to a general discussion about different types of
computer documentation and the learning needs of computer users.

The Linux Documentation Project



When one talks of Linux documentation, one should think first and
always of
the Linux Documentation Project.
Linux started to congeal and take off in 1992, and it was around the
same time that the LDP appeared, founded by Matt Welsh (who soon wrote
Running Linux for O'Reilly). The LDP is an impressive
organization that has editors, guidelines for reviewers, procedures
for updating documents, translators--in short, an it's an organization
that has tried to reproduce everything about conventional publishers,
but in an open and volunteer manner.



The LDP is a phenomenon we should all be following as a model for
documentation in an open source community. The quality of its output,
as one might expect, varies widely. At the high end, a few documents
outrank the information in most conventional, published books. At the
low end one finds mushy, vague descriptions of no interest to anybody,
sometimes written by people for whom English is not only not a first
language, but not a language.



The LDP is a fine achievement, but it's fragile. One would be overly
generous to say it runs on a shoestring budget. It basically has
nothing but a Web site and a few tremendously dedicated and inspired
volunteers who manage to keep the documentation flowing from multiple
contributors. They need an infrastructure, including the kinds of
legal protections for which such open source projects as Apache,
GNOME, and Eclipse set up foundations. People who care about
documentation should talk to the companies pouring money into Linux
and get them to set aside a little bit of those funds to put the LDP
on firm ground. Even though the LDP could be viewed in some sense as
competition to O'Reilly, I feel they are very important.

Three audiences



Let's look now at Linux documentation as a whole. I won't treat the
valuable but amorphous range of informal documentation in the form of
journal articles, newsgroups, and even chat rooms; it's enough in this
talk to stick to the conventional books that are understood when one
uses the term documentation.



The computer field has long divided documentation into three
categories based on audience--end users, system administrators, and
programmers--a division that applies well to Linux.

End users



There are some good beginner's books for end users that succeed in the
sense that they make Linux seem simple. If anything, I'm afraid they
oversimplify. Sometimes things on Linux just don't work right. A
configuration file has the wrong default setting, or you install a
program and it fails to put a menu item on your start-up menu, or
conversely, an item on your start-up menu fails because some file
association is incorrect. And people need help fixing these problems.



It's traditional to say that problems like these should be solved by
making the software systems themselves more robust and by planning to
make everything work together in advance. Just as when you buy a
Macintosh, you should be able to install Linux and have everything
just work. The end-user documentation should not have to deal with
incongruities.



Certainly, standards for installing software and making programs
interoperate correctly are valuable. But I fear the results of putting
too many constraints on programs. Something of the openness of systems
might be lost. In theory, one should be able to define standards that
make all software work together robustly without excluding innovative
developers. But in practice, I think developers on the fringe would
lose out if Linux became like a Macintosh. So let's allow some
flakiness around the edges and give end users documentation that help
them deal with it--more on this later.

System administrators



System administration documentation is the largest category in Linux,
and it addresses a lot of areas well. It's gratifying to see so many
books that provide background, take readers through complex processes
step by step, and, most significantly, allow them to backtrack and
trouble-shoot what they're doing. For instance, one regularly sees
books warn users to run ps to make sure a background daemon
is running before carrying out steps that assume the presence of that
daemon. That's a simple example of good, belt-and-suspenders system
administration, and a lot of authors know what to do.

Programmers



There is much less documentation for programmers on Linux, and the
books I do see on the shelves don't sell very well compared to the
other kinds of Linux books. I've spent a lot of time--since it's my
job to figure out where to put my company's resources--trying to
figure out why people in the Linux and open source areas don't buy
many books on programming, and I can merely cite a few hypotheses.



One possibility is that there just aren't many people writing programs
for Linux, KDE, GNOME, etc. If that's true, it's a big problem that's
going to bite Linux in the future. But it's a situation that could
also change rapidly. I don't think this is necessarily true, and in
any case it doesn't strike me as the major reason for poor sales of
programming books.



Another hypothesis I've considered is that, after all, Linux is
modeled after Unix and the desktops offer programming models similar
to other toolkits used on the X Window system, so there are lots of
people who have learned the basic programming techniques
already. Maybe all the Linux/X programmers are old hacks who don't
need new documentation. But when I look at the age of the people doing
the coding, I know that can't be the explanation.



Finally it occurred to me that the lack of interest in programming
books may be caused by the availability of other sources of
information. If programmers can get what they need elsewhere, they
won't buy many books.



And what do programmers always say is the best documentation? Source
code! So that may ultimately be the explanation. When so many open
source applications are out there for everyone to see--including the
efforts of the best coders in the industry--programmers may be finding
enough to get them going.

What the Linux field needs



So our field is doing pretty well, in terms of both formal and
informal documentation. But we still could do better. I'll give a bit
of history of computer documentation to show where we're still weak.

Reference manuals



The first computer documentation was awful three-ring-binder reference
listings filled with sentences such as, "The /x25 option invokes the
X.25 protocol on the line." Nearly all documentation (which, I'm told,
has its roots in military technical manuals) was like this from the
beginnings of the computer industry until the 1980s, when
nonprofessionals started to get their hands on computers. But
certainly, reference documentation is still important, and some of the
best-selling O'Reilly books (including Linux in a Nutshell)
fall in that category.

User-friendly documentation



In the 1980s a revolt broke out among technical writers, who insisted
on writing what they called user-friendly manuals. This led to sleek
and glossy books with lots of little icons and and tricks of layout
(facilitated by the simultaneous rise of desktop publishing) and
sentences such as "To print your document, pull down the File menu and
choose Print." Hmm, that sounds pretty much like the old X.25 example,
doesn't it? Often, under the friendly exterior, the user-friendly
documentation wasn't much better than the old stuff.



But this is being a bit unfair. The writers of user-friendly
documentation tried to be task-oriented. When done properly, the
documentation helped readers understand what their systems were
capable of doing, and even sometimes helped them decide on their
needs.



User-friendly documentation is represented today at its best in the
Missing Manual series. I say not simply because they're put out by
O'Reilly, but because sales figures show how wildly popular they
are. Missing Manuals are often at the top of the charts, particularly
the Mac OS X Missing Manual, which has kept its best-seller status for
years.

Model-building documentation



During the craze to do user-friendly documentation with lines such as
"To print your document, pull down the File menu and choose Print,"
few technical writers tried to think further or deeper about how to
educate users. But another experiment was tried during the 1980s, a
little-known movement called minimalist documentation.



The goal of minimalist documentation was not to convey facts--as was
the goal of all computer documentation up to that time--but to help
readers build the mental models that would help them solve problems
they encountered by themselves. The movement was called "minimalist"
because the documents were short and actually told the user as little
as possible. Typically, they'd show some menu or dialog box and
encourage the reader to play around with it herself. The psychologists
who invented this experimental documentation (I know of no commercial
examples) deliberately wanted to set users free and force them to take
responsibility for themselves.



Mental models help you figure out where to go to solve a problem. They
help you guess that a certain setting must be missing from a certain
file, or that a certain daemon is not running, or that a certain data
format is not being recognized by the recipient of the data.



It's very hard to form mental models. Readers certainly need
background information, but often they are not helped by simply having
information thrown at them. One must empower them to explore their
systems. Metaphors and exercises--fixtures of the Head First series at
O'Reilly, which are credited by many readers for helping them
understand complicated subjects that other books didn't explain--are
often critical. And that's where the Linux field, like many other
areas of computing, can do better.

The person who doesn't want to learn



I've been asked what to do for users who just don't want to
learn. Documentation is clearly of little value in this case. First of
all, not everybody should have to learn their systems in depth; just
like stoves or television sets, systems should provide basic
functionality in intuitive ways. But people often want to learn when
someone sits down with them and shows them the power of the
system. One needs to induce a "Wow, I could do that" feeling.



This is a general topic for the field of education. Lots of dollars
are spent trying to figure out how to teach the hard-to-teach
children. And like the computer users who don't want to learn, the
main hurdle to overcome with hard-to-teach children is motivation.
Because of socio-economic stresses or learning disabilities or other
underlying problems, these children can't take advantage of the
resources teachers offer because the children don't see a reason to.



I think the arts are a major motivator for learning. Show the children
exciting art and play wonderful music for them. Teach them to make
their own art and music. Along the way they'll come to realize that
they need mathematics to understand rhythm and harmony or
perspective. They need physics to understand musical timbre and
chemistry to understand how materials form artworks. They need to
study history to understand what led their favorite musicians and
artists to make the works they love.



So the desire to learn has to be nurtured individually in each person,
but there are certain experiences that have a high likelihood of
turning people into learners.


What general areas would you like to see more documentation for?


6 Comments

boud
2004-02-04 08:56:52
Why programmers don't by books on Linux programming

I can tell you why I at least haven't bought any books specifically on the topic of programming for Linux. The all-round books from publishers like Wrox' "Beginning Linux Programming" are really bad, giving almost no useful information (but then, so didn't "Programming with GNU tools"), and the more specific books like "Linux Application Development" or "Advanded Linux Programming" weren't all that impressive either, judging from a quick scan in a friend's copy. I didn't buy them...


But even if I haven't bought any books on programming for Linux specifically, I've bought a lot of books on programming in general: "Java, C++ & Python in a nutshell", "Python Cookbook", "Practical C++ Programming" (but that was a dud, too, at least, in the first edition), "Accelerated C++", more Java books, more Python books, books on computer graphics, on algorithms -- in fact, the only book with Linux in the title that I have, and I have been using Linux since 1993, is "Running Linux" which I bought in 1995, excited that there finally was a book on the Linux.


In the end, source code is seldom very Linux specific -- which means that the books I need aren't very Linux specific either. So, maybe you should count every other copy of "C++ in a Nutshell" as a Linux programming book sold...

btakita
2004-02-04 13:29:21
Gimmicks as a learning tool
Excellent article Andy.


It seems like to reach people who don't necessarily want to learn about a topic, you have to relate what you want to teach with what they want to learn.


Here is my
post about teaching to people who dont want to learn what you're teaching.

Jonathan Gennick
2004-02-04 14:19:03
Mental models
Good article Andy. I was struck by your comment on mental models, which is an area I've thought about too. Mental models are very important, in that they free you from having to look up every little thing. Having the wrong model in your mind of how something works can send you down the wrong path. I'm not sure I agree with the minimalist documentation approach that you described, but certainly a good mental model reduces the need to have every little thing explained in detail.


On a related note, your article reminds me how important it is to have a clear mental model in mind when you *design* a system. If designers don't have a clear model, than users won't even have the chance.

GerardM
2004-02-05 11:34:57
OSDL perhaps ?
In your article, you mention the need for at least some minimal support, hardware bandwith.. As the OSDL is about promoting Linux, one instrument is good documentation. When they include a great framework by sponsoring it, it will definetly enhance everything OSDL stands for.
Thanks,
Gerard
deltazed
2004-03-30 10:04:58
We don't need no stinking paper
Obviously NOT the message a publisher wants to hear. SORRY. IN 10 years of working with Linux systems, I've aquired a LARGE number of books. My collection of O'Reilly titles fills a 6' by 3' bookcase. They are used to impress visitors to my office. These days, when I am evaluating a 'solution', I am less concerned about the ability of a publisher to produce an outdated collection of pages. I look for an active support forum, IRC channels, ON-LINE docs, and then I test the responsiveness of the technical support folks aligned with the product. Don't get me wrong, I am a VORACIOUS reader, but even overnight delivery from Amazon can't help me get past the "I want the answer NOW...." I think many of the book publishers suffer the same 'denial' as the music distributors. The physical media is just not worth the cost/effort unless it is the ULTIMATE reference.
Of course, O'Reilly HAS figured it out and your subscription service to on-line copies of your library is on my list of THINGS THAT ARE RIGHT IN THE WORLD.
fraterm
2004-04-16 13:31:41
Why programmers don't by books on Linux programming
Wrox Publishing made the only linux specific programming book that was worth buying... Beginning Linux Programming.


All of the others were quite unimpressive. If publishers were to follow this model they would get it right. But Bouds point would be highly accurate.