Knoppix 3.8 and UnionFS. Wow. Just Wow.

by Kyle Rankin

Related link: http://www.heise.de/english/newsticker/news/56987



Klaus has released the latest version of Knoppix, 3.8, to the crowd at CeBIT 2005. This version includes the normal round of updates including the 2.6.11 kernel by default, KDE 3.3.2, and Firefox and Thunderbird instead of Mozilla. The exciting news, however, is the addition of UnionFS. UnionFS stacks your Knoppix ramdisk on top of the read-only filesystem on the CD, the effect being that you can apt-get install, and otherwise modify all of the files on the system as though they were all writeable. Here I'll go over why I think this is going to change Knoppix in a major way.

One of the struggles with using Knoppix day-to-day as a portable distribution has always been the fact that the majority of the CD was read-only. You had a nice ramdisk sandbox that let you write within the /home directory, and tinker in /etc if you changed some of the symlinks. If you wanted persistence, there was a nice script that created a small filesystem on writeable media like a usb thumbdrive and used it to store your home directory.

The Problem



The problem, of course, is that you couldn't install new programs with this method. All of /usr is read-only. Over time various methods have been introduced to try to hack around this by redirecting binaries into the /home directory and changing various system paths. The live software installer that came with previous versions of Knoppix is one example, and klik is another. These worked okay, depending on the program, but in some cases relied on hacks such as using scripts to search and replace file paths within binaries.

The Solution



Enter UnionFS. With this system, basically every segment of the filesystem is read-write. UnionFS keeps track of which files you have modified (which are modified and stored within the ramdisk) and which are unmodified, all on the fly. When you access a file in the filesystem that has been modified, UnionFS points you to the copy in ramdisk instead of the copy on the CD-ROM. This means that if you, say, want to install the enlightenment window manager all you would need to do is:


knoppix@2[knoppix]$ sudo apt-get update

knoppix@2[knoppix]$ sudo apt-get install enlightenment


You can install programs just like with Debian. They all get installed to their regular locations, and if you check out the /ramdisk directory on the CD, you will see a mirror of the root directory structure (or at least the directories that have been modified) and can see all of the files that you have changed. You can even use your favorite graphical packaging program (if you have one) to install packages.

This also means that if you set up a persistent home directory, you can potentially back up all of these changes and restore them at boot. In fact, Knoppix 3.8 already has a modified persistent home script that does just this (although it's still a beta feature so expect to see bugs here and there). Knoppix can even detect when you have done this on a drive when it boots, and will let you choose what features to restore from this image (if any). If you let it time out, Knoppix does the safe thing and doesn't load any of the settings.

The latest persistence script also automates the process of setting up an AES256-encrypted home directory, so if you are concerned about the security of your persistent image you can enable this option, type in a passcode, and Knoppix does the rest.

I really think UnionFS is going to take Knoppix to a whole new level of ubiquitous computing. You can truly use the CD just like any other installed CD, and with the prices of usb thumbdrives continuing to drop, you can basically install anything you would like. Major changes, such as replacing KDE with Gnome, probably will still work better if you do a full remaster, but I expect all of the Knoppix-based live CDs (if they are smart) will incorporate this technology into future releases.

Can you tell that I'm excited about this? Like I said, currently it seems to be a bit buggy (on a few of my machines I seem to get some instability after I restore from an image), but I imagine that sort of thing will be ironed out once the "download version" of Knoppix 3.8 hits http://www.knoppix.org.


13 Comments

jdub
2005-03-14 17:32:46
Ubuntu Hoary LiveCD
Hey, did you know that the Ubuntu Hoary LiveCD provides the same functionality as UnionFS in the new Knoppix, only it uses standard, shipping components of the Linux kernel? :-)
AI9NL
2005-03-14 18:28:54
3.8 Remastering
A "bit torrent" of the CeBIT version is now downloading on my other machine. Any re-master tips peculiar to this version you'd care to pass along?


Maybe as an online addendum to your book?


Thanks


Harv

Saiyine
2005-03-15 08:53:08
:(
I thought the new knoppix 3.8 could save the changed files! Writing them in ram is a minor enhancement.
greenfly
2005-03-15 10:12:39
:(
As I said in the blog entry, if you set up a persistent home directory, you can save any changes from the ramdisk to a usb drive or the like.
lilsirecho
2005-03-19 08:52:26
Unionfs
Please advise the limitations that accompany the use of Unionfs.


Does Unionfs allow exceeding the limits of the CD of ~700MB?


If applied to a DVD of Knoppix, will the Unionfs extend the usefulness of the DVD format from the present limit of 2.1GB on disc to the full 4.4GB of storage capacity by allowing the loading thru Unionfs of an additional 2.3GB of compressed files(assuming all of Unionfs is compressed, or is it?).?? If it isn't compressed, can it be?


Correct answers to these queries should be helpful to all users, stupid questions or no.

lilsirecho
2005-03-19 08:58:54
Unionfs
What effects will the use of TORAM mode have on the use of Unionfs?


lilsirecho
2005-03-19 09:29:30
Unionfs
Should the user use TORAM mode and install programs via Unionfs, can the DVD+RW recorder be used to advantage in recording what has been added? Maybe even the CDRW recorder?
lilsirecho
2005-03-19 11:37:46
Unionfs
What effect does use of the UNIONFS have on the storage requirements for persistent home in view of the added installs with Unionfs?
lilsirecho
2005-03-19 11:40:00
Unionfs
What happens if the install is corrupted?
lilsirecho
2005-03-19 12:01:09
Unionfs
From some of the details, it seems that Unionfs uses the feature ..copy-on-write which is the COWLOOPfs.
ojsimpson
2005-03-20 08:19:25
Yawn
Damn Small Linux (DSL) livecd has been using a writable file system for almost a year now where you can apt-get stuff with little effort. If you have a decent swapfile or swap partition, you can apt-get almost anything even large size downloads.


I am happy that KNOPPIX is heading in this direction but the functionality is not groundbreaking for a livecd distro.

williamselden
2005-05-20 14:36:14
Persistant Home Directory
I would like to edit files on my harddrive and can't seem to get the permissons worked out to read/write.


If a persistant home directory is the answer, how is one set up?


I am using the Knoppix disc.


Bill Selden

angrykeyboarder
2005-10-10 05:43:14
Ubuntu Hoary LiveCD
Apparently he didn't.... ;-)