Building a Web Cluster with FreeSBIE
Pages: 1, 2
World - BuildWorld - Create a New World
This step runs the
2.buildworld.sh script. It requires that
you've installed the
src-all package (FreeBSD sources for
everything). I suggest that you perform a full upgrade for the whole system
through CVSup. It can save you a lot of time and nerves instead of spending
hours figuring out why something is not compiling. After the world building has
finished, you can look at the results in the default
/usr/obj/usr/src directory. In our case it takes up about 419
Mbytes of space.
Install - InstallWorld - Install New World
This option runs the
3.installworld.sh script to install the
freshly built world. This step will take less time than the build process. The
/usr/local/livefs will then resemble:
-rw-r--r-- 2 root wheel 802 23 Feb 19:41 .cshrc -rw-r--r-- 2 root wheel 251 23 Feb 19:41 .profile -r--r--r-- 1 root wheel 4735 23 Feb 19:41 COPYRIGHT drwxr-xr-x 2 root wheel 1024 23 Feb 19:42 bin drwxr-xr-x 3 root wheel 512 23 Feb 19:43 boot drwxr-xr-x 2 root wheel 512 23 Feb 19:43 bootstrap drwxr-xr-x 2 root wheel 512 23 Feb 17:44 cdrom drwxr-xr-x 16 root wheel 2048 23 Feb 19:41 etc drwxr-xr-x 3 root wheel 18432 23 Feb 19:41 dev drwxr-xr-x 2 root wheel 512 23 Feb 17:44 home drwxr-xr-x 2 root wheel 512 23 Feb 17:44 mfs drwxr-xr-x 37 root wheel 1024 23 Feb 17:44 mnt drwxr-xr-x 2 root wheel 512 23 Feb 19:41 modules dr-xr-xr-x 2 root wheel 512 23 Feb 19:41 proc drwxr-xr-x 2 root wheel 512 23 Feb 19:41 root drwxr-xr-x 2 root wheel 2048 23 Feb 19:42 sbin drwxr-xr-x 5 root wheel 1024 23 Feb 19:43 stand lrwxr-xr-x 1 root wheel 11 23 Feb 19:41 sys -> usr/src/sys drwxrwxrwt 2 root wheel 512 23 Feb 19:41 tmp drwxr-xr-x 13 root wheel 512 23 Feb 19:41 usr drwxr-xr-x 20 root wheel 512 23 Feb 19:41 var
Now our LiveFS file system is 131 Mbytes in size.
Kernel - Build & Install Kernel - Create and Install Kernel
This option runs the
4.kernel.sh script to compile and install
a kernel to our LiveFS file system. The default FreeSBIE kernel configuration
/usr/local/share/freesbie/FREESBIE. It's based on the
GENERIC kernel configuration file with a lot of added options, so
you should customize this file. If you're not familiar with the kernel
configuration procedure, back up the original file as
The default configuration produces a kernel of about 5 Mbytes in size, which is definitely too much. After removing support for SCSI, RAID, missing network cards, and ISDN, my kernel size decreased to 2.2 Mbytes. You can do better.
I advise you to do a kernel compile and install using the
/usr/local/share/freesbie/kernel.sh script, but not the visual
dialog interface. If any errors appear during the compilation or
installation it's hard to see because
dialog clears all debug
information too fast.
After you've finished this step, copy the kernel to the
/usr/local/livefs kernel directory.
FIXIT - PatchFiles - Copy Cool and Useful Files
This option runs
5.patchfiles.sh, which adds a lot of different
scripts to the LiveFS file system, especially in the
Patching files in /etc directory... Making vnodes in /dev... Copying cool scripts (from sysutils/livecd) cp: /usr/local/share/freesbie/files/livecd/scripts/lang is a directory (not copied). /usr/local/share/freesbie/files/livecd/scripts/create_vnodes.sh -> /usr/local/livefs/scripts/create_vnodes.sh /usr/local/share/freesbie/files/livecd/scripts/lang/vnodes_En -> /usr/local/livefs/scripts/lang/vnodes_En Copying FloppyBackup and cardDetect scripts
Packages - Packages - Choose Packages to Install
This options runs the
6.packages.sh script to install
additional packages to your LiveFS file system. Figure 3 shows the menu of
packages to choose.
Figure 3: The package menu.
The script generates this package list with information from
pkg_info. In other words, LiveFS can install only those packages
already installed on your build system. If you have installed some of your
software manually without using the ports or pkg_add utility, you won't see it
in the generated list. As Figure 4 shows, this operation is very similar to
/stand/sysinstall utility for tuning your FreeBSD
Figure 4: Package selection.
While choosing necessary packages, keep in mind that you're limited by the space of your storage medium. A CD-R or CD-RW normally holds between 650 and 800 Mbytes. If you use a DVD, you'll have much more room for your software.
User - Custom User - Install Shell Script Files and Others
This option runs
7.customuser.sh. This copies user profiles and
software configurations from
/usr/local/share/freesbie/files/foobar to the LiveFS file system.
The defaults are:
Installing customized files... /usr/local/share/freesbie/files/foobar/irssi_config -> /usr/local/livefs/root/.irssi/config .muttrc .mutt/aliases .mutt/auto_views .mutt/bindings .mutt/colours .mutt/gpg .mutt/headers .mutt/macros cp: /usr/local/share/freesbie/files/foobar/.. is a directory (not copied). cp: /usr/local/share/freesbie/files/foobar/. is a directory (not copied). /usr/local/share/freesbie/files/foobar/.xcustom.sh -> /usr/local/livefs/root/.xcustom.sh /usr/local/share/freesbie/files/foobar/.vimrc -> /usr/local/livefs/root/.vimrc /usr/local/share/freesbie/files/foobar/.tcshrc.complete -> /usr/local/livefs/root/.tcshrc.complete /usr/local/share/freesbie/files/foobar/.tcshrc.bindkey -> /usr/local/livefs/root/.tcshrc.bindkey /usr/local/share/freesbie/files/foobar/.tcshrc -> /usr/local/livefs/root/.tcshrc /usr/local/share/freesbie/files/foobar/.shrc -> /usr/local/livefs/root/.shrc /usr/local/share/freesbie/files/foobar/.rhosts -> /usr/local/livefs/root/.rhosts /usr/local/share/freesbie/files/foobar/.xsession -> /usr/local/livefs/root/.xsession /usr/local/share/freesbie/files/foobar/.mailrc -> /usr/local/livefs/root/.mailrc /usr/local/share/freesbie/files/foobar/.mail_aliases -> /usr/local/livefs/root/.mail_aliases /usr/local/share/freesbie/files/foobar/.login_conf -> /usr/local/livefs/root/.login_conf /usr/local/share/freesbie/files/foobar/.login -> /usr/local/livefs/root/.login /usr/local/share/freesbie/files/foobar/.custom.sh -> /usr/local/livefs/root/.custom.sh /usr/local/share/freesbie/files/foobar/.cshrc -> /usr/local/livefs/root/.cshrc /usr/local/share/freesbie/files/foobar/.Xauthority -> /usr/local/livefs/root/.Xauthority /usr/local/share/freesbie/files/foobar/.profile -> /usr/local/livefs/root/.profile /usr/local/share/freesbie/files/foobar/.xinitrc -> /usr/local/livefs/root/.xinitrc /usr/local/share/freesbie/files/foobar/mutt.tar.bz2 -> /usr/local/livefs/root/mutt.tar.bz2 /usr/local/share/freesbie/files/foobar/irssi_config -> /usr/local/livefs/root/irssi_config
This is not very useful for server configuration.
Build - Build ISO - Create .iso File
This option runs the
8.mkiso.sh script to create the ISO image
file using the already configured path. In our case, this will generate the
/usr/local/share/freesbie/FreeSBIE.iso. We are almost
There are two different ways to customize the build. The first way is to
edit all files in the
/usr/local/share/freesbie directory before
./freesbie. The second way is to create a LiveFS file
structure and then edit necessary files. Certainly, you can combine both of
them, but I find the second approach much more sensible.
Customizing the System
Now that you're familiar with the build process, I can describe how to customize the system.
First, we need to customize the
/etc directory on the LiveFS. The
/etc/rc.conf file holds hostname and interface configuration.
We have no easy way to keep this file on an NFS drive, except for patching the
startup scripts. Hopefully GUFI, the main
Italian FreeBSD User Group developing FreeSBIE, will simplify this eventually.
We can either prepare a different configuration for each server in
edit the scripts to mount
nfs01. You can also try to follow the
/usr/share/examples/diskless/clone_root instructions of cloning a
diskless configuration, although I believe it'll also require a lot of hacking.
I have not looked at it closely, but it seemed more oriented to diskless
Once we've set a system hostname and have connected to the network, we must
start our services. This means we'll need to edit
nfs01:/startup/usr/local/etc/rc.d/cluster /usr/local/etc/rc.d/cluster nfs ro 0 0
/usr/local/etc/rc.d/cluster directory holds a startup
script for local services. FreeBSD's
/etc/rc script mounts NFS
file systems before it starts local software, so we won't need any hacks there.
Also we need to mount a data directory and software configuration, too:
nfs01:/storage/var/web /var/web nfs Ro 0 0
/var/web directory holds data for our web services. This
could be a directory with a complex file structure. Though we shouldn't
normally make changes there, we can do this if we mount it in
nfs01:/startup/usr/local/etc /usr/local/etc nfs ro 0 0
/usr/local/etc directory holds our software configuration
To crown it all,
nfs01 must store:
/startup/usr/local/etc /startup/usr/local/etc/rc.d /storage/var/web
This isn't a big deal, except for the data at
This won't be a big load for our NetApp NFS server.
When we've finished customizing our system, we can prepare an ISO image to burn. It took me no more than two hours to reach this point from the very beginning. I hope it normally would take no more than four or five hours for any professional familiar with FreeBSD to prepare a completely new, highly customized cluster system.
You can now burn the ISO image to disk using any operating system or
software you like. Under FreeBSD I like the
burncd one and on the Windows
platform I use Ahead Nero. Here's how to burn the system on a CD-RW disc under
# burncd -s max -f /dev/acd0a blank
This will make a blank image. To burn a CD:
# burncd -s max -f /dev/acd0a /usr/local/share/freesbie/FreeSBIE.iso fixate
It's also worth mentioning how to keep your systems up-to-date. It's both easy and difficult. I believe that the most correct way to stay current is to keep your build system updated. As a rule, your production system needs infrequent updates — only when FreeBSD has important changes. So, while your build system is up-to-date, you can pass through the build process from the very beginning to create an image for the most recent version.
Now you can take a freshly burned CD disk and put it into the CD-ROM drive of your cluster machines. Boot it, then share your happiness at using such an easy and cost-effective solution with others!
Alexander Prohorenko is a certified professional, who holds Sun Certified System Administrator and Sun Certified Java Programmer certifications.
Return to the BSD DevCenter.