Creating a Data Mirror
by Dru Lavigne
Related link: http://2005.meetbsd.org/
On Saturday, I'll be giving a tutorial on "Installing, Securing and Maintaining FreeBSD Servers" at meetBSD 2005 in Krakow, Poland. The gist of the talk is how to install the minimum possible required to keep a server operational and fully patched.
Since this is a hands-on tutorial and we weren't sure how reliable the Internet access would be in the lab, I promised to mirror the necessary files on my laptop. My personal goal was to mimic Internet availabilility within the classroom environment. For example, pkg_add -r should just work; so should cvsup.
I'm providing my working notes as they may be of use to others who find themselves with multiple FreeBSD systems behind a slow or intermittent Internet connection.
On the system hosting the files:
I initially experimented with /usr/ports/net/cvsup-mirror, but it seemed overkill for my purposes. (Michael Lucas wrote a how-to on this utility in his Big Scary Daemons column.)
My laptop already uses cvsup to keep the entire ports collection and all src up-to-date. Meaning, I already had all the tools I needed to create a mirror without installing additional software.
In addition, the other FreeBSD systems in the lab will only require the minimal ports tools but will need full src in order to rebuild world and compile a custom kernel. I wanted to be able to easily control which files students could receive via my mirror.
I started by creating a directory structure to hold the 2 required mirror config files:
# mkdir -p /usr/meetbsd/sup/test
# cd /usr/meetbsd/sup/test
# vi releases
cvs list=list.cvs prefix=/usr
# vi list.cvs
Note that releases should be appropriate for any site wishing to mirror files contained somewhere within /usr. list.cvs should be customized to
indicate exactly which subdirectories you wish to be available via cvsup.
To make the mirror available, simply refer to your base like so:
# cvsupd -b /usr/meetbsd
On each system to receive the files:
Prepare the cvsup client. Note that the supfile is similar to the examples found in the handbook except for the sections I've bolded. You'll want to replace IP_address with the IP of the system hosting the mirror:
# pkg_add -r cvsup-without-gui
# mkdir /usr/local/etc/cvsup
# vi supfile
*default release=cvs delete use-rel-suffix
You'll also want to prepare the missing directories:
# mkdir /usr/ports /usr/src
# chmod 755 /usr/ports /usr/src
Then, run cvsup:
# cvsup -L 2 supfile
There you go. You now have your own customized cvsup mirror. If you ever feel like adding or removing available files, simply edit your list.cvs.
Next, I wanted to prepare my laptop to host the packages students would need in the class.
On the system hosting packages, create the packages directory:
# mkdir /usr/ports/packages
Then, make the desired packages:
# cd /usr/ports/misc/cvsup-without-gui
# make package-recursive
Note, you'll have to make deinstall first if this application is already installed on the system on which you're building the packages.
Once you've installed the necessary packages, you'll want to configure anonymous FTP since pkg_add connects to FTP servers. This can be easily done using sysinstall:
/usr/ports/packages (FTP root directory)
remove upload subdirectory
21 (for group ID)
No (to welcome message)
When you're finished, start the FTP server in daemon mode, for anonymous access over IPv4:
# /usr/libexec/ftpd -D -A -4
Note: I haven't demonstrated how to create a secure FTP server. These directions are suited for internal LAN use where FTP is not allowed through a firewall. If this isn't your scenario, read up on how to secure your anonymous FTP server.
Accessing the package repository
On the systems you wish to install packages, change the default environment. Again, substitute IP_address with the IP address of the system hosting the packages:
# setenv PACKAGESITE ftp://IP_Address/Latest/
# pkg_add -r cvsup-without-gui
Instead of going on the Internet, your packages will install flawlessly from your own package server. Since you created those packages using the make package-recursive command, all dependencies are taken care of for you.
Well, I need to finish packing for the trip. I'll be blogging about meetBSD as I get a chance. I'll also let you know when the PDF for the tutorial is available, as well as the slides and PDF for the talk I'm giving on BSD Certification on Friday.