Pages: 1, 2
First off, here's how the FreeBSD CVSup mirror system works.
There's a master CVS repository on
freefall.freebsd.org. This is the
absolutely authoritative source of FreeBSD code. Users cannot, under
any circumstances, update their systems from
freefall. It's also a
hard-working machine. As the authoritative repository, it must check
to see if files have been changed by some program other than
This is done with
stat (2) which, while not particularly expensive used
one at a time, devours disk resources when it has to check every
single file that has ever been in FreeBSD. Every time someone updates
freefall, disk usage climbs.
The CVSup server on
freefall has only one client,
cvs-master.freebsd.org. Its purpose is to serve as a main source for
official mirrors. "The master mirror is the most efficient by far,"
says Polstra. "It doesn't have to do much disk I/O [those
and it doesn't have to do too much thinking." The files are only
cvsupd knows perfectly well what files it's
used. As such,
cvs-master can support many main mirrors. Since it's
a key part of the FreeBSD infrastructure, however, access to
cvs-master is tightly restricted to mirror sites only.
These final mirrors are what us lowly users can access. Mirror
machines pull their updates from
every 6 minutes, code becomes available on a user mirror not more
than 66 minutes after it appears on
freefall. That's not bad
for a worldwide data distribution system.
If you want to be a part of all this, it's not that hard.
To run an official mirror, you need to first check your hardware. Polstra recommends at least a 400MHz Pentium II, 256MB of RAM, and a good, fast hard disk. This would support 8 to 10 users most of the time, unless there's a recent release. (A release touches every file in the repository, increasing the amount of work needed to update, hence boosting the time required.)
One particular mirror server used to have a single disk, 128MB of RAM, and a PII-400 MHz CPU. It could handle up to eight clients simultaneously, but interactive performance was horrendous when the system was under load. That same system has been upgraded to two Ultra2 SCSI disks -- one for the system files and one for the repository, and 512MB of RAM. Its limit has been upped to 20 clients at a time, and interactive performance is always excellent.
Adding RAM is the best way to increase a CVSup mirror's performance. Updates will happen more quickly -- users will stay connected for a shorter time, so the system load will drop and it can handle more users. This is something of the opposite of the "death spiral" an overloaded machine can suffer from.
Second, you need to check your bandwidth. One mirror chosen at random transferred about 1.1 megabits of traffic out and a quarter-megabit in during a randomly-chosen day. You can't quite serve this over an ISDN line, but a T1 has plenty of capacity to spare.
If you have the hardware and the bandwidth to donate, contact Polstra at email@example.com. Include your location, as he tries to balance mirrors by geography. If your area is short on mirrors and you fit his other requirements, he might take you up on it. John chooses the mirror sites carefully, balancing the needs against the locations offered and the capacity. "I would be unlikely to allow 30 mirror sites in, say, Iceland," he says.
Next time, we'll look at the software involved in setting up a mirror.
Hopefully, we can force John to soon change the naming scheme on the
mirrors to something like
cvsup14.yourstatehere.freebsd.org. And even
if we can't do that, you can set up a local mirror to meet the needs
of your organization.
Read more Big Scary Daemons columns.
Return to the BSD DevCenter.