Using Qpsmtpdby Matt Sergeant
The Dark Ages
When it comes to web programming, Perl developers are spoiled. They have an incredible array of choices for extending web-based content delivery, whether plain old CGI scripts, FastCGI and SpeedyCGI, or mod_perl and PerlIS. Beyond that, they have a large selection of frameworks with which to build their dynamic content, from Maypole and Catalyst to AxKit and OpenInteract.
The same is not true of email. Those who administer an email server more than likely have put up with the pain of adding dnsbl lookups to something like Sendmail or Qmail, or adding recipient validation for their custom user database in Postfix. Extending email servers is painful, and for the most part you can't do it easily in Perl.
Wouldn't it be nice if you could do something like mod_perl in a mail server?
Qpsmtpd, the mod_perl of Email
In 2001, Ask Bjørn Hansen needed to improve the spam detection on perl.org's mail server. He decided the option of patching the existing Qmail installation was too painful. Instead he turned to the code in Jim Winstead's Colobus NNTP server for inspiration to re-create qmail-smtpd in Perl. The first version he hacked together comprised just 300 lines of code.
Others soon started using qpsmtpd due to its malleability. In version 0.10, they refactored the code, factoring out some of the core subsystems and creating a flexible plugin system.
Ask is very coy about his project, but he proudly states that when apache.org switched from qmail to qpsmtpd, the server load actually dropped, simply because it was rejecting so much spam. As an example of the scalability of qpsmtpd, the apache.org mail servers currently process more than 2 million emails a day on their primary MX, rejecting more than 80 percent of it as junk--all thanks to the power and flexibility of qpsmtpd.
The goals of the Qpsmtpd project are:
- To allow simple setup and integration into an existing setup
- To make it easy to create new plugins for filtering or queuing email
- To maintain great performance while still running completely in Perl
- To maintain email stability and integrity
It is important to note here that qpsmtpd is just the SMTP component of an email setup. You still need something to perform onward delivery--either to your end users, or to remote sites when you are relaying mail. Most setups use either qmail or postfix to do this, but any mail server will do.
The initial design of qpsmtpd was to talk to qmail for onward processing (hence the Q in the name); however, qpsmtpd speaks natively to qmail, postfix, or exim, and for other servers you can use SMTP to deliver mail. While this sounds like you have to maintain two systems, the additional capabilities of qpsmtpd make it worth it. The next section shows how to make qpsmtpd talk to your other email server.
Setting up qpsmtpd
You can run qpsmtpd in several ways, depending on how you want it to manage the connections:
qpsmtpd-forkserver, which forks once per connection
- Via Apache::Qpsmtpd. This makes use of mod_perl 2.0's ability to create a protocol handler for protocols other than HTTP.
- For very high concurrency, you can use it as a multiplexing server. Two large antispam companies' high-traffic spam traps have used this effectively since the second quarter of 2005, delivering concurrency as high as 10,000 on some occasions.
Each possible configuration has its pros and cons, and you should evaluate your needs depending on your current setup. The example in this article uses
qpsmtpd-forkserver, because this approach does not require any extra software.
Required Perl modules
The following Perl modules are required:
If you use a version of Perl older than 5.8.0, you will also need:
The easiest way to install modules from CPAN is with the CPAN shell. Run it with
$ perl -MCPAN -e shell
Getting it all running
After installing the prerequisite modules, getting qpsmtpd running is very simple. Download the tarball from the qpsmtpd web site and extract it in the location of your choosing. Then
cd into the directory and type:
$ mv config.sample config $ ./qpsmtpd-forkserver -u $USER
Congratulations! You now have qpsmtpd running on port 2525. Telnet in to see it running:
$ telnet localhost 2525 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 jester.sergeant.org ESMTP qpsmtpd 0.31-dev ready; send us your mail, but not your spam.
QUIT to end the session.
Running as a daemon
There are multiple ways to set up qpsmtpd as a daemon. The most common is to use Dan Bernstein's daemontools kit (a
run file is included in the distribution); however, it is also quite easy to start by using rc scripts. I will leave running it as a daemon as an exercise for the user--people get very protective of their own particular way of doing this. See the read-me file in the distribution for details of how to do this for daemontools.