The Death of A Killer App

by Dale Dougherty

The killer application of the Internet is being strangled to death. Over the last year I had seen numbers that spam accounted for as much as 50% of incoming mail. Current spam levels are much higher, according to
Bob Amen, O'Reilly's system administrator, based on his experience. Bob sent mail today describing our statistics for the last 24 hours, saying that he wanted our users to know how much spam he was keeping out of our mailboxes.

There were 82,300 incoming email connections to our mail server.


Of those, we blocked 10,000 messages through a local blocking list that I maintain and for improperly formed SMTP commands. The latter are mostly due to viruses and trojaned home systems on DSL or cable that are being used as spam robots.


We blocked 13,000 attempts to deliver the MyDoom virus.


Our spam software (SpamAssassin) identified 49,000 spam messages and 11,000 clean messages (after alias expansion). That means we get more than 4 spam messages for each good message!


Of course, this doesn't mean that O'Reilly users were not seeing spam. A good amount does get through. I'd estimate that I get 1 piece of spam for every 4 messages that get through to my mailbox. Less than 10% of the incoming email is any good, an unhealthy ratio of real meat to spam.


As much as spam filtering has improved, I'm still spending a portion of my day rooting through trash to find something good to read. The only consolation is it could be worse, if not for the efforts of admins like Bob.

Isn't it too bad to see email trashed?


10 Comments

macrat
2004-02-05 18:08:12
Don't post your e-mail address
The fact that the O'Reilly staff posts their e-mail addresses in an easy to harvest format doesn't help your sys admin much. ;-)


On of the reasons why I like to run my own mail server is so that I can create multiple e-mail accounts.


The account used to register a domain gets harvested pretty quickly.


The account used for purchases hasn't been spammed yet.


The account I send to friends is pretty quiet until my more naive friends start entering the address into web sites with a joke and a link that says 'e-mail this to your friend.'


A purchased account from major servers like pobox.com, msn.com, earthlink.net have bots that run random account names to harvest but those don't build up fast.


An old account from 1996 that I had plastered on various web pages gets absolutely flooded. Which goes back to my original point of why the O'Reilly staff gets more than the usual amount of SPAM.

jwenting
2004-02-06 03:40:27
Don't post your e-mail address
All true in part, but of course the entire idea of an email address is that people can contact you whom you've not had contact with before (at least, that should be the idea).


As you have discovered, the only way to prevent an email address from getting spammed is to not have anyone else know about it which kinda limits the usefulness of the entire idea of email.


And also remember that spammers ever more are resorting to sending a million emails to random addresses at any one of a list of ISPs in the hope that some get through.
simply loop through 'a'.. 'zzzzzzzzz' @ and you're sure to reach someone.
Having a well known domain name like O'Reilly almost ensures getting hit (especially if your name is Tim).


I get (occasional) spam on an email address I've never yet used (I'm starting to set it up to replace my heavily spammed one that originated in 1997, but still need to make a list of people and companies to notify, a list which will probably number a hundred or so making me consider splitting it up into several addresses).
I've heard other stories of people who got spammed on an address 5 minutes after they'd started accepting mail on it (and before even telling someone it existed).

jimothy
2004-02-06 06:16:49
Assassination
I think it's time we take "Spam Assassin" literally and give the spammers their due.


Honestly, do the spammers think we'll buy their enhancement pills from the same marketers who have made it their mission to harrass, deceive, and insult us while generally making the Internet an increasingly unpleasant place?


Never before have I held so much scorn for someone I don't even know, but these cowardly con artists are simply despicable.

sysadmn
2004-02-06 08:33:58
One million spam messages a day
From the corporate intranet news page:


What is "spam"? What does the company do about it?
Spam is usually defined as unsolicited commercial email. The company filters out approximately 1 million email messages with known characteristics of spam (target addresses, subject lines, etc). every day, which amounts to over 50% of the email received daily via the Internet. This is an increase of about 20% over last year and is a problem for all companies, as well as home users.

andy-lester
2004-02-06 08:44:52
Don't post your e-mail address
Why should the ORA folks not post their addresses? They want legitimate email. The spammers have made the vast majority of netizens take a defensive posture, when it shouldn't be necessary to do so.


Spam will not go away until we take the offensive and adopt the attitude that we CAN solve the spam problem. Projects like SPF are going forward with strong initiatives to get us down the path. It's not a silver bullet, but we know there isn't one anyway.


Refusing to publish your email address is like taking the long way to school to avoid the bully. It's time we stop taking that attitude.

terrie
2004-02-06 08:50:46
Easy to harvest?
Hi -- we try to keep email addresses on the site munged with javascript (though I admit sometimes folks post their own mailto links in their text without realizing that we have a better way). Are these now easy-to-harvest or did you just not realize that they were munged?
dale
2004-02-06 10:17:12
Don't post your e-mail address
Thanks for a great reply, Andy. As Terrie later points out, we try not to make it easy for the spiders to harvest the email addresses on our site. The point is that if you are going to post something on the net, don't you want to let people contact you to reply directly?


I don't want to go into hiding because of spammers. I like the analogy of going out of your way to avoid the bully. We need to deal with the bullies. I also think we need to deal with the architecture of our email systems that make them so vulnerable. It's not an end-user problem.

jwenting
2004-02-06 10:20:37
Assassination
sad fact is that some people DO buy those products and DO fall for the scams despite constant warnings not to.


Given the cost of sending they need only about 1 in 100.000 or worse success ratio to make a profit, and it's estimated they get 1 in 10.000 or better.

macrat
2004-02-07 18:43:03
Don't post your e-mail address
Dale and Terrie.


I was not saying that you should try to hide.


I was only pointing out that putting your e-mail addresses on web pages in plain text so any high school kid could capture with a script.


For example right on the page:
http://www.oreillynet.com/pub/au/26


Right there is Dale's address in plain text that can easily be captured by a script.


Many people these days use formats like 'account at something.com' or even 'account at something dot com'


Formats that are easy for humans to read but not your average script.

terrie
2004-02-08 07:53:35
It isn't in plain text
View the source of that page, and I believe you'll find that it's not there in plain text as you describe.


Your point is a good one, but I don't think it applies in this case.