Blocking spam is an arms race between spam detection and detection avoidance techniques. Lately spammers had the upper hand but the tide has turned with new PTR record blocking techniques. This is how implementing PTR record filtering has reduced our spam to nearly zero. Reducing Spam to Nearly Zero with PTR Record Filtering
Interesting article on how a big company, using a Barracuda SPAM filter, reduced their SPAM significantly by filtering out email originating from cable modem users on the Shaw Cable network in Canada.
There is really no good reason why legitimate email would be originating directly from home users on the Shaw network. It is pretty safe to assume that 99% of email being sent directly from a consumer broadband connection (and not relayed through Shaw’s SMTP server) is SPAM of some flavor.
If home users are geek savvy enough to be running their own home mail server on the Shaw network (like me!) then they should also be smart enough to know that they should be using Shaw’s SMTP server as a relay host for their outbound mail!
The only “problem” with this approach is the scalability and maintainability of the solution. Trying to compile and maintain a comprehensive collection of regular expressions to match all the potentially SPAMmy PTR records is daunting. Having said that, anything that can reduce the amount of SPAM reaching my inbox is probably worth pursuing.
In the last couple of weeks of testing, I’ve found that good real time black lists and PTR filtering have made a big difference.
It’s not perfect though. Spam still makes it through but it has dropped down drastically.
The spam that does make it through usually comes from hosts without PTR records or spammers with their own subnets. REGEX to filter out PTR isn’t too hard, see: http://barracuda.pbwiki.com.
The tough ones are the image SPAM. I’ve still got some of those leaking through. I like what Spamhaus is doing trying to build up a list of consumer IP blocks from the ISPs. The idea being that the ISPs are partnering with Spamhaus to ensure that SPAM gets blacklisted if it originates directly from a home DSL customer. Not unlike what you’re doing with the PTR stuff.
These are the RBLs that I use.
combined.njabl.org
list.dsbl.org
zen.spamhaus.org
Here’s some interesting statistics:
Yesterday we got about 469,000 messages, this is fairly average for us. About 400,000 was blocked by RBLs (~85%), 20,000 by SPF (~4%), and the rest by Barracuda’s OCR, finger printing, Bayesian and intent analysis.
Spam filtering has really followed the law of diminishing returns (kind of). The first 90% is pretty easy, the last 10% really hard. However, people really only notice the last 10%.
The psychology component plays a huge part here. If you normally get 50 spams a day, and I reduce that to 30, you probably won’t notice. However, if I reduce it from 25 to 5 (still 20 less emails), and you’ll notice.
That’s what SPF and Barracuda’s new fingerprint/ocr blocking did for us. Make it ‘perceptually’ better, even though we’re only blocking an extra 5%.
BTW: regarding image spam, after upgrading to the 3.4 Barracuda firmware and implementing SPF filtering, I’ve received zero image spams, versus the 2 or 3 I previously got daily.
I haven’t gotten my hands on a Barracuda. Big bucks! I’m firmly in the open source category for my clients if only just because of budgetary constraints.
Every time I get a SPAM complaint from a client I pull out the SPAM filtering stats. As soon as they realize how much SPAM I’m actually stopping, they usually stop complaining.
In my opinion, we should all be looking at a two layered approach anyway. Stop the bulk of it at the firewall, but don’t be so aggressive that we start blocking false positives.
I make sure that my clients are all running some sort of anti-SPAM on their local machines, that they can train, to stop the stuff that sneaks through.