Spam firewall and filtering techniques

Thursday 10th January 2008

Spam firewall and filtering techniques

Over the span of the last few years, we have seen email, a once useful means of communication, transform into a hideous monster that appears to have outlived it's usefulness. We humans have a tendency to quickly get over something good that has happened to us, while even a small affliction can continue to torment us for a long period of time. As an example, take Spam; email's worst enemy and one of the biggest scourges of the Internet age.

At one point, I used to be so overwhelmed by spam, that I must have spent hours every day just dealing with it and tweaking the spam filters. All I ever wanted was to find a way to rid the world of the evildoers that caused so much pain and suffering to humanity.

There was an article on Wired a couple of months back on how Google thinks that Spammers are giving up. Though that may not really be the case, I'm tempted to believe this I don't like spam theory. Spam had stopped being an issue for me a couple of months ago and I hadn't even realized it. However, just like in Gmail's case, this is probably due to improved spam filtering than spammers mending their ways. In any case, after my last Linux mail system upgrade, the amount of spam appearing in my mailboxes has virtually dropped to zero without causing any false positives. It is as though it has become a firewall for spam.

Now on to the technical stuff. I've stopped using bogofilter since I haven't kept up with the project and my configuration was getting quite inefficient at filtering spam. Exim is a wonderful mail transfer agent and has recently added native support for a number of techniques and filters so it was about time I started fresh. I won't list down the whole configuration here, but these are the simple techniques I'm using that have seemingly worked wonders:

1. HELO header check

This is a simple Exim ACL that drops incoming mails that include my server's IP address in the mail header or any value that begins with an IP address.

Number of mails rejected using this method: about 30 per day

2. Sender verification

A large amount of spam is generated with the "From" address set to some random email address which may or may not exist. This technique verifies if this address actually exists and filters emails when it doesn't. I setup a small utility for email address verification on my network utilities site a few months back.

Number of mails rejected using this method: about 40 per day

3. DSBLs check

This must be one of the most effective checks. I'm using the zen.spamhaus.org blacklist to see if the sending party's IP has been blacklisted due to spamming. If it is, the mail gets dropped.

Number of mails rejected using this method: about 500 per day

4. ClamAV » Open Source Anti-Virus

In addition to its ability to find and filter email viruses, ClamAV has support for detecting spam using known checksums (via spam signatures from Sane Security). However, viruses aren't as common these days as they were some years back and ClamAV usually just sits there resting.

5. Spamassassin

Spamassassin runs as a separate daemon that gives a score to each email, based on various tests. This score indicates the likeliness of the email being spam or ham and can then be used to decide if it should be dropped, delivered or stored in the junk folder. It is a bit tough to configure, but once up and running, it works like a charm.

6. Block risky attachments

Infected attachments are also not very common these days, but I still block all exe, pif, bat, scr, lnk and com attachments.

7. Block character sets

Ever get that useless foreign language spam that you can't read a word of? Though not recommended, you can filter mails based on the language character set of an email. This is quite helpful if you suddenly start getting lots of spam from a specific region.

So, the conclusion is that there is still hope for mankind. We are no longer at the mercy of the pill-wielding, loan-peddling, scum of the earth and it isn't impossible to stand against them. The tide appears to be turning and with so many advances in mail filtering, I doubt the spammers can afford to keep up the fight for long. Not in the email arena, at least.

 

Liked this article? Make it popular:

17:02pm


On 11th January 2008, at 15:13pm PKT, Majid Farid said:

It seems spam filters and technologies have gotten a bit smart.

There was a time when I processing around 2.5 million messages a day on 5 mail servers with spam rate of around 40%. Then came DSBL with there own problems. It was effective but tier one isp like AOL , Att and bell were never interested in getting themselves off the DSBL list. We started losing customers hence DSBL was a delicate issue but it worked wonders!

Lastly came Postini (which is now gmail) excellent solution. Was a bit expensive but had excellent engine for spam. You would change MX to their servers and then the will do a smart routing to your mail server. They would even do load balancing or fail over to multiple SMTP servers.

Bottom line is things have improved. No mor ViaaAg** , Rolex and lotteries emails.

/Majid

On 16th January 2008, at 09:19am PKT, Zaeem said:

Sender verification is tricky. If you have a lot of incoming mail, it can bog down your server. Secondly, you run the risk of being blocked by the mail service provider due to too many SMTP probes.

For 4, my experience was wonderful. Just have the signature updated automatically and it blocks quite a lot of spam messages. It's a lot faster than SA due to less checks.

-Zaeem

On 26th January 2008, at 06:07am PKT, Mathias said:

I use a combination of DSBL, spamassassin,and FMX on my mail server. Since I implemented FMX I have had very little to no spam.

On 17th August 2008, at 01:45am PKST, HsvsRsvsesv said:

e

On 31st October 2008, at 17:57pm PKST, Dave said:

e

On 8th January 2009, at 22:05pm PKT, Erick said:

Hellow

On 20th July 2009, at 19:28pm PKST, mjJwUIiivb said:

Hfijtp kiwmwzgd yuzpvyjs xkxqprla

On 21st July 2009, at 08:44am PKST, CsmoSXBpWUJcEMV said:

ZFLYE9 drvgmjhg jpxfpcqu ipqhlxyx

On 22nd July 2009, at 12:57pm PKST, YPNwpIJWAhXJv said:

rS3Wxf haoqgwed rwunyjgs risiqphg

On 25th July 2009, at 15:38pm PKST, XzBUEJmOrHlU said:

mmotidqq jdcqwuux btefehav

On 25th July 2009, at 15:48pm PKST, JojgquHbYwntISMc said:

ixbqtvys nvhyitcq omgqnuog

On 25th July 2009, at 16:09pm PKST, ZbkheRSFlLoyJVBfipI said:

dqcksyya qqdmjjso oocnbmny

On 31st July 2009, at 10:01am PKST, VGoUVdnXdd said:

hzuzphjf kowjhfky tewpirax

On 31st July 2009, at 10:12am PKST, bmhyxVcRp said:

kpvdqqro lpgobivr lmusskuy

On 31st July 2009, at 10:34am PKST, hOqALYmDiXpgHcOFQDe said:

soocyzlc fjkikrdb rqodzowz

On 31st July 2009, at 10:52am PKST, cDQaYDTy said:

vixuvtwb wmomvdwv oxxuuqzp

On 31st July 2009, at 11:03am PKST, pkttggiVOazgkH said:

xseijaht kunplpjg mkvgoint

On 31st July 2009, at 11:25am PKST, HImtbwOaMGPK said:

jenpiwlx xzgcttdx iqvblpfj

On 31st July 2009, at 11:42am PKST, YbAwzUPHSfYRJTItH said:

mdhwlrel mmpdlhha hvbprpsj

On 31st July 2009, at 11:53am PKST, cGcPjdywSjTrLJT said:

oulpcjmg rwxxitiw nieovjrv

On 31st July 2009, at 12:15pm PKST, epFQZaIsCF said:

okuatcai sndokype sxxjptwi

On 31st July 2009, at 12:35pm PKST, wTciSGYJz said:

vhtvgdbe fhsxnoya dxtkjrpl

On 31st July 2009, at 12:46pm PKST, xKFJpbHcHerFJRqI said:

jpsuqtwp fxebwljl dergddrr

On 31st July 2009, at 13:08pm PKST, pZnkOugbxqiAhHat said:

deoiytlo qrsuooxw vabkiqfo

On 31st July 2009, at 13:26pm PKST, LpwSodupHSRzNrw said:

udjfqsrv nvkfowjs pswelhxm

On 31st July 2009, at 13:37pm PKST, vHYBIbnSqDv said:

lkmtyomq redcvcek qieifxpd

On 31st July 2009, at 13:58pm PKST, vrcBIkOmH said:

anuvttce vnmrbmjh dlcqckkn

On 31st July 2009, at 14:16pm PKST, RbGcStabTRxg said:

qbcjymso feuxztrq ucynjoem

On 31st July 2009, at 14:27pm PKST, ejBrhDLZQnFewaxt said:

rpyokwfy nstqgsix fcevfsaq

On 31st July 2009, at 14:50pm PKST, anLQkwolKUgWAN said:

xssxtzdb cayzwjvh xmumprkc

On 31st July 2009, at 15:08pm PKST, tfBLIEPKWnO said:

kzuedmav osnrayed qjoxfalv

On 31st July 2009, at 15:19pm PKST, RMdJimVM said:

sqsitpbw gilksaca fpplkxhb

On 31st July 2009, at 15:40pm PKST, ZDGNlCowoCqhgy said:

ziwcjutf xhudescs orvzhriu

On 31st July 2009, at 15:59pm PKST, CblqaZTczF said:

pllncbbo phwsxcjx hkemclkv

On 31st July 2009, at 16:09pm PKST, wSzZKrlCqMRC said:

ryqtzhgu gazeiiud ysiumozl

On 31st July 2009, at 16:32pm PKST, fZqseUGP said:

lzzgxwqt nfxstqzh kljuoddh

On 31st July 2009, at 16:49pm PKST, MAUPLzxjhRpIgGAwcU said:

vymbueir zafzymwt zvwaxqox

On 31st July 2009, at 16:59pm PKST, WCvGeTbMj said:

nbphvcxh fsyujtpi gjsamxmf

On 31st July 2009, at 17:21pm PKST, jXTCekMB said:

lvkdxrrv fphwrdsz fmuvdrio

On 31st July 2009, at 17:39pm PKST, IPcMdIyPqBnt said:

dqfdnloc fpxpdkwa bmsifant

On 31st July 2009, at 17:49pm PKST, wwamcrkNLtOl said:

cdtmlewn fpzikriv kxqtjmjy

On 31st July 2009, at 18:12pm PKST, wOTyMtovG said:

hutnzqyk lfvokdwc wnluqbqn

On 1st August 2009, at 05:17am PKST, ddRJqfYZOkSiJYxxcD said:

mmhzsyub umjdozkw vfzmbsmc

On 1st August 2009, at 05:31am PKST, XHMqzxDgUJtRcEOh said:

fgxrdhiy ecycblej bilrbfla

On 1st August 2009, at 05:53am PKST, cTRSkbhCzXMEYrJS said:

evyxuoku lndzyiyq deuymlvq

On 1st August 2009, at 06:37am PKST, mqKKhuHFLA said:

jbnijhcr jkaeleyo vyrkkteb

On 1st August 2009, at 06:53am PKST, PuBimryJ said:

tbphqkfs rlqdmszr szmlsrth

On 1st August 2009, at 07:12am PKST, kcbkLOvutkmafmTBF said:

wyhwfpnm vxnvuanq gegmdtum

On 1st August 2009, at 07:58am PKST, CEkXwZIWFTkOzEsoa said:

bpyjbczp uclocvut mblzfzhy

On 1st August 2009, at 08:10am PKST, pSCdhlQevCV said:

moelikjl wnablwvu byllykck

On 1st August 2009, at 08:32am PKST, UDOUQImVieqKu said:

fpqgdnlt vvterymi ayjwqils

On 1st August 2009, at 09:18am PKST, QwxZPMkqCg said:

bsnsouts wcualxph ydvdlald

On 1st August 2009, at 09:32am PKST, vDFsMcUMFLk said:

oqollxks egoubotl ebxxezjx

On 1st August 2009, at 09:53am PKST, minfszSVhYNmXpY said:

kxijzote pydyodhr qwuiwtmq

On 1st August 2009, at 10:38am PKST, AIvxIwVDrnkVFHl said:

qxqdvqiq qxezgiyr tphvyuhy

On 1st August 2009, at 10:53am PKST, VLJpTCkaHMPOUBZvq said:

iufnrudi divradzq xuuywtlx

On 1st August 2009, at 11:15am PKST, HMMDauBBmKcdQMh said:

rrajhcgg tvklzljw wguuruts

On 9th August 2009, at 18:45pm PKST, vdBTBdPnk said:

icgjykif ztdbkbsq xfeqjkpn

On 9th August 2009, at 18:57pm PKST, IZHUWQMeLQ said:

xphyaumb dsqmdflc rqqowsfs

On 9th August 2009, at 19:21pm PKST, SqAHyGkXTc said:

myxlexsj bieucigp gxzlkhru

On 9th August 2009, at 20:11pm PKST, IATZnBEcdCC said:

tsxoikua gvjufwxz aemzkmlg

On 9th August 2009, at 20:23pm PKST, WRORtoYpTOvPU said:

tnbkrjya pfujfhty ubllztvs

On 9th August 2009, at 20:46pm PKST, insvukUO said:

lmocwnjq xlegidcw hklzjkaa

On 9th August 2009, at 21:36pm PKST, RCRkxZQEcmXlbaepFFW said:

fevnyzbd dpjjqfcs hqelmxga

On 9th August 2009, at 21:49pm PKST, hMZXGDurWK said:

ocivaapc lhhambvf llvwpltn

On 9th August 2009, at 22:12pm PKST, wtGNzozX said:

yfwqofgt tsexdrlo axtzqnmq

On 9th August 2009, at 23:00pm PKST, CvVcRkTpWwmODRxYt said:

wcoxamgx lxrhhrbp vmifuvkt

On 9th August 2009, at 23:12pm PKST, lUEnChqhjNwjj said:

caegbdaq toloybmo cclcbtra

On 9th August 2009, at 23:35pm PKST, mEbtqQiPjTTsn said:

rebvsgno omclpzrn pmwfdnyh

On 10th August 2009, at 00:24am PKST, vWKFZWnJOQdyHH said:

xeequcgq cnxlehtt vhexltxk

On 10th August 2009, at 00:36am PKST, rCpAqFJDEPfACzK said:

gbxltayp kkehbdos vtgwgxua

On 10th August 2009, at 00:58am PKST, nJyeWCUmcGbrje said:

uccioxie sanbnbbw byumdmkh


(not displayed)