Spam firewall and filtering techniques
Thursday 10th January 2008
On 11th January 2008, at 15:13pm PKT, Majid Farid said:
It seems spam filters and technologies have gotten a bit smart.
There was a time when I processing around 2.5 million messages a day on 5 mail servers with spam rate of around 40%. Then came DSBL with there own problems. It was effective but tier one isp like AOL , Att and bell were never interested in getting themselves off the DSBL list. We started losing customers hence DSBL was a delicate issue but it worked wonders!
Lastly came Postini (which is now gmail) excellent solution. Was a bit expensive but had excellent engine for spam. You would change MX to their servers and then the will do a smart routing to your mail server. They would even do load balancing or fail over to multiple SMTP servers.
Bottom line is things have improved. No mor ViaaAg** , Rolex and lotteries emails.
/Majid
On 16th January 2008, at 09:19am PKT, Zaeem said:
Sender verification is tricky. If you have a lot of incoming mail, it can bog down your server. Secondly, you run the risk of being blocked by the mail service provider due to too many SMTP probes.
For 4, my experience was wonderful. Just have the signature updated automatically and it blocks quite a lot of spam messages. It's a lot faster than SA due to less checks.
-Zaeem
On 26th January 2008, at 06:07am PKT, Mathias said:
I use a combination of DSBL, spamassassin,and FMX on my mail server. Since I implemented FMX I have had very little to no spam.
On 17th August 2008, at 01:45am PKST, HsvsRsvsesv said:
e
On 31st October 2008, at 17:57pm PKST, Dave said:
e
On 8th January 2009, at 22:05pm PKT, Erick said:
Hellow
On 20th July 2009, at 19:28pm PKST, mjJwUIiivb said:
Hfijtp kiwmwzgd yuzpvyjs xkxqprla
On 21st July 2009, at 08:44am PKST, CsmoSXBpWUJcEMV said:
ZFLYE9 drvgmjhg jpxfpcqu ipqhlxyx
On 22nd July 2009, at 12:57pm PKST, YPNwpIJWAhXJv said:
rS3Wxf haoqgwed rwunyjgs risiqphg
On 25th July 2009, at 15:38pm PKST, XzBUEJmOrHlU said:
mmotidqq jdcqwuux btefehav
On 25th July 2009, at 15:48pm PKST, JojgquHbYwntISMc said:
ixbqtvys nvhyitcq omgqnuog
On 25th July 2009, at 16:09pm PKST, ZbkheRSFlLoyJVBfipI said:
dqcksyya qqdmjjso oocnbmny
On 31st July 2009, at 10:01am PKST, VGoUVdnXdd said:
hzuzphjf kowjhfky tewpirax
On 31st July 2009, at 10:12am PKST, bmhyxVcRp said:
kpvdqqro lpgobivr lmusskuy
On 31st July 2009, at 10:34am PKST, hOqALYmDiXpgHcOFQDe said:
soocyzlc fjkikrdb rqodzowz
On 31st July 2009, at 10:52am PKST, cDQaYDTy said:
vixuvtwb wmomvdwv oxxuuqzp
On 31st July 2009, at 11:03am PKST, pkttggiVOazgkH said:
xseijaht kunplpjg mkvgoint
On 31st July 2009, at 11:25am PKST, HImtbwOaMGPK said:
jenpiwlx xzgcttdx iqvblpfj
On 31st July 2009, at 11:42am PKST, YbAwzUPHSfYRJTItH said:
mdhwlrel mmpdlhha hvbprpsj
On 31st July 2009, at 11:53am PKST, cGcPjdywSjTrLJT said:
oulpcjmg rwxxitiw nieovjrv
On 31st July 2009, at 12:15pm PKST, epFQZaIsCF said:
okuatcai sndokype sxxjptwi
On 31st July 2009, at 12:35pm PKST, wTciSGYJz said:
vhtvgdbe fhsxnoya dxtkjrpl
On 31st July 2009, at 12:46pm PKST, xKFJpbHcHerFJRqI said:
jpsuqtwp fxebwljl dergddrr
On 31st July 2009, at 13:08pm PKST, pZnkOugbxqiAhHat said:
deoiytlo qrsuooxw vabkiqfo
On 31st July 2009, at 13:26pm PKST, LpwSodupHSRzNrw said:
udjfqsrv nvkfowjs pswelhxm
On 31st July 2009, at 13:37pm PKST, vHYBIbnSqDv said:
lkmtyomq redcvcek qieifxpd
On 31st July 2009, at 13:58pm PKST, vrcBIkOmH said:
anuvttce vnmrbmjh dlcqckkn
On 31st July 2009, at 14:16pm PKST, RbGcStabTRxg said:
qbcjymso feuxztrq ucynjoem
On 31st July 2009, at 14:27pm PKST, ejBrhDLZQnFewaxt said:
rpyokwfy nstqgsix fcevfsaq
On 31st July 2009, at 14:50pm PKST, anLQkwolKUgWAN said:
xssxtzdb cayzwjvh xmumprkc
On 31st July 2009, at 15:08pm PKST, tfBLIEPKWnO said:
kzuedmav osnrayed qjoxfalv
On 31st July 2009, at 15:19pm PKST, RMdJimVM said:
sqsitpbw gilksaca fpplkxhb
On 31st July 2009, at 15:40pm PKST, ZDGNlCowoCqhgy said:
ziwcjutf xhudescs orvzhriu
On 31st July 2009, at 15:59pm PKST, CblqaZTczF said:
pllncbbo phwsxcjx hkemclkv
On 31st July 2009, at 16:09pm PKST, wSzZKrlCqMRC said:
ryqtzhgu gazeiiud ysiumozl
On 31st July 2009, at 16:32pm PKST, fZqseUGP said:
lzzgxwqt nfxstqzh kljuoddh
On 31st July 2009, at 16:49pm PKST, MAUPLzxjhRpIgGAwcU said:
vymbueir zafzymwt zvwaxqox
On 31st July 2009, at 16:59pm PKST, WCvGeTbMj said:
nbphvcxh fsyujtpi gjsamxmf
On 31st July 2009, at 17:21pm PKST, jXTCekMB said:
lvkdxrrv fphwrdsz fmuvdrio
On 31st July 2009, at 17:39pm PKST, IPcMdIyPqBnt said:
dqfdnloc fpxpdkwa bmsifant
On 31st July 2009, at 17:49pm PKST, wwamcrkNLtOl said:
cdtmlewn fpzikriv kxqtjmjy
On 31st July 2009, at 18:12pm PKST, wOTyMtovG said:
hutnzqyk lfvokdwc wnluqbqn
On 1st August 2009, at 05:17am PKST, ddRJqfYZOkSiJYxxcD said:
mmhzsyub umjdozkw vfzmbsmc
On 1st August 2009, at 05:31am PKST, XHMqzxDgUJtRcEOh said:
fgxrdhiy ecycblej bilrbfla
On 1st August 2009, at 05:53am PKST, cTRSkbhCzXMEYrJS said:
evyxuoku lndzyiyq deuymlvq
On 1st August 2009, at 06:37am PKST, mqKKhuHFLA said:
jbnijhcr jkaeleyo vyrkkteb
On 1st August 2009, at 06:53am PKST, PuBimryJ said:
tbphqkfs rlqdmszr szmlsrth
On 1st August 2009, at 07:12am PKST, kcbkLOvutkmafmTBF said:
wyhwfpnm vxnvuanq gegmdtum
On 1st August 2009, at 07:58am PKST, CEkXwZIWFTkOzEsoa said:
bpyjbczp uclocvut mblzfzhy
On 1st August 2009, at 08:10am PKST, pSCdhlQevCV said:
moelikjl wnablwvu byllykck
On 1st August 2009, at 08:32am PKST, UDOUQImVieqKu said:
fpqgdnlt vvterymi ayjwqils
On 1st August 2009, at 09:18am PKST, QwxZPMkqCg said:
bsnsouts wcualxph ydvdlald
On 1st August 2009, at 09:32am PKST, vDFsMcUMFLk said:
oqollxks egoubotl ebxxezjx
On 1st August 2009, at 09:53am PKST, minfszSVhYNmXpY said:
kxijzote pydyodhr qwuiwtmq
On 1st August 2009, at 10:38am PKST, AIvxIwVDrnkVFHl said:
qxqdvqiq qxezgiyr tphvyuhy
On 1st August 2009, at 10:53am PKST, VLJpTCkaHMPOUBZvq said:
iufnrudi divradzq xuuywtlx
On 1st August 2009, at 11:15am PKST, HMMDauBBmKcdQMh said:
rrajhcgg tvklzljw wguuruts
On 9th August 2009, at 18:45pm PKST, vdBTBdPnk said:
icgjykif ztdbkbsq xfeqjkpn
On 9th August 2009, at 18:57pm PKST, IZHUWQMeLQ said:
xphyaumb dsqmdflc rqqowsfs
On 9th August 2009, at 19:21pm PKST, SqAHyGkXTc said:
myxlexsj bieucigp gxzlkhru
On 9th August 2009, at 20:11pm PKST, IATZnBEcdCC said:
tsxoikua gvjufwxz aemzkmlg
On 9th August 2009, at 20:23pm PKST, WRORtoYpTOvPU said:
tnbkrjya pfujfhty ubllztvs
On 9th August 2009, at 20:46pm PKST, insvukUO said:
lmocwnjq xlegidcw hklzjkaa
On 9th August 2009, at 21:36pm PKST, RCRkxZQEcmXlbaepFFW said:
fevnyzbd dpjjqfcs hqelmxga
On 9th August 2009, at 21:49pm PKST, hMZXGDurWK said:
ocivaapc lhhambvf llvwpltn
On 9th August 2009, at 22:12pm PKST, wtGNzozX said:
yfwqofgt tsexdrlo axtzqnmq
On 9th August 2009, at 23:00pm PKST, CvVcRkTpWwmODRxYt said:
wcoxamgx lxrhhrbp vmifuvkt
On 9th August 2009, at 23:12pm PKST, lUEnChqhjNwjj said:
caegbdaq toloybmo cclcbtra
On 9th August 2009, at 23:35pm PKST, mEbtqQiPjTTsn said:
rebvsgno omclpzrn pmwfdnyh
On 10th August 2009, at 00:24am PKST, vWKFZWnJOQdyHH said:
xeequcgq cnxlehtt vhexltxk
On 10th August 2009, at 00:36am PKST, rCpAqFJDEPfACzK said:
gbxltayp kkehbdos vtgwgxua
On 10th August 2009, at 00:58am PKST, nJyeWCUmcGbrje said:
uccioxie sanbnbbw byumdmkh
Spam firewall and filtering techniques
Over the span of the last few years, we have seen email, a once useful means of communication, transform into a hideous monster that appears to have outlived it's usefulness. We humans have a tendency to quickly get over something good that has happened to us, while even a small affliction can continue to torment us for a long period of time. As an example, take Spam; email's worst enemy and one of the biggest scourges of the Internet age.
At one point, I used to be so overwhelmed by spam, that I must have spent hours every day just dealing with it and tweaking the spam filters. All I ever wanted was to find a way to rid the world of the evildoers that caused so much pain and suffering to humanity.
There was an article on Wired a couple of months back on how Google thinks that Spammers are giving up. Though that may not really be the case, I'm tempted to believe this
theory. Spam had stopped being an issue for me a couple of months ago and I
hadn't even realized it. However, just like in Gmail's case, this is
probably due to improved spam filtering than spammers mending their
ways. In any case, after my last Linux mail system upgrade, the amount
of spam appearing in my mailboxes has virtually dropped to zero without
causing any false positives. It is as though it has become a firewall for spam.
Now on to the technical stuff. I've stopped using bogofilter since I haven't kept up with the project and my configuration was getting quite inefficient at filtering spam. Exim is a wonderful mail transfer agent and has recently added native support for a number of techniques and filters so it was about time I started fresh. I won't list down the whole configuration here, but these are the simple techniques I'm using that have seemingly worked wonders:
1. HELO header check
This is a simple Exim ACL that drops incoming mails that include my server's IP address in the mail header or any value that begins with an IP address.
Number of mails rejected using this method: about 30 per day
2. Sender verification
A large amount of spam is generated with the "From" address set to some random email address which may or may not exist. This technique verifies if this address actually exists and filters emails when it doesn't. I setup a small utility for email address verification on my network utilities site a few months back.
Number of mails rejected using this method: about 40 per day
3. DSBLs check
This must be one of the most effective checks. I'm using the zen.spamhaus.org blacklist to see if the sending party's IP has been blacklisted due to spamming. If it is, the mail gets dropped.
Number of mails rejected using this method: about 500 per day
4. ClamAV » Open Source Anti-Virus
In addition to its ability to find and filter email viruses, ClamAV has support for detecting spam using known checksums (via spam signatures from Sane Security). However, viruses aren't as common these days as they were some years back and ClamAV usually just sits there resting.
5. Spamassassin
Spamassassin runs as a separate daemon that gives a score to each email, based on various tests. This score indicates the likeliness of the email being spam or ham and can then be used to decide if it should be dropped, delivered or stored in the junk folder. It is a bit tough to configure, but once up and running, it works like a charm.
6. Block risky attachments
Infected attachments are also not very common these days, but I still block all exe, pif, bat, scr, lnk and com attachments.
7. Block character sets
Ever get that useless foreign language spam that you can't read a word of? Though not recommended, you can filter mails based on the language character set of an email. This is quite helpful if you suddenly start getting lots of spam from a specific region.
So, the conclusion is that there is still hope for mankind. We are no longer at the mercy of the pill-wielding, loan-peddling, scum of the earth and it isn't impossible to stand against them. The tide appears to be turning and with so many advances in mail filtering, I doubt the spammers can afford to keep up the fight for long. Not in the email arena, at least.
Liked this article? Make it popular:
17:02pm