NRAO Home  >  Green Bank  |  Wiki Topic:    GB > Computing > SpamMitigationstuff

Changes | Index | Search | Go

Spam Quarantining Stuff

This page is now obsolete. See the GOLD BOOK.

Basic idea is to have a central procmail rule that diverts email tagged by spamassassin to a users webmail directory. Users can then access the quarantine using squirrelmail or any other mua that understands and can access the mbox file in which such messages are stored. As part of the system users get a daily digest email of what is currently in their quarantine.

The scheme will be opt-in for existing users and opt-out for new users. Whether the scheme is active for a user will depend on the existence of a file in their home directory called ".QUARANTINE_SPAM" The existence of this file indicates that the user is opted in. Conversley it's absence indicates the user is opted out.

The Procmailrc

The second rule checks for the existence of $HOME/.QUARANTINE_SPAM file and only proceeds if it does exist.

# Whitelist the digests!
:0H:
* ^Subject: .*(NRAO Daily Quarantine Digest)
$DEFAULT

# quarantine spam
:0:
* ? test  -f $HOME/.QUARANTINE_SPAM
* ^X-MailScanner-(SpamScore: sssss|SpamCheck:.(ORDB|INfinite-Monkeys|SpamAssassin| SBL+XBL))
$HOME/webmail/QUARANTINE

The matching rule above is pretty conservative, matching anything that scores 5 or more or matches one of the tests for known spam relays etc. I have not set it to do any logging as it produces reams of output that can be gleaned from /var/log/maillog anyway.

The scheme can easily be extended, for instance a user could have messages tagged as virus sent to the quarantine as well by a second rule in the global procmail that only runs if another file .QUARANTINE_VIRUS exists in the users home directory.

:0:
* ? test  -f $HOME/.QUARANTINE_VIRUS
* ^Subject: .*\{VIRUS?\}
$HOME/webmail/QUARANTINE

Others that spring to mind are:

• .DELETE_SPAM
• .DELETE_VIRUS
• .QUARANTINE_FOREIGN (Foreign char sets)
• .DELETE_FOREIGN

A suitable procmailrc is in /home/nraosoft/apps/quarantine/promailrc just copy it to /etc/procmailrc on the mail delivery host.

Daily Digest

Update:

Sample output:

61 quarantined messages:

Tehran@techsoftamerica.com      {SPAM?} FW: Shy Lady in prrrevet action
yenalykyf@info.com.tr           {SPAM?} Busty amateur on table
hwang@news.com                  {SPAM?} Girl in nude pantyhose
name@hsuchi.net                 {SPAM?} Babe Hardcore Pussy Fucked & Cum Covered
pmk@sesmail.com                 {SPAM?} Dirty Bitch Suck & Messy Facial Cumshot
ZEPKTI@radiance-ind.com         {SPAM?} Re: [IMPORTANT] Notice to Home Owner [531
wqfgnujhqdlbwf@yahoo.com        {SPAM?} Pay Less For Branded Watches 4Dv1
TAJRAXYWSBPH@alti-byg.dk        {SPAM?} valium
Lisha@marshjewelers.com         Your limited time savings code, don't delay!
bernard@radiomexico.com         {SPAM?} Asian Babe Blwojob Hardcroe scrutiny
dulfer@earthlink.net            {SPAM?} helen Clark it's happeend derivate

A single cronjob to fire off something like the following will take care of sending all opted in people a digest:

# Modified 09/14/05 to not send digest if quarantine is empty
#
for i in /users/*

do

name=`basename $i`

  if
     [[ -a /netapp/users/$name/.QUARANTINE_DIGEST && -s /users/$name/webmail/QUARANTINE ]]
  then
     awk -f /opt/services/quarantine/spam-digest /users/$name/webmail/QUARANTINE  | /bin/mail -s "NRAO Daily Quarantine Digest" $name
  fi

done

This script will need to be customised for each site the "test -a" line needs to be looking at the real /users NOT an automounted area.

Again, a copy of this is in /home/nraosoft/apps/quarantine/digest-cronjob. Copy to /opt/services/quarantine/digest-cronjob on the delivery host and make crontab entry: " 0 7 * * 1-5 /opt/services/quarantine/digest-cronjob" to mail out digests at 07:00 Mon-Fri or customise to your preference.

Rotation of quarantine

The sequence goes:

1. rotate QUARANTINE daily
2. run cronjob that appends the newly rotated QUARANTINE.1 to OLDQUARANTINE daily
3. delete QUARANTINE.1 daily
4. rotate OLDQUARANTINE weekly

Find the files in /home/nraosoft/apps/quarantine/quarantine ..../quarantine-weekly and ..../logrotate-cronjob

Copy quarantine and quarantine-weekly to /etc/logrotate.d/ and logrotate-cronjob to /etc/cron.daily

Design questions

• Digest frequency
  Daily. More frequently is annoying, less often is also bad imho.
• Digest contents
  Should it list only messages that arrived since the last digest was sent or everything currently in the quarantine? I favour the latter as it is much simpler to do. Listing the contents in reverse order of arrival makes more sense to me than oldest to newest.
• Digest sanitation
  Should the digest script try and sanitize the subject lines? Some users may be offended by the contents. To be honest I don't think we can really do much about it.
• Logrotate frequency
  Weekly. Doing it more often, especially if we use the compress option in logrotate, makes acessing quarantine folders older than a day or two a royal pain. Think about getting back from vacation! This may in fact be an argument for not compressing with logrotate.
• Personal procmailrc files
  If a message is shunted into quarantine by the global procmail then the users procmail is not run. Otherwise it is.
• Quarantine folder name
  Just call it QUARANTINE and have done.

How to get a user opted in.

1. Ensure a webmail folder exists. If the user is already using an MUA that understands mbox and can access the users unix filespace use a link or whatever is most appropriate.
2. Create the QUARANTINE and OLDQUARANTINE folders (Simply touching them into existence is fine)
3. Ensure ownership and permissions of webmail and *QUARANTINE are right
4. Touch .QUARANTINE_SPAM (optionally .QUARANTINE_FOREIGN)
5. If user desires the daily digest also touch .QUARANTINE_DIGEST into existence.
6. Check ownership/permissions on the above 2 'trigger' files.
7. echo webmail/QUARANTINE >> .mailboxlist (again check ownership/permissions) to subscribe the folder.
8. Educate user on squirrelmail basics. How to access QUARANTINE, how to 'deliver' a false positive. Actually run them through it.
9. Ensure user is aware of location of documentation on squirrelmail and spam quaranting setup.
10. Relax in the knowledge of a job well done

Update 09/12/05

Green Bank now has 8 alpha testers. So far so good. Of the 8, 6 are squirrelmail users already. One is a vm user and one is on thunderbird.

09/13/05 add nradziwi
09/14/05 added koneil, degan & bmckean. Also modified the digest cronjob to not send an email if a users quarantine folder is empty.

09/15/05 If a user accesses their quarantine through an mua other than webmail and they have filtering within the mua then simply opening the quarantine folder can cause it to be emptied to wherever their filters say. This generates a helpdesk ticket pointing out the inaccuracy of the digest email.

09/16/05 Added tweadon, mholstin. Added line about subscribing the quarantine foler.

09/28/05 Modified logrotation to do it daily with wekkly archive.

-- ChrisClark - 11 Sep 2005