Procmail configuration

Home | UNIX | Oracle | Code | Practical | Private

Procmail configuration

pbraun.nethence.com/doc/mail/postoffice.html
pbraun.nethence.com/doc/mail/vm-pop3d.html
pbraun.nethence.com/doc/mail/dovecot.html
pbraun.nethence.com/doc/mail/spamass-milter.html
pbraun.nethence.com/doc/mail/procmail.html

pbraun.nethence.com/doc/mail/fetchmail.html
pbraun.nethence.com/doc/mail/getmail.html
pbraun.nethence.com/doc/mail/procmail.html


Introduction
If you're getting message for several POP accounts, it's preferable to make a clean cut separation between the procmail configs and call procmail with -m.


Prerequesties
Make sure procmail is installed,
which procmail
#rpm -q procmail
#pkg_info | grep procmail


Procmail rules
Here's some minimal and advanced configuration examples,
cd ~/
wget pbraun.nethence.com/mail/.procmail.minimal
wget pbraun.nethence.com/mail/.procmail.advanced
and a wrapper to validate the rules,
mkdir -p ~/bin
cd ~/bin
wget pbraun.nethence.com/mail/mailcheck.ksh

Note. you might want to process the delivery failures,
#|From:.*MAILER-DAEMON@\

Note. other possible mailing-list tweaks,
#|List-Id:.*\/[^<> ]+\
#|X-Mailing-List:.*<\/[^@<> ]+@[^@<> ]+\

Todo. see if we can get the MATCH from the 'From' or 'Return-Path' header when dealing with 'List-Unsubscribe', e.g. with something like,
#* ^List-Unsubscribe
#* ^From:.*\/[^@<> ]+@[^@<> ]+


Spamassassin
Don't forget configure Spamassassin system-wide (disable -c),
required_hits 5
report_safe 0
#rewrite_header Subject [SPAM]

required_score 5
score HTML_COMMENT_8BITS 0
score UPPERCASE_25_50 0
score UPPERCASE_50_75 0
score UPPERCASE_75_100 0
score OBSCURED_EMAIL 0
score SUBJ_ILLEGAL_CHARS 0
Note. we do not rewrite subject
Note. score* here to permit asian and non-english languages
Note. with -c you would have to edit user prefs after first run,
#vi /var/lib/spamass-milter/.spamassassin/user_prefs
Note. we're dropping the tagged spam messages but otherwise to rewrite the subject,
#rewrite_subject 1
#subject_tag ***SPAM***
Note. nested blocks should not be locked, hence ':0' and not ':0:'


Ready to go
To enable procmail, either use your SMTP,
vi ~/.forward
like,
"|IFS=' '&&p=/usr/local/bin/procmail&&test -f $p&&exec $p -f-||exit 75"
Ref. mirror.ncsa.illinois.edu/procmail-faq/mini-faq.html#forward

Or configure a POP client to send the messages to procmail directly e.g. with fetchmail or getmail.


Reprocess an mbox
Reprocess script
Get that script to reprocess your mboxes (www.uq.edu.au/~suter/software/reprocess-mbox-via-procmail/),
cd /usr/local/bin
wget www.uq.edu.au/~suter/software/reprocess-mbox-via-procmail/reprocess-mbox-via-procmail
mv reprocess-mbox-via-procmail reprocess.sh
chmod +x reprocess.sh
Note. if you're using specific procmailrcs per account, you need to edit it to call the right one,
vi reprocess.sh
change,
## Reprocess our copy via procmail
formail -s procmail -m /etc/procmailrc.mailuser < $tmpfile \
|| die $0: formail failed - data in $tmpfile.

Now as user, temporarly disable the backup feature from your procmail and reprocess an mbox,
cd /var/spool/virtual/example.net/user.imap/lists
/usr/local/bin/reprocess.sh 0unsorted

Alternatives
Use 'formail -s' to split the messages,
cat mbox | formail -ds | procmail -m
Note. '-d' permits to handle non-standard message formats.

Or just resend the messages,
cat mbox | usr/sbin/sendmail -oi user@example.com


Remove duplicates
Get that perl script,
cd /usr/local/bin
wget wdr1.com/hacks/mbox-dedup.pl
eventually fix the perl path,
vi mbox-dedup.pl
and fix the perms,
chmod +x mbox-dedup.pl

Note. for the next step you might have to enable mail user's shell temporarly,
chsh -s /bin/sh mail
su - mail

Remove duplicates,
cd /var/spool/virtual/example.net/user.imap/backup
#mbox-dedup.pl --help
mbox-dedup.pl 201008
rm -f 201008.bak

Note. now disable mail user's shell,
chsh -s /sbin/nologin mail

Note. we didn't had much success with this one,
:0 Wh: msgid.lock
| /usr/bin/formail -D 8192 /var/log/mailuser/msgid.cache


Redeliver everything
We're using the archives to sort all messages again.

Note. for the next step you might have to enable mail user's shell temporarly,
chsh -s /bin/sh mail
su - mail

Make sure your mail archive has no duplicates,
cd /var/spool/virtual/example.net/user.imap/backup
mbox-dedup.pl 201008
rm -f 201008.bak

Temporarly disable the backup feature from your procmail,
vi /etc/procmail.mailuser
comment those lines,
#:0 c
#backupwda/$yearmonth

Clean-up everything,
cd /var/spool/virtual/example.net/user.imap
rm -f inbox 0bcc 0reports 0unsorted lists/*

Redeliver all messages,
cd backup
cp -f 201008 201008.proc
/usr/local/bin/reprocess.sh 201008.proc
ls -l 201008.proc  # should be empty
rm -f 201008.proc

Reenable the backup feature,
vi /etc/procmail.mailuser
comment those lines,
:0 c
backupwda/$yearmonth

Note. now disable mail user's shell,
chsh -s /sbin/nologin mail


Usage
Note. for the next step you might have to enable mail user's shell temporarly,
chsh -s /bin/sh mail
su - mail

Mailing-lists are handled automaticly but you need to create their mbox if you want them to be sent to it. This might help to find out the right name for that lists/mbox,
grep -i unattended 0unsorted \
| egrep '^Sender|^List-Id|^X-BeenThere|^X-BeenThere|^Delivered-To|^X-Loop|^List-Unsubscribe'
touch unattended-info@lists.sourceforge.net

Note. now disable mail user's shell,
chsh -s /sbin/nologin mail


Additional notes
Note. Using SHELL=/bin/ksh, you may enhance like you want e.g.,
:0
* !? [[ -f lists/file1 || -f lists/file2 ]]
{

Note. ':0 chfw' for getting variables out of headers
Note. ':0 Hf' for piping to formail

You may use condition checks against the Received header,
#:0
#* ^Received:.*IP_OR_HOSTNAME
#{

Note. '!' for "NOT"
Note. ':0 Wic' for piping to mkdir (man procmailex)
Note. ':0E' for procmail 'else if' condition

Exclude some messages e.g. messages having a spam-like Reply-To,
* !^Reply-To:.*@spamdomaine.com

You may want to enable some bayesian filter. E.g. BMF,
:0 fw
| bmf -p
:0:
* ^X-Spam-Status: Yes
spam

Don't use those common rules. This may brake PDF attached documents,
# DO NOT USE : THIS BRAKES PDF ATTACHMENTS
# Fix signature dashes
#:0 fBw
#* ^--$
#| sed -e 's/^--$/-- /'

Note. to edit the subject for CC messages,
:0
* ^Cc:.*email@example.net
{
:0 fh
* Subject: \/.*
| formail -I "Subject: CC/ $MATCH"

:0:
$DEFAULT
}

Note. to change the subject for unknown mailing-lists,
{
:0
* ? [[ -f lists/$MATCH ]]
{
:0:
lists/$MATCH
}

:0 chfw
subject=| /usr/bin/formail -xSubject:
:0 Hf
| /usr/bin/formail -i "Subject: $MATCH/ $subject"
:0:
lists/0unsorted
}
Note. conditions explained :
- if the matching mbox exists, we're devlivering the message into it
- if the matching mbox doesn't exist already, we're only editing the Subject line and send it to lists/0unsorted


References
userpages.umbc.edu/~ian/procmail.html
partmaps.org/era/procmail/mini-faq.html
lipas.uwasa.fi/~ts/info/proctips.html#orrules
pm-doc.sourceforge.net/doc/
sial.org/howto/procmail/
Great guide : partmaps.org/era/procmail/quickref.html
Sweet regex manual : www.perlcode.org/tutorials/procmail/regular_expressions.pod
Some mail headers : people.dsv.su.se/~jpalme/ietf/mail-headers/mail-headers.txt
Gentoo procmail : gentoo-wiki.com/Procmail_configuration
Timo's procmail tips and recipes : lipas.uwasa.fi/~ts/info/proctips.html
lists.debian.org/debian-user/2001/12/msg02530.html
luv.asn.au/overheads/procmail.html
tldp.org/LDP/LG/issue14/procmail.html


Home | UNIX | Oracle | Code | Practical | Private | Donate | Print | html/css
© 2010 Pierre-Philipp Braun